Pipelines & automation
Custom pipeline assembly
Compose your own pipeline from typed, validated steps on a canvas, and run it as one recorded job.
The named pipelines cover the common routes through an analysis, but a real project has its own order: pick, fit, transfer an assignment, then measure rates and take the result to a tensor. Chaining those by hand means re-uploading intermediate files and keeping track of which output went where, which is exactly the bookkeeping that gets lost.
Workflow Assembly gives you a canvas of blocks. Each block declares the types of what it consumes and produces, and the canvas refuses to connect ports of different types, so a chain that cannot work cannot be drawn. The platform applies the same rules again when you submit, rather than trusting the browser, then resolves the graph into a plan.
The plan runs as a single durable job — one run, one place to watch it, one set of results — instead of a string of separate submissions. Because it uses the platform's own steps, what comes out carries the same provenance as a result from any named pipeline.
What it needs
- Datasets already stored on your account
- A graph of typed blocks
What you get
- One recorded run covering every step
- Results filed and linked exactly as the named pipelines file theirs
What does the work
Temporal
Open source, redistributed under its licenceMITDurable execution of the assembled chain