Pipeline Editor
The Pipeline Editor at frontend/data-workbench/src/components/PipelineBuilder/ provides a visual DAG editor for constructing data pipelines with drag-and-drop node composition.
Components
| Component | File | Purpose |
|---|---|---|
PipelineCanvas | PipelineCanvas.tsx | React Flow canvas for DAG editing |
PipelineNode | PipelineNode.tsx | Custom node for pipeline steps |
NodePalette | NodePalette.tsx | Draggable node type catalog |
NodeConfigPanel | NodeConfigPanel.tsx | Node configuration side panel |
Types
// frontend/data-workbench/src/components/PipelineBuilder/types.ts
interface PipelineNodeData {
id: string;
type: 'source' | 'transform' | 'sink' | 'quality' | 'branch' | 'join';
name: string;
config: Record<string, unknown>;
status?: 'idle' | 'running' | 'success' | 'failed';
}Node Types
| Category | Node Types |
|---|---|
| Source | Database, File, API, Stream |
| Transform | SQL, Python, dbt, Filter, Aggregate |
| Sink | Database, File, API, Data Lake |
| Quality | Validation, Profiling, Alerting |
| Flow | Branch, Join, Loop |
PipelineCanvas
Features:
- Drag nodes from palette to canvas
- Connect nodes by drawing edges
- Validate DAG (no cycles, required inputs)
- Execute pipeline with real-time status
- View execution logs per node
- Integration with
PipelineApiClient