Creating Lineage

Lineage edges can be created through three primary mechanisms: direct API calls, batch operations, and OpenLineage event ingestion. This page covers all creation methods.

Create a Lineage Edge

Create a single lineage edge between two entities:

POST /v1/lineage/edges

curl -X POST "http://localhost:8086/v1/lineage/edges" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '{
    "tenantId": "550e8400-e29b-41d4-a716-446655440000",
    "sourceEntityId": "550e8400-e29b-41d4-a716-446655440001",
    "sourceEntityFqn": "warehouse.raw.orders",
    "sourceEntityType": "table",
    "targetEntityId": "550e8400-e29b-41d4-a716-446655440002",
    "targetEntityFqn": "warehouse.analytics.dim_orders",
    "targetEntityType": "table",
    "lineageType": "DIRECT",
    "lineageSource": "PIPELINE",
    "pipelineId": "550e8400-e29b-41d4-a716-446655440010",
    "pipelineRunId": "550e8400-e29b-41d4-a716-446655440011",
    "sqlQuery": "INSERT INTO analytics.dim_orders SELECT * FROM raw.orders",
    "description": "ETL: raw orders to dimension table",
    "confidence": 1.0,
    "manual": false,
    "columnLineage": [
      {
        "sourceColumn": "order_id",
        "targetColumn": "order_id",
        "transformation": "DIRECT_COPY"
      },
      {
        "sourceColumn": "amount",
        "targetColumn": "total_amount",
        "transformation": "CAST(amount AS DECIMAL(18,2))"
      }
    ],
    "metadata": {
      "pipeline.name": "orders-etl",
      "pipeline.schedule": "daily"
    },
    "tags": ["etl", "production"],
    "createdBy": "data-engineering"
  }'

Response (201 Created)

{
  "id": "edge-new-001",
  "tenantId": "550e8400-...",
  "sourceEntityId": "550e8400-...",
  "sourceEntityFqn": "warehouse.raw.orders",
  "targetEntityFqn": "warehouse.analytics.dim_orders",
  "lineageType": "DIRECT",
  "lineageSource": "PIPELINE",
  "confidence": 1.0,
  "createdAt": "2026-02-12T10:30:00Z",
  "createdBy": "data-engineering"
}

Required Roles

Edge creation requires CATALOG_ADMIN or DATA_ENGINEER role.

Request Fields

Field	Required	Description
`tenantId`	Yes	Tenant UUID
`sourceEntityId`	Yes	Source entity UUID
`sourceEntityFqn`	Yes	Fully qualified name of source (e.g., `db.schema.table`)
`sourceEntityType`	Yes	Type: `table`, `view`, `pipeline`, `dashboard`
`targetEntityId`	Yes	Target entity UUID
`targetEntityFqn`	Yes	Fully qualified name of target
`targetEntityType`	Yes	Type of target entity
`lineageType`	Yes	`DIRECT`, `INDIRECT`, or `DERIVED`
`lineageSource`	Yes	`PIPELINE`, `QUERY_ANALYSIS`, `MANUAL`, `OPENMETADATA`, `INTEGRATION`
`pipelineId`	No	Pipeline that creates this lineage
`pipelineRunId`	No	Specific pipeline run
`sqlQuery`	No	SQL that defines the transformation
`description`	No	Human-readable description
`confidence`	No	Confidence score (0.0 to 1.0)
`manual`	No	Whether this was manually created
`columnLineage`	No	Column-level lineage mappings
`metadata`	No	Additional key-value metadata
`tags`	No	Tags for categorization
`createdBy`	No	Creator identifier

Batch Create Edges

Create multiple lineage edges in a single request:

POST /v1/lineage/edges/batch

curl -X POST "http://localhost:8086/v1/lineage/edges/batch" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $TOKEN" \
  -d '[
    {
      "tenantId": "550e8400-e29b-41d4-a716-446655440000",
      "sourceEntityId": "tbl-raw-customers",
      "sourceEntityFqn": "warehouse.raw.customers",
      "sourceEntityType": "table",
      "targetEntityId": "tbl-dim-customers",
      "targetEntityFqn": "warehouse.analytics.dim_customers",
      "targetEntityType": "table",
      "lineageType": "DIRECT",
      "lineageSource": "PIPELINE",
      "confidence": 1.0
    },
    {
      "tenantId": "550e8400-e29b-41d4-a716-446655440000",
      "sourceEntityId": "tbl-raw-products",
      "sourceEntityFqn": "warehouse.raw.products",
      "sourceEntityType": "table",
      "targetEntityId": "tbl-dim-products",
      "targetEntityFqn": "warehouse.analytics.dim_products",
      "targetEntityType": "table",
      "lineageType": "DIRECT",
      "lineageSource": "PIPELINE",
      "confidence": 1.0
    }
  ]'

Response (201 Created)

{
  "totalRequested": 2,
  "totalCreated": 2,
  "edges": [
    { "id": "edge-batch-001", "sourceEntityFqn": "warehouse.raw.customers", "targetEntityFqn": "warehouse.analytics.dim_customers" },
    { "id": "edge-batch-002", "sourceEntityFqn": "warehouse.raw.products", "targetEntityFqn": "warehouse.analytics.dim_products" }
  ]
}

OpenLineage Event Ingestion

Ingest lineage from external systems that emit OpenLineage events (Spark, Airflow, dbt, Trino, Flink):

POST /v1/lineage/openlineage

curl -X POST "http://localhost:8086/v1/lineage/openlineage" \
  -H "Content-Type: application/json" \
  -H "X-Tenant-ID: 550e8400-e29b-41d4-a716-446655440000" \
  -d '{
    "eventType": "COMPLETE",
    "eventTime": "2026-02-12T10:00:00Z",
    "job": {
      "namespace": "spark://analytics-cluster",
      "name": "orders_etl_job"
    },
    "run": {
      "runId": "run-12345-abcde"
    },
    "inputs": [
      {
        "namespace": "warehouse",
        "name": "raw.orders",
        "facets": {}
      },
      {
        "namespace": "warehouse",
        "name": "raw.customers",
        "facets": {}
      }
    ],
    "outputs": [
      {
        "namespace": "warehouse",
        "name": "analytics.dim_orders",
        "facets": {
          "columnLineage": {
            "fields": {
              "order_id": {
                "inputFields": [
                  { "namespace": "warehouse", "name": "raw.orders", "field": "order_id" }
                ]
              },
              "customer_name": {
                "inputFields": [
                  { "namespace": "warehouse", "name": "raw.customers", "field": "name", "transformationType": "RENAME" }
                ]
              }
            }
          }
        }
      }
    ]
  }'

Response

{
  "status": "accepted",
  "edgesCreated": 2,
  "eventType": "COMPLETE",
  "jobName": "spark://analytics-cluster.orders_etl_job",
  "runId": "run-12345-abcde"
}

Job Namespace Mapping

The controller automatically maps job namespaces to lineage sources:

Namespace Pattern	Lineage Source
Contains `spark`, `airflow`, `dag`, `dbt`, `flink`	`PIPELINE`
Contains `trino`, `presto`	`QUERY_ANALYSIS`
Contains `openmetadata`	`OPENMETADATA`
Other	`INTEGRATION`

Column Lineage Facet Extraction

When OpenLineage events include the columnLineage facet on outputs, the controller automatically extracts column-level lineage mappings from the fields structure.

Batch OpenLineage Ingestion

Ingest multiple OpenLineage events at once:

POST /v1/lineage/openlineage/batch

curl -X POST "http://localhost:8086/v1/lineage/openlineage/batch" \
  -H "Content-Type: application/json" \
  -H "X-Tenant-ID: 550e8400-e29b-41d4-a716-446655440000" \
  -d '[
    {
      "eventType": "COMPLETE",
      "eventTime": "2026-02-12T10:00:00Z",
      "job": { "namespace": "airflow://prod", "name": "dag_orders" },
      "run": { "runId": "run-001" },
      "inputs": [{ "namespace": "warehouse", "name": "raw.orders" }],
      "outputs": [{ "namespace": "warehouse", "name": "staging.orders_clean" }]
    },
    {
      "eventType": "COMPLETE",
      "eventTime": "2026-02-12T10:05:00Z",
      "job": { "namespace": "airflow://prod", "name": "dag_customers" },
      "run": { "runId": "run-002" },
      "inputs": [{ "namespace": "warehouse", "name": "raw.customers" }],
      "outputs": [{ "namespace": "warehouse", "name": "staging.customers_clean" }]
    }
  ]'

Response

{
  "status": "completed",
  "totalEvents": 2,
  "processedEvents": 2,
  "totalEdgesCreated": 2,
  "errors": null
}

Manage Existing Edges

Get Edge

GET /v1/lineage/edges/{edgeId}

Delete Edge (Soft Delete)

DELETE /v1/lineage/edges/{edgeId}

curl -X DELETE "http://localhost:8086/v1/lineage/edges/edge-001" \
  -H "Authorization: Bearer $TOKEN"

Verify Edge

Mark an edge as verified by a data steward:

POST /v1/lineage/edges/{edgeId}/verify

curl -X POST "http://localhost:8086/v1/lineage/edges/edge-001/verify" \
  -H "Authorization: Bearer $TOKEN"

Deactivate Entity Lineage

Deactivate all lineage edges for a deleted entity:

DELETE /v1/lineage/entity/{entityId}?tenantId={tenantId}

curl -X DELETE "http://localhost:8086/v1/lineage/entity/550e8400-e29b-41d4-a716-446655440001?tenantId=550e8400-e29b-41d4-a716-446655440000" \
  -H "Authorization: Bearer $TOKEN"

Response:

{
  "entityId": "550e8400-...",
  "deactivatedEdges": 7
}

Source Reference

Component	File
Create edge	`LineageController.java` -- `createEdge()`
Batch create	`LineageController.java` -- `createEdgesBatch()`
OpenLineage ingestion	`LineageController.java` -- `ingestOpenLineageEvent()`
OpenLineage batch	`LineageController.java` -- `ingestOpenLineageEventsBatch()`
Edge management	`LineageController.java` -- `getEdge()`, `deleteEdge()`, `verifyEdge()`
Entity deactivation	`LineageController.java` -- `deactivateEntityLineage()`
Lineage service	`LineageService.java`

Visualization & Export Overview