Data Plane Architecture

The Data Plane is the execution engine of the MATIH platform. It consists of 14 polyglot services that process tenant-specific workloads -- executing queries, orchestrating AI agents, training ML models, managing data pipelines, rendering visualizations, and enforcing data governance. Unlike the homogeneous Control Plane, the Data Plane deliberately uses a polyglot architecture to match each service's technology to its problem domain.

The ops-agent-service requires the most resources due to its AI workloads and observability data processing. The ai-service and ml-service also have elevated memory limits for LLM context windows and model loading.

Data Plane services use different health check patterns based on their technology stack:

Java services provide deep health checks via Spring Actuator that verify database connectivity, Redis connectivity, and Kafka broker availability. Python services implement simpler health endpoints that verify the application is running and can handle requests.

The Data Plane services form an analytical pipeline where services chain together:

                              +-------------+
                              |  ai-service |
                              +------+------+
                                     |
                    +----------------+----------------+
                    |                |                |
           +--------v------+  +-----v-------+  +----v----------+
           | query-engine  |  |semantic-layer|  |catalog-service|
           +--------+------+  +-----+-------+  +----+----------+
                    |                |                |
                    +--------+-------+                |
                             |                        |
                    +--------v------+          +------v----------+
                    |     Trino     |          | ontology-service|
                    +---------------+          +-----------------+

The core "Intent to Insights" flow traverses: ai-service --> catalog-service (schema context) + semantic-layer (metric definitions) --> query-engine --> Trino (execution) --> back to ai-service (analysis) --> optionally render-service (visualization).

Sub-Pages

• Query Architecture -- Query engine and Trino integration
• AI Architecture -- Multi-agent orchestrator
• ML Architecture -- MLOps pipeline
• Catalog Architecture -- Metadata and semantic layer
• Pipeline Architecture -- Data pipelines and CDC
• Agent Architecture -- Data plane agent coordination