Stage 16: Data Plane Services

Stage 16 deploys all data plane application services to the matih-data-plane namespace. This includes Python, Java, and Node.js services that form the data processing and AI/ML layer of the platform.

Source file: scripts/stages/16-data-plane-services.sh

Services Deployed

Service	Technology	Port	Description
ai-service	Python/FastAPI	8000	Conversational AI, agent orchestration
ml-service	Python/FastAPI	8000	ML model management and serving
data-quality-service	Python/FastAPI	8000	Data quality validation and scoring
pipeline-service	Java	8092	Pipeline orchestration
catalog-service	Java	8086	Data catalog API
semantic-layer	Java	8086	Semantic model service
bi-service	Java	8084	Business intelligence API
query-engine	Java	8080	SQL generation and execution
render-service	Node.js	8098	Chart and report rendering
data-plane-agent	Java	8085	Data plane management agent
ontology-service	Python	8101	Semantic ontology management
governance-service	Python	8080	Data governance policies
ops-agent-service	Python	8080	Operations agent

Image Tag Resolution

Same pattern as Stage 08, with ACR validation and fallback:

Read from build metadata JSON
Validate tag exists in ACR (using catalog-service as sample)
Fall back to latest ACR tag if not found
Write resolved tag to tag file for downstream stages

Deployment Pattern

Each service follows the standard Helm deployment pattern:

helm upgrade --install <service> \
    infrastructure/helm/<service> \
    --namespace matih-data-plane \
    --set image.repository=${ACR_URL}/matih/<service> \
    --set image.tag=${IMAGE_TAG} \
    --wait --timeout 10m

Libraries Used

Library	Purpose
`core/config.sh`	Configuration and Terraform outputs
`k8s/namespace.sh`	Namespace management
`k8s/secrets.sh`	Secret management
`helm/deploy.sh`	Helm deployment
`acr/deploy.sh`	ACR image verification

Dependencies

Requires: 05b-data-plane-infrastructure, 15a-matih-operator
Required by: 17-data-plane-frontend, 18-validate

Dependency Verification

kubectl get pods -n matih-data-plane
kubectl get deployments -n matih-data-plane

Stage 15: AI Infrastructure Stage 17: DP Frontend