Orchestration and IaC

The MATIH Platform is fully Kubernetes-native, with all services packaged as Helm charts and all infrastructure defined as code. This section documents the orchestration technologies, infrastructure-as-code tools, and deployment patterns used across the platform.

Orchestration Stack

Technology	Version	Purpose
Kubernetes	1.29+	Container orchestration and workload scheduling
Helm	3.14+	Kubernetes package management
Terraform	1.6+	Infrastructure provisioning on cloud providers
Crossplane	Latest	Kubernetes-native infrastructure management
ArgoCD	Latest	GitOps continuous delivery
cert-manager	Latest	TLS certificate lifecycle management
External Secrets Operator	Latest	Secret synchronization from external vaults
Strimzi Operator	0.38+	Kafka cluster management on Kubernetes
KubeRay Operator	Latest	Ray cluster management on Kubernetes
Temporal	Latest	Durable workflow orchestration for pipelines

Kubernetes

Kubernetes is the foundation of the platform's deployment model:

Aspect	Details
Minimum version	1.29
Namespaces	7 platform namespaces + per-tenant namespaces
NetworkPolicies	Per-namespace ingress/egress rules
ResourceQuotas	Per-tenant resource limits
RBAC	Service accounts with least-privilege bindings
Pod Security	Non-root containers, read-only filesystems

Helm

All 55+ platform components are packaged as Helm charts:

Aspect	Details
Chart structure	One chart per service with `values.yaml` base + environment overrides
Environment overrides	`values-dev.yaml`, `values-staging.yaml`, `values-prod.yaml`
Secret management	All credentials via `secretKeyRef`, never hardcoded in values
Deep merge behavior	Environment overrides merge into base values
Validation	`helm template` + `helm lint` before every deployment

Helm Chart Organization

infrastructure/helm/
  +-- ai-service/
  +-- bi-service/
  +-- catalog-service/
  +-- iam-service/
  +-- tenant-service/
  +-- ingress-nginx/
  +-- ... (55+ charts)

Terraform

Terraform provisions cloud infrastructure across multiple providers:

Provider	Resources
Azure	AKS cluster, Azure DNS, Key Vault, Storage Account
AWS	EKS cluster, Route53, Secrets Manager, S3
GCP	GKE cluster, Cloud DNS, Secret Manager, GCS

Terraform modules are organized by cloud provider:

infrastructure/terraform/
  +-- modules/
  |     +-- azure/
  |     +-- aws/
  |     +-- gcp/
  +-- environments/
        +-- azure-matihlabs/
        +-- aws-production/

cert-manager

cert-manager automates TLS certificate management:

Aspect	Details
Challenge type	DNS-01 via Azure DNS (production)
Issuers	`letsencrypt-staging-dns01` (dev), `letsencrypt-prod-dns01` (production)
Per-tenant certificates	Automated per-tenant certificate provisioning
Renewal	Automatic renewal before expiry

External Secrets Operator

External Secrets Operator synchronizes secrets from cloud vaults:

Provider	Secret Store
Azure	Azure Key Vault
AWS	AWS Secrets Manager
GCP	GCP Secret Manager

In development, secrets are created via the scripts/lib/k8s/dev-secrets.sh script.

Deployment Workflow

Step	Tool	Command
Build service	Build script	`./scripts/build.sh`
Build and deploy single service	Deploy script	`./scripts/tools/service-build-deploy.sh`
Full CD pipeline	CD script	`./scripts/cd-new.sh all dev`
Infrastructure only	CD script	`./scripts/cd-new.sh infra dev`
Check platform status	Status script	`./scripts/tools/platform-status.sh`

Compute Engines -- Trino, Spark, Flink
Backend Stack -- Service technologies
Architecture: Service Topology -- Service deployment topology
Platform Admin Persona -- Operations workflow

ML & AI Infrastructure Compute Engines