Feature Store Architecture

The MATIH Unified Feature Store provides an integrated platform for feature management, connecting feature declaration, lifecycle management, offline/online storage, materialization, streaming features, embedding features, and serving into a single cohesive system.

Architecture Overview

  +-------------------+     +--------------------+     +-------------------+
  | Feature           |     | Streaming Pipeline |     | Embedding Pipeline|
  | Declaration       |     | (Flink/Kafka)      |     | (OpenAI/Custom)   |
  +--------+----------+     +---------+----------+     +--------+----------+
           |                          |                         |
  +--------v----------+     +---------v----------+     +--------v----------+
  | Feature Registry  |     | Online Store       |     | Vector Index      |
  | (Lifecycle FSM)   +---->| (Redis/Aerospike)  |<----+ (Pinecone-style)  |
  +--------+----------+     +---------+----------+     +-------------------+
           |                          ^
  +--------v----------+               |
  | Offline Store     | Materialization
  | (Iceberg tables)  +---------------+
  +--------+----------+
           |
  +--------v----------+
  | Point-in-Time     |
  | Join Engine       |
  +-------------------+

Core Components

Component	Class	Purpose
`UnifiedFeatureStore`	Main entry point	Connects all feature store subsystems
`RedisOnlineStore`	Online backend	Low-latency feature serving via Redis
`AerospikeOnlineStore`	Online backend	High-throughput serving via Aerospike
`IcebergOfflineStore`	Offline backend	Historical features with time travel
`MaterializationEngine`	ETL	Moves data from offline to online store
`StreamingPipeline`	Real-time	Flink SQL-based streaming aggregations
`EmbeddingPipeline`	Vectors	Feature embedding and similarity search
`PointInTimeJoinEngine`	Training	Point-in-time correct feature retrieval
`FeatureServingLayer`	Serving	Unified lookup with online/offline fallback

Feature Lifecycle States

Features follow a state machine from declaration through production:

State	Description
`draft`	Initial declaration, not yet validated
`validating`	Validation in progress
`pending_approval`	Awaiting human approval
`approved`	Approved, ready for materialization
`materializing`	Initial data load in progress
`active`	Serving production traffic
`suspended`	Temporarily disabled
`deprecated`	Marked for removal
`archived`	Fully decommissioned

Factory Function

from src.features.unified_feature_store import create_unified_feature_store
 
store = create_unified_feature_store(
    online_store_type="aerospike",
    aerospike_config={"hosts": [("aerospike", 3000)], "namespace": "features"},
    iceberg_config={"catalog_name": "hive", "warehouse_location": "s3://warehouse"},
    streaming_config={"kafka_bootstrap_servers": "kafka:9092"},
    embedding_config={"model_type": "openai", "dimension": 1536},
)
await store.initialize()

Section Contents

Page	Description
Feast Integration	Feast registry, feature groups, materialization
Online Store	Aerospike/Redis online serving
Offline Store	Iceberg offline store with time travel
Streaming Features	Real-time feature computation
Embedding Features	Vector embedding feature management
Feature Versioning	Version management and compatibility
Agentic Interface	Agent-driven feature discovery

Source Files

File	Path
UnifiedFeatureStore	`data-plane/ml-service/src/features/unified_feature_store.py`
Feature Store API	`data-plane/ml-service/src/api/feature_api.py`
Feast Registry	`data-plane/ml-service/src/features/feast_registry_service.py`

Feature Store Feast Integration