Context Management

Production - Context window optimization and management

The Context Management service optimizes how conversation history, schema context, and system prompts are packed into the LLM context window.

12.6.3.1Context Window Configuration

max_context_tokens: int = 8000   # Maximum tokens for context
max_schema_tables: int = 50      # Maximum tables in schema context
max_sample_rows: int = 5         # Sample rows per table

Optimization Strategies

Strategy	Description
Truncation	Remove oldest messages when exceeding token limit
Summarization	Replace old messages with LLM-generated summary
Relevance Filtering	Keep only messages relevant to current query
Schema Pruning	Include only tables/columns relevant to the question

# Optimize context
curl -X POST http://localhost:8000/api/v1/llm/context/optimize \
  -H "Content-Type: application/json" \
  -H "X-Tenant-ID: acme-corp" \
  -d '{
    "messages": [...],
    "max_tokens": 8000,
    "strategy": "relevance_filtering",
    "current_query": "What is total revenue?"
  }'

Response Cache Context Intelligence