Decision Ranking

The DecisionRankingService provides multi-factor ranking of past decisions for precedent search. It combines semantic similarity, temporal relevance, outcome confidence, and contextual matching to surface the most useful historical precedents.

Overview

When agents or users face a decision, the ranking service retrieves and ranks similar past decisions based on multiple factors. This enables data-driven decision-making by learning from historical outcomes.

Source: data-plane/ai-service/src/context_graph/services/decision_ranking_service.py

Ranking Factors

Factor	Default Weight	Description
`SEMANTIC_SIMILARITY`	0.35	Cosine similarity between decision rationale embeddings
`TEMPORAL_RELEVANCE`	0.20	Exponential decay based on decision age
`OUTCOME_CONFIDENCE`	0.25	Success rate of the historical decision
`CONTEXT_MATCH`	0.10	Overlap between current and historical context
`ENTITY_TYPE_MATCH`	0.05	Whether subject entity types are the same
`DOMAIN_MATCH`	0.05	Whether decisions are in the same domain

Configuration

from context_graph.services.decision_ranking_service import RankingConfig
 
config = RankingConfig(
    semantic_weight=0.35,
    temporal_weight=0.20,
    outcome_weight=0.25,
    context_weight=0.10,
    entity_type_weight=0.05,
    domain_weight=0.05,
    temporal_half_life_days=90,
    min_combined_score=0.3,
)

Outcome Statuses

The outcome of past decisions directly impacts their ranking score:

Status	Score Multiplier	Description
`SUCCESSFUL`	1.0	Full outcome confidence
`PARTIALLY_SUCCESSFUL`	0.6	Moderate confidence
`FAILED`	0.2	Low confidence (still returned for learning)
`PENDING`	0.4	Outcome not yet known
`UNKNOWN`	0.3	Outcome was never recorded

Scoring Formula

The final ranking score is computed as:

score = (semantic_weight * semantic_similarity)
      + (temporal_weight * temporal_decay)
      + (outcome_weight * outcome_confidence)
      + (context_weight * context_overlap)
      + (entity_type_weight * type_match)
      + (domain_weight * domain_match)

Where temporal_decay = exp(-age_days / half_life_days).

API Usage

ranked_decisions = await ranking_service.rank_precedents(
    query_embedding=query_vector,
    tenant_id="acme",
    decision_type="model_deployment",
    top_k=10,
    config=config,
)

Pattern Inference

Beyond ranking individual decisions, the service can infer patterns from clusters of similar decisions:

Common decision sequences (e.g., "approve data access" often follows "data quality review")
Decision outcome correlations (e.g., decisions with peer review have higher success rates)
Temporal patterns (e.g., schema changes are more successful when done mid-week)

Integration Points

Component	Integration
Hybrid Store	Provides embedding vectors for semantic similarity
BiTemporal Store	Stores and retrieves decision history
Pattern Mining	Feeds decision patterns into the mining pipeline
Feedback Scoring	Outcome signals update decision confidence scores

Pattern Mining Feedback Scoring