Explainability (SHAP/LIME)

The Explainability module provides tools for understanding model predictions through feature attribution methods. It supports both global explanations (understanding overall model behavior) and local explanations (understanding individual predictions) using SHAP (SHapley Additive exPlanations) and LIME (Local Interpretable Model-agnostic Explanations).

Explanation Methods

Method	Scope	Approach	Best For
SHAP TreeExplainer	Global + Local	Exact Shapley values for tree models	XGBoost, LightGBM, Random Forest
SHAP KernelExplainer	Global + Local	Model-agnostic kernel approximation	Any model type
SHAP DeepExplainer	Global + Local	Deep learning attribution	Neural networks
LIME	Local	Local linear approximation	Any model, single predictions
Permutation Importance	Global	Feature permutation impact	Quick global importance

Generate Explanation

POST /api/v1/governance/explain

{
  "model_id": "model-xyz789",
  "method": "shap",
  "explanation_type": "global",
  "dataset": {
    "source": "sql",
    "query": "SELECT * FROM ml_features.customer_churn LIMIT 1000"
  },
  "options": {
    "max_samples": 1000,
    "top_features": 10
  }
}

Global Explanation Response

{
  "method": "shap",
  "type": "global",
  "feature_importance": [
    {"feature": "tenure", "importance": 0.35, "direction": "negative"},
    {"feature": "monthly_charges", "importance": 0.28, "direction": "positive"},
    {"feature": "total_charges", "importance": 0.22, "direction": "negative"},
    {"feature": "contract_type", "importance": 0.15, "direction": "mixed"}
  ],
  "interaction_effects": [
    {"features": ["tenure", "contract_type"], "interaction_strength": 0.08}
  ],
  "samples_analyzed": 1000
}

Local Explanation Request

{
  "model_id": "model-xyz789",
  "method": "shap",
  "explanation_type": "local",
  "instance": {
    "tenure": 3,
    "monthly_charges": 89.50,
    "total_charges": 268.50,
    "contract_type": "month-to-month"
  }
}

Local Explanation Response

{
  "prediction": 1,
  "probability": 0.87,
  "base_value": 0.26,
  "feature_contributions": [
    {"feature": "tenure", "value": 3, "shap_value": 0.25},
    {"feature": "contract_type", "value": "month-to-month", "shap_value": 0.18},
    {"feature": "monthly_charges", "value": 89.50, "shap_value": 0.12},
    {"feature": "total_charges", "value": 268.50, "shap_value": 0.06}
  ],
  "explanation_text": "This customer is predicted to churn (87% probability) primarily due to very short tenure (3 months) and month-to-month contract type."
}

SHAP Computation

The ModelExplainabilityService computes SHAP values using the appropriate explainer:

class ModelExplainabilityService:
    def explain_global(self, model, data, method="shap"):
        if method == "shap":
            explainer = shap.TreeExplainer(model)
            shap_values = explainer.shap_values(data)
            return self._format_global(shap_values, data.columns)
 
    def explain_local(self, model, instance, method="shap"):
        explainer = shap.TreeExplainer(model)
        shap_values = explainer.shap_values(instance)
        return self._format_local(shap_values, instance)

Visualization Support

Explanation results include data formatted for frontend visualization:

Visualization	Description	Data Provided
Feature importance bar chart	Top features ranked by importance	Sorted feature list with values
SHAP summary plot	Distribution of SHAP values per feature	Per-feature SHAP distributions
Waterfall plot	Individual prediction breakdown	Cumulative feature contributions
Force plot	Local explanation visualization	Base value and feature forces
Dependence plot	Feature value vs. SHAP value	Scatter data for selected features

Configuration

Environment Variable	Default	Description
`SHAP_MAX_SAMPLES`	`1000`	Maximum samples for SHAP computation
`SHAP_BACKGROUND_SIZE`	`100`	Background dataset size for KernelExplainer
`LIME_NUM_FEATURES`	`10`	Number of features in LIME explanation
`EXPLAIN_TIMEOUT_SECONDS`	`300`	Maximum computation time

ML Governance Overview Fairness & Bias