Compliance and Audit

The Compliance module provides comprehensive audit trails, model documentation, regulatory reporting, and governance workflows for ML models. It generates model cards, tracks all model lifecycle events, and produces compliance reports aligned with GDPR, CCPA, ECOA, and internal data governance policies.

Compliance Architecture

The ComplianceAuditService in src/compliance/compliance_audit_service.py manages audit events and documentation:

Model Lifecycle Events --> Compliance Audit Service --> Audit Store (PostgreSQL)
                                                    --> Model Cards (Artifact Store)
                                                    --> Compliance Reports (Export)

Audit Trail

Every significant model lifecycle event is recorded in the audit trail:

Event Type	Trigger	Data Captured
`model.trained`	Training job completion	Parameters, metrics, data snapshot
`model.registered`	Model registration	Artifact URI, metadata, tags
`model.stage_transition`	Stage change	From/to stage, approver, reason
`model.deployed`	Production deployment	Endpoint, traffic config
`model.prediction`	Inference request (sampled)	Input features, output, latency
`model.retired`	Model archival	Reason, replacement model
`model.fairness_check`	Fairness assessment	Metrics, pass/fail, groups
`model.explanation`	Explanation generated	Method, feature attributions

Get Audit Trail

GET /api/v1/governance/audit?model_id=model-xyz789

Query Parameters

Parameter	Type	Required	Description
model_id	string	yes	Model identifier
event_type	string	no	Filter by event type
date_from	string	no	Start date (ISO format)
date_to	string	no	End date (ISO format)
limit	integer	no	Max events (default 100)

Response

{
  "events": [
    {
      "event_id": "audit-001",
      "event_type": "model.trained",
      "model_id": "model-xyz789",
      "timestamp": "2025-03-15T10:00:00Z",
      "actor": "user-456",
      "details": {
        "algorithm": "xgboost",
        "dataset_size": 5000,
        "metrics": {"f1_score": 0.912}
      }
    }
  ]
}

Model Cards

Model cards provide standardized documentation following the Google Model Cards framework:

GET /api/v1/governance/model-card?model_id=model-xyz789

Model Card Contents

Section	Description
Overview	Model name, version, owner, purpose
Intended Use	Primary and out-of-scope use cases
Training Data	Dataset description, size, features
Evaluation	Metrics on test set, sliced performance
Fairness	Fairness metrics across protected groups
Limitations	Known limitations and failure modes
Ethical Considerations	Potential risks and mitigations
Deployment	Serving configuration, SLA targets

Response

{
  "model_name": "churn-predictor",
  "version": "v3",
  "owner": "ml-team",
  "purpose": "Predict customer churn probability",
  "intended_use": {
    "primary": "Customer retention campaign targeting",
    "out_of_scope": "Credit decisioning, employment screening"
  },
  "training_data": {
    "source": "customer_features table",
    "samples": 50000,
    "date_range": "2024-01 to 2024-12"
  },
  "evaluation": {
    "test_set_size": 10000,
    "metrics": {"f1_score": 0.912, "auc_roc": 0.97}
  },
  "fairness": {
    "assessed": true,
    "overall_fair": true,
    "details": "Passes demographic parity across gender and age groups"
  }
}

Regulatory Reporting

The compliance module generates reports for specific regulatory frameworks:

Framework	Report Contents
GDPR (Article 22)	Automated decision-making documentation, right to explanation
CCPA	Data usage disclosure, opt-out compliance
ECOA	Fair lending analysis, disparate impact assessment
SOX	Model change control, approval workflows
Internal	Model risk tier, validation status, monitoring coverage

Configuration

Environment Variable	Default	Description
`COMPLIANCE_AUDIT_ENABLED`	`true`	Enable audit trail logging
`COMPLIANCE_PREDICTION_SAMPLE_RATE`	`0.01`	Prediction sampling rate for audit
`COMPLIANCE_RETENTION_DAYS`	`2555`	Audit record retention (7 years)
`COMPLIANCE_MODEL_CARD_AUTO`	`true`	Auto-generate model cards on registration

Fairness & Bias Model Debugging