A/B Testing

The A/B Testing system enables controlled experimentation across the platform. The ABTestingController at /api/v1/experiments provides a full experiment lifecycle -- from creation through variant assignment, event tracking, and statistical result analysis. Experiments are tied to feature flags, allowing seamless integration between feature rollout and data-driven decision making.

ABTestingController

@RestController
@RequestMapping("/api/v1/experiments")
@Tag(name = "A/B Testing", description = "A/B experiment management and tracking")
public class ABTestingController {
    private final ABTestingService abTestingService;
}

Experiment Lifecycle

DRAFT -> RUNNING -> COMPLETED -> ARCHIVED
            |
            v
          PAUSED -> RUNNING (resumed)

Status	Description
`DRAFT`	Experiment created but not yet started
`RUNNING`	Actively assigning variants and collecting data
`PAUSED`	Temporarily stopped; can be resumed
`COMPLETED`	Experiment finished with a declared winner
`ARCHIVED`	Historical record; no longer active

Create an Experiment

Endpoint: POST /api/v1/experiments

curl -X POST http://localhost:8888/api/v1/experiments \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "X-Tenant-Id: 550e8400-e29b-41d4-a716-446655440000" \
  -d '{
    "flagId": "880e8400-e29b-41d4-a716-446655440000",
    "experimentKey": "checkout-flow-v2",
    "name": "Checkout Flow Redesign",
    "description": "Testing the new streamlined checkout flow",
    "hypothesis": "The simplified checkout will increase conversion rate by 15%",
    "variants": {
      "control": {
        "name": "Current Checkout",
        "description": "Existing multi-step checkout",
        "weight": 50,
        "isControl": true
      },
      "treatment": {
        "name": "Simplified Checkout",
        "description": "New single-page checkout",
        "weight": 50,
        "isControl": false
      }
    },
    "trafficAllocation": 100,
    "primaryMetric": "conversion_rate",
    "secondaryMetrics": ["average_order_value", "time_to_purchase"],
    "targetingRules": [],
    "mutualExclusionGroup": "checkout-experiments",
    "scheduledStart": "2026-02-15T00:00:00Z",
    "scheduledEnd": "2026-03-15T00:00:00Z",
    "createdBy": "550e8400-e29b-41d4-a716-446655440001"
  }'

Response (201 Created):

{
  "id": "exp-001",
  "tenantId": "550e8400-...",
  "experimentKey": "checkout-flow-v2",
  "name": "Checkout Flow Redesign",
  "status": "DRAFT",
  "variants": {
    "control": { "name": "Current Checkout", "weight": 50, "isControl": true },
    "treatment": { "name": "Simplified Checkout", "weight": 50, "isControl": false }
  },
  "trafficAllocation": 100,
  "primaryMetric": "conversion_rate",
  "createdAt": "2026-02-12T10:00:00Z"
}

Start an Experiment

Transition an experiment from DRAFT to RUNNING.

Endpoint: POST /api/v1/experiments/{experimentId}/start

curl -X POST http://localhost:8888/api/v1/experiments/exp-001/start \
  -H "Authorization: Bearer ${TOKEN}"

Assign User to Variant

When a user encounters the experiment, assign them to a variant. The assignment is deterministic -- the same user always gets the same variant.

Endpoint: POST /api/v1/experiments/{experimentId}/assign

curl -X POST http://localhost:8888/api/v1/experiments/exp-001/assign \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${TOKEN}" \
  -d '{
    "userId": "user-001",
    "context": {
      "browser": "chrome",
      "platform": "web",
      "country": "US"
    }
  }'

Response:

{
  "experimentId": "exp-001",
  "userId": "user-001",
  "variantKey": "treatment",
  "variantName": "Simplified Checkout",
  "assignedAt": "2026-02-15T10:30:00Z"
}

Track Events

Track Impression

Record when a user sees the experiment variant.

Endpoint: POST /api/v1/experiments/{experimentId}/track/impression

curl -X POST http://localhost:8888/api/v1/experiments/exp-001/track/impression \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${TOKEN}" \
  -d '{
    "userId": "user-001",
    "variantKey": "treatment",
    "metadata": { "page": "checkout", "session": "sess-123" }
  }'

Track Conversion

Record a conversion event with an optional metric value.

Endpoint: POST /api/v1/experiments/{experimentId}/track/conversion

curl -X POST http://localhost:8888/api/v1/experiments/exp-001/track/conversion \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${TOKEN}" \
  -d '{
    "userId": "user-001",
    "variantKey": "treatment",
    "metricName": "conversion_rate",
    "value": 1.0,
    "metadata": { "orderTotal": 99.99 }
  }'

Track Custom Event

Record any custom event for secondary metric analysis.

Endpoint: POST /api/v1/experiments/{experimentId}/track/event

curl -X POST http://localhost:8888/api/v1/experiments/exp-001/track/event \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${TOKEN}" \
  -d '{
    "userId": "user-001",
    "variantKey": "treatment",
    "eventName": "time_to_purchase",
    "value": 45.2,
    "metadata": { "step_count": 3 }
  }'

Get Experiment Results

Retrieve statistical analysis of the experiment results.

Endpoint: GET /api/v1/experiments/{experimentId}/results

curl http://localhost:8888/api/v1/experiments/exp-001/results \
  -H "Authorization: Bearer ${TOKEN}"

Response:

{
  "experimentId": "exp-001",
  "status": "RUNNING",
  "duration": "P14D",
  "totalParticipants": 5000,
  "variantResults": {
    "control": {
      "participants": 2500,
      "impressions": 8750,
      "conversions": 375,
      "conversionRate": 0.15,
      "metrics": {
        "conversion_rate": { "mean": 0.15, "stddev": 0.02 },
        "average_order_value": { "mean": 87.50, "stddev": 23.40 }
      }
    },
    "treatment": {
      "participants": 2500,
      "impressions": 8900,
      "conversions": 450,
      "conversionRate": 0.18,
      "metrics": {
        "conversion_rate": { "mean": 0.18, "stddev": 0.02 },
        "average_order_value": { "mean": 92.30, "stddev": 25.10 }
      }
    }
  },
  "statisticalSignificance": {
    "pValue": 0.003,
    "confidenceLevel": 0.95,
    "isSignificant": true,
    "recommendedWinner": "treatment",
    "uplift": 0.20
  }
}

Recalculate Results

Force a recalculation of experiment results with the latest data.

Endpoint: POST /api/v1/experiments/{experimentId}/results/recalculate

Complete an Experiment

Declare a winner and complete the experiment.

Endpoint: POST /api/v1/experiments/{experimentId}/complete

curl -X POST http://localhost:8888/api/v1/experiments/exp-001/complete \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${TOKEN}" \
  -d '{
    "winnerVariant": "treatment",
    "conclusion": "Simplified checkout increased conversion by 20% with statistical significance"
  }'

Pause and Archive

Pause: POST /api/v1/experiments/{experimentId}/pause

Archive: POST /api/v1/experiments/{experimentId}/archive

List Experiments

All experiments (paginated)

Endpoint: GET /api/v1/experiments

By status

Endpoint: GET /api/v1/experiments/status/{status}

curl http://localhost:8888/api/v1/experiments/status/RUNNING \
  -H "Authorization: Bearer ${TOKEN}" \
  -H "X-Tenant-Id: 550e8400-e29b-41d4-a716-446655440000"

Mutual Exclusion Groups

Experiments in the same mutualExclusionGroup will never assign the same user to multiple experiments simultaneously. This prevents interference between related experiments.

{
  "mutualExclusionGroup": "checkout-experiments"
}

When a user is already assigned to one experiment in a mutual exclusion group, they will be excluded from all other experiments in the same group.

Data Model

ABExperiment Entity

Field	Type	Description
`id`	UUID	Experiment identifier
`tenantId`	UUID	Tenant scope
`flagId`	UUID	Associated feature flag
`experimentKey`	String	Unique key within tenant
`status`	Enum	DRAFT, RUNNING, PAUSED, COMPLETED, ARCHIVED
`variants`	JSON	Map of variant configurations
`trafficAllocation`	Integer	Percentage of traffic in experiment (0-100)
`primaryMetric`	String	Primary success metric
`mutualExclusionGroup`	String	Exclusion group for preventing interference
`winnerVariant`	String	Declared winner (after completion)
`conclusion`	String	Summary of findings

Feature Flags Targeting Rules