Alerting
The AlertingController manages alert rules and active alerts for the observability system. Alert rules define conditions based on metrics, logs, or traces that trigger notifications when thresholds are breached.
Alert Rules
Create Alert Rule
Endpoint: POST /api/v1/observability/alerts/rules
curl -X POST http://localhost:8088/api/v1/observability/alerts/rules \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${TOKEN}" \
-H "X-Tenant-ID: 550e8400" \
-d '{
"name": "High Error Rate",
"description": "Alert when error rate exceeds 5%",
"query": "rate(http_requests_total{status=~\"5..\"}[5m]) / rate(http_requests_total[5m]) > 0.05",
"duration": "5m",
"severity": "critical",
"labels": {"team": "backend"},
"annotations": {"summary": "Error rate is above 5%"},
"notificationChannels": ["email", "slack"]
}'List Alert Rules
Endpoint: GET /api/v1/observability/alerts/rules
Update Alert Rule
Endpoint: PUT /api/v1/observability/alerts/rules/:ruleId
Delete Alert Rule
Endpoint: DELETE /api/v1/observability/alerts/rules/:ruleId
AlertRule Structure
| Field | Type | Description |
|---|---|---|
id | String | Rule identifier |
name | String | Rule name |
description | String | Rule description |
query | String | PromQL expression that defines the alert condition |
duration | String | How long the condition must be true before firing |
severity | String | info, warning, critical |
labels | Map | Additional labels for routing |
annotations | Map | Human-readable context |
notificationChannels | List | Channels to notify on alert |
enabled | boolean | Whether the rule is active |
Active Alerts
List Active Alerts
Endpoint: GET /api/v1/observability/alerts/active
Returns all currently firing alerts for the tenant.
Acknowledge Alert
Endpoint: POST /api/v1/observability/alerts/:alertId/acknowledge
Resolve Alert
Endpoint: POST /api/v1/observability/alerts/:alertId/resolve
Alert Structure
| Field | Type | Description |
|---|---|---|
id | String | Alert instance ID |
ruleName | String | Originating rule name |
severity | String | Alert severity |
status | String | firing, acknowledged, resolved |
startsAt | Instant | When the alert started firing |
endsAt | Instant | When the alert was resolved |
value | double | Current metric value |
labels | Map | Alert labels |
annotations | Map | Alert annotations |
Alert History
Endpoint: GET /api/v1/observability/alerts/history
Returns historical alerts for trend analysis and post-incident review.
Notification Channels
Alerts can be delivered through multiple channels:
| Channel | Description |
|---|---|
email | Email notifications to configured recipients |
slack | Slack webhook notifications |
pagerduty | PagerDuty incident creation |
webhook | Custom HTTP webhook |