Redis Pub/Sub

Redis Pub/Sub provides ephemeral, real-time messaging for scenarios where durability is not required but low latency is critical. It complements Kafka by handling fire-and-forget notifications, configuration broadcasts, and real-time streaming to browser clients.

When to Use Redis Pub/Sub vs Kafka

Criterion	Redis Pub/Sub	Kafka
Message durability	Not persisted	Persisted with configurable retention
Delivery guarantee	At-most-once	At-least-once (with idempotent producers)
Latency	Sub-millisecond	Low milliseconds
Use case	Ephemeral notifications	Durable event processing

Channel Patterns

Channel	Publisher	Subscriber	Purpose
`config:changes`	Config Service	All services	Configuration update broadcast
`ai:stream:{sessionId}`	AI Service	WebSocket bridge	AI response token streaming
`dashboard:refresh:{id}`	BI Service	Frontend WebSocket	Live dashboard data refresh
`health:status:{namespace}`	All services	Observability API	Real-time health updates

Configuration Change Propagation

When the Config Service updates a value, it publishes to Redis Pub/Sub:

Config Service: PUBLISH config:changes
  {"key": "ai.model.default", "value": "gpt-4o", "scope": "global"}

All services: SUBSCRIBE config:changes
  --> Receive notification
  --> Reload local configuration cache
  --> Apply new value without restart

This enables zero-downtime configuration changes across the platform.

AI Response Streaming

The AI Service streams response tokens through Redis Pub/Sub:

AI Service:
  PUBLISH ai:stream:sess-abc {"token": "Revenue", "index": 0, "type": "text"}
  PUBLISH ai:stream:sess-abc {"token": " was", "index": 1, "type": "text"}
  PUBLISH ai:stream:sess-abc {"token": " $2.4M", "index": 2, "type": "text"}
  PUBLISH ai:stream:sess-abc {"done": true, "sql": "SELECT ...", "data": [...]}

WebSocket Bridge:
  SUBSCRIBE ai:stream:sess-abc
  --> Forward each message to browser via WebSocket/SSE

Dashboard Live Refresh

When dashboard data changes, the BI Service notifies connected clients:

BI Service:
  PUBLISH dashboard:refresh:dash-123
    {"widgetId": "w-1", "dataVersion": 42}

Frontend WebSocket:
  SUBSCRIBE dashboard:refresh:dash-123
  --> Refetch widget data from API
  --> Update chart in browser

Operational Considerations

Aspect	Details
Message loss	Expected (at-most-once); critical events must use Kafka
No replay	Messages not stored; late subscribers miss messages
No consumer groups	All subscribers receive all messages (no load balancing)
Pattern matching	Supports pattern subscriptions (e.g., `ai:stream:*`)

Kafka Topology -- Durable event streaming
WebSocket Architecture -- Browser real-time communication
Data Stores: Redis -- Redis infrastructure

Event Schemas WebSocket Architecture