Load Balancing

The API Gateway manages load balancing through Kong upstreams. The LoadBalancerService supports multiple algorithms including round-robin, weighted distribution, consistent hashing (sticky sessions by header or cookie), and latency-based routing. Each upstream contains targets (backend instances) with configurable weights and health checking.

Algorithms

Algorithm	Kong Value	Description
`ROUND_ROBIN`	`round-robin`	Distributes requests evenly across targets
`CONSISTENT_HASHING`	`consistent-hashing`	Routes requests based on a hash key (header, cookie, IP)
`LATENCY`	`latency`	Routes to the target with lowest latency (Kong Enterprise)

Upstream Configuration

UpstreamConfig Properties

Property	Type	Default	Description
`name`	String	required	Unique upstream identifier
`algorithm`	Algorithm	`ROUND_ROBIN`	Load balancing algorithm
`slots`	int	10000	Number of hash slots for consistent hashing
`hashOn`	HashOn	`NONE`	What to hash on for consistent hashing
`hashOnHeader`	String	null	Header name when `hashOn` is `HEADER`
`hashOnCookie`	String	null	Cookie name when `hashOn` is `COOKIE`
`hashOnCookiePath`	String	`/`	Cookie path for consistent hashing
`hashFallback`	HashOn	null	Fallback hash strategy
`healthchecks`	HealthcheckConfig	null	Active and passive health check config

HashOn Options

Value	Description
`NONE`	No hashing (use with round-robin)
`CONSUMER`	Hash on authenticated consumer
`IP`	Hash on client IP address
`HEADER`	Hash on a specific request header
`COOKIE`	Hash on a specific cookie
`PATH`	Hash on the request path
`QUERY_ARG`	Hash on a query parameter
`URI_CAPTURE`	Hash on a URI capture group

Preset Configurations

Round-Robin

Endpoint: POST /api/v1/gateway/upstreams/preset/round-robin

curl -X POST "http://localhost:8080/api/v1/gateway/upstreams/preset/round-robin?name=ai-upstream" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${TOKEN}" \
  -d '["ai-service-1:8000", "ai-service-2:8000", "ai-service-3:8000"]'

Weighted

Endpoint: POST /api/v1/gateway/upstreams/preset/weighted

curl -X POST "http://localhost:8080/api/v1/gateway/upstreams/preset/weighted?name=ai-upstream" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${TOKEN}" \
  -d '{"ai-service-1:8000": 60, "ai-service-2:8000": 30, "ai-service-3:8000": 10}'

Sticky Sessions (Header)

Endpoint: POST /api/v1/gateway/upstreams/preset/sticky-header

curl -X POST "http://localhost:8080/api/v1/gateway/upstreams/preset/sticky-header?name=ai-upstream&headerName=X-Session-ID" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${TOKEN}" \
  -d '["ai-service-1:8000", "ai-service-2:8000"]'

Sticky Sessions (Cookie)

Endpoint: POST /api/v1/gateway/upstreams/preset/sticky-cookie

curl -X POST "http://localhost:8080/api/v1/gateway/upstreams/preset/sticky-cookie?name=ai-upstream&cookieName=session_id" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${TOKEN}" \
  -d '["ai-service-1:8000", "ai-service-2:8000"]'

Least Connections

Endpoint: POST /api/v1/gateway/upstreams/preset/least-connections

curl -X POST "http://localhost:8080/api/v1/gateway/upstreams/preset/least-connections?name=ai-upstream" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${TOKEN}" \
  -d '["ai-service-1:8000", "ai-service-2:8000", "ai-service-3:8000"]'

Health Checks

Active Health Checks

Active health checks periodically probe targets to determine availability:

Property	Default	Description
`type`	`http`	Health check protocol
`timeout`	1s	Probe timeout
`concurrency`	10	Concurrent probe connections
`httpPath`	`/health`	Path to probe
`healthyInterval`	5s	Probe interval for healthy targets
`healthySuccesses`	2	Successes needed to mark healthy
`healthyHttpStatuses`	200, 302	HTTP statuses considered healthy
`unhealthyInterval`	5s	Probe interval for unhealthy targets
`unhealthyTcpFailures`	2	TCP failures to mark unhealthy
`unhealthyHttpFailures`	5	HTTP failures to mark unhealthy
`unhealthyTimeouts`	3	Timeouts to mark unhealthy
`unhealthyHttpStatuses`	429, 500, 503	HTTP statuses considered unhealthy

Passive Health Checks

Passive health checks monitor real traffic responses to determine target health without additional probe requests.

Target Management

Add a Target

Endpoint: POST /api/v1/gateway/upstreams/:upstreamName/targets

curl -X POST http://localhost:8080/api/v1/gateway/upstreams/ai-upstream/targets \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${TOKEN}" \
  -d '{"target": "ai-service-4:8000", "weight": 100}'

Update Target Weight

Endpoint: PATCH /api/v1/gateway/upstreams/:upstreamName/targets/:targetAddress/weight

curl -X PATCH "http://localhost:8080/api/v1/gateway/upstreams/ai-upstream/targets/ai-service-4:8000/weight?weight=50" \
  -H "Authorization: Bearer ${TOKEN}"

Check Upstream Health

Endpoint: GET /api/v1/gateway/upstreams/:upstreamName/health

curl http://localhost:8080/api/v1/gateway/upstreams/ai-upstream/health \
  -H "Authorization: Bearer ${TOKEN}"

Response returns the health status of each target: HEALTHY, UNHEALTHY, or HEALTHCHECKS_OFF.

Traffic Mirroring Plugins