Agent Pools and Scaling
Beta - AgentPool, PooledAgent, AgentPoolManager for task queuing and horizontal scaling
Agent Pools provide a mechanism for managing groups of identical agent instances that process tasks from a shared queue. This enables horizontal scaling of agent processing capacity and load balancing across multiple agent instances.
12.2.4.1Pool Architecture
Task Queue
+----------+
Task 1 -------->| |
Task 2 -------->| Queue |
Task 3 -------->| |
+----+-----+
|
+-------------+-------------+
| | |
+-----v-----+ +----v------+ +----v------+
| Agent #1 | | Agent #2 | | Agent #3 |
| (idle) | | (busy) | | (idle) |
+-----------+ +-----------+ +-----------+Pool Configuration
from src.agents.pools import AgentPoolManager, PoolConfig
pool_manager = AgentPoolManager()
# Create a pool of SQL agents
pool_id = pool_manager.create_pool(
name="sql-agents",
agent_config=AgentConfig(
name="SQL Agent",
agent_type=AgentType.SPECIALIST,
role=AgentRole.ANALYST,
system_prompt="You are an expert SQL analyst.",
),
config=PoolConfig(
min_agents=2, # Minimum warm agents
max_agents=10, # Maximum agents under load
scale_up_threshold=0.8, # Queue utilization threshold
scale_down_threshold=0.2,
idle_timeout_seconds=300, # Agent idle before removal
),
)Submitting Tasks
curl -X POST http://localhost:8000/api/v1/agents/pools/{pool_id}/submit \
-H "Content-Type: application/json" \
-H "X-Tenant-ID: acme-corp" \
-d '{
"message": "Generate a query for monthly revenue by region",
"priority": "normal",
"timeout_seconds": 120
}'Response:
{
"task_id": "task-uuid-456",
"pool_id": "pool-uuid-123",
"status": "queued",
"position": 0,
"estimated_wait_ms": 250
}Pool Statistics
curl http://localhost:8000/api/v1/agents/pools/{pool_id}/stats?tenant_id=acme-corp{
"pool_id": "pool-uuid-123",
"name": "sql-agents",
"active_agents": 3,
"idle_agents": 1,
"busy_agents": 2,
"queue_depth": 5,
"tasks_completed": 142,
"avg_task_duration_ms": 2340,
"utilization": 0.67
}