Resource Pools & Rotation — Orch8 Docs

Resource Pools & Rotation#

Distribute work across multiple resources — API keys, email accounts, proxy servers, or any rate-limited service. Built-in rotation strategies, daily caps, and warmup ramps ensure even distribution without overloading any single resource.

Why Resource Pools#

Many workflows interact with rate-limited external services. Without rotation, a single API key or mail server becomes a bottleneck:

●Email deliverability — rotate across SMTP accounts to avoid sender reputation damage
●API rate limits — distribute calls across multiple keys to stay under per-key quotas
●Proxy rotation — cycle through IPs to avoid blocks during scraping
●Load balancing — spread traffic across servers with weight-based distribution
●New resource onboarding — warmup ramps prevent new accounts from hitting limits on day one

Core Concepts#

Concept	Description
Pool	A named collection of resources with a rotation strategy. Tenant-scoped.
Resource	A single unit in a pool (e.g., one API key, one mail server). Has a unique `resource_key`.
Strategy	How the next resource is selected: round_robin, weighted, or random.
Daily Cap	Maximum assignments per day per resource. Resets at midnight. 0 = unlimited.
Warmup	Gradual daily cap increase over N days for newly added resources.
Assignment	Result of requesting a resource: Assigned(key), Exhausted, or Empty.

Rotation Strategies#

round_robin — Default

Assigns resources in order, cycling through all enabled resources with capacity. Tracks position via round_robin_index. Skips resources that have hit their daily cap.

weighted

Assigns proportional to each resource's weight. A resource with weight 3 receives ~3x more assignments than weight 1. Respects daily caps — exhausted resources are excluded from the weighted selection.

random

Random uniform selection among enabled resources with remaining capacity. Useful when distribution doesn't need to be perfectly even.

Assignment results: When all resources hit their daily cap, assignment returns Exhausted. If no resources are enabled, it returns Empty. Workflows should handle these cases (retry later or fail).

Warmup Ramps#

New email accounts or freshly provisioned API keys often have lower limits initially. Warmup ramps gradually increase the daily cap from a starting value to the full cap over a configurable number of days.

How It Works

The effective daily cap is linearly interpolated:

effective_cap = warmup_start_cap + (daily_cap - warmup_start_cap) * days_active / warmup_days

Example: warmup_start_cap=10, daily_cap=100, warmup_days=10

Day 0:  cap = 10
Day 3:  cap = 10 + (90 * 3 / 10) = 37
Day 5:  cap = 10 + (90 * 5 / 10) = 55
Day 10: cap = 100 (warmup complete)
Day 15: cap = 100 (stays at full)

Configuration

Field	Type	Description
warmup_start	date (YYYY-MM-DD)	When the warmup period begins
warmup_days	u32	Number of days to ramp from start cap to full cap. 0 = no warmup.
warmup_start_cap	u32	Starting daily cap on day 0 of warmup

Pool API#

Create Pool

JSON

POST /pools

{
  "tenant_id": "tenant-1",
  "name": "Email Senders",
  "strategy": "round_robin"
}

Response (201):
{
  "id": "550e8400-...",
  "tenant_id": "tenant-1",
  "name": "Email Senders",
  "strategy": "round_robin",
  "round_robin_index": 0,
  "created_at": "2024-01-15T10:00:00Z",
  "updated_at": "2024-01-15T10:00:00Z"
}

List Pools

JSON

GET /pools?tenant_id=tenant-1

Add Resource

JSON

POST /pools/{pool_id}/resources

{
  "resource_key": "smtp-account-1",
  "name": "Primary SMTP",
  "weight": 2,
  "daily_cap": 500,
  "warmup_start": "2024-01-15",
  "warmup_days": 14,
  "warmup_start_cap": 50
}

Response (201):
{
  "id": "...",
  "pool_id": "...",
  "resource_key": "smtp-account-1",
  "name": "Primary SMTP",
  "weight": 2,
  "enabled": true,
  "daily_cap": 500,
  "daily_usage": 0,
  "daily_usage_date": null,
  "warmup_start": "2024-01-15",
  "warmup_days": 14,
  "warmup_start_cap": 50,
  "created_at": "2024-01-15T10:00:00Z"
}

Update Resource

JSON

PUT /pools/{pool_id}/resources/{resource_id}

{
  "enabled": false,
  "daily_cap": 1000
}

Delete Resource

JSON

DELETE /pools/{pool_id}/resources/{resource_id}

Response: 204 No Content

List Resources

JSON

GET /pools/{pool_id}/resources

Response: Array of PoolResource objects with current daily_usage

Validation: resource_key andname must be 1-255 characters. weight must be at least 1.

Using Pools in Workflows#

Workflows request a resource from a pool before executing rate-limited steps. The assigned resource_key is used to look up the corresponding credential or configuration.

Example: Email Rotation

JSON

[
  {
    "id": "get-sender",
    "type": "pool_assign",
    "params": {
      "pool_id": "email-senders-pool-id"
    }
  },
  {
    "id": "send-email",
    "type": "http_request",
    "params": {
      "url": "https://api.sendgrid.com/v3/mail/send",
      "headers": {
        "Authorization": "Bearer credentials://{{steps.get-sender.output.resource_key}}"
      },
      "body": {
        "from": "{{steps.get-sender.output.resource_key}}@example.com",
        "to": "{{context.recipient}}",
        "subject": "Hello"
      }
    }
  }
]

Handling Exhaustion

JSON

{
  "id": "get-api-key",
  "type": "pool_assign",
  "params": { "pool_id": "openai-keys-pool" },
  "retry": {
    "max_attempts": 3,
    "backoff": "exponential",
    "initial_delay_ms": 60000
  }
}

When all resources are exhausted, the step fails with a transient error. A retry policy with delay gives resources time to reset at midnight.

Production Patterns#

Multi-Provider LLM Pool

Create a pool with weighted resources: OpenAI (weight: 3, cap: 10000), Anthropic (weight: 2, cap: 5000), Groq (weight: 1, cap: 2000). The engine distributes calls proportionally while respecting per-provider rate limits.

SMTP Warmup

Add new mail accounts with warmup_start_cap: 20,daily_cap: 500, warmup_days: 30. Over a month, sending volume gradually increases, building sender reputation without triggering spam filters.

Graceful Degradation

Disable a resource via PATCH enabled: false when it's experiencing issues. The pool automatically routes to remaining resources. Re-enable when resolved — no workflow changes needed.

Capacity Monitoring

Poll GET /pools/{id}/resources to checkdaily_usage vs daily_cap across resources. Alert when total remaining capacity drops below threshold.

Previous← Signals & LLM Usage NextStreaming & Approvals →