Core Concepts
Sequences
A sequence is a versioned template that defines a series of steps, delays, branches, and control flow blocks. You define it once as a JSON definition and then schedule many task instances against it. Sequences are scoped by tenant and namespace.
Task instances
An instance is one execution of a sequence — one contact in a campaign, one user in an onboarding flow, one invoice in a dunning sequence. Each instance has its own state, timezone, priority, metadata, context, and position in the sequence. Instances survive crashes and restarts.
Instance states:
scheduledWaiting to fire at next_fire_atrunningCurrently executing a stepwaitingBlocked on a signal or external eventpausedPaused via signal, can be resumedcompletedAll steps finished successfullyfailedFailed after retries exhaustedcancelledCancelled via signal
Priority levels (highest first):
- ✓Critical (3)
- ✓High (2)
- ✓Normal (1)
- ✓Low (0)
Blocks
Blocks are the building blocks of a sequence. They can be nested recursively to create complex control flow:
stepRun a single handler function. Supports delay, retry, timeout, and rate limit keyparallelRun multiple branches concurrently, wait for all to completeraceRun multiple branches, first to complete wins, cancel the restloopRepeat a body block while a condition is true (max_iterations default: 1000)for_eachIterate over a collection from context, executing the body for each itemrouterEvaluate conditions and route to the first matching branch, with optional defaulttry-catch-finallyExecute try block; on failure run catch block; always run finally blocksub_sequenceCreate a child instance from another sequence. Parent waits for child completion, then merges child context.data into parent
Execution tree
When an instance starts, the engine builds an execution tree from the sequence definition. Each node tracks its block type, state (pending, running, completed, failed, cancelled, skipped), parent relationship, and timestamps. The tree is persisted in the execution_tree table.
Signals
Signals let you interact with running instances from outside. Persisted in signal_inbox and processed before each step execution. Each signal is delivered exactly once.
| Signal | Effect |
|---|---|
| pause | Pause execution — transitions instance to Paused state |
| resume | Resume a paused instance back to Scheduled |
| cancel | Cancel the instance permanently (terminal) |
| update_context | Merge payload into instance context.data |
| custom:* | Application-defined signal with arbitrary JSON payload |
Context
Every instance carries a structured, multi-section context through its entire execution lifetime.
| Section | Purpose | Mutability |
|---|---|---|
| data | Shared data between steps — handler inputs/outputs, user info | Read/write by handlers |
| config | Instance configuration set at creation | Read-only after creation |
| audit | Append-only event log (timestamp, event, details) | Append-only |
| runtime | Current step, attempt count, timestamps — engine-managed | Read-only to handlers |
Rate limits
Each resource (mailbox, API key, phone number) gets its own sliding window rate limit, scoped by tenant. When a task hits the limit, the engine defers it — reschedules to the next available slot instead of dropping or rejecting it. Rate limit checks are atomic (single DB round-trip).
Resource pools
Assign multiple resources to a campaign. The engine distributes instances across them using weighted round-robin. If a resource hits its rate limit, the engine tries the next one. If all are exhausted, it defers. You add or remove resources without changing sequence logic.
Warmup ramps
New resources start at low capacity and gradually ramp up over a configured schedule. A new mailbox might start at 5 sends/day and reach full capacity over 3-4 weeks. The engine enforces the current capacity automatically.
Send windows
Tasks fire only during configured hours in the recipient's timezone. If a step is due outside the window, it defers to the next opening. Combined with per-instance timezone, this ensures every contact receives messages during their business hours.
Delays and business time
Each instance carries its own timezone. Step delays support:
- ✓Relative delays (e.g., 2 days after previous step)
- ✓Business days only — skips weekends (Sat→Mon, Sun→Mon)
- ✓Randomized jitter — ± offset to spread sends naturally
- ✓Timezone-aware calculation using the instance's timezone
Recovery
On startup and periodically, the engine scans for stale instances — those stuck in running state longer than a configurable threshold (default: stale_instance_threshold_secs). Stale instances are reset to scheduled for re-execution.
Webhooks
The engine emits webhook events to configured URLs on instance lifecycle changes (instance.completed, instance.failed). Webhooks are non-blocking (spawned tasks) with automatic retry and exponential backoff.
Dive deeper
Defining Sequences
Sequence structure and definition syntax
Block Types
ForEach, Parallel, Race, Router, TryCatch
Built-in Handlers
http_request, log, sleep, llm_call
Signals & LLM Usage
Pause, resume, cancel, custom signals
Rate Limits
Sliding-window rate limiting per resource
Resource Pools
Multi-resource distribution and warmup