Signals & LLM Usage — Orch8 Docs

Signals & LLM Usage#

Signals let you control running workflow instances from the outside — pause, resume, cancel, inject data, or trigger custom logic. Combined with LLM usage tracking, you get full observability into AI-powered orchestrations.

Signal Overview#

A signal is an asynchronous message sent to a running workflow instance. Signals are stored in an inbox and processed by the scheduler on the next tick — they are durable, idempotent, and order-preserving.

Durable

Signals are persisted in the database. They survive server restarts and are never lost.

Idempotent

Each signal has a unique ID and a delivered flag. Once processed, it won't be re-delivered.

Atomic

Signals are enqueued with an atomic check — if the target instance is already in a terminal state, the signal is rejected.

Interceptable

Sequences can define an on_signal interceptor to run custom logic when any signal arrives.

Signal Types#

Signal	Effect	Payload	Reversible
pause	Transitions instance to Paused state. Execution aborts after current step completes.	None	Yes (resume)
resume	Transitions from Paused back to Scheduled. Execution continues.	None	N/A
cancel	Marks cancellable nodes as Cancelled. Respects non-cancellable scopes.	None	No
update_context	Merges payload into the instance's execution context.	JSON object (ExecutionContext)	Yes (send again)
custom	User-defined. Delivered to `on_signal` interceptor or `wait_for_input` steps.	Any JSON	Depends on logic

Sending Signals#

Via REST API

JSON

POST /instances/{instance_id}/signals

{
  "signal_type": "pause"
}

Response:
{
  "signal_id": "550e8400-e29b-41d4-a716-446655440000"
}

Update Context Signal

JSON

POST /instances/{instance_id}/signals

{
  "signal_type": "update_context",
  "payload": {
    "approved": true,
    "reviewer": "alice@example.com",
    "notes": "Looks good, proceed with deployment."
  }
}

The payload is merged into the instance context. Subsequent steps can access these values via template interpolation.

Custom Signal

JSON

POST /instances/{instance_id}/signals

{
  "signal_type": "approval_received",
  "payload": {
    "decision": "approved",
    "amount": 5000
  }
}

Via CLI

Bash

# Pause a running instance
orch8 signal <instance-id> pause

# Resume a paused instance
orch8 signal <instance-id> resume

# Cancel an instance
orch8 signal <instance-id> cancel

# Update context with JSON payload
orch8 signal <instance-id> update_context '{"approved": true}'

# Send custom signal
orch8 signal <instance-id> my_custom_event '{"key": "value"}'

Scoped Cancellation#

Not all work should stop when a cancel signal arrives. Orch8 supports three mechanisms for protecting critical work:

1. Per-Step Flag: cancellable: false

Mark individual steps as non-cancellable. They will continue to execute even after a cancel signal.

JSON

{
  "type": "step",
  "id": "send-receipt",
  "handler": "http_request",
  "cancellable": false,
  "params": { "url": "https://api.stripe.com/v1/receipts", "method": "POST" }
}

2. CancellationScope Block

Wrap a group of steps in a cancellation scope. All steps inside are protected from cancel signals.

3. Finally Branch (try-catch)

Steps in the finally_block branch of a try-catch block are always non-cancellable — they run regardless of success, failure, or cancellation.

When a cancel signal arrives, the scheduler identifies all non-cancellable nodes. Only cancellable nodes transition to Cancelled. If non-cancellable work remains active, the instance continues running until that work completes.

Human-in-the-Loop#

The wait_for_input step type pauses execution until a custom signal matching a specific pattern is received. This enables approval workflows, human review gates, and interactive processes.

Sequence Definition

JSON

{
  "type": "step",
  "id": "wait-for-approval",
  "handler": "human_review",
  "params": { "review_data": "Pending approval" },
  "wait_for_input": {
    "prompt": "Review and approve or reject.",
    "timeout": 86400000
  }
}

Completing the Wait

JSON

POST /instances/{instance_id}/signals

{
  "signal_type": "human_input:approval",
  "payload": {
    "approved": true,
    "reviewer": "manager@company.com"
  }
}

The signal payload becomes the step output, accessible to subsequent steps via template interpolation.

Example: Expense Approval

JSON

[
  {
    "type": "step",
    "id": "notify-manager",
    "handler": "http_request",
    "params": {
      "url": "https://slack.com/api/chat.postMessage",
      "method": "POST",
      "body": {
        "channel": "#approvals",
        "text": "Expense ${{context.data.amount}} needs approval."
      }
    }
  },
  {
    "type": "step",
    "id": "wait-approval",
    "handler": "human_review",
    "params": { "review_data": "Expense approval pending" },
    "wait_for_input": {
      "prompt": "Approve or reject the expense.",
      "timeout": 86400000,
      "store_as": "expense_decision"
    }
  },
  {
    "type": "router",
    "id": "process-decision",
    "routes": [
      {
        "condition": "context.data.expense_decision == 'approve'",
        "blocks": [{ "type": "step", "id": "reimburse", "handler": "reimburse_expense" }]
      }
    ],
    "default": [{ "type": "step", "id": "reject-email", "handler": "send_rejection" }]
  }
]

Inter-Workflow Signals#

A running workflow can signal another workflow instance using the built-in send_signal step handler. This enables coordination between independent workflow instances.

JSON

{
  "type": "step",
  "id": "notify-parent",
  "handler": "send_signal",
  "params": {
    "instance_id": "{{context.data.parent_instance_id}}",
    "signal_type": "child_completed",
    "payload": {
      "result": "{{steps.process.output}}",
      "child_id": "{{context.data.instance_id}}"
    }
  }
}

Safety: The handler validates that the target instance exists, belongs to the same tenant, and is not in a terminal state. An atomic enqueue_signal_if_active operation prevents race conditions.

LLM Usage Tracking#

Every llm_call step returns token usage data alongside the model response. This enables cost tracking, quota enforcement, and usage analytics across all supported providers.

Response Format

JSON

// Step output from any llm_call step
{
  "provider": "openai",
  "model": "gpt-4o",
  "message": {
    "role": "assistant",
    "content": "Here is your summary...",
    "tool_calls": []
  },
  "finish_reason": "stop",
  "usage": {
    "prompt_tokens": 1250,
    "completion_tokens": 340,
    "total_tokens": 1590
  }
}

Failover Response

When using the providers failover array, the response includes which providers were attempted:

JSON

{
  "provider": "openai",
  "model": "gpt-4o",
  "message": { "role": "assistant", "content": "..." },
  "finish_reason": "stop",
  "usage": {
    "prompt_tokens": 800,
    "completion_tokens": 200,
    "total_tokens": 1000
  },
  "tried": ["anthropic", "openai"]
}

The tried array shows that Anthropic was attempted first (and failed with a transient error) before OpenAI succeeded.

Supported Providers

OpenAI

Anthropic

Deepseek

Gemini

Groq

Together

Mistral

Perplexity

Qwen

OpenRouter

All OpenAI-compatible providers use the same protocol. Anthropic uses the native Messages API. Usage fields are normalized to the same format regardless of provider.

Usage Aggregation Patterns#

LLM usage is stored in step outputs. Here are patterns for aggregating and acting on usage data within workflows.

Pattern 1: Accumulate in Context

Use a post-LLM step to accumulate token counts in the execution context:

JSON

[
  {
    "type": "step",
    "id": "ai-summarize",
    "handler": "llm_call",
    "params": {
      "provider": "openai",
      "model": "gpt-4o",
      "api_key_env": "OPENAI_API_KEY",
      "messages": [{ "role": "user", "content": "Summarize: {{context.data.document}}" }]
    }
  },
  {
    "type": "step",
    "id": "track-usage",
    "handler": "transform",
    "params": {
      "output": {
        "total_tokens_used": "{{context.data.total_tokens_used + steps.ai-summarize.output.usage.total_tokens}}",
        "calls_made": "{{context.data.calls_made + 1}}"
      }
    }
  }
]

Pattern 2: Cost Gate

Halt execution if accumulated cost exceeds a budget:

JSON

{
  "type": "router",
  "id": "check-budget",
  "routes": [
    {
      "condition": "context.data.total_tokens_used > 100000",
      "blocks": [{ "type": "step", "id": "budget-exceeded-alert", "handler": "log",
                   "params": { "message": "Budget exceeded", "level": "warn" } }]
    }
  ],
  "default": [{ "type": "step", "id": "next-ai-step", "handler": "llm_call",
                "params": { "provider": "openai", "model": "gpt-4o" } }]
}

Pattern 3: Report Usage via Webhook

Send usage data to an external analytics service after each LLM call:

JSON

{
  "type": "step",
  "id": "report-usage",
  "handler": "http_request",
  "params": {
    "url": "https://analytics.internal/api/llm-usage",
    "method": "POST",
    "body": {
      "provider": "{{steps.ai-summarize.output.provider}}",
      "model": "{{steps.ai-summarize.output.model}}",
      "tokens": "{{steps.ai-summarize.output.usage}}"
    }
  }
}

Pattern 4: Per-Instance Usage Summary

Query step outputs after completion to get total usage for an instance:

Bash

GET /instances/{instance_id}/outputs

# Filter for llm_call steps and sum usage.total_tokens
# across all step outputs where type = "llm_call"

CLI Reference#

The orch8 signal command sends signals from your terminal.

Bash

Usage: orch8 signal <INSTANCE_ID> <SIGNAL_TYPE> [PAYLOAD]

Arguments:
  <INSTANCE_ID>   Target instance UUID
  <SIGNAL_TYPE>   One of: pause, resume, cancel, update_context, or any custom string
  [PAYLOAD]       Optional JSON string (required for update_context)

Examples:
  orch8 signal abc-123 pause
  orch8 signal abc-123 resume
  orch8 signal abc-123 cancel
  orch8 signal abc-123 update_context '{"key": "value"}'
  orch8 signal abc-123 human_input:approval '{"approved": true}'

Tip: Custom signal types with the prefix human_input: are routed towait_for_input steps matching that pattern. Other custom signals go to the on_signal interceptor.

Previous← Credentials & Keys NextResource Pools →