How automations work
In brief: Automations match entity events under optional conditions, aggregate them by policy, optionally wait for human approval, then dispatch a catalog workflow to a cloud agent with a stable
workflow-execution-id.
This page explains the mechanics behind every automation: what can trigger a run, how conditions and templates work, when the trigger is considered met, and how executions are tracked.
Trigger: entity + event + conditions
A trigger is an event on a project entity, optionally narrowed by conditions.
Supported entities and events
| Entity | Events |
|---|---|
| Issue | Created, status / severity / assignee updated, label added / removed / labels updated, comment added |
| Story | Created, status / priority / assignee / labels updated |
| Scenario | Created, status / priority / assignee / labels updated, story linked |
| Test execution | Status updated, completed, failed |
| Test execution batch | Status updated, completed, failed |
| Release | Created, status updated (Draft → In Progress → Ready → Completed → Archived) |
Conditions
Conditions are optional and AND-ed. Each condition is:
field + value side (current_value or previous_value) + comparator + comparison value
Comparators include equals, not equals, in / not in, contains, greater / less than, and changed (current ≠ previous; comparison value ignored).
Comparison values are free-form text and may use the meta tokens current_value / previous_value (for example, express “status changed” as field status, current ≠ previous).
Popular condition templates in the wizard cover common cases such as:
- Issue severity = HIGH
- Issue status changed
- Issue
reported_by≠ ExploreChimp - Story / scenario status → ready
- Batch completed with
failed_count> 0
Action: execute a workflow
Today the only action type is execute an agent workflow. The action configures:
| Field | Purpose |
|---|---|
| Workflow | Catalog id such as fix-issue, implement, run-qa, fix-test-execution |
| Task template | Markdown with {{…}} placeholders filled from the triggering event |
| Policy | Optional *.policy.md (defaults to the workflow’s default policy) |
| Invocation strategy | OpenHands or Via GitHub Issue |
| Human intervention | Optional gates before invoke and/or before plan execute |
When the automation fulfills, TestChimp builds a cloud-agent prompt of the form:
/testchimp <workflow> --policy <policy-file> Tasks: <rendered task templates>
--workflow-execution-id <ulid> [--mode=non-interactive]
--mode=non-interactive is appended for fully autonomous runs so the skill auto-approves the plan and continues. If you require approval before plan execute, the first invoke plans only; after the plan is upserted the execution moves to Plan ready, and Approve re-invokes with execute.
Task templates
Entity fields from the trigger are available as templated strings with dot notation:
Fix issue {{issue.id}} — {{issue.title}}
Implement story {{story.id}} — {{story.title}}
Investigate failing test {{test_execution.job_id}} on {{test_execution.branch_name}}
Also available:
- Event context — e.g.
{{event.entity_type}},{{event.entity_id}} - Project — e.g.
{{project.repo_url}},{{project.default_branch}} - Branch — when known on the event / entity
The task-template editor autocompletes {{ entity roots and field paths. Defaults are prefilled per workflow (for example fix-issue → Fix issue {{issue.id}} — {{issue.title}}).
When aggregation batches multiple matching events into one run, rendered task descriptions are concatenated (separated by blank lines) into a single Tasks: block. Aggregation is per workflow action.
Aggregation: when the trigger is met
Matching events do not always invoke the agent immediately. The automation’s aggregation policy decides when the trigger is fulfilled:
| Policy | Behaviour |
|---|---|
| Immediate | Fulfill on every matching event (still subject to the project rate limit) |
| Buffered (default) | Wait for a quiet period (grace, minimum 5 minutes) with no new matching events. Optional max window (hours since first buffered event; default 1 hour) and max event count can fulfill earlier |
| Windowed | Collect events and fulfill on fixed hour boundaries (window_hours) |
A background flush (every 5 minutes) drains eligible buffered / windowed rows so pending events are not silently dropped.
Buffered defaults: grace 5 minutes, max window 1 hour, max event count unset.
Use Immediate for rare, high-signal events (for example a single high-severity issue). Prefer Buffered when bursts of related events should collapse into one agent run (for example several failing tests in one CI push). Use Windowed for periodic digests (for example hourly upkeep of accumulated ready stories).
Human intervention
Two independent gates (both off by default):
- Require approval before invoke — execution stays Awaiting invoke approval until someone Approves in Workflow Executions. Reject cancels that run.
- Require approval before plan execute — cloud agent is invoked to Plan first; once the plan file is upserted, status becomes Plan ready. Approve re-invokes Execute.
When either gate is on, you can choose who gets notified (email today): a specific user, a role, or the assignee of the triggering issue / story / scenario.
From a pending execution you can also copy the prompt and run it with a local agent. The copied prompt includes workflow-execution-id, so status updates still land on the same execution row.
Rate limiting
Every outbound cloud-agent call (initial invoke and plan-approval continuations) counts toward the project’s max agent invocations per hour (default 5). When the cap is hit, the execution stays Queued until the next flush window; it is not dropped. Adjust the cap under Project Settings → Automations.
Tracking and IDs
| ID | Meaning |
|---|---|
| Automation id | Stable identity of the automation definition |
| Automation execution id (ULID) | One fulfillment of that automation (may cover multiple buffered events) |
| Workflow execution id (ULID) | One catalog workflow run (the action). Passed in the agent prompt and reported via report-agent-action |
Agents should prefer the prompt’s --workflow-execution-id over minting a second ULID so the Workflow Executions timeline stays accurate.
Lifecycle of a run
- Event emitted on an entity (create, status change, batch complete, …).
- Enabled automations whose trigger entity + event match are evaluated; conditions AND-ed.
- Matching events enter the automation’s aggregation buffer (or fulfill immediately).
- When the aggregation window fulfills → an automation execution is created and workflow execution(s) are queued.
- If invoke approval is required → wait for Approve; else dispatch.
- Cloud agent receives the prompt via OpenHands or GitHub Issue.
- Agent reports progress with the workflow execution id; UI shows Running → Done / Failed.
- If plan-execute approval is required → pause at Plan ready until Approve.
Related
FAQ
Which entities can trigger an automation?
Issue, story, scenario, test execution, test execution batch, and release—each with entity-specific events such as created, status updated, failed, or completed.
What is buffered aggregation?
Buffered (the default) waits for a quiet period of at least five minutes with no new matching events, bounded by an optional max window and max event count, then fires one agent run for the batch.
What human approval gates exist?
You can require approval before invoking the cloud agent, and/or after the agent produces a plan before plan execute. Pending runs appear in Workflow Executions.
How do task templates work?
Markdown templates use {{entity.field}} placeholders filled from the triggering event—for example Fix issue {{issue.id}}—with autocomplete in the wizard.