Cloud agents

In brief: Automations reach cloud agents three ways—ChimpHands on your GitHub Actions runner (recommended), labelled GitHub issues your agent watches, or a JSON POST to a named webhook—configured once per project and selected per automation.

Automations need a path from “trigger met” to “agent running /testchimp …”. That path is the invocation strategy. Configure strategies at the project level under Project Settings → Automations (and ChimpHands workflow setup under ChimpHands), then pick one on each automation’s action step.

ChimpHands and Via GitHub Issue deliver a text /testchimp … prompt. Webhook delivers a JSON body with the same task prompt plus a project API key (and backend URL) so a receiver can authenticate CLI/MCP calls.

:::tip Prefer ChimpHands ChimpHands is TestChimp's native cloud agent—one-click CI setup, full repo context, seamless async-to-chat human-in-the-loop, and QA-specific metering. Use GitHub Issue or Webhook only when you must keep an external agent product as the executor. :::

Choose a strategy

	ChimpHands	Via GitHub Issue	Via Webhook
How it works	TestChimp dispatches your repo's chimphands.yml workflow with session id + prompt	TestChimp opens a GitHub issue whose body is the prompt	TestChimp POSTs JSON to a named webhook URL with an Authorization header
When available	GitHub App + repo mapped + ChimpHands workflow installed	TestChimp GitHub App installed + repository selected	At least one webhook (nickname, URL, auth header) under Automations
Good fit	Default for QA automations—native session, PR output, platform chat	Bring-your-own agent (Copilot, Codex Action, Cursor Cloud, custom bots)	Custom / self-hosted runners that accept HTTP webhooks
Continuation (plan approve → execute)	Same ChimpHands session / new dispatch with execute prompt	Comment on the same GitHub issue	Re-POST to the same webhook nickname
Default in wizard	Preferred when ChimpHands is configured	When GitHub is connected but ChimpHands is not	—

If an automation requests a strategy that is not configured, TestChimp falls back to whichever integration is ready (ChimpHands → GitHub Issue → Webhook). If none are ready, the invoke fails with a clear reason.

Project rules (shared)

Under Project Settings → Automations:

• Max agent invocations per hour — shared budget across all automations (default 5). Includes initial invokes and plan-approval continuations. Over-cap runs stay queued until capacity frees.
• Strategy panels for ChimpHands, GitHub Issue, and WebHook show configured vs needs-setup state.

ChimpHands (recommended)

Prerequisites

1. Install the TestChimp GitHub App and select the repository for this project.
2. Open ChimpHands in the sidebar and install .github/workflows/chimphands.yml (direct commit or via PR).
3. Add TESTCHIMP_API_KEY as a GitHub Actions secret on the repo (TestChimp surfaces this during setup).

See ChimpHands introduction for architecture, human-in-the-loop, and credits.

What TestChimp creates

When an automation using ChimpHands fulfills:

1. TestChimp creates a ChimpHands session and workflow execution row.
2. It dispatches the ChimpHands GitHub Actions workflow with session_id, the rendered /testchimp … prompt, and optional branch.
3. The job checks out the repo, runs OpenCode + TestChimp CLI + skill, and tunnels progress back to TestChimp.
4. The workflow execution stores the session id so you can open the ChimpHands chat or GitHub Actions run from Workflow Executions.

When a plan-execute approval continues the run, TestChimp dispatches again with the execute prompt (or revives the session per product rules).

Setup checklist

1. Connect GitHub and map the repository.
2. Install the ChimpHands workflow from the ChimpHands page.
3. Confirm TESTCHIMP_API_KEY (and TESTCHIMP_BACKEND_URL if non-prod) on the repo.
4. In the automation wizard, choose ChimpHands as invocation strategy—card should show Configured.

Via GitHub Issue

Use this when you operate a third-party coding agent that watches labelled issues rather than ChimpHands.

Prerequisites

1. Install the TestChimp GitHub App and select the repository for this project.
2. Optionally add extra issue labels under Integrations → GitHub (the built-in testchimp label is always applied).

What TestChimp creates

When an automation using this strategy fulfills:

1. An issue is opened on the mapped repository.
2. Title resembles TestChimp <workflow-id>: automated run.
3. Body contains the full /testchimp … prompt plus a hidden marker with the workflow execution id.
4. Labels include testchimp and any extra labels from project GitHub config.

Point your cloud agent at the labels

Configure your coding agent to:

• Watch issues labelled testchimp (and any extras you added), or
• React to a mention / assignment on those issues

The issue body already contains everything needed. The agent should run the prompt (with the TestChimp skill + MCP / CLI for that project) and report agent actions so the workflow execution updates.

This approach keeps agent hosting outside TestChimp: you bring Copilot coding agent, Codex, Cursor Cloud, Claude Code Action, or an in-house runner—as long as it can open the repo, run /testchimp, and use your TESTCHIMP_API_KEY.

Step-by-step setup (skill, mcp.json, secrets, and per-provider triggers): Setting up cloud agents.

:::tip Same repo The GitHub repository mapped in TestChimp must be the one your agents check out. :::

Via Webhook

Prerequisites

Under Project Settings → Automations → Invoke via WebHook, add one or more endpoints. Each requires:

• Nickname — unique per project (case-insensitive). The first webhook defaults to default.
• Webhook URL — absolute HTTPS (or HTTP) URL that accepts POST with JSON.
• Authorization header value — sent as the full Authorization header (e.g. Bearer …).

The same manage UI is available from the automation wizard Configure modal so you do not have to leave setup.

What TestChimp sends

When an automation using this strategy fulfills (or continues after plan approval), TestChimp POSTs JSON:

Field	Meaning
task_prompt	Full /testchimp … prompt (same as other strategies)
testchimp_project_api_key	Active project data API key for CLI/MCP auth
testchimp_backend_url	Featureservice base URL for this environment
generic_instructions	Reminder to install the latest TestChimp skill + CLI/MCP

The automation action stores a webhook nickname. With one webhook, the wizard selects it automatically; with two or more, you pick the nickname on the action step.

Setup checklist

1. Add at least one webhook under Project Settings → Automations → Invoke via WebHook.
2. Ensure the project has an active API key (Key management).
3. In the automation wizard, choose Via Webhook (and a nickname if you have several).
4. Point your receiver at the payload fields above and run the task_prompt with the provided key / backend URL.

Comparing strategies for QA automations

• Prefer ChimpHands for the default path—native sessions, CI checkout, platform chat, and QA metering.
• Prefer GitHub Issue when multiple external agent products may pick up work, or you already route agent jobs through GitHub issues / labels.
• Prefer Webhook for custom runners or platforms that ingest HTTP callbacks rather than GitHub issues.
• You can mix strategies across automations in one project (for example ChimpHands for fix-issue, Webhook for nightly upkeep).

Troubleshooting

Symptom	What to check
Strategy shows “not configured”	ChimpHands workflow + secrets; GitHub App + repo for Issue; at least one webhook for Webhook
Execution stuck Queued	Hourly invocation cap; wait for the 5-minute drain or raise the limit
ChimpHands job fails immediately	TESTCHIMP_API_KEY secret; TESTCHIMP_BACKEND_URL matches project environment; workflow on default branch
Agent never starts on GitHub Issue	Agent not watching testchimp (or extra) labels; issue created in wrong repo
Webhook fails on invoke	URL reachable; Authorization header accepted; nickname still exists in project config; active project API key present
Status never updates in TestChimp	Agent must report with the prompt’s --workflow-execution-id

Related

• ChimpHands — Native QA cloud agent
• Setting up cloud agents — Bring-your-own agents (GitHub Issue listeners)
• How automations work
• Create an automation
• GitHub integration