Cloud agents

In brief: Automations reach cloud agents two ways—OpenHands conversations started by TestChimp, or labelled GitHub issues that your own agent watches—configured once per project and selected per automation.

Automations need a path from “trigger met” to “agent running /testchimp …”. That path is the invocation strategy. Configure strategies at the project level under Project Settings → Automations, then pick one on each automation’s action step.

Both strategies deliver the same kind of prompt: a TestChimp skill command plus the rendered task text and a workflow-execution-id so progress and completion report back into Executions → Workflow Executions.

Choose a strategy

	OpenHands	Via GitHub Issue
How it works	TestChimp creates an OpenHands conversation with the prompt (and optional branch)	TestChimp opens a GitHub issue whose body is the prompt
When available	OpenHands API key configured + GitHub repo mapped	TestChimp GitHub App installed + repository selected
Good fit	You already run OpenHands cloud or self-hosted for coding agents	You want any label-watching agent (Copilot coding agent, OpenHands resolver, Codex, custom bots)
Continuation (plan approve → execute)	New OpenHands conversation with the execute prompt	Comment on the same GitHub issue
Default in wizard	—	Preferred when GitHub is connected

If an automation requests a strategy that is not configured, TestChimp falls back to whichever integration is ready (OpenHands first, then GitHub Issue). If neither is ready, the invoke fails with a clear reason.

Project rules (shared)

Under Project Settings → Automations:

• Max agent invocations per hour — shared budget across all automations (default 5). Includes initial invokes and plan-approval continuations. Over-cap runs stay queued until capacity frees.
• Strategy panels for GitHub Issue and OpenHands show configured vs needs-setup state.

Via GitHub Issue

Prerequisites

1. Install the TestChimp GitHub App and select the repository for this project.
2. Optionally add extra issue labels under Integrations → GitHub (the built-in testchimp label is always applied).

What TestChimp creates

When an automation using this strategy fulfills:

1. An issue is opened on the mapped repository.
2. Title resembles TestChimp <workflow-id>: automated run.
3. Body contains the full /testchimp … prompt plus a hidden marker with the workflow execution id.
4. Labels include testchimp and any extra labels from project GitHub config.

Point your cloud agent at the labels

Configure your coding agent to:

• Watch issues labelled testchimp (and any extras you added), or
• React to a mention / assignment on those issues

The issue body already contains everything needed. The agent should run the prompt (with the TestChimp skill + MCP / CLI for that project) and report agent actions so the workflow execution updates.

This approach keeps agent hosting outside TestChimp: you bring Copilot coding agent, an OpenHands issue resolver, Codex, Cursor Cloud, Claude Code Action, or an in-house runner—as long as it can open the repo, run /testchimp, and use your TESTCHIMP_API_KEY.

Step-by-step setup (skill, mcp.json, secrets, and per-provider triggers): Setting up cloud agents.

:::tip Same repo as OpenHands The GitHub repository mapped in TestChimp must be the one your agents check out. For OpenHands-as-resolver setups, that is also the repo OpenHands is connected to. :::

OpenHands

Prerequisites

1. Map a GitHub repository on the project (OpenHands receives the repo full name from that integration).

2. Configure OpenHands under Project Settings → Automations (or Integrations → OpenHands):

   • API key from OpenHands (or your self-hosted instance)
   • Installation type: Cloud (default https://app.all-hands.dev) or self-hosted URL

OpenHands integration for automations follows the same project config as one-click bug fixes. Teams tier is required.

What TestChimp creates

When an automation using OpenHands fulfills:

1. TestChimp calls the OpenHands API to start a conversation.
2. initial_user_msg is the rendered workflow prompt.
3. repository is the project’s mapped GitHub repo; selected_branch is set when the triggering event carries a branch.
4. The workflow execution stores the OpenHands conversation id so you can open it from the executions UI.

When a plan-execute approval continues the run, TestChimp starts a new conversation with the execute prompt (OpenHands conversations are not resumed in-place for this path).

Setup checklist

1. Create / sign in to OpenHands and connect the same GitHub repository as TestChimp.
2. Copy an API key from OpenHands settings.
3. In TestChimp → Project Settings → Automations → Invoke via OpenHands, paste the key and save.
4. Confirm the strategy shows Configured when creating an automation.
5. In OpenHands, register TestChimp MCP (npx + @testchimp/cli@latest + mcp) with TESTCHIMP_API_KEY, and install the TestChimp skill under .agents/skills/testchimp in the repo — details in Setting up cloud agents → OpenHands.

Comparing the two for QA automations

• Prefer GitHub Issue when multiple agent products may pick up work, or when you already route agent jobs through GitHub issues / labels.
• Prefer OpenHands when you want TestChimp to start the agent session directly and you standardize on OpenHands for cloud coding.
• You can mix strategies across automations in one project (for example OpenHands for fix-issue, GitHub Issue for nightly upkeep).

Troubleshooting

Symptom	What to check
Strategy shows “not configured”	GitHub App + repo for Issue strategy; OpenHands API key for OpenHands
Execution stuck Queued	Hourly invocation cap; wait for the 5-minute drain or raise the limit
Agent never starts on GitHub Issue	Agent not watching testchimp (or extra) labels; issue created in wrong repo
OpenHands fails on invoke	API key, installation URL, and repo access in OpenHands match TestChimp’s mapped repo
Status never updates in TestChimp	Agent must report with the prompt’s --workflow-execution-id

Related

• Setting up cloud agents — Cursor, Claude Code, Codex, Copilot, OpenHands
• How automations work
• Create an automation
• OpenHands integration
• GitHub integration