Skip to main content

What is TestChimp?

TestChimp is a QA workflow layer for agents. It orchestrates a tight feedback loop so tools like Claude continuously understand product risk, coverage gaps, and where users actually spend time—then act. It supports web (Playwright) and native mobile (iOS and Android via Mobilewright); see Mobile testing for platform-specific capabilities.

  • Update test suite to cover under-tested areas of the app, informed by real user behaviour and requirement documentations
  • Handle QA infra updates necessary (seed / probe endpoints, authoring fixtures that emulate real world scenarios)
  • Run exploratory tests on high impact areas
  • Instrument code to understand real user behaviour - to help inform future QA infra updates

Humans stay in control of intent (what the product should do). Agents do the heavy lifting of execution and upkeep against that intent and real user behaviour - continuously learning, identifying gaps and addressing them.

Overall workflow: plans, agents, tests, CI, and TrueCoverage

How the loop fits together​

  1. Planned reality — Teams maintain test plans in the TestChimp web app (kanban, forms, priorities). Plans are stored as Markdown in a mapped plans/ folder and synced to your Git repo, so agents read the same source of truth as the platform.
  2. One-time setup — Install the TestChimp skill on Claude (or Cursor, and other hosts that load skills). Run /testchimp init to wire MCP (@testchimp/cli), SmartTests layout, CI, seed / teardown / read endpoints, fixtures, mocking strategy, TrueCoverage instrumentation, and environment strategy—see QA on Autopilot.
  3. Per PR — After development, run /testchimp test. The agent uses TestChimp APIs via MCP for QA intelligence, updates seeds and fixtures, instruments user event emits where needed, authors API and UI tests, and links them to scenarios (e.g. // @Scenario: comments). If you are not on a PR, pass a focus area (story, folder, or scenario scope).
  4. In CI — Run tests using standard playwright runner (with the @testchimp/playwright runtime plugin installed - this is done by the skill automatically for you) : it reports runs and traces, tags user events with test identity for TrueCoverage, and loops in AI during runtime to execute natural-language steps enabling more resilient scripts—see Playwright runtime plugin.
  5. Periodically — Run /testchimp evolve (for example after deploys) to pull summarized insights from production behaviour, requirement traceability, test execution, and plans—then expand coverage, update fixtures and seeds, explore high-engagement surfaces for UX frictions, and retire obsolete tests.

The TestChimp web app is the control plane: test planning, bugs found from explorations, test coverage gaps aligned with requirements, TrueCoverage drill-downs, and workflow visibility—while Plans → Git → Claude + MCP → CI stays the operational spine.

  1. Connect your Git repo in TestChimp and map two folders: one for tests (SmartTests root) and one for plans (markdown test plans).
  2. Install the TestChimp skill on your agent host (e.g. Claude Code / Cursor).
  3. Run /testchimp init once per repository to establish QA infra and integration.
  4. Run /testchimp test when a PR is ready (or with an explicit scope if not branch-based).

Then follow the Quick Start for step-by-step setup (Git mapping, skill, MCP, init, test) and Onboarding to TestChimp for org and project admin.

Why not “just use Claude”?​

Claude can write tests. TestChimp tells which tests to write and why, bringing in requirement traceability (planned reality) and real user behaviour (product reality), compared against tested reality (through CI reported test executions and user event emits during test runs).

Claude can explore the app. TestChimp tells which areas to probe first—aligned to risk, coverage gaps, and how users actually move through the product.

Claude can tackle individual QA tasks. TestChimp orchestrates them into one seamless workflow: plans in repo, data-backed intelligence, seed/probe endpoints, fixtures, mocks, environment provisioning, event instrumentation, and evolve cycles—so the suite and infra stay aligned with the product in a continuous loop instead of drifting after the first green run.

Execution time intelligence. Claude outputs test scripts - which are by default plain Playwright scripts. They are UI-selector bound, which means the same brittleness of standard Playwright scripts still remains. TestChimp enables adding "natural language steps" in scripts - so that the step is executed intelligently at run time. This allows Claude to "defer intelligent translation of intent -> Playwright command" to runtime, making the scripts more resilient to slight product variances.

In short: we help Claude (and your team)

  • Ship the right automated QA
  • Keep it updated against plans, production behaviour, and what runs in CI.
  • Make tests resilient by enabling intelligence during execution time

Deep dive​

Get started​

  1. Quick Start — Get running in a few steps.
  2. Onboarding to TestChimp — Team and project setup.