AI Test Generation Tool
Short answer
TestChimp generates tests as Playwright SmartTests in your repo—not proprietary cloud scripts. Agents use markdown plans + TrueCoverage + PR diffs to decide what to generate; ai.act / ai.verify handle brittle UI surgically while Arrange and Assert stay on seeds and probes.
Who this is for
You need an AI test generation tool because hand-writing every Playwright spec cannot keep up with daily merges—or because chat-generated scripts flake in CI and lack requirement traceability. TestChimp is for teams that want generation plus maintenance: tests born in PRs, linked to scenarios, and repaired by agents when UI or backend rules change.
Why most AI test generation fails
Generic generators optimize happy-path demos:
- No Arrange discipline — shared coupons, users, and orders collide in parallel CI
- UI-only Assert — green toasts while database state is wrong
- One-shot output — no evolve loop when production behaviour shifts
- Vendor lock-in — English SaaS or cloud runners you cannot debug locally
TestChimp treats generation as continuous portfolio work scoped by plans, diffs, and TrueCoverage—not a single export from chat.
Inputs that drive generation
| Input | Role in generation |
|---|---|
| Markdown scenarios | Defines what must stay covered (test planning) |
| PR diff | Defines what changed this merge |
| TrueCoverage | Defines what users do that tests still miss (TrueCoverage) |
| Manual capture | Grounds agents in real UI state (Chrome extension) |
| Prior CI runs | Informs repair, not regen from scratch (test runs) |
How TestChimp generates tests
1. Per-PR generation (/testchimp test)
On every feature PR, agents read your plans folder, identify affected scenarios, and author or update SmartTests with // @Scenario: links. Seed routes and probes come from /testchimp init (test).
2. Capture-assisted authoring
Manual testers record flows in the Chrome extension; agents convert sessions into SmartTest prompts with deterministic steps where possible (creating SmartTests).
3. Hybrid runtime (not pure agentic)
Most steps stay fast Playwright. Optional ai.act and ai.verify cover volatile copy or layout—see pure agentic vs SmartTests.
4. Post-deploy evolution (/testchimp evolve)
Generation does not stop at merge. TrueCoverage and run history drive the next wave of tests after deploy (evolve).
Example scenario
Situation: ChatGPT writes a checkout test that clicks through UI but never validates payment capture.
Expected outcome: SmartTest uses seed coupon (Arrange), UI checkout (Act), probe confirms order row (Assert).
Why UI-only automation breaks: Generated script passes while webhook handler is stubbed—production charges fail silently.
- Arrange: `/testchimp init` exposes seed route for run-scoped user and cart.
- Act: Agent-generated Playwright submits checkout.
- Assert: Probe endpoint returns order ID and payment status; `@Scenario` links roll up coverage.
TestChimp workflow: Evolve adds tests for prod-only payment methods TrueCoverage flagged untested.
Same Arrange/Act/Assert pattern as expired-coupon checkout.
vs record-replay and English SaaS
| Approach | Asset you own | CI fit | Requirement links |
|---|---|---|---|
| Record-replay | Opaque recordings | Weak | None |
| English SaaS (testRigor, Testsigma, …) | Vendor runner | Lock-in | Often external |
| TestChimp SmartTests | Playwright in Git | Native traces/reporters | // @Scenario: in code |
Compare: record-replay vs TestChimp · TestChimp vs testRigor · TestChimp vs Testsigma.
Review before merge
Generated tests are normal Playwright files in Git. Reviewers see diffs, traces on failure, and scenario impact in the same PR—same bar as hand-written automation. Agents do not merge autonomously unless you configure that policy.
Import existing Playwright
Link current specs to markdown scenarios, add TestChimp fixtures via /testchimp init, and let agents extend coverage from plans and TrueCoverage gaps. Generation becomes portfolio growth, not a rip-and-replace.
Related reading
- AI test generation explained
- AI testing tool for startups
- QA on Autopilot
- Expired-coupon Arrange/Act/Assert pattern
Frequently asked questions
Are AI-generated SmartTests editable and reviewable?
Yes—they are Playwright files in Git. Agents open PRs; you review like any code. `// @Scenario:` links make requirement coverage visible in the same diff as the test changes.
UI changes weekly—do we re-record or regenerate everything?
Neither. `/testchimp test` on the PR updates affected SmartTests using scenario context. Hybrid `ai.act` handles volatile copy; seed routes and probes keep Arrange/Assert stable through refactors.
Can we import existing Playwright and grow with AI?
Yes. Link specs to markdown scenarios, add TestChimp fixtures, and let agents extend coverage from plans and TrueCoverage gaps—generation becomes portfolio maintenance, not one-shot files.
Our eng team maintains tests but we have no QA—does TestChimp fit?
Yes. Most Playwright teams rely on developers to patch flaky suites between features. TestChimp gives them a structured loop: markdown scenarios in Git, `/testchimp test` on each PR to author or repair SmartTests, requirement roll-ups from `// @Scenario:`, and `/testchimp evolve` after deploy via TrueCoverage—reliable releases without hiring QA first.
Generate Playwright you own—not vendor scripts
Run /testchimp init, link markdown scenarios, and let agents author SmartTests on every PR with TrueCoverage-driven evolve after deploy.