Testing Apps Built with Replit
Short answer
Replit accelerates build and deploy with agents and instant previews. TestChimp adds Playwright SmartTests in Git, preview/staging targets, and /testchimp test on PRs so Replit velocity does not skip backend validation.
Who this is for
Teams exporting Replit projects to Git—or running CI against Repl preview URLs—who need E2E that survives environment drift and agent regen.
How teams ship with Replit
Agent-assisted coding, Repl deployments, and quick forks. Previews prove a path worked once—not that CI stays green after the next agent edit.
Common QA gaps
| Risk | What goes wrong |
|---|---|
| Preview-only verification | No Git-native E2E gate on every PR |
| Environment drift | Repl DB differs from production schema or rules |
| Agent-generated routes | Missing seed/probe endpoints for parallel CI |
| No scenario traceability | Multiple contributors prompt the same Repl without linked plans |
Why E2E with probes is non-negotiable
Replit apps mix serverless handlers and client UI. UI-only checks miss handler bugs; parallel CI needs run-scoped data (multi-environment).
The TestChimp loop on every PR
TestChimp does not replace your builder—it orchestrates QA on what agents ship:
| Phase | Command | Outcome |
|---|---|---|
| Bootstrap | /testchimp init | Seed/probe routes, fixtures, Playwright CI, TrueCoverage (init) |
| Per-PR QA | /testchimp test | Agents read markdown plans, author/repair SmartTests, wire // @Scenario: (test) |
| UX risk | /testchimp explore | ExploreChimp on SmartTest pathways (explore) |
| Post-deploy | /testchimp evolve | Close TrueCoverage and plan gaps (evolve) |
Install the TestChimp skill in your agent IDE. SmartTests remain Playwright in Git—standard traces, reporters, and CI (SmartTests).
Three realities TestChimp aligns
| Reality | Without orchestration | With TestChimp |
|---|---|---|
| Planned | Scenarios live in chat or Notion | Markdown plans in Git (test planning) |
| Tested | Session-scoped agent tests | CI SmartTests + test runs (test runs) |
| Production | Unknown coverage holes | TrueCoverage RUM ↔ runs (TrueCoverage) |
Mismatch signals drive the next /testchimp test cycle—not another ad hoc prompt.
Example scenario
Situation: Replit agent adds checkout that works in preview while webhook validation is stubbed.
Expected outcome: Invalid webhook signature rejects checkout; no charge occurs.
Why UI-only automation breaks: Preview uses test keys; production webhook secret mismatch ships unnoticed.
- Arrange: Configure staging baseURL; seed cart via API on preview host.
- Act: Playwright submits checkout against preview deployment.
- Assert: Probe validates webhook handler received signed payload.
TestChimp workflow: TrueCoverage compares prod checkout errors vs test coverage.
Same Arrange/Act/Assert pattern as expired-coupon checkout.
Worked example
A Replit agent stubs webhook signature validation in preview. SmartTests with probe asserts catch missing validation before production—checkout testing.
Related
Git integration · Lovable · Flaky E2E fixes
Frequently asked questions
We ship from Replit previews—how does TestChimp fit?
Point Playwright baseURL at your Repl preview or staging host, map markdown plans in Git, and run `/testchimp test` on PRs. Agents author SmartTests that survive regen churn because Arrange uses seed routes and Assert uses probes—not brittle UI-only checks tied to a single preview session.
Can tests target Repl preview URLs?
Yes—configure Playwright baseURL per environment. SmartTests use seed/probe Arrange/Assert so preview churn does not flake like UI-only recordings.
We already use coding agents—do we still need TestChimp without QA?
Agents alone produce session-scoped tests. TestChimp orchestrates Replit with markdown plans, CI history, ExploreChimp, and TrueCoverage—`/testchimp test` on every PR so developers drive QA without a separate org.
Agent-written tests failed overnight—how does TestChimp recover?
Because SmartTests live in Git with scenario links, the next `/testchimp test` run sees CI history and TrueCoverage gaps, then opens a fix PR—not a fresh chat thread. Deterministic Arrange/Assert steps fail fast; hybrid AI steps absorb copy or layout churn without rerunning entire agent sessions.
Apply these patterns in your repo
Run `/testchimp init` to connect TestChimp to your repo, then `/testchimp test` on PRs to turn these patterns into maintained SmartTests. Use `/testchimp evolve` when you want to expand coverage as your app grows.