Testing Apps Built with Lovable
Short answer
Lovable helps you ship UI fast—but demos often cover only the happy path. TestChimp adds per-PR E2E QA with Playwright SmartTests, requirement traceability, and TrueCoverage so production matches intent—not just what looked fine in preview.
Who this is for
Founders and small teams using Lovable (often with Supabase) who ship UI weekly and need CI-grade E2E before scale—not preview-only happy paths.
How teams ship with Lovable
Prompt-driven UI generation, Supabase backends, and rapid exports to Git. Lovable optimizes time-to-preview; backend edge cases and parallel-safe test data are rarely generated automatically.
Common QA gaps
| Risk | What goes wrong |
|---|---|
| Happy-path-only | Expired coupons, declined cards, and edge cases never built |
| Hallucinated wiring | Demo Supabase rules or API calls that fail under real data |
| Regen churn | Selectors break when you regenerate components |
| No merge gate | Preview works once; CI never runs E2E |
Why E2E with probes is non-negotiable
Lovable demos prove a path worked once. Production needs Arrange/Act/Assert with run-scoped data—see why record-replay falls short.
The TestChimp loop on every PR
TestChimp does not replace your builder—it orchestrates QA on what agents ship:
| Phase | Command | Outcome |
|---|---|---|
| Bootstrap | /testchimp init | Seed/probe routes, fixtures, Playwright CI, TrueCoverage (init) |
| Per-PR QA | /testchimp test | Agents read markdown plans, author/repair SmartTests, wire // @Scenario: (test) |
| UX risk | /testchimp explore | ExploreChimp on SmartTest pathways (explore) |
| Post-deploy | /testchimp evolve | Close TrueCoverage and plan gaps (evolve) |
Install the TestChimp skill in your agent IDE. SmartTests remain Playwright in Git—standard traces, reporters, and CI (SmartTests).
Three realities TestChimp aligns
| Reality | Without orchestration | With TestChimp |
|---|---|---|
| Planned | Scenarios live in chat or Notion | Markdown plans in Git (test planning) |
| Tested | Session-scoped agent tests | CI SmartTests + test runs (test runs) |
| Production | Unknown coverage holes | TrueCoverage RUM ↔ runs (TrueCoverage) |
Mismatch signals drive the next /testchimp test cycle—not another ad hoc prompt.
Example scenario
Situation: Lovable ships a checkout screen that succeeds with a hard-coded test coupon in Supabase.
Expected outcome: Expired or invalid coupons are rejected; no order is created.
Why UI-only automation breaks: Shared demo coupon expires; CI flakes for weeks without product changes.
- Arrange: Seed route mints run-scoped coupon with past `expires_at` for this worker only.
- Act: Playwright applies coupon and submits checkout.
- Assert: Probe confirms zero orders; UI error is optional.
TestChimp workflow: TrueCoverage flags prod checkout paths Lovable never generated tests for.
Same Arrange/Act/Assert pattern as expired-coupon checkout.
Worked example
Your Lovable app may demo checkout successfully while expired coupons still create orders. /testchimp test on pricing PRs adds probe Assert and scenario links—cart and coupons guide.
Related
Vibe-coded overview · Replit · AI testing tool
Frequently asked questions
Lovable generates our app—do we need separate QA tooling?
Lovable optimizes shipping UI; TestChimp closes the E2E gap. After `/testchimp init`, `/testchimp test` on each PR validates payment, auth, and API side effects with probe assertions while linking tests to markdown scenarios—so vibe-coded velocity does not trade away release confidence.
Lovable ships fast—when should we add TestChimp?
As soon as you have payment, auth, or multi-step flows. `/testchimp init` before scale prevents demo-only quality from reaching production.
We already use coding agents—do we still need TestChimp without QA?
Agents alone produce session-scoped tests. TestChimp orchestrates Lovable with markdown plans, CI history, ExploreChimp, and TrueCoverage—`/testchimp test` on every PR so developers drive QA without a separate org.
AI or recorded tests from Lovable fail after UI changes—then what?
TestChimp keeps deterministic Playwright steps wherever possible; optional `ai.act`/`ai.verify` handles volatile UI. `/testchimp test` on the PR that changed the screen updates selectors and probes together. You are not re-recording opaque sessions—agents patch reviewable Git diffs.
Apply these patterns in your repo
Run `/testchimp init` to connect TestChimp to your repo, then `/testchimp test` on PRs to turn these patterns into maintained SmartTests. Use `/testchimp evolve` when you want to expand coverage as your app grows.