Testing Apps Built with Bolt
Short answer
Bolt.new helps you ship full-stack previews fast—often React + Vite with Supabase backends. Demos prove the happy path once; TestChimp adds per-PR E2E with /testchimp init seed/probe routes, Playwright CI, and SmartTests linked to markdown scenarios so Bolt velocity does not trade away release confidence.
Who this is for
Founders and small teams using Bolt.new (StackBlitz-backed previews, frequent Git export) who need CI-grade E2E before scale—not preview-only flows that break when prompts regenerate UI.
How teams ship with Bolt
Prompt-driven full-stack generation, Supabase Auth and Postgres, rapid export to GitHub. Bolt optimizes time-to-working-preview; RLS edge cases, parallel-safe fixtures, and webhook timing are rarely generated automatically—same gap as Lovable stacks.
Common QA gaps
| Risk | What goes wrong |
|---|---|
| Happy-path-only | Auth, billing, and empty states never generated |
| Supabase wiring | Demo RLS or anon-key misuse passes preview |
| Export drift | Preview works; exported repo missing env or migrations |
| Regen churn | Selectors break when you re-prompt UI |
| No merge gate | Green Bolt preview; red production |
Why E2E with probes is non-negotiable
Bolt demos prove a path worked in the sandbox. Production needs Arrange/Act/Assert with run-scoped data—see why record-replay falls short and Supabase E2E patterns.
Bolt preview ──export──► Git repo ──/testchimp init──► seed/probe + CI
│
└── /testchimp test on each PR
Bolt + Supabase stack
Most Bolt apps pair Vite/React with Supabase. Apply service-role seed routes server-side only:
// supabase/functions or Vite API route — never expose service role to browser
import { createClient } from '@supabase/supabase-js';
const admin = createClient(
import.meta.env.VITE_SUPABASE_URL,
process.env.SUPABASE_SERVICE_ROLE_KEY!,
);
export async function seedRunUser(runId: string) {
const email = `bolt+${runId}@test.local`;
const { data, error } = await admin.auth.admin.createUser({
email,
password: `pw-${runId}`,
email_confirm: true,
});
if (error) throw error;
return { email, password: `pw-${runId}`, userId: data.user!.id };
}
Probe orders, profiles, and RLS-enforced tables—never assert toast copy alone (UI-only gotcha).
The TestChimp loop on every PR
TestChimp does not replace Bolt—it orchestrates QA on what you export:
| Phase | Command | Outcome |
|---|---|---|
| Bootstrap | /testchimp init | Seed/probe routes, fixtures, Playwright CI, TrueCoverage (init) |
| Per-PR QA | /testchimp test | Agents read markdown plans, author/repair SmartTests, wire // @Scenario: (test) |
| UX risk | /testchimp explore | ExploreChimp on SmartTest pathways (explore) |
| Post-deploy | /testchimp evolve | Close TrueCoverage and plan gaps (evolve) |
Install the TestChimp skill in Cursor or Claude. SmartTests remain Playwright in Git—standard traces, reporters, and CI (SmartTests).
Export checklist before CI
| Step | Why |
|---|---|
| Pin Supabase migrations | Preview DB ≠ prod schema |
Add E2E_TEST_MODE gate | Seed routes safe in staging only |
data-testid on auth/checkout | Survives Bolt regen better than CSS |
| Env vars in GitHub Actions | VITE_* + server secrets separated |
| Link to agent-built workflow | Repeatable loop, not one-off agent test |
Worked example
Your Bolt app may demo signup while RLS still allows cross-tenant profile reads. /testchimp test on auth PRs adds probe Assert and // @Scenario: auth/tenant-isolation—patterns from Supabase guide.
// @Scenario: auth/tenant-isolation
test('user cannot read other tenant profile', async ({ page, request, runId }) => {
const victim = await request.post('/api/test/seed-user', { data: { runId: `${runId}-v` } }).then(r => r.json());
const attacker = await request.post('/api/test/seed-user', { data: { runId: `${runId}-a` } }).then(r => r.json());
await page.goto('/login');
await page.getByLabel('Email').fill(attacker.email);
await page.getByLabel('Password').fill(attacker.password);
await page.getByRole('button', { name: 'Sign in' }).click();
await page.goto(`/profile/${victim.userId}`);
await expect.poll(async () => {
const res = await request.get(`/api/test/probe-profile-access?viewer=${attacker.userId}&target=${victim.userId}`);
return (await res.json()).allowed;
}).toBe(false);
});
Anti-patterns
| Anti-pattern | Why it fails | Better approach |
|---|---|---|
| Test only in Bolt preview | Export breaks env | CI on Git export |
| Anon key for privileged seed | Key leak + wrong RLS | Server seed routes |
| Shared Bolt demo user | Parallel collision | Per-run bolt+${runId}@test.local |
ai.act on every step | Non-deterministic CI | test ids + probes on critical paths |
| Skip init after export | No probe layer | /testchimp init once per repo |
External references
Example scenario
Situation: Bolt regenerates checkout UI; coupon field moves but expired-coupon bug returns.
Expected outcome: Expired coupons rejected; probe shows zero orders for runId.
Why UI-only automation breaks: Shared demo coupon works in preview; CI uses different data and flakes.
- Arrange: Seed route mints run-scoped coupon with past expires_at.
- Act: Playwright applies coupon and submits checkout.
- Assert: Probe confirms zero orders; UI error optional.
TestChimp workflow: TrueCoverage flags prod checkout paths Bolt never generated tests for.
Same Arrange/Act/Assert pattern as expired-coupon checkout.
Related
- Apps built with Lovable
- Supabase Playwright
- Seed routes and probe Assert
- Agent-built QA workflow
- Vibe-coded overview
Frequently asked questions
Bolt generates our app—do we need separate QA tooling?
Bolt optimizes preview velocity; TestChimp closes the E2E gap. After export, /testchimp init adds seed/probe routes and CI; /testchimp test on each PR validates auth, Supabase RLS, and payments with probe assertions.
When should we add TestChimp after Bolt?
As soon as you export to Git and have auth, Supabase, or payments. Init before scale prevents demo-only quality from reaching production.
Bolt preview vs exported repo—which do tests target?
Run Playwright against the exported Git repo in CI—that is what you deploy. Preview URLs are useful for exploration; probes and seeds live in exported code.
We use Supabase from Bolt—same patterns as Lovable?
Yes—service-role seed routes server-side, per-run users, probe Assert on tables. See Supabase and Lovable guides; Bolt export may need migration sync.
UI regen broke our tests—what now?
/testchimp test on the regen PR repairs selectors with scenario context. Add data-testid on auth and checkout during init to reduce future drift.
Can we run E2E inside StackBlitz only?
Possible for dev; production CI should run on GitHub Actions or similar with secrets, emulators, and parallel workers—init scaffolds that harness.
Do coding agents replace TestChimp for Bolt apps?
Agents produce session-scoped tests. TestChimp orchestrates markdown plans, CI history, ExploreChimp, and TrueCoverage—/testchimp test on every PR.
What is the first command after Bolt export?
/testchimp init in the exported repo—then add markdown scenarios and run /testchimp test on your next feature PR.
Apply these patterns in your repo
Run `/testchimp init` to connect TestChimp to your repo, then `/testchimp test` on PRs to turn these patterns into maintained SmartTests. Use `/testchimp evolve` when you want to expand coverage as your app grows.