Missing Teardown Leaks Test Data
Short answer
afterAll delete that never runs—crashed worker, skipped hook, flaky network—leaves orphan users and orders on staging. Prefer runId isolation so leaked rows are inert, and probe counts in Assert to catch pollution before the next suite. Teardown is a safety net, not the primary isolation strategy.
Part of Common E2E testing gotchas.
Symptom
- Staging fills with
test-user-*accounts; quotas and billing alerts fire - Spec fails on "email already taken" without code changes
- Order or inventory counts drift; unrelated specs start failing
afterAllcleanup passes locally but workers crash in CI—rows remain- Finance or support finds fake charges from E2E checkouts
Root cause
Tests assume mutable shared world and best-effort cleanup:
- Creates global
test@example.comwithout namespace afterEach/afterAllDELETE races with parallel workers deleting wrong rows- No Assert on whether data was actually removed
- Teardown skipped when test fails before hook runs (unless
test.afterEachwith careful ordering) - Shared DB between preview and manual QA—leaks are human-visible
This is world-state mutation without isolation—see mutating shared world-state and parallel CI collisions.
Fix: runId isolation over teardown
1. Scope every created entity by runId
test('checkout creates paid order', async ({ page, request }) => {
const runId = `order-${test.info().workerIndex}-${Date.now()}`;
await request.post('/api/test/seed-cart', {
data: { runId, cartId: `cart-${runId}`, items: [{ sku: 'TEE', qty: 1 }] },
});
await page.goto(`/checkout?cartId=cart-${runId}`);
await page.getByRole('button', { name: 'Pay' }).click();
await expect.poll(async () => {
const res = await request.get(`/api/test/probe-order?runId=${runId}`);
return (await res.json()).paidOrderCount;
}).toBe(1);
});
Other tests ignore rows tagged with alien runId values—leaked data does not collide. Full pattern: seed routes and probe Assert.
2. Probe counts catch leaks in the same spec
// After business Assert—optional hygiene check
const probe = await request.get(`/api/test/probe-run-footprint?runId=${runId}`);
const { orderCount, userCount } = await probe.json();
expect(orderCount).toBe(1);
expect(userCount).toBe(1);
If Act created extras (double-submit bug), probe counts fail even when UI toast is green—authoritative like UI-only assertions.
3. Teardown as optional safety net
test.afterEach(async ({ request }, testInfo) => {
const runId = testInfo.annotations.find(a => a.type === 'runId')?.description;
if (!runId) return;
await request.delete(`/api/test/purge-run?runId=${runId}`).catch(() => {});
});
Use purge routes for compliance or cost (PII, paid sandbox charges)—not as the only isolation mechanism. Purge must be idempotent and keyed by runId, never DELETE FROM orders WHERE email LIKE 'test%' while parallel workers run.
4. When teardown still matters
| Scenario | Prefer runId | Add teardown |
|---|---|---|
| Parallel CI on shared staging | Yes | Optional nightly purge job |
| Paid sandbox charges | Yes + probe | Purge orders with runId |
| PII in test DB | Yes | Hard delete after spec |
| Local ephemeral SQLite | Either | test.describe.configure({ mode: 'serial' }) rare |
Anti-patterns
| Anti-pattern | Why it fails | Better approach |
|---|---|---|
Global test@example.com | Collisions; leaks block next run | user-${runId}@test.local |
afterAll deletes "all test data" | Kills parallel worker rows | Purge by runId only |
| No Assert on row counts | Silent duplicates | probe-order paidOrderCount |
| Teardown-only isolation | Hook skip on crash | runId namespace first |
| Reuse production-like SKUs | Inventory drift | Seed catalog per runId |
TestChimp workflow
/testchimp init scaffolds seed and probe routes with mandatory runId on fixtures—SmartTests stop depending on teardown hooks that CI skips. /testchimp test adds probe footprint asserts when markdown scenarios state "exactly one order" or "no duplicate charge"; agents prefer isolation over fragile afterEach DELETE chains.
Related
- World-state mutation
- Parallel CI collisions
- Hardcoded test data
- UI-only assertions
- Seed routes and probe Assert
- Stripe test mode
Frequently asked questions
Should E2E tests clean up data in afterEach?
Teardown helps compliance and cost, but it is unreliable when tests fail or workers crash. Prefer runId-scoped seeds so leaked rows do not collide with the next spec; use idempotent purge routes as a safety net.
What is runId isolation?
Every test run gets a unique runId; all users, carts, and orders created in Arrange are tagged with it. Probes filter by runId—parallel workers and leaked data from a crashed test do not share namespaces.
Our staging DB is full of test users—how do we fix?
Stop creating global test emails; add runId to seed routes; run a one-time archival job on rows older than N days with a test marker. Going forward, probe counts catch duplicate creates in the same spec.
Can afterAll delete run in parallel CI safely?
Only if deletion is scoped to that worker runId. Broad DELETE WHERE email LIKE test% will delete another worker active rows and cause random failures.
How do probe counts detect leaks?
probe-order and probe-run-footprint return counts for a runId. If UI shows one success but paidOrderCount is 2, you have a double-submit or missing isolation bug—not a teardown problem alone.
Is teardown the same as test isolation?
No—isolation prevents cross-test interference; teardown removes rows afterward. Isolation via runId works even when teardown never runs. Design for isolation first.
Does TestChimp scaffold runId and probe routes?
Yes—/testchimp init generates seed and probe stubs with mandatory runId so SmartTests do not rely on shared staging users or fragile afterAll cleanup. /testchimp test adds footprint probes when scenarios specify exact counts.
We use Stripe test mode—do we still need runId?
Yes—charges and customers accumulate; webhooks may fan out to shared handlers. Tag checkout sessions with runId and probe paidOrderCount; purge sandbox objects by runId on a schedule if needed.
Apply these patterns in your repo
Run `/testchimp init` to connect TestChimp to your repo, then `/testchimp test` on PRs to turn these patterns into maintained SmartTests. Use `/testchimp evolve` when you want to expand coverage as your app grows.