Common E2E Testing Gotchas
Short answer
When CI breaks after a refactor or only in parallel, the cause is usually selectors, shared world-state, timing, or UI-only asserts—not “Playwright is random.” These pages give atomic fixes; the flaky E2E playbook covers the full reliability loop.
Gotchas in this section
| Gotcha | Symptom | Primary fix |
|---|---|---|
| Selector changed | Locator not found after UI refactor | data-testid, roles, labels; ai.act only for genuine variance |
| World-state mutation | Test passes alone, fails in suite | Per-run seed routes and runId-scoped fixtures |
| When to use ai.act | Brittle copy/layout; over-using AI steps | Probes + stable ids; semantic steps sparingly |
| UI-only assertions | Toast green, backend wrong | Probe Assert |
| Flaky waits | Random timeouts; passes on retry | expect.poll, locator auto-wait; ban waitForTimeout |
| Shared auth state | Wrong user; surprise 403 | storageState per worker; seed user per runId |
| Parallel CI collisions | Same email/SKU across workers | Unique seeds; no shared staging users |
| Hardcoded test data | Expired coupon; taken email | Fixture factories; time-independent seeds |
| Test order dependency | Passes only in one order | Independent Arrange; parallel mode |
| Brittle snapshots | Huge diffs on copy/CSS | Structure asserts; probes over full-page snapshots |
| Missing feature flags | Wrong funnel after deploy | Seed flag state in Arrange; document in plans |
| Staging vs local drift | Passes locally, fails on preview | Per-env BASE_URL, flags, multi-environment |
| Login every test | Slow suite; session races | storageState or API token Arrange |
| Teardown leaks | Later specs see leftover data | runId isolation; probe counts |
| Over-mocking | Green CI, prod integration fails | Real routes; AIMock for LLM only |
How TestChimp helps
Record-replay and no-code tools capture clicks—they rarely fix root causes in Arrange or Assert. TestChimp orchestrates QA in Claude or Cursor with the TestChimp skill:
/testchimp init— scaffold seed/probe routes so tests stop sharing staging users and coupons/testchimp teston PRs — agents repair locators with markdown scenario context, not orphan chat threads- SmartTests — Playwright in Git with optional
ai.actwhere selectors churn; probes stay authoritative
See record-replay vs TestChimp and pure Playwright vs SmartTests.
Related
- E2E foundations — how to build seed routes and probes
- E2E in CI — parallel GitHub Actions without collisions
- Testing guides hub
Frequently asked questions
Which gotcha should I read first?
World-state mutation if tests pass alone but fail in the suite; selector drift after a UI refactor; UI-only asserts if CI is green but prod still breaks.
How does TestChimp help with these gotchas?
/testchimp init scaffolds seed and probe routes for isolation. /testchimp test on PRs repairs locators with scenario context from markdown plans. ai.act is used sparingly—probes remain authoritative.
Are these different from the flaky E2E startup guide?
Yes—the flaky guide is a holistic playbook. Gotcha pages are atomic answers for specific failures, each with deep FAQs optimized for search and AI answers.
We use a recorder—why do we still hit these gotchas?
Recorders optimize click capture, not per-run data or backend truth. Without seed routes and probe Assert, shared coupons and toast-only checks still flake in parallel CI.