Skip to main content

TrueCoverage

TrueCoverage is the QA Intelligence surface that connects real user behaviour with your test efforts. By tracking the same events in production (via testchimp-rum-js) and during test runs, TestChimp aligns test execution results with the events real users trigger—so you can see coverage gaps and prioritize testing where it matters most.

What It Delivers​

TrueCoverage answers QA intelligence questions that are grounded in real usage:

  • Where do users drop off, and is that path tested? — Identify steps where users leave or stall, and whether you have tests (or exploratory coverage) for those steps.
  • Where do users dwell or struggle? — See long “time to next event” or “time to first event” (discovery friction), and overlay exploratory findings (e.g. usability or performance bugs) at those same steps.
  • Which event variants are high-frequency but untested? — Use metadata slicing (e.g. payment method, checkout step) to see which variations of an event are common in production and which are covered in your test/coverage scope.
  • What do users do after a given action? — Event transitions show the funnel of follow-up actions (e.g. after “add-to-cart”) so you can align test scenarios with real user paths.

You can use frequency metrics to prioritize test design and close high-impact gaps—e.g. “users are dropping off here, and there is no test covering that path”—and tie exploratory agent findings to the same behaviour.

How It Works (High Level)​

  • Events (key user actions like checkout, add-to-cart) are emitted in user sessions via testchimp-rum-js. The Playwright TestChimp runner augments event emits with test identification so the same events are recorded in test runs.
  • You set a base execution scope (e.g. production) for event funnels and stats, and a coverage execution scope (e.g. test runs). TrueCoverage overlays coverage on top of base behaviour to show what is and isn’t covered.
  • We compute session-level and event-level metrics (including position in session, time to next event, time to end of session) and event transitions (A → B) to power funnel views, drop-off/dwell/discovery indicators, and metadata-based slicing.

For full detail on events, metrics, and execution scopes, see How it works. For the Overview and Event pages (chips, funnel, sample sessions, metadata slicing), see Dashboard.

Where to Use It​

TrueCoverage is available as its own area in the app. Configure the base and coverage execution scopes, then use the Overview for the event funnel and the Event page (by clicking an event chip) for per-event summary, funnel next steps, sample sessions, and metadata slicing.

ROI and Use Cases​

  • Prioritize test efforts by relative event frequency and metadata variants—focus on high-traffic paths and variants that lack coverage.
  • Close coverage gaps where users actually behave: “users are dropping off here—and there is no test covering that path.”
  • Connect exploratory findings to behaviour: see usability or performance bugs at the same steps where users dwell or drop off, so you can fix high-impact issues first.

For a full overview of purpose, capabilities, and how it differs from typical product analytics, see the TrueCoverage Introduction.