Skip to main content

TrueCoverage

TrueCoverage is the QA Intelligence surface that connects real user behaviour with your test efforts. By tracking the same events in production (via @testchimp/rum-js) and during test runs, TestChimp aligns test execution results with the events real users trigger—so you can see coverage gaps and prioritize testing where it matters most.

TrueCoverage UI

What It Delivers: The 4Ds of RUM-based QA Strategy​

TrueCoverage organizes event-level insights into four categories. Each is described below with the stats we track and how they can inform your QA strategy.

Demand​

Which actions users perform most often.

  • Relative frequency — this event’s share of all event emits in the scope.
  • Number of occurrences — total times this event was emitted.
  • Number of unique sessions — sessions that emitted this event at least once.

Covering high-demand events in tests protects core user journeys from regressions. Use these stats to prioritize test design on the actions that dominate real usage.

Duration​

Where users spend the most time.

  • Time to next event — how long from this event until the next action in the same session.
  • Time since previous event — how long from the previous event to this one.

Events where users dwell (long time to next or since previous) are good candidates for deeper testing, including variations and alternative paths.

Drop-off​

Where users abandon sessions.

  • Percentage of sessions where this event is terminal — share of sessions that ended on this event.
  • Reverse index in session — index from the end of the session (e.g. often last or near-last).
  • Time from this event to end of session — time from this event until the session ends.

Use these stats to find likely drop-off points. Check test coverage there to rule out bugs and investigate potential usability issues; add or adjust tests where users often leave and coverage is missing.

Depth​

How deep into a session users typically reach a given feature.

  • Position in session — where this event tends to occur in the sequence (1st, 2nd, 3rd event, etc.).
  • Time to first occurrence in session — how long until users reach this action from session start.

Top-of-funnel events (early position, short time to first occurrence) gate the rest of the user journey. Ensure they are well tested so regressions don’t block users from reaching deeper features.

TrueCoverage also answers:

  • Which event variants are high-frequency but untested? — Use metadata slicing (e.g. payment method, checkout step) to see which variations are common in production and which are covered in your test/coverage scope.
  • What do users do after a given action? — Event transitions show the funnel of follow-up actions (e.g. after "add-to-cart") so you can align test scenarios with real user paths.

You can use the 4Ds framework to prioritize test design and close high-impact gaps—e.g. "users are dropping off here, and there is no test covering that path"—and tie exploratory agent findings to the same behaviour.

How It Works (High Level)​

  • Events (key user actions like checkout, add-to-cart) are emitted in user sessions via @testchimp/rum-js. The @testchimp/playwright augments event emits with test identification so the same events are recorded in test runs.
  • You set a base execution scope (e.g. production) for event funnels and stats, and a coverage execution scope (e.g. test runs). TrueCoverage overlays coverage on top of base behaviour to show what is and isn’t covered.
  • We compute session-level and event-level metrics mapped to the 4Ds (Demand, Duration, Drop-off, Depth)—e.g. relative frequency, position in session, time to next event, percentage terminal event, time to first occurrence—and event transitions (A → B) to power funnel views, drop-off/dwell/discovery indicators, and metadata-based slicing.

For full detail on events, metrics, and execution scopes, see How it works. For the Overview and Event pages (chips, funnel, sample sessions, metadata slicing), see Dashboard.

Where to Use It​

TrueCoverage is available as its own area in the app. Configure the base and coverage execution scopes, then use the Overview for the event funnel and the Event page (by clicking an event chip) for per-event summary, funnel next steps, sample sessions, and metadata slicing.

ROI and Use Cases​

  • Prioritize test efforts by relative event frequency and metadata variants—focus on high-traffic paths and variants that lack coverage.
  • Close coverage gaps where users actually behave: “users are dropping off here—and there is no test covering that path.”
  • Connect exploratory findings to behaviour: see usability or performance bugs at the same steps where users dwell or drop off, so you can fix high-impact issues first.

For a full overview of purpose, capabilities, and how it differs from typical product analytics, see the TrueCoverage Introduction.