How It Works

TrueCoverage is built on events emitted in user sessions, collected by testchimp-rum-js, and aligned with test runs via the Playwright TestChimp runner. This page explains those core ideas and the analytics metrics we compute.

Core Ideas

Events in User Sessions

• Events represent key user actions—e.g. send-message, checkout, add-to-cart—not low-level UI interactions like click-button or close-modal. You instrument your app to emit these events so they can be analyzed across sessions.
• Events are emitted in user sessions (browser sessions). Each session has a sequence of events with timestamps; we aggregate and analyze them per day.

Test Alignment

The playwright-testchimp-runner augments event emits automatically with test identification details (e.g. test file, test name, run id). That allows TestChimp to:

• Record the same event taxonomy during test execution as in production (or any other environment).
• Align test execution results with the events recorded across different environments.
• Overlay “coverage” (which events were emitted during test runs) on top of “base” behaviour (e.g. production) to show coverage gaps.

So you get one consistent event model for both real users and tests, and can see exactly which user behaviours are covered by tests.

Analytics Metrics We Calculate

All metrics below are computed per day (daily aggregates). They power the TrueCoverage Dashboard and event-level insights.

Session-Level (per day)

Metric	Description
Number of sessions	Count of sessions in the selected base scope.
Session duration	Histogram of session length (e.g. time from first to last event).
Event count per session	Histogram of how many events each session contains.

These help you understand session length and activity distribution before drilling into specific events.

Event-Type Level (per day)

For each event type (e.g. checkout, add-to-cart), we compute:

Metric	Description
Number of occurrences	Total times this event was emitted.
Number of unique sessions	Sessions that emitted this event at least once.
Position in session	Histogram of the event’s index in the session (1st, 2nd, 3rd event, etc.).
Reverse index in session	Histogram of “index from end of session”—helps identify drop-off likelihood (e.g. often last or near-last event).
Time to first occurrence in session	Histogram of time from session start to the first emit of this event—indicates discovery friction (how long until users reach this action).
Time from event to end of session	Histogram of time from this event to session end—indicates drop-off after the action.
Time to next event in session	Histogram of time from this event to the next event in the same session—indicates dwell (how long users stay before continuing).

These metrics drive the event chips (relative frequency, position), and the dwell / drop-off / discovery indicators on the Overview and Event pages.

Event Transitions (per day, aggregated)

We record event transitions: when event B occurs after event A in the same session (A → B). For each transition we maintain:

Metric	Description
Time to transition	Histogram of time from event A to event B.
Number of occurrences	How often this transition happened.
Number of unique sessions	Sessions that contained this transition.

Transitions are used to show funnel next steps for a given event: what users do after doing X, and the distribution of follow-up actions (e.g. after add-to-cart, how often do they go to checkout vs remove-item).

Metadata Fields

You can attach metadata to events (e.g. form_of_payment, checkout_step, plan_type). TrueCoverage uses metadata to:

• Break down event counts by metadata key values (e.g. for payment, break down by form_of_payment: card, PayPal, etc.).
• Show which value combinations are covered in the coverage execution scope and which are not—so you can prioritize testing for high-frequency variants that lack coverage.

Good Usage of Metadata for Slicing

• Categorical dimensions: form_of_payment, plan_type, checkout_step—slicing reveals which variants are common and which need test coverage.
• Feature flags or segments: e.g. experiment_variant, user_tier—to see behaviour and coverage by segment.
• Outcome or context: e.g. success/failure, source (modal vs inline)—to prioritize testing on failure paths or specific entry points.

Avoid using metadata for high-cardinality or unique values (e.g. user id, session id); use it for dimensions you want to slice and prioritize by.

Next Steps

• Dashboard — How the Overview and Event pages use these metrics, and how to use base vs coverage execution scope.