How performance tests are organized

In brief: Performance tests live in a reserved k6/ tree under the mapped tests root—journeys and composites as one k6 run each, plus profiles, datasets, and wrappers that ingest summaries and timeseries into TestChimp Executions.

After /testchimp init-perf, the mapped SmartTests / tests root (the folder with .testchimp-tests) contains a reserved tree:

<tests root>/k6/
  journeys/      # one user journey per file — one k6 run
  composites/    # weighted mix of journeys — one k6 run
  profiles/      # smoke / load / volume options
  datasets/      # versioned manifests (no secrets)
  lib/           # shared helpers (gitignored reporter download)
  scripts/       # prepare, run, seed, related-selection wrappers
  artifacts/     # generated related selections and local output

Playwright must ignore this tree (testIgnore: ['**/k6/**']). k6 files are never SmartTests.

Journeys vs composites

A journey is one user path (checkout, search, invoice export). It may link several platform scenario ordinals (#TS-101, #TS-140). Run journeys when you want isolated degrade detection—the right default after a PR.

A composite is a weighted mix of journeys. It answers “can the system handle a typical overall load,” not “did this one path get slower.” Membership and relative weights need explicit approval. Adding a journey to a composite does not approve a new VU/RPS/duration number.

Every journey and composite file exports stable metadata:

export const testchimp = {
  id: 'checkout-journey',          // stable id (logical key with path)
  kind: 'journey',                 // or 'composite'
  scenarios: ['#TS-101'],          // journeys only
  testTypes: ['load'],             // 'volume' and/or 'load'
  members: ['checkout-journey'],   // composites only
};

k6 cannot read that sibling export from handleSummary. Always run through k6/scripts/run.sh (which calls run-journey.sh, and the staircase helper for volume) so ingest gets TESTCHIMP_PERF_META. Bare k6 run is not a TestChimp run—no ingest, and no Executions charts.

Load vs volume (keep the axes separate)

Axis	Controls	Does not control
Load	Concurrent users (VUs / arrival / duration) via profiles and a load identity-pool dataset	How much data each tenant already has
Volume	Records / objects / bytes per tenant via a volume manifest. Staircases seed 10% / 50% / 100% as isolated tenants, then one k6 run via k6/scripts/run.sh holds 1 VU at each size and emits volume_size	How many users arrive at once

Profiles (smoke.js, load.js, volume.js) own concurrency and duration. Datasets own shape and cardinality. Smoke is the safe authoring/contract default (max_vus: 1). Load and volume examples in the scaffold are not production capacity claims—override them only from approved policy.

k6 has no dataset-size executor. Load climbs with built-in ramping-vus; volume cannot. The TestChimp convention — so Executions can chart volume size the way it charts VUs — is one run, 1 VU, real tenants at 10% / 50% / 100% of target. Do not ingest three separate 1-VU plateaus, and do not step a volume_size gauge against one unchanged dataset.

TrueCoverage demand and redacted REAL E2E timings help pick which journeys to write and how to weight a composite. They never become absolute VUs, RPS, duration, dataset size, or SLOs.

Scripts you actually run

From the tests root (after k6/scripts/prepare.sh, which downloads @testchimp/k6 latest into gitignored k6/lib/):

# suite (all journeys — load then volume)
SEED_COMMAND=<project seed> k6/scripts/run.sh

# named files — paths relative to k6/
k6/scripts/run.sh journeys/foo.js journeys/nested/bar.js

# PR / CI: related-perf-tests.json (sibling of related-tests.json)
k6/scripts/run.sh --impacted

# authoring validate (skips volume staircase)
K6_PROFILE=smoke k6/scripts/run.sh journeys/foo.js

/testchimp run QA and /testchimp create-perf-tests write plans/smart-smoke/<branch>/related-perf-tests.json from the change scope (select-related.sh). --impacted reads that file. Paths are k6-relative (journeys/foo.js). A missing file warns and runs all journeys (set PERF_IMPACTED_STRICT=1 to fail instead). selected: [] means nothing to run — that is what run-qa writes when journeys exist but the PR hits none. Merge-gate CI should skip (or set PERF_IMPACTED_STRICT) when the file is missing — do not let portable fallthrough run the full suite on every PR.

Always go through run.sh (or the leaf run-journey.sh it calls). Bare k6 run is not a TestChimp run—no ingest, and no Executions charts. Volume staircases are an internal helper; do not call run-volume-staircase.sh from CI.

If a manifest needs seed data, call k6/scripts/seed.sh "$K6_DATASET" first. Seed never guesses a product endpoint—it uses an explicit SEED_COMMAND or SEED_URL. Pair teardown for generated state.

External dependencies

When the system under test calls payments, email, CRMs, identity providers, or LLMs, performance tests must not hit live vendors by default and must not stub them at 0 ms.

• Inventory outbound systems on the journey path.
• Stub them in the harness (env-level doubles preferred so real client code still runs). Use k6/lib/mock-external.js / mock-llm.js when the script needs a deterministic double.
• Respond with realistic latency (redacted REAL E2E timing distributions when available; otherwise documented typicals in policy).

Zero-latency stubs hide queueing, pool exhaustion, and timeout budgets. Live externals add cost and flakiness. Either mode is false confidence. Changing a stub from 50 ms to 300 ms is a new comparison key, not a silent win.

What TestChimp captures during a run

The wrapper (not a raw k6 run) does three jobs:

1. Metadata — folder path, file name, testchimp.id, kind, scenarios, test types, members, plus env you exported (TESTCHIMP_ENV, TESTCHIMP_RELEASE, TESTCHIMP_GIT_COMMIT_SHA, TESTCHIMP_BRANCH_NAME, profile, dataset, LLM mode).
2. Summary ingest — handleSummary posts the run. Stderr should include TestChimp perf ingest ok … runId=.
3. Timeseries attach — k6 run --out json (once, from the wrapper) → Node downsample (downsample.js, default 5 s buckets) → attach by that run id. Every k6 metric in the dump is kept (HTTP trends, rates, counters, gauges, custom metrics, VUs). When HTTP samples include tags.status, attach also writes failed-request class rates (http_req_failed.5xx.rate, .4xx, .3xx, .0xx) alongside combined http_req_failed.rate. Attach is non-fatal: a failed chart upload does not change the k6 exit code.

Those fields are why the Executions detail page can show threshold, p95, fail rate, duration, environment, release, profile, and overlay another run.

Captured field	Why it exists
Threshold passed / failed	Policy SLO gate for this profile
HTTP request duration p95	Headline latency for the run
HTTP fail rate	Error budget signal (k6 http_req_failed)
HTTP 5xx / 4xx / 3xx / no-response rates	Why the fail rate moved — server errors vs client 4xx vs redirects vs dropped connections
Duration	Wall-clock length of the k6 invocation
Timeseries (p95, VUs / volume size, and other k6 metrics)	Shape over time—spikes vs a slow grind; load charts VUs, volume charts volume_size
Profile / dataset / LLM mode / environment / dependency mocks	Comparison keys—mismatched runs are incomparable
Release, branch, git SHA, batch id	Trace the run to a cut, a PR, or a CI batch
Linked scenarios / operations / paths	Requirement coverage (--include-perf) and related selection

If attach is skipped (missing downsample.js or no runId), the summary still ingested but the detail page shows No timeseries was attached. Do not sample metrics inside journey JS and POST them as timeseries. Do not teach CI k6 run script.js --out json=….

Comparison keys (do not mix apples)

A later run is only a fair compare when these match:

• Environment (and envClass used at baseline promote)
• Profile name
• Dataset manifest / version
• LLM mode
• External mock inventory and latency profile
• Materially equivalent code and config for the path under test

The results UI recommends comparable candidates. CLI compare-perf-to-baseline exits nonzero when comparison.regressed is true. A missing or mismatched baseline is incomparable, never a pass.

Related selection artifacts

plans/smart-smoke/<branch>/related-perf-tests.json

Sibling of Playwright related-tests.json. file paths are k6-relative (journeys/foo.js). /testchimp run QA and /testchimp create-perf-tests both write it when k6/journeys exists. Execute with k6/scripts/run.sh --impacted.

Platform list-related-perf-tests returns journeys plus parent composites by default. Reasons include scenario, operation, and path.

Next

• Workflows for authoring and running
• Viewing results and comparisons
• Concept: performance test metrics (why p95, 4xx, and 5xx) · load vs volume · k6 vs JMeter vs Locust