Performance Testing for Agentic Teams: Journeys, Composites, and Comparable Runs
Agent coding is very good at getting the feature to work.
A user can check out. The API returns 200. The SmartTest is green. You merge.
Then ten concurrent checkouts queue behind one chatty query. Or the reports page, which was snappy with three invoices in the seed DB, falls over on a tenant that actually uses the product.
That gap is not a mystery. Agents optimize for the path in front of them—one user, empty-ish data, mocked collaborators that return in 0 ms. Concurrency and data volume are different questions. Functional tests do not answer them.
So we shipped performance testing as a first-class TestChimp surface: k6 in your repo, agentic workflows that author the right journeys (and keep composites honest), related runs after a PR or a release, and an Executions view that compares this run to a prior one.
Full product guide: Performance Testing.
The questions that actually matter
Not “did k6 print a chart.” The questions product and eng already have:
| You want to know | What we run |
|---|---|
| Will this path hold if traffic shows up together? | Load journeys (many VUs)—not a bigger dataset |
| Does this page still work when the tenant already has history? | Volume journeys (cardinality / records)—not more users |
| Did this PR make checkout slower than last week? | Related journeys, then compare to a matching prior run |
| Can the evening mix still breathe if we add this journey? | A composite with an explicit membership/weight—absolute load stays a separate decision |
We keep load and volume as separate axes. “Make it heavier” by turning both knobs hides which one broke.
Agents author journeys. You still own capacity.
/testchimp create-perf-tests does not invent a k6 file from vibes.
It ranks real scenarios (priority, semantic coverage, get-requirement-coverage --include-perf). It uses redacted REAL E2E interaction shapes—method, path template, schema, status class, timing distribution—not cookies, tokens, or production bodies. When TrueCoverage is mature, relative demand helps order the queue and suggest composite weights.
What it will not do: copy a production RPS into load.js. TestChimp telemetry tells you what is worth testing. You (or run-perf-tests.policy.md) still set VUs, duration, and dataset size. Smoke is the default while authoring. Load/volume wait on an explicit capacity decision.
Composites are a weighted mix of journeys—“typical overall load,” not isolated degrade detection. Adding a journey to a composite is always a prompted approval. Silent membership is how you accidentally change the mix and then argue about the chart.
Outbound deps (payments, email, LLMs, partner APIs) get harness mocks with realistic latency. Hitting Stripe in a load test is expensive and flaky. Stubbing it at 0 ms is worse: you never see pool exhaustion. Either outcome is false confidence.
Folder layout, metadata, and wrappers: How performance tests are organized.
Related runs after a PR. Release range when you cut.
Perf load/volume does not execute inside /testchimp test. Functional smoke should stay fast. k6 execution is /testchimp run-perf-tests or CI k6/scripts/run.sh. Run QA does write plans/smart-smoke/<branch>/related-perf-tests.json when k6/journeys exists, so the next CI --impacted run picks up the affected journeys.
On a PR, we select related journeys from the change set—scenarios, operations, path templates—then run them through k6/scripts/run.sh --impacted. Bare k6 run is not a TestChimp run: no ingest, no Executions charts.
On a release, the prompt from the release page is:
/testchimp run performance tests for release 1.2.0
The git range is prior SHA → cut SHA, not whatever happens to be checked out. Ingest stamps TESTCHIMP_RELEASE so the release panel and Executions list light up. If existing journeys do not cover that range, the agent asks whether to author (nested create-perf-tests on the same range) instead of silently inventing a suite or pretending coverage exists.
/testchimp upkeep-perf is the long loop: stale scenario links, drifted contracts, composite membership, thresholds that should not be quietly weakened to make compare green.
/testchimp init-perf scaffolds k6/ once. /testchimp import-perf-tests brings Locust/JMeter/Gatling/Artillery/k6-from-elsewhere into that tree. Source VU counts stay out of load profiles until you approve capacity.
Workflow walkthrough: Performance testing workflows.
Results you can overlay, not a one-off HTML report
Wrapper runs ingest a summary and attach downsampled timeseries (p95, fail rate, VUs, and the rest of the k6 dump). Executions → Performance Tests lists them—optionally grouped by batch. Open a run for threshold, p95, fail rate, duration, metric-over-time, and VUs.
Then click Add Comparison Run. Recommended candidates share the comparison keys (environment, profile, dataset, LLM mode, mock/latency profile). The overlay re-bases both series on elapsed time so you are not comparing Tuesday 4pm wall clock to Wednesday 9am.
That is the degrade question: same kind of run, previous comparable result, is this worse? Green against a loose threshold can still be a regression versus last Tuesday. CLI compare-perf-to-baseline is the same contract for agents and CI—it exits nonzero when comparison.regressed is true. A mismatched baseline is incomparable, not a pass.
How this fits the rest of TestChimp
If you’ve been following along:
- SmartTests (now grouped as Functional Testing in the docs) prove the path works for a user
- Smart Smoke keeps that suite runnable in a CI budget
- TrueCoverage says which journeys real people actually take
- Performance testing asks whether those journeys still hold when many people take them—or when the tenant already has history
Planned reality → functionally tested reality → load/volume-tested reality → production reality. The middle was the hole for teams whose agents ship faster than anyone can stare at an EXPLAIN plan.
Frequently asked questions
What is TestChimp performance testing?
Grafana k6 scripts in your mapped tests folder (k6/journeys and k6/composites), authored and selected by agentic workflows, ingested into Executions so you can chart a run and overlay a comparable prior run.
Do I need this if my SmartTests are green?
Yes if you care about concurrency or tenant data volume. Agents (and humans) routinely ship designs that are correct for one user on a small seed database and fail under the conditions production actually creates.
Does /testchimp test run k6?
No. Keep functional PR smoke fast. Use /testchimp run-perf-tests (or a dedicated CI job that calls the same wrappers).
Will TestChimp pick my VU count from production traffic?
No. TrueCoverage and interaction timings are relative signals—what to test, how to weight a mix. Absolute VUs, RPS, duration, and dataset size come from policy or an explicit approval.
How do I see if we got slower?
Open the run → Add Comparison Run → pick a recommended candidate. Or compare-perf-to-baseline in CI. Compare only matching environment / profile / dataset / mock profile.
Docs: Intro · Organization · Workflows · Results.
Concept: What is load testing? · Volume testing · k6 vs JMeter vs Locust · Compare across releases.