Performance Testing Guides
Short answer
Performance testing answers whether the product still works when many users arrive at once (load testing) or when a tenant already has a lot of data (volume testing). Functional tests prove a single user can complete checkout; performance tests prove checkout still completes under concurrency, dataset size, and load patterns that match real demand. TestChimp runs that work as scenario-linked k6 journeys in Git, then compares runs across releases so a p95 or 5xx spike is a ship-blocking signal—not a screenshot of a one-off Grafana board.
Part of Testing Guides. Product how-to: Performance Testing.
Who this is for
Engineering and QA leads who need canonical answers to “what is load testing?”, “load vs volume vs stress?”, and “k6 vs JMeter vs Locust?”—then a clear path to run those tests on every PR and release without a separate load-testing silo.
Guides in this section
| Guide | What you'll learn |
|---|---|
| What is load testing? | Concurrent users (VUs / arrival rate), think time, SLOs—and why a tight request loop is not a user journey |
| What is volume testing? | Large datasets vs many users; indexes, N+1, pagination that only fail with history |
| Load testing patterns | Ramp, spike, soak, stress, closed vs open models, constant arrival rate |
| Performance test metrics | p95 latency, HTTP 4xx / 5xx / fail rate, VUs over time—and what each one is for |
| k6 vs JMeter vs Locust | Why most Git-native teams pick k6, when JMeter or Locust still fit, how to import |
| Performance regression testing | Compare this cut to the last one; auto-tag runs to releases via git SHA |
Load vs volume vs other “perf” words
These words get used interchangeably. They are different questions:
| Term | Question | Control |
|---|---|---|
| Load testing | Can N concurrent users complete this journey? | Virtual users / arrival rate + duration |
| Volume testing | Does the same journey still hold when the tenant already has M records? | Dataset size (cardinality, bytes)—not more VUs |
| Stress testing | Where does the system break? | Push past expected load until errors or latency explode |
| Soak (endurance) | Does it hold for hours? | Long duration at realistic load (leaks, queue backup) |
| Spike | What happens if traffic jumps suddenly? | Steep ramp, short peak, recover |
Raising both users and data size in one script hides which axis failed. Keep the axes separate.
Why one-off k6 charts are not a QA program
A dashboard from last Tuesday does not tell you whether 1.3.0 is slower than 1.2.0. You need:
- The same journey (stable test identity) on matching comparison keys (environment, profile, dataset, mock latency)
- Metrics that explain the fail — p95 up with a 5xx spike is a server incident; p95 up with a 4xx spike is usually the client or the script
- A release attachment so performance is part of the same ship story as SmartTests, manual runs, and release checks
That is the gap TestChimp performance testing is built for: k6 in your mapped tests folder, ingest into Executions → Performance Tests, overlay a prior run, and stamp the run onto the release from git SHA.
How TestChimp applies to these guides
| Layer | What it does |
|---|---|
| k6 in Git | Journeys and composites under tests/k6/—not a SaaS recorder. Playwright ignores **/k6/**. |
| Scenario links | The same #TS-… ordinals as SmartTests, so requirement coverage can include perf (--include-perf) |
| Agent workflows | /testchimp init-perf · create-perf-tests · run-perf-tests · upkeep-perf — not inside /testchimp test |
| Captured signals | Threshold, p95, fail rate, 4xx / 5xx class rates, duration, VU timeseries — what we capture |
| Release story | Wrapper runs stamp TESTCHIMP_GIT_COMMIT_SHA; TestChimp auto-tags those runs onto the release whose git cut contains that SHA, so the release Performance Tests panel sits next to functional evidence |
TrueCoverage ranks which journeys matter. It never silently sets VUs, RPS, duration, or dataset size—those come from policy or an explicit approval.
Product docs (how to run it)
- Performance Testing — why it exists in TestChimp
- k6 layout and capture
- Authoring and run workflows
- View results and compare runs
Frequently asked questions
What is the difference between load testing and volume testing?
Load testing varies concurrent users (or arrival rate) against a typical dataset. Volume testing keeps concurrency low and varies how much data already exists (rows, objects, bytes). Mixing both in one run hides which axis broke.
Is performance testing the same as stress testing?
No. Load testing checks expected concurrent demand. Stress testing pushes past that demand to find the breaking point. Soak testing holds realistic load for a long time. They share tools (k6, JMeter, Locust) but answer different questions—see load testing patterns.
Does TestChimp replace k6, JMeter, or Locust?
TestChimp standardizes on Grafana k6 in your repo and can import existing Locust, JMeter, Gatling, or Artillery suites into tests/k6/. It does not replace the executor: it authors, runs, ingests, compares, and attaches those runs to releases and scenarios.
Does /testchimp test run performance tests?
No. Functional SmartTests stay fast in /testchimp test. Performance testing uses dedicated workflows (init-perf, create-perf-tests, run-perf-tests, upkeep-perf) so load jobs do not block PR smoke.
How do performance tests show up on a release?
Wrapper runs record the git HEAD SHA. TestChimp stamps untagged runs onto the release whose git cut (prior release SHA → current SHA) contains that commit, and release-scoped /testchimp run-perf-tests also set TESTCHIMP_RELEASE. Open the release Performance Tests panel or Executions.
Make performance part of the release story
Run `/testchimp init-perf` once, author scenario-linked k6 journeys, and compare this cut to the last one—p95, 4xx/5xx, and fail rate on the same Executions page as the rest of QA.