Performance Test Metrics: Latency, 4xx, 5xx, and Fail Rate
Short answer
A useful performance run records latency (p95 / p99 of HTTP duration), reliability (overall fail rate), and why it failed (HTTP 5xx vs 4xx vs redirects vs no-response), plus VUs over time so you know the load actually arrived. Thresholds tell you pass/fail against an SLO. Comparisons tell you whether this release is worse than a comparable prior run even when the SLO still passes. TestChimp ingest keeps those series on Executions and on the release, so performance data sits in the same QA story as functional tests.
Part of Performance testing guides.
Who this is for
Anyone reading a k6 summary (or a JMeter Aggregate Report) who needs to know which numbers are ship gates, which are diagnostics, and which are noise.
The metric set that actually changes decisions
| Metric | What it is | Utility |
|---|---|---|
| Threshold passed / failed | Did this profile’s SLO checks pass? | Binary gate for CI / release |
| HTTP duration p95 (p99) | 95th (99th) percentile request latency | User-visible slowness; catches long-tail, not just the mean |
| HTTP fail rate | Share of requests k6 treats as failed (http_req_failed) | Combined error budget |
| 5xx rate | Server / gateway / unhandled errors | “The system is breaking under this load or dataset” |
| 4xx rate | Client / auth / validation / business-rule errors | Often a journey or data bug, not raw capacity |
| 3xx rate | Redirects | Surprise hops that inflate duration; misconfigured base URL |
| No-response (0xx) rate | Dropped connections, resets, timeouts with no HTTP status | Saturation, killed pods, network, or a mock that never answered |
| Duration (wall clock) | Length of the k6 process | Distinguishes a 2-minute smoke from a 2-hour soak |
| VUs over time | Intended vs actual concurrency | A p95 explosion with flat VUs is an env/script failure, not a product degrade |
| Custom / business metrics | e.g. checkout completions per minute | Capacity in user outcomes, not only HTTP |
Averages hide the tail. p50 can look fine while p95 is the outage your customers feel. Prefer percentiles for latency SLOs.
Why you must split 4xx and 5xx
http_req_failed collapsing everything into one number is how teams ship the wrong fix.
| If you see… | Likely story | What not to do |
|---|---|---|
| 5xx up, p95 up | Server overload, dependency timeout, 502/503/504, query death | Ignore it because “a few errors are fine” |
| 4xx up, p95 flat | Auth token expired in the VU, sold-out SKU, validation, wrong dataset | “Add more boxes” — this is usually the script or seed, not hardware |
| 4xx up, p95 up | Chatty retries, 401 storms, or 429 rate limits you did not plan for | Treat 429 as 200 just to green the chart |
| 0xx / no response | Timeouts, connection reset, k6 never got a status | Raise thresholds; fix the environment first |
| Fail rate up, classes flat | Older reporter without status tags, or non-HTTP failures | Re-run via k6/scripts/run.sh so prepare.sh fetches npm @testchimp/k6 latest |
4xx is not automatically a capacity problem. A load test that “fails” on 401 because tokens were not refreshed is a harness bug. A volume test that 400s on “result set too large” may be the product bug you wanted.
Latency: which percentile?
| Percentile | Use |
|---|---|
| p50 | Sanity that the happy path is not absurd; never the only SLO |
| p95 | Standard user-facing SLO; matches how TestChimp headlines a run |
| p99 | Extra-tail (payments, search). Noisy on short tests—read it on soak/load plateaus |
| max | Outliers and single retries; do not gate on max alone |
Utility: p95 up vs last release with matching profile/dataset is a performance regression even if the threshold is still green (“soft degrade”). That is often the whole point of comparing runs.
Timeseries vs a single summary number
A 12-minute average p95 can hide a 2-minute 5xx storm in the middle of a spike. Timeseries (p95, fail classes, VUs) let you ask:
- Did latency track the ramp (expected) or jump while VUs were flat (env)?
- Did 5xx recover after a spike (healthy shed) or stay high (wedged)?
- Did we even reach target VUs?
TestChimp wrappers dump k6 JSON once, downsample (default 5 s buckets), and attach every metric in that dump to the run. Bare k6 run skips charts. See what we capture.
How TestChimp stores and shows these metrics
On Executions → Performance Tests (and the release Performance Tests panel):
- Headline: threshold, p95, fail rate, duration — with the comparison run underneath when you overlay
- Metric over time: default
http_req_duration.p95; switch to Failed request rate for any failure vs 5xx / 4xx / 3xx / no response - VU chart so latency is never read without concurrency
- Git commit and release columns so the same numbers attach to the ship candidate
CLI/MCP: get-perf-run, list-perf-runs, compare-perf-to-baseline (nonzero exit when comparison.regressed). Product UI: viewing results.
SLOs vs regressions
- SLO / threshold — “Is this profile acceptable in absolute terms?”
- Regression — “Are we worse than last Tuesday / last release on a comparable run?”
Green against a weak threshold can still be a product incident. TestChimp recommends overlays only when environment, profile, dataset, LLM mode, and mock latency match. A missing baseline is incomparable, not a pass.
Related
Frequently asked questions
What metrics should I track in a load test?
At minimum: HTTP duration p95 (and p99), overall fail rate, HTTP 5xx and 4xx rates separately, wall-clock duration, and virtual users over time. Percentiles beat averages. Split 4xx from 5xx so you do not scale hardware to fix a bad seed or expired token.
Why are 5xx errors important in performance testing?
5xx (and many timeouts) mean the server or an upstream failed under concurrency or data volume—the capacity and reliability story. They are the primary “the system is unhealthy” signal alongside p95 latency.
Are 4xx errors a load-test failure?
Sometimes. 429 may be expected shedding; 401/403 often means the harness lost auth; 422/409 can be business rules or depleted test data. Split 4xx from 5xx. Do not hide 4xx by marking them successful just to green a chart.
Why use p95 instead of average latency?
Average (mean) is pulled by both very fast cached hits and rare disasters. p95 is the latency most users still sit under and is the usual SLO; p99 describes the tail. TestChimp headlines HTTP request duration p95 on each run.
What performance metrics does TestChimp keep?
Wrapper ingest stores threshold result, p95, fail rate, duration, VU and metric timeseries, and status-class rates (5xx, 4xx, 3xx, no-response) when http samples are tagged. Runs also store git SHA, release, profile, dataset, and scenario links for comparison and the release QA story.
See p95 and 5xx next to the last release
Run k6 through TestChimp wrappers so Executions stores latency, 4xx/5xx class rates, and VUs—and overlay a comparable prior run instead of eyeballing a one-off summary.