Programmatic Release Gating
Short answer
Release gating means deciding whether a version is safe to ship from data, not spreadsheets. TestChimp exposes POST /api/mcp/get_release_details (API key auth) so CI jobs and agents can load in-scope test status, open issues, and release checks for a release label and apply your own pass/fail policy.
When to use this API
| Use case | API |
|---|---|
| ExploreChimp / agent needs cut SHA, prior release, focus areas | get_release (POST /api/mcp/get_release) — thin catalog |
| CI or agent quality gate needs stats + detail | get_release_details (POST /api/mcp/get_release_details) — this page |
The gate API does not encode thresholds (e.g. “block if any P0 failed”). Your pipeline reads the JSON and decides.
Authentication
Same as other MCP tools: project-scoped API key.
| Header / env | Value |
|---|---|
TestChimp-Api-Key | Project API key |
| CLI | TESTCHIMP_API_KEY (and optional TESTCHIMP_BACKEND_URL) |
Resolve keys the same way as other @testchimp/cli / MCP tools (project MCP config walk-up). Never commit keys.
Request
POST /api/mcp/get_release_details
Content-Type: application/json
{ "version": "1.2.0" }
version is the release label in the catalog (same string you see in the Releases UI).
Response shape (overview)
| Field | Meaning |
|---|---|
label | Echo of the requested version |
scope | Release focus areas (FileScope entries from the catalog) |
summary.environmentStats | Per environment that has test activity: priority × execution-status counts for in-scope scenarios |
summary.issueStats | Open-issue counts by severity×category and severity×reported-by (release_scanners / explorechimp / humans) |
summary.scanSummaries | Each release check scan: status, bug severity counts, checker config |
detailedResults.scenarios | In-scope scenarios with metadata + latest result per named test run on the release |
detailedResults.openIssues | Open issues tagged to the release (title + metadata only; no description/attachments) |
Execution statuses match scenario coverage: successful, failed, not attempted, blocked, skipped (out-of-scope scenarios are excluded from in-scope aggregations).
Aggregation rules
| Rule | Behavior |
|---|---|
| In scope | Union of scenarios from non-archived named test runs for the release (plus release-tagged automation), minus scenarios whose winning result is out of scope |
| Per environment | Within each environment, latest execution wins per scenario across that env’s test runs |
| Detail matrix | For each in-scope scenario, one result per named test run that includes it; within a run, latest execution only |
| Open issues | Tagged to the release (app_release_id); statuses Active, In progress, Blocked (fixed / ignored / duplicate / archived excluded) |
CLI
export TESTCHIMP_API_KEY=… # from MCP config; never echo
testchimp get-release-details --version '1.2.0'
MCP tool name: get-release-details (same body: { "version": "…" }).
Example: curl gate stub
export TESTCHIMP_API_KEY=…
export TESTCHIMP_BACKEND_URL=https://api.testchimp.io # or your host
RESP=$(curl -sS -X POST "${TESTCHIMP_BACKEND_URL}/api/mcp/get_release_details" \
-H "Content-Type: application/json" \
-H "TestChimp-Api-Key: ${TESTCHIMP_API_KEY}" \
-d '{"version":"1.2.0"}')
# Example policy: fail CI if any in-scope scenario is FAILED in any environment
echo "$RESP" | jq -e '
.summary.environmentStats[]?.priorityStatusCounts[]?
| select(.status == "FAILED_SCENARIO_COVERAGE" and (.count // 0) > 0)
| empty
' && echo "GATE PASS" || { echo "GATE FAIL: failed in-scope scenarios"; exit 1; }
Adjust the jq policy for your team (P0-only, max open criticals, required completed scans, etc.).
Related
- Release Management API Guide
- Interactive API Reference (Scalar)
- API authentication and request basics
- Release management overview
- Release Checks
- Linking automation batches
- Manual sessions → test runs
Frequently asked questions
What is programmatic release gating in TestChimp?
It is using POST /api/mcp/get_release_details with a project API key to load in-scope test aggregations, open-issue stats, and release-scan summaries for a version label, then applying your own pass/fail policy in CI or an agent.
How is get_release_details different from get_release?
get_release returns thin catalog metadata (git SHA, prior release, focus areas). get_release_details returns gate-oriented summary and detail: per-environment priority×status counts, open issues, scan configs, and per-scenario test-run results.
Does TestChimp enforce a fixed gate policy?
No. The API returns the data. Your pipeline or agent decides thresholds—for example zero failed P0 scenarios or no open critical security issues.
Gate the release from real TestChimp data
Call get_release_details in CI, apply your policy, and ship only when in-scope tests, issues, and release checks meet your bar.