Skip to main content

API Contract Governance: Cover What Your Spec Promises

· 6 min read
Nuwan Samarasekera
Nuwan Samarasekera
Founder & CEO, TestChimp

TL;DR: We’ve shipped API contract governance in TestChimp. Point us at your OpenAPI roots, opt in to API payload capture on SmartTest runs, and the APIs surface shows which operations—and which request/response fields and status codes—your automation actually exercised. Uncovered holes become Create issue, Create test, or Ignore gap—accountable work, not a spreadsheet of “we should cover that someday.”

The contract nobody audits​

Your OpenAPI (or Swagger) file is not documentation fluff. It is the business contract of the product: methods, paths, request shapes, response codes, and fields clients depend on.

UI E2E already touches that contract. Checkout hits POST /orders. Declined payment hits POST /payments. A rare enum or a 422 body? Often never.

Without a contract lens, teams hit the same failure modes:

Failure modeWhat goes wrong
Happy-path blind spotsGreen UI suites; error enums, optional filters, and rare response codes never run
Spec driftTraffic hits paths or fields the OpenAPI root never declared
Silent API debt“We have E2E”—whole services still have zero covering tests
Unowned gapsSomeone notices a hole; nothing is filed, authored, or consciously deferred

API contract governance is not a separate API-testing product. It is the audit and triage layer that turns traffic your SmartTests already generate into accountable coverage of the contract.

Full product guide: API Contract Governance.

How it works​

Three inputs. One surface.

  1. Discover the contract — Configure OpenAPI root path(s) from your connected repo (APIs → Configure). TestChimp parses them into services and operations.
  2. Observe traffic — Run SmartTests with TESTCHIMP_ENABLE_API_CAPTURE=1 (and @testchimp/playwright ≥ 0.2.17). Matching HTTP interactions upload with the batch; coverage denorm runs after—not on the hot ingest path.
  3. Govern gaps — List view for the portfolio; detail view for schema fields and response codes. From any hole: file an issue, copy an agent prompt to create/update tests, or ignore the gap when it is intentional.

Capture is off by default so ordinary runs stay light. Opt in when you want Operations coverage—same pattern as other optional runtime features. Setup detail: Set up API contract governance and the runtime plugin.

From endpoint scores to schema fields​

The APIs list is the portfolio view: service picker, coverage-band summary, method/path, spec-defined vs observed-only, coverage score, latency, covering-test chips.

Click a row and you get the interesting part—field-level truth:

API operation detail with request schema tree, undocumented fields, and Create issue / Create test / Ignore gap

  • Request schema and query parameters with covering tests per field
  • Response schemas nested by status code
  • undocumented tags when traffic (or denorm) sees a field or code the OpenAPI root never declared
  • Empty covering tests when the contract exists but automation never exercised it

That is the difference between “we hit /orders” and “we never sent delivery.type = same-day, and nobody owns that hole.”

List and detail walkthroughs: Operations list · Operation detail.

Governance that sticks: issue, test, or ignore​

Coverage without triage is another dashboard nobody opens.

Wherever coverage is missing—operation, field, or response code—TestChimp offers the same decision set:

ActionWhen to use
Create issueTrack the gap as work—pre-filled title/description with method, path, operation id, and field or response code
Create testCopy an agent prompt scoped for /testchimp create tests—agents may update existing UI or API specs, not only invent a dedicated api/ folder
Ignore gapAcknowledge intentional out-of-scope or deprecated surface without filing or authoring

Agents already understand operation-id / field / response-code scopes. The UI just stops the gap from living only in someone’s head.

How this fits the rest of TestChimp​

If you’ve followed us:

  • Requirement quality governance hardens the plan contract before agents spend tokens
  • Release governance hardens the ship decision for a version
  • API contract governance hardens what your automation proved against the HTTP contract—the surface clients and integrations actually depend on

SmartTests remain the traffic source. Issues remain the backlog. Create-tests remains the authoring path. The APIs page is where contract debt becomes visible and governable—next to TrueCoverage for behaviour-aligned RUM, not instead of it.

Planned reality → tested reality → production reality. The API surface was the missing middle for teams whose “coverage” stopped at green UI journeys.

Frequently asked questions​

What is API contract governance in TestChimp?​

API contract governance maps OpenAPI operations to real and mocked SmartTest traffic so you can see endpoint and schema-field coverage, then triage gaps with create issue, create test, or ignore gap.

Do I need dedicated API tests?​

No. Coverage is inferred from HTTP traffic your SmartTests already generate when capture is enabled. You can still author dedicated API specs; the governance surface tracks both.

Why is my coverage empty after configuring OpenAPI?​

Parsing the spec discovers operations. Scores and covering tests need runs with TESTCHIMP_ENABLE_API_CAPTURE=1. Capture is off by default on purpose.

What does Ignore gap do?​

It marks an uncovered operation, field, or response code as intentionally ungoverned for now—without creating an issue or a test—so teams can acknowledge debt without false urgency.

Spec defined No vs undocumented—what’s the difference?​

Spec defined No on an operation means the whole method/path is missing from the OpenAPI root (observed in traffic only). An undocumented tag on a field or response code means that leaf was seen but not declared—classic contract drift.

Try it​

  1. Connect git and open APIs → Configure — select your OpenAPI root file(s)
  2. Wait for parse/sync; pick the service in the list
  3. Run SmartTests with TESTCHIMP_ENABLE_API_CAPTURE=1
  4. Refresh APIs — open a low-score or uncovered operation
  5. Create issue, Create test, or Ignore gap on the holes that matter

Start here:

Know the contract. Cover the edge cases—before unaudited branches ship.

Further reading​

TestChimp

Related posts