Configuration

In brief: Smart Smoke is opt-in per run via environment variables—configure budgets and tags in playwright.config defaults, write related-tests.json for the branch, then run normal Playwright.

Requirements

Requirement	Notes
@testchimp/playwright	≥ 0.2.20 (smart-smoke selection + skip reason)
TESTCHIMP_API_KEY	On the Playwright runner process (not only MCP)

Install / reporter base setup: Run SmartTests in CI.

---

Enable a run (required)

Smart Smoke is opt-in per run. Do not hard-enable it in playwright.config as always-on.

export TESTCHIMP_SMART_SMOKE_ENABLED=true   # or 1
npx playwright test

---

Modes

Mode	When	How
Related-tests-only	Tight PR confidence; safe default for agents	TESTCHIMP_SMART_SMOKE_RELATED_TESTS_ONLY=true
Budgeted smoke	Broader ROI within a size cap	Set a budget

Related-tests-only skips the selection API and runs only locators listed in related-tests.json.

---

Budget and tags

Environment variables (win over config when set)

Variable	Purpose
TESTCHIMP_SMART_SMOKE_ENABLED	true / 1 to enable for this run
TESTCHIMP_SMART_SMOKE_RELATED_TESTS_ONLY	true / 1 → related-tests-only
TESTCHIMP_SMART_SMOKE_MAX_TIME_BUDGET_MINS	Time packing (minutes)
TESTCHIMP_SMART_SMOKE_MAX_TESTS	Cap at N tests
TESTCHIMP_SMART_SMOKE_SUITE_PERCENTAGE	Cap at ~N% of the suite
TESTCHIMP_SMART_SMOKE_INCLUDE_TAGS	Comma-separated tags (smoke or @smoke)
TESTCHIMP_BRANCH_NAME	Branch used to load plans/smart-smoke/<branch>/related-tests.json

If budgeted mode is on and no size constraint is set, the plugin defaults to suitePercentage = 20 and logs a warning.

When more than one size constraint is set, selection uses the most restrictive effective cap.

Project defaults (playwright.config)

Optional defaults only under use (templates often seed these). Never set enabled / related-only here as always-on:

// playwright.config.js (SmartTests root)
const { defineConfig } = require('@playwright/test');

module.exports = defineConfig({
  use: {
    // …
    testchimpSmartSmoke: {
      suitePercentage: 20,
      includeTags: ['smoke'],
      // maxTests: 40,
      // maxTimeBudgetMins: 20,
      // relatedTestsOnly: false,  // prefer env per run
    },
  },
  reporter: [
    ['list'],
    ['@testchimp/playwright/reporter'],
  ],
});

TypeScript projects use the same use.testchimpSmartSmoke shape.

---

Tags

Budgeted smoke includes tagged tests as seeds (union with related + newly authored).

• Use Playwright tag: '@smoke' (and other tags from plans/knowledge/policies/global.policy.md → tags:).
• Do not use { type: 'group', description: 'smoke' } annotations for suite membership—Playwright CLI and smart-smoke matching use tags, not group annotations.

Example:

test(
  'guest can complete checkout',
  {
    tag: '@smoke',
    annotation: [{ type: 'scenario', description: '#TS-42' }],
  },
  async ({ page }) => {
    // …
  }
);

---

related-tests.json

Path

plans/smart-smoke/<branch>/related-tests.json

• <branch> is the current git branch name.
• If the branch contains / (e.g. feat/checkout), keep path segments literally (nested directories).

Body

Either a JSON array of TestLocators, or an object:

{
  "relatedTests": [
    {
      "folderPath": ["auth"],
      "fileName": "login.spec.ts",
      "testSuite": [],
      "testName": "user can log in"
    }
  ]
}

Also accepted: related_tests, and snake_case locator fields (folder_path, file_name, test_suite, test_name).

Locator rules

Field	Meaning
folderPath	Path segments under the mapped tests root — do not prefix tests/
fileName	Spec basename (e.g. login.spec.ts)
testSuite	Nested test.describe titles (array; empty if none)
testName	The test title

Agents produce this file during /testchimp run smart smoke and as Phase 5 of /testchimp test. It is also required at the end of Create tests.

---

CI examples

Related-tests-only on PRs

# GitHub Actions sketch
- name: Smart Smoke (related only)
  working-directory: tests   # or your SmartTests root
  env:
    TESTCHIMP_API_KEY: ${{ secrets.TESTCHIMP_API_KEY }}
    TESTCHIMP_SMART_SMOKE_ENABLED: "true"
    TESTCHIMP_SMART_SMOKE_RELATED_TESTS_ONLY: "true"
    TESTCHIMP_BRANCH_NAME: ${{ github.head_ref || github.ref_name }}
  run: npx playwright test

Ensure the job checks out plans/smart-smoke/<branch>/related-tests.json (committed by the agent / prior workflow step) before the run.

Budgeted smoke (20 minutes)

- name: Smart Smoke (budgeted)
  working-directory: tests
  env:
    TESTCHIMP_API_KEY: ${{ secrets.TESTCHIMP_API_KEY }}
    TESTCHIMP_SMART_SMOKE_ENABLED: "true"
    TESTCHIMP_SMART_SMOKE_MAX_TIME_BUDGET_MINS: "20"
    TESTCHIMP_SMART_SMOKE_INCLUDE_TAGS: "smoke"
    TESTCHIMP_BRANCH_NAME: ${{ github.head_ref || github.ref_name }}
  run: npx playwright test

Honor config reporters—do not pass Playwright CLI --reporter / -r (it replaces reporters and can drop TestChimp ingest).

---

What you should see

Observation	Meaning
Log: smart-smoke enabled / selection size	Plugin resolved config and selected locators
Tests skipped with reason smart-smoke	Outside the selected set (expected)
Warning about default 20% suite	Budgeted mode with no size constraint
Selection API / 401 errors	Re-check TESTCHIMP_API_KEY + backend URL on the runner

---

Agent workflows

Prompt	Role
/testchimp run smart smoke	Standalone: impact → related-tests → enable → run → fix
/testchimp test	Composite: Smart smoke is Phase 5 after Validate
Legacy /testchimp run smart regression	Same workflow — prefer smart smoke going forward

Authoritative product behaviour is this section; the workflow pages are the agent playbooks.

See also

• Introduction
• How it works
• Linking scenarios
• Run SmartTests in CI