Skip to main content

Run Smart Smoke

In brief: Smart smoke identifies related SmartTests for a change set, writes related-tests.json, enables selection with TESTCHIMP_SMART_SMOKE_ENABLED (defaults from use.testchimpSmartSmoke), runs the related∪tagged∪budget subset, and helps fix failures.

:::tip Authoritative product docs For how Smart Smoke works, when to use it, and full configuration reference, see Smart Smoke Runs — How it works · Configuration. This page is the agent workflow playbook. :::

:::note Legacy id Older policies and prompts may still say run-smart-regression / /testchimp run smart regression. Prefer run-smart-smoke going forward. :::

When to use​

After code or test changes when you want a high-ROI subset (not the whole suite), or as Phase 5 inside Run QA. For a known failing execution id, prefer Fix test execution.

Prompt​

/testchimp run smart smoke
/testchimp run smart smoke for checkout
/testchimp run smart smoke plans/stories/billing

Scope​

Optional. Feature branch → related tests from the PR/diff scope. Default branch → since last run or a history range. Or name an area / plans path.

Policy​

Optional policy can encode selection heuristics, flake rules, and validation requirements.

Prerequisites​

Depends on connect-to-test-env.

What it does​

  1. Identifies related SmartTests for the scope and writes plans/smart-smoke/<branch>/related-tests.json.
  2. Runs Playwright with TESTCHIMP_SMART_SMOKE_ENABLED=true so the reporter selects related ∪ tagged ∪ budget (defaults from playwright.config use.testchimpSmartSmoke — typically suitePercentage: 20 and includeTags: ['smoke']).
  3. Helps repair failures while respecting policy and reporting agent actions.

Prefer related-tests-only (TESTCHIMP_SMART_SMOKE_RELATED_TESTS_ONLY=true) as the safe default unless the user asks for budgeted smoke. Full env and locator rules: Configuration.

See also​