Skip to main content

Linking Scenarios from SmartTests

Overview​

Linking your SmartTests to scenarios enables requirement traceability, allowing you to see which user stories and scenarios are covered by your tests and track their execution status. TestChimp makes this process simple with scenario comments and autocomplete support in the Web IDE.

Adding Scenario Comments​

To link a test to a scenario, simply add a comment in your test script:

test('login test', async ({ page }) => {
  // @Scenario: User can log in with valid credentials
  await page.goto('https://example.com/login');
  await page.fill('#username', 'testuser');
  await page.fill('#password', 'password123');
  await page.click('button[type="submit"]');
  await expect(page).toHaveURL('https://example.com/dashboard');
});

The comment format is:

// @Scenario: <scenario title>

Autocomplete Support in Web IDE​

The TestChimp Web IDE provides autocomplete support for scenario titles:

  1. Type the comment: Start typing // @Scenario:
  2. Autocomplete appears: A dropdown list shows available scenarios from your project
  3. Select a scenario: Choose the scenario from the list
  4. Exact match: The autocomplete ensures exact matching of scenario titles

This autocomplete feature:

  • Prevents typos: Ensures scenario titles match exactly
  • Shows available scenarios: See all scenarios in your project
  • Saves time: No need to remember or look up scenario titles manually

How Linking Works​

When you add a // @Scenario: comment to your test:

  1. Test execution tracking: TestChimp identifies the scenario comment during test execution
  2. Automatic linking: The test execution is automatically linked to the specified scenario
  3. Coverage calculation: The scenario's coverage is updated based on test execution results
  4. Roll-up to user stories: Coverage is aggregated at the user story level

Viewing Linked Coverage​

Once tests are linked to scenarios, you can view coverage in:

User Stories View​

  1. Navigate to the User Stories section
  2. Click "Show Test Coverage"
  3. Select the execution scope (environment, release, time range)
  4. View coverage indicators on user stories and scenarios

For more details, see Requirement Traceability.

Insights Tab - Requirement Coverage​

  1. Navigate to SmartTests section
  2. Open the Insights tab
  3. Select Requirement Coverage pane
  4. View which user stories are covered by your tests

Requirement Coverage in SmartTests

Best Practices​

1. Use Consistent Scenario Titles​

  • Match exactly: Scenario titles in comments must match scenario titles in TestChimp exactly (case-insensitive)
  • Use autocomplete: Leverage the Web IDE autocomplete to ensure exact matching
  • One scenario per test: Generally, one scenario comment per test provides the clearest traceability
  • Multiple scenarios when appropriate: Link multiple scenarios if a single test genuinely covers multiple scenarios
  • Cover all scenarios: Ensure every scenario has at least one test linked to it (use coverage dashboards to identify gaps)

Integration with CI/CD​

When you run tests in CI with playwright-testchimp-reporter configured, scenario comments are automatically extracted and test executions are linked to scenarios. No additional configuration is needed.

For details on setting up CI execution, see Run SmartTests in CI with Playwright Runner.

Troubleshooting​

Tests Not Showing in Coverage​

  • Check scenario titles: Ensure the scenario title in the comment matches exactly (case-sensitive)
  • Verify scenario exists: Confirm the scenario exists in your TestChimp project
  • Check test execution: Ensure tests have been executed (either directly or via CI)
  • Review execution scope: Check that the selected execution scope includes your test runs

Autocomplete Not Working​

  • Refresh the page: Available scenarios are fetched at page load to avoid slow loads. If you recently added a scenario, do a page refresh.
  • Verify scenarios exist: Make sure you have scenarios created in your project

Linking scenarios from SmartTests is a simple but powerful way to enable requirement traceability and gain visibility into how your tests cover your requirements.