2 posts tagged with "manual-testing"

Manual testing still finds what automation misses—but too often, the path from a good manual run to a reliable automated test is broken.

Teams try Playwright codegen or record-replay tools, get a script quickly, and then spend weeks fighting flakes: shared data, missing assertions, no link back to the scenario, and no fit with POMs or fixtures already in the repo.

Today we’re announcing a workflow we recommend for turning manual sessions into SmartTests: capture with traceability, then let a coding agent upskilled with TestChimp author automation that actually belongs in your codebase.

---

The problem with “just record it”​

Record-replay—including Playwright codegen—optimizes for mirroring UI clicks. That is not the same as authoring a repeatable test.

Real automation needs:

• Arrange: seed data, fixtures, run-scoped entities
• Act: the journey that matters (often shorter than what a human clicked through)
• Assert: UI checks and backend state where outcomes live

Recorders capture the act layer well. They usually skip arrange and assert, and they never know which business scenario you were proving.

The result is familiar: tests that pass once on a developer machine, then fail in CI because the world-state was never set up—or because the script asserts the wrong thing (or nothing at all).

---

What we do instead​

TestChimp connects manual execution, test planning, and agent-authored Playwright in one loop.

1) Capture the manual session—with scenario context​

Use the TestChimp Chrome extension Manual tab to record a session while exercising your app. Start from Test Planning so the scenario is pre-linked (recommended), or link a scenario as part of the workflow.

What gets stored:

• Step-by-step actions and screenshots
• Linked scenarios (business context)
• Environment and release metadata
• Pass/fail outcome and optional bugs/notes

The session is auditable manual evidence and the reference for automation—not a throwaway recording.

2) Generate prompt → coding agent​

Open the session in TestChimp (Executions → Manual Sessions) and click Copy test generate prompt. Paste it into your agent host (Cursor, Claude Code, etc.) with the TestChimp skill installed.

The agent pulls rich context via get-manual-session-details (CLI or MCP):

• Recorded steps
• Linked scenarios and scenario steps
• Screenshots for visual grounding
• Project layout and existing POMs, fixtures, seed/probe endpoints

It uses the manual walkthrough as reference, navigates the app to validate selectors, and writes a SmartTest that reuses your harness—not a blind replay file.

3) Continuous improvement—not one-shot codegen​

Authoring does not stop at the first green run. TestChimp’s feedback loop surfaces coverage gaps (planned scenarios and TrueCoverage behaviour signals). Your agent runs /testchimp test on PRs and /testchimp evolve on a schedule or after deploys to close gaps, extend fixtures, and keep tests aligned with how users actually behave (QA on Autopilot).

The Web IDE is where you view tests, run them, and see insights aligned with your test folder structure—not where we expect most authoring to happen anymore.

---

How this differs from record-replay vendors​

Tools like mabl, Katalon, and Testim (and codegen at the framework level) center on capture → replay. They can speed up first script creation, but they typically:

• omit fixture-backed world-state
• lack in-repo scenario traceability at authoring time
• rarely generate backend probe assertions
• produce tests that do not compose with your existing Playwright patterns

TestChimp’s manual-to-auto path is informed agent authoring: session + scenario + screenshots + your repo conventions → repeatable Playwright in Git. See the full comparison: Record-replay vs TestChimp.

---

When to use which path​

Situation	What we recommend
Exploratory selector discovery	Playwright codegen or inspector—disposable output
Turning a validated manual scenario into CI automation	Manual capture → generate prompt → TestChimp agent
Ongoing suite maintenance and gap closure	/testchimp evolve + coverage insights
Viewing tests and folder-aligned insights	TestChimp Web IDE

---

Get started​

1. Install the Chrome extension and add the TestChimp skill to your coding agent.
2. Capture a manual session from a linked scenario (manual test capture guide).
3. Copy test generate prompt and let the agent author the SmartTest (Creating SmartTests).
4. Wire /testchimp test into your PR flow and schedule /testchimp evolve for portfolio upkeep.

Manual testing stays human. Automation becomes engineering-grade—because the agent authors like an engineer who read the scenario, not like a recorder that only heard the clicks.

:::tip Updated workflow Since this post, the Manual tab also supports open-ended sessions (objective instead of scenario), Add bug on steps (deferred until session finish), and Atlas-backed screen/state selection. See the current guide: Manual test session capture. :::

Manual testing is still where teams catch the “this feels wrong” stuff:

• confusing UX
• copy bugs
• edge-case flows
• integration weirdness that doesn’t show up in clean scripted runs

But there’s a persistent problem: manual testing evidence doesn’t stay connected to what was planned.

Test plans live in one place. Notes and screenshots live in another. Pass/fail outcomes live in Slack threads or Jira comments. And the mapping back to the scenario is usually… memory.

Today we’re announcing a new workflow in TestChimp: Manual testing with traceability.

---

The idea​

If your team already does test planning (user stories → scenarios), then manual execution should be:

• tied to a specific scenario
• captured as step-by-step evidence
• recorded with environment + release context
• marked passed / failed
• queryable later as execution history (not a dead document)

That’s what this feature does.

---

Manual Test Session Capture (in the Chrome extension)​

In the TestChimp Chrome extension, there’s now a Manual tab.

It lets a tester record a manual session while they execute a planned scenario, with traceability stored directly on the record.

What gets captured:

• Steps: each interaction is captured as a step
• Screenshots: uploaded automatically
• Notes: add notes to the latest step, optionally highlighting a UI element/area
• Outcome: mark the session as passed or failed
• Context: environment + release (and optional git branch context)

---

How it works (quick walkthrough)​

1. Open the Chrome extension and switch to the Manual tab
2. Click Create Manual Test Record
3. Select the test scenario you’re executing (required)
4. (Optional) pick the git branch context
5. Click Start Capture and run the test as usual
6. Add notes when needed (with optional element/area attachments)
7. Click End capture, then mark passed or failed
8. Open View execution to see the full record in TestChimp

If you want the full documentation, see Manual Test Session Capture.

---

Why this matters (beyond “we captured a GIF”)​

Manual testing isn’t going away. But it needs to stop being unstructured.

With scenario-linked manual records, you can answer real questions without archaeology:

• Which scenarios were manually executed for this release?
• Where are we relying on manual validation because automation doesn’t exist yet?
• What’s the evidence behind a “pass” when something regresses later?
• What’s failing on a specific branch or environment?

It’s manual testing… but operationalized.

---

Unified coverage: manual + automated, in one view​

The bigger win is what happens after you capture manual execution.

Because manual sessions are linked to the same scenarios as your SmartTests, TestChimp can provide unified requirement coverage insights across:

• Automated runs (SmartTests in CI or in the Web IDE)
• Manual runs (scenario-linked manual sessions with evidence + pass/fail)

So instead of two separate worlds (a test management tool for manual, and CI dashboards for automation), you get one requirement-centric view:

• scenario coverage status
• recent execution history (pass/fail)
• evidence trail for manual validation
• clear gaps where scenarios have no automated coverage yet

This is the foundation for keeping your suite honest: manual validation is visible and it doesn’t get conflated with “we have automation”.

---

How coding agents consume this to prioritize test authoring​

Once coverage and history are unified at the scenario layer, agents can treat it as an ordered backlog—especially in workflows like /testchimp test (PR-level) and /testchimp evolve (portfolio-level).

In practice, the agent pulls:

• Requirement coverage (what scenarios are covered vs missing tests)
• Execution history (what’s failing or flaky right now)
• (Optionally) TrueCoverage signals (what real users do most, where they drop off)

Then it prioritizes authoring work where it has the highest leverage:

• uncovered high-priority scenarios first
• gaps in the exact folder/feature area the team owns
• high-traffic paths with low coverage (when TrueCoverage is enabled)

The end result is a tighter loop: manual + automated executions feed the same insights, and those insights drive agents toward the most important missing tests—rather than “write more tests” as a generic goal.

---

Manual vs Script Gen (important distinction)​

This workflow is intentionally not about generating automation.

• Use Manual when you want an auditable record of a tester executing a scenario (pass/fail, notes, screenshots).
• Use Script Gen when you want to capture steps and generate a Playwright SmartTest.

---

Try it​

• Install the Chrome extension
• Plan scenarios in Test Planning
• Run your next manual regression session through the Manual tab

If you have feedback on what would make manual execution records more useful (branch/release filtering, richer notes, better rollups), we’re actively iterating.