How to Test Canvas, Charts, and Visual Widgets
Short answer
Canvas, WebGL charts, whiteboards, and AI-generated visual widgets resist stable Playwright selectors—pixel clicks flake on resize and devicePixelRatio. Prefer probe Assert on underlying series data, chart library test hooks where available, and hybrid ai.act for semantic targets ("select Q3 bar", "draw rectangle around region") with ai.verify on visible outcomes—not coordinate math in CI.
Part of Testing Guides by AI and conversational UX.
Who this is for
Teams shipping dashboards, analytics charts (ECharts, Chart.js, D3, Recharts), configurators with canvas previews, whiteboard/collaboration surfaces, or AI copilots that manipulate visual widgets.
Why testing visual widgets matters
- Wrong decisions — drill-down shows stale series; filter not applied to chart data
- Silent render failures — WebGL context loss → blank chart with no error toast
- AI copilot lies — assistant claims "highlighted Q3" but selection state empty
- Accessibility gaps — keyboard users cannot reach chart actions tested only via coordinates
Complexity map
| Scenario | Edge case | Why tests break | Approach |
|---|---|---|---|
| Coordinates | Responsive resize | Click misses bar | ai.act semantic target or probe |
| WebGL context loss | GPU flake in CI | Blank canvas | Probe chart data API |
| Drill-down | Click bar → detail | Wrong series | Probe selected series id |
| Tooltip | Hover-only data | Headless miss | Probe or library API |
| Animation | Assert mid-transition | Flake | Wait animation end flag |
| Time series | Timezone shift | Wrong bucket | Seed fixed timestamps |
| Legend toggle | Series hidden | Stale assert | Probe visible series ids |
| Export chart | PNG download | Untested pipeline | Download + optional parse |
| AI highlight | Natural language select | No stable selector | ai.act + probe selection |
| Map overlap | See Google Maps guide | iframe + canvas | frameLocator + probe |
Testing pyramid for visuals
1. Unit: chart data transforms, scales
2. Probe/API: series payload for seeded dashboard
3. E2E: ai.act interaction + ai.verify + probe selection state
4. Visual regression (optional): layout-critical only, not every CI run
Default PR CI: 2 + 3, not pixel diff on every chart.
Probe Assert on chart data (preferred)
await page.goto('/dashboard/sales?runId=' + runId);
await expect.poll(async () => {
const res = await request.get(`/api/test/probe-chart-series?dashboard=sales&runId=${runId}`);
const { series } = await res.json();
return series.find(s => s.id === 'q3')?.value;
}).toBe(125000);
Seed dashboard data in Arrange so probe values are deterministic.
Chart library test hooks
Some libraries expose DOM or APIs:
| Library | Hook |
|---|---|
| Recharts | data-testid on Bar/custom components |
| ECharts | chart.getOption() in page.evaluate (fragile—prefer probe) |
| Chart.js | Plugin callbacks in test build |
Prefer instrumentation you control in test builds over minified internal selectors.
ai.act for semantic interaction
When coordinates are unstable:
await ai.act('Click the Q3 bar in the revenue chart');
await ai.verify('Q3 detail panel opens or Q3 is highlighted in the chart');
await expect.poll(() => probeSelectedSeries(runId)).toBe('q3');
Use ai.act/ai.verify for visual targeting; probe for whether the right data slice activated.
WebGL blank chart detection
await expect(page.getByTestId('chart-container')).toBeVisible();
const pixelBlank = await page.evaluate(() => {
const canvas = document.querySelector('canvas');
if (!canvas) return true;
const ctx = canvas.getContext('2d') || canvas.getContext('webgl');
return !ctx;
});
expect(pixelBlank).toBe(false);
// Authoritative: probe
await expect.poll(() => probeChartError(runId)).toBeNull();
Drill-down and filter integration
| Act | Assert |
|---|---|
| Apply date filter | Probe series point count changes |
| Click legend off | Probe excluded series id |
| Drill to detail route | URL + probe detail record id |
Link to search and filters when charts share filter state with tables.
AI + canvas copilots
When users ask copilot to "circle the anomaly" or "compare these two bars":
- AIMock planner returning
highlight_seriestool - ai.verify visual outcome at high level
- probe
selection_stateon backend—never trust model description alone
See conversational UI and agent workflows.
Anti-patterns
| Anti-pattern | Why it fails | Better approach |
|---|---|---|
page.mouse.click(x,y) | Breaks on resize | ai.act or probe |
| Screenshot every PR | Noise on fonts | Probe data; visual on release |
| Assert tooltip text only | Hover flake | Probe underlying point |
| Skip WebGL failure path | Blank dashboard | Probe error + empty state UI |
| Real-time live data | Nondeterministic | Seed fixtures per runId |
| ai.verify without probe | Copilot hallucination | selection probe |
Example scenario
Situation: User drills from monthly chart into March detail for seeded account.
Expected outcome: Detail view shows March metrics matching seeded probe data—not empty chart.
Why UI-only automation breaks: Chart renders bars but wrong month data due to timezone bucket bug.
- Arrange: Seed series with known March value via test API.
- Act: ai.act('Open March from the monthly revenue chart') or click data-testid hook.
- Assert: Probe selectedMonth=2025-03 and detail total matches seed.
TestChimp workflow: Track widget_type × interaction_mode in TrueCoverage when drill-down dominates prod.
Same Arrange/Act/Assert pattern as expired-coupon checkout.
Evals vs E2E: when each layer helps
| Layer | Best for | Limitations |
|---|---|---|
| Offline evals | AI-generated chart summaries, natural language descriptions of data | Does not test WebGL render, click routing, or drill-down routes |
| E2E SmartTests (ai.act/ai.verify + probe series data) | Interaction wiring, filters, drill-down navigation | Not for validating chart pixel aesthetics at scale |
| Hybrid | Evals on copilot narrative; E2E on selection state and data probes | Standard for AI analytics dashboards |
Numeric data correctness belongs in probes and unit tests on aggregations. LLM-as-judge may grade narrative insights offline. E2E confirms user actions change the right series. TestChimp does not ship eval tooling.
Connect scenarios to your QA workflow
Capture business rules in markdown test plans and enforce them with seed routes and probe Assert. Link SmartTests with // @Scenario: for requirement traceability. Use /testchimp test on PRs; /testchimp explore on SmartTest paths for non-functional gaps (ExploreChimp).
Related scenarios
- AI web apps — hybrid SmartTests
- Google Maps — geo visual overlap
- Product configurators — canvas previews
- Data grids — linked table/chart views
External references
- Playwright mouse — last resort for coordinates
- Playwright visual comparisons — optional layout checks
- SmartTests intro — ai.act and ai.verify
Frequently asked questions
Can Playwright click canvas coordinates reliably?
Often no on responsive layouts—use data-testid hooks on chart components, ai.act for semantic targets, or probe API for underlying series values. Coordinates are last resort.
How do I assert chart data without reading pixels?
Expose probe endpoint or test-only API returning series ids and values for seeded runId. Assert JSON—not canvas pixels—in CI default jobs.
When should I use ai.act on charts?
When library lacks stable selectors and interaction is semantic (select Q3, open legend menu). Always pair with probe on selection or filtered data.
How do I detect blank WebGL charts?
Assert container visible plus probe chart data non-empty and chart_error null. Optional canvas context check as secondary signal.
Should I snapshot charts in CI?
Reserve visual snapshots for release or layout-critical marketing dashboards. Default PR CI uses probes to avoid font/OS flake.
How do I test AI copilot chart highlights?
AIMock tool plan for highlight action; ai.verify high-level outcome; probe selection_state authoritative.
Complex visual widgets broke after release—how do hybrid SmartTests help?
Compare prod vs test-run across widget_type × interaction_mode in TrueCoverage. Reserve ai.act for canvas; keep Arrange/Assert on probes. /testchimp evolve when prod interaction modes lack scenarios.
Apply these patterns in your repo
Run `/testchimp init` to connect TestChimp to your repo, then `/testchimp test` on PRs to turn these patterns into maintained SmartTests. Use `/testchimp evolve` when you want to expand coverage as your app grows.