How to Test Webhooks and Async Event Processing
Short answer
Event-driven systems fail on ordering assumptions, missing idempotency, and asserting before handlers finish—not on JSON parsing alone. Test generic webhooks with signed fixture POSTs, duplicate event.id replay, out-of-order sequences, expect.poll probes up to 20–30s, and dead-letter visibility—not fixed sleeps or UI toasts as source of truth.
Part of Testing Guides by integrations.
Who this is for
Teams with inbound webhooks (partners, Stripe, GitHub, Shopify) or internal async queues (SQS, RabbitMQ, Bull) where UI or API returns before processing completes.
Why testing async processing matters
- Double processing — duplicate webhook charges or ships twice
- Poison messages — one bad payload blocks queue
- Out-of-order — state machine stuck in impossible transition
- Silent drops — 200 returned but worker never ran
- Flaky E2E — success UI before handler completes
Complexity map
| Scenario | Edge case | Why tests break | Approach |
|---|---|---|---|
| Out of order | Event B before A | Wrong DB state | Tolerate partial; probe eventual |
| Idempotency | Duplicate event id | Double write | Unique constraint; probe count |
| Slow handler | >5s processing | Assert too early | expect.poll probe |
| Signature | Invalid HMAC | 401 untested | Negative POST spec |
| Poison message | Schema valid, business invalid | Queue stuck | DLQ probe + alert |
| Retry storm | Handler 500 | Duplicate side effects | Idempotent retry |
| At-least-once | Three deliveries | Triple charge | Idempotency key |
| Partial batch | 2 of 5 events fail | Inconsistent | Probe per-item status |
| Clock skew | timestamp ordering | Sort bugs | Monotonic event version |
| Webhook + UI | User sees success early | Race | Poll aggregate status |
Handler contract to test
POST /webhooks/partner
→ verify signature
→ persist raw event (optional)
→ enqueue or process
→ return 2xx quickly
→ worker updates authoritative state
E2E asserts step 5 via probe—not HTTP 200 alone.
Signed fixture POST
import crypto from 'crypto';
function sign(body: string, secret: string) {
const sig = crypto.createHmac('sha256', secret).update(body).digest('hex');
return `sha256=${sig}`;
}
const body = JSON.stringify({ id: 'evt_' + runId, type: 'order.shipped', data: { orderId: '123' } });
await request.post('/api/webhooks/partner', {
headers: { 'X-Signature': sign(body, process.env.WEBHOOK_SECRET!) },
data: body,
});
Mirror production verification logic in test helper.
Polling probe (mandatory)
await expect.poll(async () => {
const res = await request.get(`/api/test/probe-order/123`);
return (await res.json()).status;
}, { timeout: 25_000, intervals: [500, 1000, 2000] }).toBe('shipped');
See Playwright polling assertions. Never page.waitForTimeout(5000).
Idempotency spec
const payload = { id: 'evt_dup_' + runId, type: 'payment.captured', ... };
await postSignedWebhook(payload);
await postSignedWebhook(payload);
await expect.poll(() => probePaymentCount('123')).toBe(1);
Out-of-order sequence
await postSignedWebhook({ id: '2', type: 'shipment.delivered', orderId: '123' });
await postSignedWebhook({ id: '1', type: 'shipment.created', orderId: '123' });
await expect.poll(() => probeOrder('123')).toMatchObject({ status: 'delivered' });
Document whether handler queues until dependencies exist or uses saga compensation.
Failed handler / retry
await enableWebhookFailureOnce(runId);
await postSignedWebhook(fixture);
await expect.poll(() => probeHandlerAttempts('evt_' + runId)).toBeGreaterThan(1);
await expect.poll(() => probeOrderStatus('123')).toBe('processed');
Dead letter queue
When max retries exceeded:
- Probe DLQ row with
event.id - Assert alerting hook fired (mock in test)
- Replay from DLQ admin UI scenario (optional E2E)
Playwright UI + async backend
User clicks "Export" → job queued → email when done:
await page.getByRole('button', { name: /export/i }).click();
await expect(page.getByText(/processing/i)).toBeVisible();
await expect.poll(() => probeExportJob(runId)).toBe('completed');
Do not assert download until probe says complete (PDF guide).
Anti-patterns
| Anti-pattern | Why it fails | Better approach |
|---|---|---|
| Assert on 200 OK | Worker crashed async | Poll probe |
| Fixed sleep 10s | Still races | expect.poll |
| No duplicate test | Double charge prod | Replay event id |
| Assume event order | Stuck states | Out-of-order fixture |
| Skip signature negative | Forged webhooks | Invalid HMAC spec |
| Shared event ids in CI | Collision | runId in event id |
Example scenario
Situation: Partner webhook marks order shipped after warehouse API lag.
Expected outcome: Order status shipped exactly once even if webhook delivered twice.
Why UI-only automation breaks: Admin UI shows shipped from cache while probe still processing.
- Arrange: Seed order processing; signed webhook fixture ready.
- Act: POST order.shipped webhook (twice for idempotency spec).
- Assert: Probe status shipped; fulfillment count 1.
TestChimp workflow: Track event_type × handler_outcome when handler_outcome=failed retries untested in prod.
Same Arrange/Act/Assert pattern as expired-coupon checkout.
Connect scenarios to your QA workflow
Capture business rules in markdown test plans and enforce them with seed routes and probe Assert. Link SmartTests with // @Scenario: for requirement traceability. Use /testchimp test on PRs; /testchimp explore on SmartTest paths for non-functional gaps (ExploreChimp).
Related scenarios
- Stripe webhooks — Stripe-specific signing and CLI
- AI agent workflows — tool async completion
- WebSockets live updates — push vs poll
External references
- Playwright polling
- Stripe webhook best practices — idempotency patterns apply broadly
- AWS SQS at-least-once delivery
Frequently asked questions
How long should E2E wait for async webhook processing?
Poll probe endpoint with expect.poll and exponential backoff up to 20–30s. Fail with handler logs on timeout—never fixed sleep.
How do I test duplicate webhook delivery?
POST same event id twice with valid signature. Probe exactly one side effect—idempotency table or natural key unique constraint.
How do I test out-of-order events?
POST fixtures in reverse dependency order. Probe eventual consistent state or assert saga compensation—document expected behavior.
Should tests assert HTTP 200 from webhook endpoint?
Only as sanity check—authoritative assert is probe on processed state. Return 200 quickly pattern means worker may lag.
How do I test poison messages?
POST valid signature but business-invalid payload. Probe DLQ entry and that queue continues processing other messages.
How do I combine UI tests with webhook processing?
UI Act triggers action; poll probe until async job complete before download or success assert. UI 'processing' state optional intermediate assert.
New event types ship often—how do we track handler coverage?
Compare event_type × handler_outcome in TrueCoverage prod vs test-run. Run /testchimp evolve after deploy when new types lack signed fixture tests.
Apply these patterns in your repo
Run `/testchimp init` to connect TestChimp to your repo, then `/testchimp test` on PRs to turn these patterns into maintained SmartTests. Use `/testchimp evolve` when you want to expand coverage as your app grows.