How to Test Plan Limits and Feature Flags
Short answer
UI hiding a premium button is not security. Entitlement E2E must probe protected APIs return 403 for free-tier tokens, metered limits block the 101st call when seeded at 99/100, and grandfathered features persist after downgrade per policy—using plan fixture users from /testchimp init seed routes.
Part of Testing Guides by business flow.
Who this is for
Tiered SaaS with feature matrices, usage caps, LaunchDarkly-style flags tied to billing, or legacy grandfathering.
Why testing entitlements matters
- Security — free users calling paid APIs causes cost and data exposure.
- Revenue — limits not enforced → unlimited usage on starter plan.
- Trust — paid users blocked by wrong flag rollout.
Complexity map
| Scenario | Edge case | Why tests break | Approach |
|---|---|---|---|
| UI hidden | API open | curl bypass | Probe 403 free tier |
| Meter at boundary | 99/100 used | 101st succeeds | Seed usage 99; Act twice |
| Plan upgrade | Immediate unlock | Stale JWT claims | Refresh token; probe 200 |
| Downgrade | Feature revoke | Still 200 | Probe 403 after period end |
| Grandfather | Legacy flag | New users blocked | Seed grandfather user |
| Flag % rollout | Non-deterministic | Flaky | Force flag on in test env |
| Org vs user | Team on pro | User free | Probe org entitlement |
| API key auth | Not session | Untested path | Probe with API key fixture |
| Webhook feature | External sync | Delay | Poll entitlement probe |
| Trial entitlements | Full pro during trial | Post-trial lock | Link trial guide |
| Admin override | Support grant | Audit | Probe temporary grant expiry |
| Entitlement cache | 5 min TTL | Flaky upgrade test | Bypass cache in test env |
API 403 for free tier
const { token } = await request.post('/api/test/seed-user-plan', {
data: { runId, plan: 'free' },
}).then(r => r.json());
const res = await request.post('/api/v1/exports/advanced', {
headers: { Authorization: `Bearer ${token}` },
data: { format: 'csv' },
});
expect(res.status()).toBe(403);
const body = await res.json();
expect(body.code).toMatch(/ENTITLEMENT|FORBIDDEN/i);
UI optional: expect(page.getByRole('button', { name: 'Export' })).toHaveCount(0).
Metered limit boundary
await request.post('/api/test/seed-usage', {
data: { runId, meter: 'api_calls', count: 99, limit: 100 },
});
const ok = await request.post('/api/v1/analyze', {
headers: { Authorization: `Bearer ${token}` },
data: { payload: 'test' },
});
expect(ok.status()).toBe(200);
const blocked = await request.post('/api/v1/analyze', {
headers: { Authorization: `Bearer ${token}` },
data: { payload: 'test2' },
});
expect(blocked.status()).toBe(429);
See usage metering for period boundaries.
Upgrade unlock
await request.post('/api/test/upgrade-plan', { data: { runId, plan: 'pro' } });
await request.post('/api/test/refresh-session', { data: { runId } });
const res = await request.post('/api/v1/exports/advanced', {
headers: { Authorization: `Bearer ${await getToken(runId)}` },
});
expect(res.status()).toBe(200);
Feature flags vs billing entitlements
Document which source wins in prod tests:
- Billing-only — disable flag; probe plan tier
- Flag-only — seed flag off for paid user; probe 403
- Combined — both must pass; test each negative
Anti-patterns
| Anti-pattern | Why it fails | Better approach |
|---|---|---|
| UI visibility only | API bypass | Probe endpoint |
| No boundary test | Off-by-one in prod | 99/100 scenario |
| Stale token after upgrade | Flaky pass/fail | refresh-session route |
| Prod LaunchDarkly | Non-deterministic | Test env overrides |
Example scenario
Situation: Free-tier user calls premium export API directly.
Expected outcome: 403 with entitlement error; no export job queued.
Why UI-only automation breaks: Export button hidden but API returns 200 and job runs.
- Arrange: Seed free plan user token for runId.
- Act: POST /api/v1/exports/advanced without UI.
- Assert: Probe 403; probe export job count zero.
TestChimp workflow: Track entitlement_key × plan_tier in TrueCoverage for API-only premium features.
Same Arrange/Act/Assert pattern as expired-coupon checkout.
Connect scenarios to your QA workflow
Capture business rules in markdown test plans and enforce them with seed routes and probe Assert. Link SmartTests with // @Scenario: for requirement traceability. Use /testchimp test on PRs; /testchimp explore on SmartTest paths for non-functional gaps (ExploreChimp).
Related scenarios
- Seat licensing — seat-gated features
- Trial to paid — trial entitlements
- Admin RBAC — admin vs plan
- RBAC permissions — role matrices
External references
Frequently asked questions
Should entitlement tests check UI and API?
Always both—hidden button is not security. Use API Arrange for token, probe protected endpoint, optionally confirm UI does not expose action.
How do I test metered limits at the boundary?
Seed usage at limit-1 via test route, perform two actions, probe first succeeds and second returns 429 or entitlement error.
Feature flag or plan tier—which wins?
Test per prod precedence document. Include scenarios where flag disables feature for paid user and where free user with flag on still blocked by billing if that applies.
JWT stale after upgrade—how to fix in tests?
Call test refresh-session route or re-login after plan change before probing premium API.
Grandfathered features after downgrade?
Seed user with grandfather flag, downgrade plan, probe feature still 200 if policy allows—or 403 if not.
Org-level vs user-level entitlements?
Seed org on pro with user role member; probe features that inherit org plan vs personal add-ons separately.
How do we know every entitlement_key is tested per plan?
TrueCoverage on entitlement_key × plan_tier. Run /testchimp evolve when feature matrix expands—tag SmartTests with // @Scenario:.
Apply these patterns in your repo
Run `/testchimp init` to connect TestChimp to your repo, then `/testchimp test` on PRs to turn these patterns into maintained SmartTests. Use `/testchimp evolve` when you want to expand coverage as your app grows.