OnRamp E2E tests (Playwright)

The E2E suites live under playwright/, one folder per domain (core, projects, playbooks, insights, workflows, integrations-adobe, integrations-docusign, library-automations, email-settings). They're authored and maintained by Claude Code subagents, not hand-written — you drive the agents and they emit TypeScript specs that follow the house conventions automatically.

Every test owns its transactional data lifecycle: it creates what it needs through the UI, uses it, and deletes it — so any suite runs in any order, in parallel, against any environment. What it runs against is not a bare login but a golden seed (devtools/db/, bootstrapped with bun run db.seed — see the README): the seeded login identities, the full lookup/reference tables, and a small set of per-tenant fixtures. The tenet still holds — tests own their transactional rows; the seed only provides the floor.

Pre-seeded fixtures belong in the golden seed, not in a test. If a spec needs data it genuinely cannot create through the UI — e.g. workflows/hubspot.spec reads a cached HubSpot or_integration schema (vendor 0) — that fixture goes into the golden seed via the golden-seed-update skill, which routes it to a migration (if prod needs it too), the seed generator (dev/test-only fixture), or nothing (if it's actually transactional → create it in the test). Never hand-add it to seed_data.sql or fake it per-test. Background: the golden-seed RFC.

Getting started (first run)

Three one-time setup steps the suites don't do for you:

1. Deps: bun install && bun run deps.dev.
2. Playwright browser — bun install does not install it: bun run test:e2e:install (= playwright install chromium). Skipping this is the "Executable doesn't exist … run playwright install" error on your first test:e2e:*.
3. Golden DB: bun run db.seed (Docker required). The suites assume this seed — they log in as the seeded admin (admin@example.com). If you point a suite at your own dev DB instead, either seed it the same way or override the creds to a user that exists there (E2E_USER_EMAIL=… E2E_USER_PASSWORD=…). An auth/login failure at the very start of a run almost always means that login user isn't in the DB you pointed at — not a test bug.

Then bring the stack up and run a suite:

bun run dev --no-hmr         # stack up: Vite (https://local.onramp.us:3000) + Flask (:5000)
bun run test:e2e:core        # in a second shell

Why --no-hmr: HMR adds nothing when a browser fleet (not you) drives the pages, and under parallel E2E load Vite's HMR websocket drops and its client answers every drop with a full location.reload() of the unbundled module graph — headless Chromium balloons by gigabytes per worker until the OS starts killing things. A stack started plain (bun run dev) works for a quick serial run, but use --no-hmr for anything parallel.

Run the suites

The stack must be up first — HTTPS on local.onramp.us:3000 (Vite), Flask on :5000. Bring it up with bun run dev --no-hmr (see Getting started; the dev-verify skill covers stack health, certs, and recovery).

# Zero-config against the local stack — defaults to the seeded admin + local URL:
bun run test:e2e:projects          # any folder name — see the `test:e2e:*` scripts in package.json

bun run test:e2e:all               # every zero-config folder, sequentially

# Watch one spec fail, serially:
bunx playwright test --config=playwright/projects/playwright.config.ts archive.spec.ts --workers=1

# Override creds / URL only for a non-local target (dev/stage):
E2E_USER_EMAIL=… E2E_USER_PASSWORD=… E2E_BASE_URL=https://… bun run test:e2e:projects

• Local is zero-config. Creds default to the seeded admin admin@example.com / OnRampLocal!23 (vendor 0); base URL defaults to https://local.onramp.us:3000 (the local dev stack). Just run bun run test:e2e:<suite> — no env prefix.
• Override via env for non-local targets (dev/stage): E2E_USER_EMAIL / E2E_USER_PASSWORD (real creds from CI/Actions secrets — never committed) and E2E_BASE_URL. CI sets E2E_BASE_URL to its preview URL and relies on the default creds.
• Other seeded identity: customer@example.com is the vendor-1 owner. Most suites assume the admin — see playwright-testing for role preconditions.
• E2E_WORKERS=N caps per-suite worker count. Set it when several suites run at once against the one shared stack so N × suites doesn't oversubscribe the box (≈cores − 2 total Chromiums is the ceiling). On a 14-core machine: WORKERS=2 per folder alone, WORKERS=1 when running all folders concurrently.
• On failure, read playwright/<folder>/test-results/<spec>/error-context.md — a full a11y snapshot at the break — before guessing selectors.
• The playbooks suite enables the playbook-versioning flag automatically (its global-setup.ts), so the versioned UX is what runs.
• The task-prepopulation suite is deliberately outside test:e2e:all: its fixtures call internal (API-key) endpoints, and the golden seed carries no vendor_api row to default E2E_API_KEY from. Run it on its own — bun run test:e2e:task-prepopulation — with the credentials its README lists.

Running in CI

.github/workflows/e2e.yml runs the suites post-merge, not pre-merge. They gate promotion, not review:

• Push to main (and hotfix/**) runs the full matrix — one parallel job per suite, so wall clock is the slowest suite. This starts alongside the dev deploy and is the only scheduled run.
• Pull requests run no suites. The suites weren't catching enough defects to justify the per-PR (let alone per-commit) cost. The e2e suites check still reports on a PR — it is a required status check on the main ruleset, and an absent required check wedges every PR — but it is a no-op that passes immediately. Once that ruleset entry is removed, the pull_request trigger comes out of the workflow.
• Want the signal on a branch anyway? Dispatch E2E (Playwright) 🎭 on it from the Actions UI (or gh workflow run e2e.yml --ref <branch>). That runs the full matrix against your branch.

Promotion reads that result. 🧪 Promote to Staging and 🚀 Promote to Production call .github/actions/main-suite-gate with suite: e2e, which looks up the newest conclusive push-to-main E2E run in the promotion target's history and refuses to promote a red main. (The same action gates the Python unit suite, as a separate step with its own override.) It never triggers a suite and never waits on one — a run still in flight on the target commit falls back to the last conclusive ancestor — so gating costs seconds and promotion wall clock is unchanged. A run cancelled by a newer push carries no verdict and is passed over, not read as a failure. If E2E itself is broken (harness/runner outage rather than a real suite failure), re-dispatch the promote workflow with skip_e2e_gate checked.

A single domain failure re-runs as one job ("Re-run failed jobs"), not the whole matrix — and re-running it to green is what clears the promotion gate.

Two CI-only quirks worth knowing when a test passes locally but not in CI:

• CI serves a built bundle (PREVIEW_MODE=1 bun run dev), not the Vite dev server. Behaviour that depends on dev-only HMR/source paths won't reproduce.
• The app process starts with CI unset. GitHub sets CI=true, which the backend maps to qa_mode and hides the org login form — so the app runs under env -u CI while Playwright itself keeps CI=true for its worker count. Retries are 0 by policy (a flaky test is quarantined, not retried — see playwright-testing).

Skills you'll use

Skill	For
playwright-testing	The conventions (the lifecycle tenet, locator policy, page-object layout) and the three authoring subagents below. Auto-loads when you touch anything under playwright/.
dev-verify	Bring up and drive the local stack; cert/colima recovery; agent-browser mechanics.
onramp-db	Inspect the local dev Postgres (uuid → numeric id, columns, seed checks). Read-only conventions.
workflow-verify	Validate CRM automation / workflow-editor changes locally with no live OAuth (HubSpot/Salesforce field mapping, dealstage).
crm-workflow-e2e	Run the CRM trigger specs against a deployed PR preview — the only target that renders the flag-on trigger panel. Preflights that the preview is serving that PR's commit.

Author or fix a test

Three subagents, invoked by name from a Claude Code session:

You have…	Use	What it does
A page/flow with no plan	playwright-test-planner	Explores the live app, writes a plan to playwright/<feature>/specs/<area>.md.
A plan or a clear scenario	playwright-test-generator	Drives the flow in a real browser, emits tests/*.spec.ts + page objects, runs the suite.
A red spec	playwright-test-healer	Runs it, reads the trace/snapshot, fixes the spec or page object.

All three load the playwright-testing skill first, so the lifecycle tenet and locator policy come for free — you don't restate them. Give the agent just: which feature folder, the route/scenario, and confirm the creds env is set.

Scale with Claude workflows

When the same treatment needs to hit many specs at once — auditing every spec against a new convention, healing a whole suite, a cross-folder migration — a single conversation can't coordinate it. Reach for a Claude workflow: a script that fans out subagents in the background and returns one structured result. Trigger it by including the word workflow in your prompt.

The pattern that fits E2E is a pipeline: audit, then heal, partitioned by folder.

1. Audit — one read-only subagent per spec classifies it against the target convention. Returns structured findings.
2. Heal — one subagent per feature folder fixes all of that folder's specs + page objects, then verifies with bun run test:e2e:<folder>.

Collision rules (non-negotiable for parallel runs):

• One subagent owns one feature folder. Page objects are shared within a folder, so two agents in the same folder corrupt each other. Partition by folder, never finer.
• playwright/shared/ is off-limits to fan-out agents (auth, helpers, base config). Only a single serialized agent touches it, and only for a genuine shared-infra root cause.
• Verify your own folder only (test:e2e:<folder>), never test:e2e:all, during a run.
• Throttle with E2E_WORKERS so concurrent folder suites don't oversubscribe the one shared stack.

Model routing. A workflow's subagents inherit the session model unless the script routes a stage to another. Keep orchestration/audit on the strong model, and route implementation/heal stages to a cheaper one — see the claude-workflows skill for the convention and the one-line agent(..., { model }) knob. Limits: ≤16 concurrent agents, 1000 per run.

Folders & layout

playwright/
├── shared/       # imported by every folder — auth, random, capture-id, base config (not a test folder)
├── core/         # login/auth + global search
├── projects/     # projects CRUD, members, data fields, tasks, archive, search
├── playbooks/    # playbooks, modules, roles, versioning modes (globalSetup enables playbook-versioning)
├── insights/     # Project Schedule Timeline (Gantt)
└── workflows/    # Ramps / workflow authoring (HubSpot trigger)

Per folder: specs in tests/, element queries in pages/*.page.ts (locators only), UI-driven create/delete helpers in fixtures/ (they return captured ids). The thin playwright.config.ts calls createBaseConfig({ name }) from playwright/shared/config.base.ts; cross-suite infra (auth.ts, random.ts, capture-id.ts) is imported from shared/, never copied or imported across folders. A new domain is a new folder — copy playwright/projects/.

Local gotcha — the Flask reloader

The dev API runs with --reload, which watches *.zip. Playwright writes trace zips into test-results/ mid-run; without an exclude that restarts the API and flakes every test. make dev.api passes --exclude-patterns "*/test-results/*/*:*/playwright-report/*/*". Watchdog's * is per-segment (not fnmatch), so a naive */test-results/* silently no-ops — keep the two-segment tail if you edit it.