Skip to content

Browser E2E (Playwright): login/dashboard, CRUD round-trip, export download — wired into CI - #135

Merged
buzz-code merged 8 commits into
mainfrom
claude/docs-test-coverage-projects-6ex6b1
Aug 13, 2026
Merged

buzz-code merged 8 commits into
mainfrom
claude/docs-test-coverage-projects-6ex6b1

Conversation

@buzz-code

@buzz-code buzz-code commented Aug 13, 2026 •

Copy link
Copy Markdown
Owner

Summary

Real-browser E2E, driving the actual client against a live server + real MySQL, not mocks/jsdom. Three flows, deliberately narrow — chosen for what only a real browser + server + database can catch, not for breadth (the jsdom-based entities.test.js covers every resource page cheaply already, see nra-client#40):

  • login-dashboard.spec.ts — register → login through the real UI form → dashboard loads.
  • crud-roundtrip.spec.ts — create/edit/delete a klass_type through the real UI, checking after each step that the change round-tripped through the actual server and database.
  • export-download.spec.ts — downloads a real Excel export and checks it's non-empty (the server actually builds .xlsx bytes; nothing to assert against with a mocked dataProvider).

Shared setup (registerTestUser, loginThroughUi, and markUserAsPaid — export is gated on isPaid, which nothing in the app can set for a fresh registration, so this reaches directly into the DB for setup, same reasoning as registerTestUser reaching around the UI via the API) lives in tests/helpers.ts.

Wired into CI as a new e2e-tests job: a MySQL service container, db/data.sql + yarn typeorm:run, server and client started and polled for readiness, then the suite. Playwright launches its own managed browser there (npx playwright install --with-deps chromium) rather than the sandbox's pre-installed one, which playwright.config.ts now only uses when that path actually exists.

Bugs found and fixed getting this green on a real CI runner (not just locally)

Local success didn't transfer directly — each of these was a real CI-only failure:

  • bash -e (GitHub Actions' default shell mode) aborted the server-readiness retry loop on its first, expected-to-fail connection attempt: code=$(curl ...) is a plain assignment, not exempt from -e the way curl ... && break is. Reproduced exactly locally before fixing (bash -e against an unreachable port: exit 7, zero loop output).
  • /opt/pw-browsers/chromium (the pre-installed Chromium referenced in the original pilot) only exists in the dev sandbox this was first built in — GitHub Actions runners don't have it. playwright.config.ts now checks for the path rather than assuming it.

Verification

Ran the full sequence locally against a from-scratch database (dropped and recreated, not reused) before pushing: mysql_user/mysql_database created, db/data.sql imported, yarn typeorm:run, server + client started and polled exactly as the CI steps do, then all 3 specs — green. CI itself is now green too (client-tests and e2e-tests both passing on the head commit).


Generated by Claude Code

First real-browser test in the repo — drives the actual client against
a live server + real MySQL, not mocks/jsdom. Scoped as a pilot for one
flow to prove the pattern before deciding whether to extend it further
or wire it into CI (see multi-repo-codespace's TEST_COVERAGE_TODO.md,
Phase 6).

- Registers a fresh user via POST /auth/register (same pattern as the
  server/test/*-crud.e2e-spec.ts suites) rather than relying on seeded
  DB fixtures, whose password hashes aren't real bcrypt hashes of any
  known password.
- Logs in through the actual UI form, confirms the dashboard loads
  (sidebar renders, no leftover login form, no error page).
- Uses the pre-installed Chromium at $PLAYWRIGHT_BROWSERS_PATH via
  launchOptions.executablePath, avoiding a browser download.
- README documents the run-without-docker stack setup needed to run
  it, including a real gotcha hit while building this: NODE_ENV=development
  gates the CORS localhost allowlist in bootstrap.util.ts, so a server
  started without it fails every browser request as a silent CORS
  error (curl/supertest don't hit this — no Origin header).

Not wired into CI — deliberately a manual pilot for now.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017YepEQ4PfrEhS2L8imLLRe
Copilot AI lite review requested due to automatic review settings August 13, 2026 05:28

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Add Playwright browser E2E pilot for real client→server→MySQL flow, covering login → dashboard, run manually (not CI).

Changes:

  • Add Playwright config + e2e package scaffolding (e2e/package.json, e2e/yarn.lock, e2e/.gitignore)
  • Add pilot spec login-dashboard.spec.ts (API register user, then UI login, then assert admin shell renders)
  • Add e2e/README.md with local stack run steps

Reviewed changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
e2e/tests/login-dashboard.spec.ts New Playwright spec for register → login → dashboard checks
e2e/playwright.config.ts Playwright config with baseURL, custom Chromium executablePath
e2e/README.md Runbook for bringing up live stack and running e2e
e2e/package.json Minimal e2e package scripts + devDependency
e2e/yarn.lock Lockfile for e2e workspace deps
e2e/.gitignore Ignore Playwright outputs + node_modules

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread e2e/tests/login-dashboard.spec.ts Outdated
Comment on lines +10 to +20
const apiContext = await request.newContext();
const username = `pw_e2e_user_${Date.now()}`;
const password = 'TestPass_123';
const res = await apiContext.post(`${API_BASE_URL}/auth/register`, {
data: { username, password, name: 'Playwright E2E' },
});
if (!res.ok()) {
throw new Error(`Failed to register test user: ${res.status()} ${await res.text()}`);
}
await apiContext.dispose();
return { username, password };
Comment thread e2e/README.md Outdated
Comment on lines +11 to +15
This environment ships a pre-installed Chromium at `$PLAYWRIGHT_BROWSERS_PATH`
(`/opt/pw-browsers`), so there's no browser download step. `playwright.config.ts`
points `launchOptions.executablePath` at it directly rather than relying on
`@playwright/test`'s own revision-matched download, which avoids a network
fetch that may not even be reachable in a sandboxed environment.
claude added 3 commits August 13, 2026 08:39
Picks up nra-client@a760e90: resource discovery now expands SubMenu
groups (real entities were never being found before, only ~8 static
utility pages), and Create/Edit crash detection now checks for the
per-route error fallback (<h1 role="alert">) instead of relying on
sidebar menuitems alone, since react-admin's per-route ErrorBoundary
doesn't take the sidebar down with it.

Verified against this app: injected a call to a never-imported
function into student.jsx's Create/Edit Inputs — the suite passed
before this fix (false negative) and failed after it.
Picks up nra-client@07c823a. This app's client/src/entities.test.js
already calls createResourceTests(App) directly; the generic fallback
in client/shared/utils/testing/entities.test.js was running the exact
same List/Create/Edit sweep over the same App a second time for zero
extra coverage. Harmless while resource discovery silently found
nothing (fixed in the prior commit), but a real cost now that it
works — this is very likely why CI's "Run client tests" job just
ran two full ~50min sweeps of the same 46 resources back to back
instead of one. Also widens the per-phase timeout further, since CI
hardware measured noticeably slower than local dev for this workload.
Picks up nra-client@98c5164. The previous fix (dedup + wider timeouts)
got CI's client-tests job through the full List/Create/Edit sweep, but
it then failed on the lightweight Common shared routes tests that run
right after — consistent with resource/DOM accumulation across ~140
renders in one process rather than any single resource being slow.
Two mitigations: createResourceTests.js now hard-resets document.body
between iterations (MUI portals can mount outside what RTL's cleanup()
tracks), and this job now runs yarn test with a larger V8 heap.

Copy link
Copy Markdown
Owner Author

CI status update

client-tests has failed twice on this branch, both times because of the client/shared bump that fixes createResourceTests's Create/Edit crash detection (see nra-client#40) — the crash-detection fix itself is solid and verified, but making resource discovery actually work surfaced a real performance problem: entities.test.js now renders the full admin app ~140 times in one process (every real resource's List/Create/Edit page, not the ~8 static pages it used to find).

Two rounds so far:

  1. Eliminated a redundant duplicate run — this app's src/entities.test.js and the shared fallback entities.test.js were both running the identical sweep. Real win (confirmed: went from "2 failed suites" to "1 skipped, 1 failed"), but CI still failed — now on the lightweight Common shared routes tests that run right after the sweep, not the sweep itself. That pattern (fast tests failing only after ~140 heavy renders precede them) looks like DOM/resource accumulation across the run rather than any one resource being slow.
  2. Just pushed: hard-reset document.body between iterations (MUI portals can mount outside what RTL's cleanup() tracks) + bumped the CI job's Node heap limit (--max-old-space-size=4096).

I don't have high confidence (2) fully resolves it — my local sandbox couldn't cleanly confirm it one way or the other before pushing (this sandbox's been under heavy load this whole session, so local timing isn't a reliable signal right now), and each CI cycle here costs over an hour. If CI is still red after this, I'll stop guess-and-check iterating and instead scope back what entities.test.js covers per run (e.g. sampling instead of exhaustive Create/Edit coverage) rather than keep burning CI cycles blindly — that's a real coverage/cost tradeoff, not a bug fix, so I'll flag it here for a decision rather than pick it myself.


Generated by Claude Code


Generated by Claude Code

Picks up nra-client@a8c10e8, which stops remounting the whole app per
page (remount cost was climbing monotonically across iterations, which
is what made this job degrade) and drops testing-library *ByRole
queries that cost ~666ms each under jsdom. Also fixes two false-pass
bugs found on the way: content was being sampled before React's
startTransition swapped in the new page, and the getOne mock's fixed
id made create-less resources look like real crashes.

Locally: 171s for the full file, all green; jest's own estimate for the
previous version was 1339s.

Copy link
Copy Markdown
Owner Author

CI runtime: ~29 min → ~3 min

My previous two rounds treated this as a timeout/memory problem and were wrong. Profiling instead of guessing found four causes:

Speed

  1. Every page remounted the entire <App/> — all providers, theme, i18n, router, ~46 <Resource>s — to test one route. And remount cost grew: successive mounts timed at 1.2s → 2.6s → 2.9s → 3.5s → 4.7s, even with cleanup() between them. Extrapolated over ~140 iterations that is exactly the ~12.5s/page CI was seeing. Each phase now mounts once and navigates; navigation is flat.
  2. *ByRole queries are pathological under jsdom — they filter matches by accessibility, running getComputedStyle per candidate. Measured 666 ms per queryAllByRole('menuitem') vs 0.7 ms for the equivalent querySelectorAll (~950×), and findAllByRole re-runs that scan on every poll.

Correctness — two bugs that were silently passing broken pages

  1. React Router navigates inside startTransition, so React keeps the previous page mounted until the new one is ready. The test was sampling the old page: observed directly with the URL at /student/1 while #main-content still rendered the previous resource's edit page (its back-link still pointed at /lesson_schedule). Related: "the DOM stopped changing" is not "the page rendered" — an Edit page waiting on getOne is perfectly stable. A deliberately broken Edit form passed while the identical broken Create form failed. Settling now waits for the content to actually swap, then stabilise, polling in separate act() calls so React can flush the transition.
  2. The getOne mock always returned id: 1. A resource with no create page makes /<resource>/create fall through to the edit route with id "create", and useEditController throws on the mismatch — a mock-induced crash indistinguishable from a real one. It produced 4 false positives (/report_group/create, /student_by_year/create, /grade_effect_by_user/create, /abs_count_effect_by_user/create). getOne/update now echo the requested id.

Verification — bug injected (call to a never-imported function in student.jsx's Inputs): create and edit phases each fail naming exactly /student/create and /student/1, no false positives elsewhere. Reverted: all 10 tests pass in 171s (jest's own estimate for the previous version was 1339s). Failures now name the offending route and list every bad page instead of stopping at the first.

Note that (3) and (4) mean the earlier "passing" runs were weaker than they looked — some pages were being checked before they rendered, or while still showing the previous page.


Generated by Claude Code


Generated by Claude Code

claude added 3 commits August 13, 2026 20:28
Fixes both outstanding review comments:
- README referenced $PLAYWRIGHT_BROWSERS_PATH but the config actually read
  PLAYWRIGHT_CHROMIUM_PATH; config now points at /opt/pw-browsers/chromium
  (a stable symlink, so it survives Chromium version bumps) and the README
  matches.
- registerTestUser()'s apiContext.dispose() wasn't called on the throw path;
  wrapped in try/finally.

Adds two flows chosen for what only a real browser + server + database can
catch, verified end-to-end against a from-scratch MySQL database (not
assumed — actually torn down and rebuilt to prove the CI steps work):

- crud-roundtrip.spec.ts: create/edit/delete a klass_type through the real
  UI, checking after each step that the change round-tripped through the
  server and database, not just that the request didn't throw. klass_type
  as a non-admin user needs only two plain fields, no reference picker.
- export-download.spec.ts: downloads a real Excel export and checks it's
  non-empty. Exporting is gated on isPaid, which nothing in the app can set
  for a fresh registration, so tests/helpers.ts's markUserAsPaid() reaches
  directly into the DB for setup — same reasoning as registerTestUser()
  already reaching around the UI via the API.

Shared setup (registerTestUser, markUserAsPaid, loginThroughUi) extracted
into tests/helpers.ts; login-dashboard.spec.ts now uses it too.

Wires all three into CI as a new e2e-tests job: a MySQL service container,
migrations from db/data.sql + yarn typeorm:run, server and client started
and polled for readiness, then the suite. Ran this exact sequence locally
against a torn-down-and-recreated database before committing.
GitHub Actions runs run: steps under bash -e. code=$(curl ...) is a plain
assignment, not exempt from -e the way 'curl ... && break' is — so on the
very first, expected-to-fail connection attempt (server not up yet) curl's
exit 7 aborted the whole step before the retry loop ran a single iteration.
Reproduced exactly locally (bash -e against an unreachable port: SCRIPT_EXIT=7,
zero loop output) and confirmed '|| true' fixes it (loop runs to completion,
SCRIPT_EXIT=0) before pushing.
/opt/pw-browsers/chromium is specific to the dev sandbox this pilot was
first built in — GitHub Actions runners don't have it, so CI failed with
'Failed to launch chromium because executable doesn't exist'. My earlier
local proof run didn't catch this because it also ran inside that same
sandbox.

playwright.config.ts now only pins executablePath when that path actually
exists on disk; otherwise it's left unset so Playwright uses its own
managed browser. CI installs one via 'npx playwright install --with-deps
chromium' in the Install E2E deps step.

Verified locally: pointing PLAYWRIGHT_CHROMIUM_PATH at a path that doesn't
exist reproduces CI's exact failure mode (Playwright's own 'please run
playwright install' error) — confirming the fallback needs that install
step, not just an unset executablePath, and that this is the same failure
CI hit.
@buzz-code buzz-code changed the title Add browser E2E pilot (Playwright): login → dashboard Browser E2E (Playwright): login/dashboard, CRUD round-trip, export download — wired into CI Aug 13, 2026
@buzz-code
buzz-code merged commit c24ec8f into main Aug 13, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants