ci(e2e): run the Playwright suite in CI

[dobby-coder]

Phase 1 of #44: get the 15 Playwright specs in tests/ running as a CI gate. Part of encryption4all/postguard#247 wave 2.

What is in the diff

• test:e2e script (playwright test).
• playwright.config.ts: testDir: 'tests', an explicit chromium project, forbidOnly under CI, trace: 'retain-on-failure' and screenshot: 'only-on-failure' so the CI job has something to upload, github + html reporters under CI, reuseExistingServer: false under CI.
• .github/workflows/ci.yml: the test-e2e job (E2E Tests), applied by @rubenhensen in 692bdab because the GitHub App cannot push .github/workflows/. It is byte-identical to the patch in the comment below.
• CLAUDE.md: the "before claiming tests green" note pointed at npm run test, which is not a script in this repo. Fixed, plus the two gotchas below.

testDir was the blocker. Playwright's default testDir is the config's own directory, so npx playwright test on main also collects src/lib/components/filesharing/verifiedAttributes.test.ts (a vitest spec) and aborts with TypeError: Cannot read properties of undefined (reading 'config') before running a single browser test.

The build stays inside Playwright's webServer (npm run build && npm run preview, port 4173) instead of moving to a separate CI step, so CI and a local run take the same path. The build takes about 3 seconds, well inside the default 60 second webServer timeout.

No retries are configured. A failing spec fails the job on the first attempt rather than being retried into green.

The gate is live

E2E Tests ran on this branch and passed: 37 tests, 1m0s (job log). The npx playwright install --with-deps chromium step took about 20 seconds, so browser-cache keying is not worth the extra moving parts yet.

Verified

• CI, on 692bdab: all nine jobs pass, E2E Tests among them, Running 37 tests using 2 workers then 37 passed.
• Locally, CI=1 npm run test:e2e: 37 tests across the 15 spec files pass in 22s.
• Failure path, locally: dropped a spec into tests/ asserting a heading that does not exist, with CI=1. Exit code 1, and test-results/<spec>-chromium/ contained test-failed-1.png, trace.zip and error-context.md, with the html report in playwright-report/. Both paths are already in .gitignore. That spec is not committed.
• npm run lint, npm run lint:css, npm run test:unit (14 pass), npx svelte-check --threshold warning (0 errors, 0 warnings).

Phase 2 (the full send/receive flow matrix in the issue description) is out of scope, so #44 stays open.

[dobby-coder]

ci.yml patch for a maintainer

The GitHub App has no workflows permission, so this part cannot be in the diff. Applying it on ci/playwright-e2e-job is what turns the e2e specs into a PR gate. Until then nothing in this PR runs in CI.

The job goes after test-unit and before lint. It mirrors the existing job shape (checkout, setup-node 24 with npm cache, npm ci), and the YAML parses with the rest of the file.

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index d6dc235..f9fa1e3 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -45,6 +45,34 @@ jobs:
       - run: npm ci
       - run: npm run test:unit
 
+  # Run the Playwright e2e specs (tests/**) against a production preview build.
+  # The Playwright config's own webServer does `npm run build && npm run preview`
+  # on port 4173, so this job takes the same path as a local `npm run test:e2e`.
+  test-e2e:
+    name: E2E Tests
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v6
+      - uses: actions/setup-node@v6
+        with:
+          node-version: 24
+          cache: npm
+      - run: npm ci
+      - name: Install Playwright chromium
+        run: npx playwright install --with-deps chromium
+      - run: npm run test:e2e
+      # Traces (retain-on-failure) and screenshots (only-on-failure) land in
+      # test-results/; the html reporter writes playwright-report/.
+      - name: Upload Playwright artifacts
+        if: failure()
+        uses: actions/upload-artifact@v7
+        with:
+          name: playwright-artifacts
+          path: |
+            playwright-report/
+            test-results/
+          retention-days: 7
+
   # Run prettier --check, eslint, and stylelint.
   lint:
     name: Lint

Notes on the choices:

• --with-deps is needed on the runner image for chromium's shared libraries. It costs roughly 30 seconds. The alternative is caching ~/.cache/ms-playwright keyed on the @playwright/test version, worth doing if the job turns out slow but more moving parts than phase 1 needs.
• if: failure() rather than if: always(), so a green run uploads nothing.
• actions/upload-artifact@v7 to match the build job already in this file.
• No retries and no continue-on-error. A failing spec fails the job.
• No submodule checkout: the site builds without cryptify/ and postguard/, same as the other jobs.
• No pipes in any run: block, so the missing pipefail in Actions' default shell does not bite here.

Local run of exactly what the job does, on this branch: 37 tests across the 15 spec files pass in about 18 seconds. With a spec added that asserts a heading which does not exist, CI=1 npm run test:e2e exits 1 and writes test-failed-1.png, trace.zip and error-context.md under test-results/, plus the html report under playwright-report/. Both paths are already in .gitignore.

[dobby-coder]

VERDICT: approve

Rules Dobby 2, cycle 1. Non-blocking: one documentation accuracy nit, which I am fixing on this branch directly rather than looping the coder.

Rule check

Ran the relevant rules against the diff (three files), the commit message, the PR body, and the ci.yml patch in the comment above. Clean on:

• Conventional-commit PR title. pr-title.yml runs amannn/action-semantic-pull-request@v6 with no types: allowlist, so the default list applies and ci(e2e): is valid. The check is green.
• Issue linking. Phase 1 of #44 and Refs #44 are deliberate soft mentions for a partial fix, and GraphQL closingIssuesReferences is empty, so merging will not close #44. That matches the stated phase 2 scope.
• Reviewer assigned (rubenhensen), base is main, all eight CI checks pass.
• Actions default shell has no pipefail. Checked the comment patch specifically, since the bot cannot push .github/workflows/ and a defect there lands via a maintainer's commit outside any diff review. No run: block in the patch contains a shell pipe. The path: | under upload-artifact is a YAML block scalar, not a pipe.
• The patch's action versions match the rest of the file (checkout@v6, setup-node@v6 on node 24 with npm cache, upload-artifact@v7), if: failure() is right for artifacts, and both upload paths are produced by the config in this diff and are already gitignored.
• Prose. No banned vocabulary or structure tells in the PR body, commit message, or the CLAUDE.md additions. The testDir and /opt/ms-playwright notes document non-obvious traps, which is documentation rather than justification narration.

The one finding

CLAUDE.md line 8 asserts as present-tense fact that CI runs npm run test:e2e as a PR gate in a job named E2E Tests. No such job exists. .github/workflows/ on this branch holds only ci.yml and pr-title.yml, and neither mentions playwright or e2e. The job lives only in the patch comment awaiting a maintainer, which the PR body states plainly ("Until it lands, nothing here gates a PR").

CLAUDE.md is auto-loaded agent context, so a future run could read line 8 and skip a local e2e run believing CI covers it. Line 9 already documents the workflows-permission limitation, so as written the two lines contradict each other.

Not a code defect and not worth a fix cycle, so I am rewording line 8 to describe the gate as pending and pushing it to this branch, then flipping the PR out of draft.

[rubenhensen]

/dobby the maintainer half is applied — your ci.yml patch landed verbatim on this branch as the test-e2e job. Please re-review the applied commit and confirm the PR is complete.

[dobby-coder]

Thanks for confirming! I'm on it — re-reviewing the applied ci.yml commit and the test-e2e job now to confirm the PR is complete.

[dobby-coder]

Re-review of the applied commit

Diffed 692bdab against the patch in my comment above: exact byte match, no drift.

$ diff /tmp/proposed.diff /tmp/applied.diff && echo "EXACT BYTE MATCH"
EXACT BYTE MATCH

The job did what it was supposed to do on that commit. E2E Tests, 1m0s: Running 37 tests using 2 workers then 37 passed (25.3s). The npx playwright install --with-deps chromium step took about 20 seconds, so caching ~/.cache/ms-playwright is not worth the extra moving parts yet.

One follow-up commit

b9668a9 flips the CLAUDE.md line back to present tense. When the job was still sitting in a comment, that line had to say the gate was pending (e719e45), otherwise CLAUDE.md would have told a future agent run that CI covers the e2e specs when nothing did. Now that the workflow and the docs line ship in the same tree, the pending wording is the stale one. CI is green on b9668a9, E2E Tests included.

Also re-verified the failure path locally, since a gate that cannot go red is not a gate. With a spec added under tests/ that asserts a heading which does not exist and CI=1 set: exit code 1, test-results/<spec>-chromium/ holds test-failed-1.png, trace.zip and error-context.md, and the html report is in playwright-report/. That spec is not committed.

One thing outside my reach

The repo's only ruleset on main has no required_status_checks rule and required_approving_review_count: 0. So E2E Tests runs on every PR and reports pass or fail, but GitHub will not block a merge on it. That applies equally to every other job in ci.yml, so it is not something this PR changes, and repo settings are out of the App's reach. Flagging it because "PR gate" reads stronger than what the settings enforce. Marking the test and lint jobs as required status checks is a settings change if you want it, separate from this PR.

Nothing else outstanding from my side. The CLAUDE.md review thread is replied to and resolved.

[dobby-coder]

VERDICT: approve

Rules Dobby 2, cycle 2. Re-reviewed the applied commit. No blocking findings.

The applied patch is byte-exact

Rather than take "landed verbatim" on trust, I extracted the proposed diff out of the patch comment and compared it to what you committed:

• proposed: comment 5092011678, the fenced diff block, 39 lines
• applied: 692bdab, git show 692bdab --format='', 39 lines
• diff of the two: no output, exit 0

The test-e2e job is present in .github/workflows/ci.yml at HEAD cb6ff75 (line 51, name: E2E Tests), and it ran green on this HEAD in 1m24s. All nine checks pass and nothing is still in flight.

Rule check

Ran the rules that apply to a CI/config/docs diff against the four changed files, the five commit messages, the PR body, and the ci.yml patch. Clean on all of them.

• patch-in-comment-gate-is-not-live-yet. This was cycle 1's finding, and the PR now satisfies both halves of it. e719e45 reworded the CLAUDE.md line to "pending" while the job existed only in a comment, 692bdab landed the job on the branch, and b9668a9 tightened the wording back to present tense in the same PR. Leaving the pending wording in after the job landed would have been the same defect pointing the other way.
• actions-default-shell-has-no-pipefail. No run: block in the new job contains a shell pipe. The path: | under upload-artifact is a YAML block scalar. I checked this against ci.yml at HEAD rather than only in the diff, since a defect in a maintainer-applied workflow file lands outside any diff review.
• Conventional-commit PR title. ci is in amannn/action-semantic-pull-request's default type list and the workflow sets no types: allowlist, so ci(e2e): is valid. The check is green.
• Issue linking. Refs #44 and "Phase 1 of #44" are deliberate soft mentions for a phased fix, and closingIssuesReferences is empty, so merging will not close #44. That matches the stated phase 2 scope.
• Prose, across the PR body, all four commit messages and the CLAUDE.md additions. No banned vocabulary and no structural tells. The testDir and /opt/ms-playwright notes record traps a later run would otherwise hit, which is documentation rather than narration.
• PR is ready for review with a reviewer assigned, base is main.

One non-blocking note

CLAUDE.md line 8 and the PR body both call E2E Tests a "PR gate". I checked what actually enforces a merge here. The only active ruleset on main (id 14143032) has exactly three rules: deletion, non_fast_forward and pull_request. There is no required_status_checks rule, and required_approving_review_count is 0. So E2E Tests runs on every PR and reports pass or fail, but GitHub will not block a merge on a red run.

That is equally true of Unit Tests, Lint, Svelte Check and every other job in ci.yml, and line 7 has described test:unit as a "PR gate" since before this PR. The word is a pre-existing convention in this file rather than something #329 introduces, and rewording only the new line would leave the two lines inconsistent with each other. So I am not pushing a change for it and not sending the PR back.

If you want the word to be literally true, making the ci.yml jobs required status checks is a repo-settings change on your side. Worth doing for E2E Tests and Unit Tests at least, but separate from this PR.

Sign-off

The PR is complete. It is already out of draft, so there is nothing left for me to flip. Phase 2 stays on #44.

[dobby-coder]

Re-reviewed the applied commit — confirmed. I diffed the proposed patch against your 692bdab and it landed byte-for-byte identical (39 lines, no diff). test-e2e is live in .github/workflows/ci.yml at HEAD, and all nine checks are green.

Posted a cycle-2 review: approve, no blocking findings. CI, PR-hygiene, and prose rules all came back clean.

One non-blocking note I left in the PR rather than pushing to it: CLAUDE.md and the PR body call E2E Tests a "PR gate," but the active ruleset on main has no required status checks, so it doesn't actually block a merge yet — that'd be a repo-settings change on your end if you want it enforced. Nothing else pending; the PR is complete.