Update to support vxsuite v4.0 and v4.1#17
Open
eventualbuddha wants to merge 25 commits into
Open
Conversation
Update vx-qa to run against vxsuite v4.0 (tag v4.0.7) and v4.1 (tag v4.1.0-alpha), dropping older versions. A QA job now specifies a `version` (matching vxsuite's SoftwareVersion values v4.0/v4.1); the git ref, patch file, and ballot data model are derived from it. - Add src/config/versions.ts as the single source of version -> ref / patch / ballot-model mapping. Replace vxsuite.ref with vxsuite.version (enum) in the config schema/types; thread it through clone, CLI (--vxsuite-version), serve (vxsuite_version payload param), and report. - Split the single vxsuite.patch into per-version patches (vxsuite-v4.0.patch, vxsuite-v4.1.patch), each verified to apply at its ref. Rework the audio mock to target getAudioCardName/pactl (the old server.ts/get_audio_info.ts block no longer exists), reduce the bootstrap hunk to the logname->HOME fix, and re-author the write-in onDraw hook against each version's marking.ts. - Normalize v4.1 election shapes on load so downstream code is version-agnostic: ballotStyles[].ballotPositions -> gridLayouts, and yes/no options[] -> yesOption/noOption. - CircleCI: parameterized per-version test-qa jobs with version-aware cache keys; qa-vxdesign accepts a vxsuite_version pipeline parameter. - Add a v4.1 test fixture converted from the existing v4.0 package using vxsuite's own convertV4p0ElectionToLatest; geometry verified to match the v4.0 original exactly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9ZJ972AKRchfzfePawpA7
pnpm@latest is now >= 11, which requires Node >= 22.13 (it imports node:sqlite). The CI image runs Node 20.16.0, so `npm install -g pnpm@latest` failed with ERR_UNKNOWN_BUILTIN_MODULE. Pin to pnpm 10.34.1 (the last line supporting Node 20) and record it via packageManager. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9ZJ972AKRchfzfePawpA7
Two fixes for the failed CI run: - test-unit failed on `pnpm lint`: the added packageManager field left package.json unformatted (lint-staged only formats *.ts/*.js). Run it through oxfmt. - test-qa-v4.0/v4.1 failed in vxsuite bootstrap: the pdi-scanner and ballot-interpreter Rust addons build with `cargo build --offline`, which needs every crate already cached. Building an arbitrary ref hits crates the CI image hasn't cached (e.g. `csv`). Run `cargo fetch` at the workspace root during bootstrap to populate the cache first. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9ZJ972AKRchfzfePawpA7
cargo fetch (online) succeeded but the subsequent `cargo build --offline` still failed to find `csv` — the offline build doesn't see the fetched cache (different cargo context). Instead of pre-fetching, strip `--offline` from the Rust addon build scripts (pdi-scanner, ballot-interpreter) after pnpm install so the build downloads any crates missing from the CI image's cache. Idempotent and skips files that don't use --offline (v4.1's ballot-interpreter uses napi). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9ZJ972AKRchfzfePawpA7
The QA run reached VxAdmin config but timed out waiting for the admin backend on port 3004. VxSuite computes each app's backend port as `FRONTEND_PORT + 1` (3001) — admin and scan both use 3001, not the per-app 3002/3004 the tool assumed. Simplify env-config to a single frontend (3000) / backend (3001) pair. Also bump the vxsuite cache key (v1 -> v2): the cached repo was built before the --offline fix and its presence made bootstrap skip, masking an incomplete scan/pdi-scanner build. A fresh key forces a clean rebuild so the fixes take effect. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9ZJ972AKRchfzfePawpA7
VxAdmin started (port fix worked) but writing the election package to the mock USB drive failed with ENOENT. Newer VxSuite moved device mock state from libs/<device>/dev-workspace to <repo>/.mock-state/<NODE_ENV>/ (see getMockStateRootDir). Update the USB data path, and rework State to clear and archive .mock-state (app backend dev-workspaces are unchanged). Also mkdir the target dir before writing to the mock USB, since Phase 2 clears it first. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9ZJ972AKRchfzfePawpA7
VxScan backend crashed on startup: `AudioCard.setVolume` -> pactl
set-sink-volume -> Connection refused. My audio mock covered the pactl
wrapper and getAudioCardName, but setAudioVolume calls `execFile('pactl',
...)` directly, bypassing the wrapper. Add an early return in
setAudioVolume for non-production. Now all three audio operations
(card-name detection, profile, volume) are mocked.
Changing the patch also changes its hash, which is part of the CI
vxsuite cache key, so this forces a clean rebuild that compiles the
corrected patch.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K9ZJ972AKRchfzfePawpA7
VxScan started/configured/opened polls, then failed inserting a ballot:
`Cannot POST /dock/pdiScannerInsertSheet` (404). The mock PDI scanner
dev-dock API was reworked to a sheet-queue model:
pdiScannerInsertSheet(s) -> pdiScannerInsertSheets({path}) (feeds a PDF's
sheets via a queue), pdiScannerGetSheetStatus -> pdiScannerGetStatus
(returns {sheetStatus, queue}), plus pdiScannerClearSheetQueue. Sheet
status values also changed (noSheetEnabled, etc.). Update the scanner
controller accordingly; clear any active queue before each insert.
Also fix the (non-fatal) mock printer path: reports moved from
libs/fujitsu-thermal-printer/dev-workspace/prints to
<repo>/.mock-state/<NODE_ENV>/prints.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K9ZJ972AKRchfzfePawpA7
Phase 6 (scanning) now works end-to-end. Phase 7 (tally) failed on `page.goto localhost:3000 ERR_CONNECTION_REFUSED` right after the admin app restarted: the frontend port can accept TCP before Vite is serving, and stopApp only waited for the backend port to free (not the frontend), so a restart races. Retry navigateToApp on connection errors, and wait for both frontend and backend ports to be free when stopping an app. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9ZJ972AKRchfzfePawpA7
Two fixes: - v4.1 build failed with `Cannot find module @rolldown/binding-linux-x64-gnu`. VxSuite v4.1 pins pnpm@9.15.9 (Vite 8/rolldown), but the tool built it under pnpm 10.34.1, which mis-resolves the optional native binding. bootstrapRepo now installs and uses the pnpm version from VxSuite's `packageManager` field (v4.0 -> 8.15.5, v4.1 -> 9.15.9). - v4.0 reached Phase 7 (tally) but `page.goto localhost:3000` was refused after the admin app restarted: a previous app's run-dev process tree lingers and holds the port, so the new Vite binds elsewhere. Clear any processes on the app ports before each startApp. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9ZJ972AKRchfzfePawpA7
- Port management relied on lsof, which the CI image lacks, so isPortFree always reported "free" and the between-app port waits were no-ops — letting a new app start while the previous run-dev tree still held :3000. Make isPortFree a TCP connection attempt and wait for the ports to actually free before starting an app. - Bump the vxsuite cache key (v2 -> v3): v4.1's cached node_modules was installed by pnpm 10 (missing the rolldown binding), and pnpm 9.15.9 treated it as up-to-date and skipped install. A fresh key forces a clean install under the pinned pnpm. - Run the QA test jobs with DEBUG=1 so app stdout / retries / port waits are visible for diagnosing the Phase 7 restart. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9ZJ972AKRchfzfePawpA7
DEBUG output confirmed Phase 7's failure: "Ports 3000/3001 still busy before starting admin" — the scan run-dev tree wasn't dying, so it kept holding :3000 and the restarted admin couldn't serve there. tree-kill alone (SIGTERM, needs ps/pgrep) left Vite/esbuild/backend children alive. Spawn each app detached (own process group) and SIGKILL the whole group on stop, with tree-kill as a fallback. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9ZJ972AKRchfzfePawpA7
v4.1's build failed because rolldown's native binding (@rolldown/binding-linux-x64-gnu) declares engines node ^20.19.0 || >=22.12.0, so pnpm skipped it as engine-incompatible under the 4.2.0 image's Node 20.16.0. VxSuite v4.1 pins Node 20.19.0 and its own CI uses cimg-debian12:4.5.0; match that image (and bump .node-version to 20.19.0). v4.0 is unaffected (Node 20.x, ABI-stable). Bust the vxsuite cache (v3 -> v4) so both versions rebuild cleanly on the new runtime. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9ZJ972AKRchfzfePawpA7
v4.1 now builds and runs, but VxAdmin config failed: "No election packages found on the inserted USB drive." v4.1's mock file USB drive nests data under a disk name (.mock-state/<NODE_ENV>/usb-drive/sdb/ mock-usb-data) and VxAdmin scans that mountpoint, but the tool wrote to v4.0's location (usb-drive/mock-usb-data). Make the mock USB data dir version-specific via VERSION_SPECS. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9ZJ972AKRchfzfePawpA7
The converted fixture reused v4.0-rendered ballots whose embedded hash no
longer matched the converted election.json, so v4.1 VxScan rejected every
sheet ("Ballot Scan Failed"). Regenerate a genuine v4.1 package by
running VxSuite's own renderer (libs/hmpb
renderAllBallotPdfsAndCreateElectionDefinition) at v4.1.0-alpha against
the v4.0 election (NhBallot template, softwareVersion v4.1), producing
election.json + ballots.jsonl with a matching ballotHash. No cloud audio
or translation (English-only; reuses the existing app strings /
systemSettings). Verified it loads through the QA tool (ballotPositions
-> gridLayouts, proof ballots render).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K9ZJ972AKRchfzfePawpA7
v4.1 rewrote the VxAdmin tally CVR screen (CvrsScreen replaces the old CastVoteRecordsTab). The button that opens the CVR import modal is now labeled "Load" instead of "Load CVRs", so the tally workflow timed out clicking "Load CVRs". Try the v4.0 label first, fall back to v4.1's. The import modal itself (role=alertdialog, per-file "Load" buttons, "New CVR Loaded" success) is unchanged. With this, v4.1 scans ballots correctly (real rendered fixture) and proceeds into CVR import; earlier phases already pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9ZJ972AKRchfzfePawpA7
The previous commit switched the CVR-import opener to
getByRole('button', {name: 'Load CVRs'}), which doesn't match v4.0's
element (the original getByText did), so v4.0 regressed: it loaded the
first CVR then timed out on the "any more?" iteration. Restore
getByText('Load CVRs') as the v4.0 path and only fall back to v4.1's
"Load" button when that text is absent.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K9ZJ972AKRchfzfePawpA7
v4.1's CVR opener is an icon button whose icon pollutes its accessible
name, so getByRole({name:'Load',exact:true}) missed it. Target its
explicit `.LoadButton` class instead. Keep getByText('Load CVRs') for
v4.0. Also dump page state (buttons/links/headings) if opening the
import modal fails, so any remaining v4.1 tally-UI difference is
diagnosable from the CI log.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K9ZJ972AKRchfzfePawpA7
v4.1 now runs the full flow (scan 24/12 like v4.0, manual tallies valid, reports generated) but QA validation failed: "Unexpected votes in CSV for <contest>/ballots-cast". v4.1's tally report CSV adds a per-contest 'ballots-cast' metric row (csv_tally_report.ts) alongside overvotes/undervotes; it's not a candidate vote. Exclude it from the vote comparison. No-op for v4.0 (whose CSV has no ballots-cast). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9ZJ972AKRchfzfePawpA7
Primary elections split ballot styles and contests by party. Two bugs prevented the manual-tally phase from working: 1. VxAdmin labels each ballot-style option with the party appended (e.g. "Bedford - Democratic"). The workflow searched for the bare precinct name, so no option matched, the style was never selected, and the Voting Method dropdown stayed disabled -> 30s click timeout. 2. getContestsForBallotStyle filtered contests by district only. In a primary, a party's ballot styles share identical districts, so the DEM and REP styles both resolved to all 25 contests instead of their 12/13 party contests. Now candidate contests are additionally matched by partyId when the ballot style has one (yes/no measures unaffected; general elections unchanged since they carry no partyId). Adds partyId/groupId to BallotStyle and partyId to CandidateContest. CI: decouple the test-qa config filename from the vxsuite version via a new optional `config` param, and add a test-qa-v4.0-primary job reusing the v4.0 vxsuite build/patch/cache with the primary fixture. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9ZJ972AKRchfzfePawpA7
The primary fixture enables the 'BlankBallot' precinct-scan adjudication reason, so VxScan returns blank ballots (and our unmarked-write-in pattern, which fills no bubbles and is therefore effectively blank) to the voter for review instead of counting them. The QA tool hardcoded these as expected-accepted, so the run failed even though VxScan behaved correctly. Derive expectedAccepted from systemSettings.precinctScanAdjudicationReasons: - blank (official): accepted only when 'BlankBallot' is not a reason - unmarked-write-in: accepted only when 'BlankBallot' is not a reason This keeps the general election (no 'BlankBallot') accepting both, and makes the primary correctly expect them rejected. Tally validation is unaffected (it already ignores rejected/unmarked-write-in sheets). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9ZJ972AKRchfzfePawpA7
Previously the tool hard-required disallowCastingOvervotes=true. Now it
handles both settings and no longer requires it:
- disallowCastingOvervotes=true: overvoted ballots can only be returned;
we scan one overvote per official ballot and verify it is rejected.
- disallowCastingOvervotes=false: the voter may cast an overvote. We scan
two overvotes per official ballot — casting one ("Cast Ballot") and
returning the other — so tallies/reports reflect a cast overvote.
To let the scanner (not just our expectations) see the setting without
maintaining a separate election package, add an optional
election.systemSettingsOverrides config field. Overrides are written into
the package's systemSettings.json before VxAdmin loads it (and re-exports
it to VxScan). The dev/test packages are unsigned, so this is safe.
Recorded scan votes now reflect what the scanner counts: a contest marked
beyond its seats is an overvote and contributes no selections, so those
contests are dropped from the recorded votes. This keeps tally validation
correct when an overvoted ballot is cast (accepted) rather than returned.
CI: add test-qa-v4.0-overvote, reusing the v4.0 general package with
disallowCastingOvervotes overridden to false.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K9ZJ972AKRchfzfePawpA7
Extend the primary-election and castable-overvote coverage to v4.1. - v4.1 overvote: reuse the rendered v4.1 general package with the systemSettingsOverrides config (disallowCastingOvervotes=false); no new fixture needed. - v4.1 primary: add a real v4.1-rendered primary package. Generated from the v4.0 primary election (Bedford, NH) by re-rendering it with vxsuite v4.1.0-alpha's hmpb renderer (NhBallot template) so the ballots carry v4.1 ballotPositions and a ballotHash that matches election.json. The v4.0 export was adapted for the v4.1 schema (county -> jurisdiction, added pollingPlaces and the clerk signature the NH template requires). Adds test-qa-v4.1-primary and test-qa-v4.1-overvote jobs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9ZJ972AKRchfzfePawpA7
The mock poll-worker card insertion before closing polls can race with the app settling after the last scanned ballot, leaving VxScan on the voter "Insert Your Ballot" screen so the close-polls prompt never appears (intermittent CI failure, e.g. the v4.0-overvote job). Retry the card insertion up to three times, re-inserting the card if the prompt doesn't show, before failing with debug state. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K9ZJ972AKRchfzfePawpA7
Passing `parseInt` directly as a Commander option coercion is a bug:
Commander calls it as `parseInt(value, previous)`, so the previous option
value becomes the radix (e.g. `--port 9100` -> `parseInt('9100', 9000)` ->
NaN, which crashed `serve` with ERR_SOCKET_BAD_PORT). Use a helper that
parses base 10 and rejects non-integers. Applies to --port, --limit-ballots,
and --limit-manual-tallies.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K9ZJ972AKRchfzfePawpA7
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Co-authored by Claude Code 🤖
Pairs with votingworks/vxsuite#8865.
I recommend verifying the reports more than looking at the code. See the latest job on this branch, and a test report for a v4.1 run.