Skip to content

fix(capacity): valid calibration run IDs and early build-artifact preflight - #248

Merged
ttbombadil merged 1 commit into
mainfrom
fix/calibration-run-id-and-build-preflight
Oct 12, 2026
Merged

ttbombadil merged 1 commit into
mainfrom
fix/calibration-run-id-and-build-preflight

Conversation

@ttbombadil

Copy link
Copy Markdown
Collaborator

Summary

Fixes two real failures of the capacity-calibration workflow found on a fresh Ubuntu 26.04 / x86_64 install. No change to capacity semantics, limits, policy, or production defaults; TEST_RUN_ID_PATTERN and the 64-character limit are untouched.

1. Invalid CAPACITY_TEST_RUN_ID

scripts/calibrate-capacity.ts generated capacity_calibration_${Date.now()}_${randomUUID()}, which is 71 characters (21 + 13 + 1 + 36). server/config.ts validates CAPACITY_TEST_RUN_ID against TEST_RUN_ID_PATTERN (^[A-Za-z0-9_-]{1,64}$), so the owned backend refused to start.

  • New exported createCalibrationRunId() produces cap_<timestamp>_<uuid> (54 characters) and re-checks the result against the central TEST_RUN_ID_PATTERN, failing loudly in the script rather than inside the backend.

2. Missing dist/public surfaced only deep inside backend startup

The calibrator spawns server/index.ts with NODE_ENV=test, which calls serveStatic(); without npm run build the backend died during startup with a serveStatic error.

  • New exported assertCalibrationBuildArtifacts() is wired into runCalibration() as the first step, before the host/Docker probes and before any Docker-control, active, or startup measurement. It checks dist/public/index.html, resolved relative to the script (the same location serveStatic() resolves).
  • Missing build: the run aborts early with Capacity calibration requires a built application (<path> not found); run npm run build.; no probe, backend, or report directory is created.
  • The script does not run npm run build itself. serveStatic() and the NODE_ENV=test gateway semantics are unchanged.
  • --dry-run starts no backend and therefore skips the check.
  • The check is injectable (verifyBuildArtifacts) so unit tests stay hermetic: the CI unit job runs before npm run build.

Docs

  • docs/INSTALL_SERVER.md (calibration section) and docs/CAPACITY_MODEL.md (host-capacity calibration workflow) now document npm ci → npm run build → npm run build:sandbox → npm run capacity:calibrate -- --expected-users <N>, note that build:sandbox equals the existing docker build -f Dockerfile.sandbox … line, and that the unosim-server image is not needed.
  • README.md: one-line note.

How the failures were reproduced

Reported on a fresh Ubuntu 26.04 / x86_64 host:

npm ci
docker build -f Dockerfile.sandbox -t unosim-sandbox:latest .
docker build -t unosim-server:latest .
npm run capacity:calibrate -- --expected-users 200
  1. Invalid CAPACITY_TEST_RUN_ID: with the build present, the owned backend rejected the generated ID because it exceeds TEST_RUN_ID_PATTERN. Verified locally by computing the old format: 71 characters vs. the 64-character limit (new format: 54).
  2. Missing dist/public before npm run build: on that host no npm run build had been run, so dist/public did not exist and the backend died in serveStatic() during startup. The installation doc did not make the build a clear precondition.

I did not have access to the Ubuntu host; I verified the fix locally in a detached git worktree without dist/: npm run capacity:calibrate -- --expected-users 200 now exits immediately (exit code 2) with the actionable message above and creates no capacity-test-results/.

Tests

tests/server/calibrate-capacity-cli.test.ts (18 tests, was 9 before this change):

  • run ID matches TEST_RUN_ID_PATTERN, length ≤ INPUT_LIMITS.webSocket.maxTestRunIdChars, longest realistic timestamp stays within the limit, runs are distinguished by timestamp and by UUID (deterministic injected clock/UUID, no random-sampling test), an invalid ID is rejected;
  • build preflight: present index accepted; missing index gives the npm run build message; runCalibration() aborts before any host/Docker probe, control measurement, backend start, scenario, or report; dry run does not call the check;
  • docs contract: the four commands appear in order in INSTALL_SERVER.md and CAPACITY_MODEL.md.

Existing calibration tests inject verifyBuildArtifacts so they do not depend on a build.

Local verification: npm run check, ESLint with --max-warnings=0, npm run check:docs, and the full ./run-tests.sh pipeline (static analysis, knip, unit, toolchain integration, Docker, Playwright E2E 28 passed, production build, security audit, Sonar gate: 0 violations) were green; the pre-push hook (unit + fresh Sonar scan) passed.

Note: the commit message states the old ID length as 74 characters; the correct value is 71 (as above). Not amended, per repo policy.

🤖 Generated with Claude Code

…flight

Two defects found on a fresh Ubuntu 26.04 host:

- The calibration run ID (capacity_calibration_<ms>_<uuid>) was 74
  characters, exceeding TEST_RUN_ID_PATTERN (64), so the owned backend
  refused CAPACITY_TEST_RUN_ID and never started. Generate cap_<ms>_<uuid>
  (54 characters) via createCalibrationRunId(), which re-checks the central
  pattern.
- The owned backend serves dist/public via serveStatic(), so a missing
  build killed it deep inside startup. runCalibration() now verifies
  dist/public/index.html before any probe, Docker control, active or
  startup measurement and aborts with "run npm run build". It does not
  build automatically; dry runs start no backend and skip the check.

Document the host calibration sequence (npm ci, npm run build,
npm run build:sandbox, npm run capacity:calibrate). No capacity policy,
limits or production defaults change.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@ttbombadil
ttbombadil marked this pull request as ready for review October 12, 2026 12:49
@ttbombadil
ttbombadil merged commit 67e476c into main Oct 12, 2026
6 checks passed
@ttbombadil
ttbombadil deleted the fix/calibration-run-id-and-build-preflight branch October 12, 2026 15:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant