Skip to content

test: bind route test servers to the loopback address they request - #247

Merged
ttbombadil merged 3 commits into
mainfrom
fix/flaky-route-tests
Oct 12, 2026
Merged

ttbombadil merged 3 commits into
mainfrom
fix/flaky-route-tests

Conversation

@ttbombadil

Copy link
Copy Markdown
Collaborator

Stacked on #246 (Sonar fixes, needed to pass the pre-push gate) – merge #246 first; GitHub then retargets this PR to main.

Problem

Two tests failed sporadically only under the full parallel suite:

  • tutor.routes.test.ts › "rejects a browser revision …": 401 instead of 400
  • examples.routes.test.ts › "passes exact repository …": timeout after 30 s

Root cause (same for both)

The route tests started their server with app.listen(0) (wildcard ::) but sent requests to 127.0.0.1. On macOS:

  1. Ephemeral ports come sequentially from one counter shared by all processes (measured: +1 per allocation); the full suite wraps it with thousands of connections.
  2. A wildcard listen(0) can be handed a port that another process already holds on 127.0.0.1 (measured: after a counter wrap, listen(0) returned the held port; lsof shows *:P and 127.0.0.1:P both LISTEN). listen(0, "127.0.0.1") and outgoing connects do skip held ports.
  3. Connections to 127.0.0.1:P go to the more specific listener, i.e. the other worker's server.

Many tests keep servers on 127.0.0.1. If the foreign server enforces the gateway or is the dev server from dev-entrypoint.test.ts (which forwards the test key to KI:connect → 401 CREDENTIAL_INVALID, reproduced exactly), the tutor test gets 401. If it has no HTTP handler (WebSocket-only test servers), the examples request never gets an answer → timeout.

Deterministic reproduction of the old failure

A helper process binds every second port from the current counter position on 127.0.0.1 (answering 401, or without request handler):

Old code Fixed
tutor test, 401 holder 5/5 expected 401 to be 400 32/32 green
examples test, no-handler holder 5/5 timeout 5/5 green
all 7 changed files, both holders, 3 runs each 7/7 files red every run 7/7 green every run

Fix

listen(0, "127.0.0.1") in the 7 test files whose clients use 127.0.0.1 (the two affected ones plus compiler.routes, routes-core, server-status, test-reset.routes, protocol-version, which had the same latent bug). Assertions, timeouts and test content are unchanged; no retries. Tests that request localhost and access-control.test.ts (deliberate loopback variants) keep the wildcard server.

Test plan

  • 7 files isolated 5×: 75/75 green
  • 7 files 6× while the full unit suite ran in parallel: 75/75 green each; suite 3052/3052
  • ./run-tests.sh – all tests green (Docker 32/32, E2E 28/28)
  • Pre-push fresh Sonar scan: gate OK, 0 new violations

🤖 Generated with Claude Code

ttbombadil and others added 2 commits October 12, 2026 11:46
Seven test files listened on the wildcard address (app.listen(0) -> ::) but
sent requests to 127.0.0.1. On macOS an ephemeral wildcard listen can be
handed a port that another Vitest worker already holds on 127.0.0.1, and
connections to 127.0.0.1:<port> then reach that other server. Under the full
parallel suite this produced 401 instead of 400 in tutor.routes.test.ts (a
dev or gateway server answered) and a 30 s timeout in
examples.routes.test.ts (a server without HTTP handler answered).

Binding to 127.0.0.1 makes the kernel reject that port. Assertions and
timeouts are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ttbombadil
ttbombadil changed the base branch from fix/sonar-new-rules to main October 12, 2026 10:43
@ttbombadil
ttbombadil merged commit 22ad201 into main Oct 12, 2026
6 checks passed
@ttbombadil
ttbombadil deleted the fix/flaky-route-tests branch October 12, 2026 11:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant