test: stop the suite from reading — and calling — the developer's own gateway

[xizhuomengcontin]

Orca-Code-Review — push 1

Severity	Count
P0	0
P1	0
P2	0
P3	0

✅ no blocking findings

record, replay, attach and compare all resolve their upstream through upstreamPlan() and
readConfig() with no environment argument. In a test, that reads the real
~/.config/orca/config.json.

On a machine where anyone has run orca setup, that file names a gateway. Any dialect a test did
not override with --upstream-* resolves to it, and the test sends a live request to a third
party's host.

How it showed up

Two tests were red on my machine and green in CI:

cli/test/compare.test.ts > forks every model from the SAME parent run, not from the previous fork
cli/test/e2e.test.ts    > forks from a checkpoint and continues live on a different model

Run alone, in 1.6 seconds:

fake-agent: upstream 401: {"type":"error","error":{"message":"unknown provider error"}}
AssertionError: expected 2 to be greater than 2
 ❯ packages/cli/test/e2e.test.ts:754:32

The test passes --upstream-anthropic <fixture>. It does not pass --upstream-openai, so
resolveUpstream falls through to config.gateway.url for openai and openai-responses —
in my case http://155.103.159.98:8317, a real host.

What saved the key is upstreamPlan's unanimity rule, and it is worth naming because it was
the only thing standing there: origins were not all the gateway, so no headers were attached and
the request came back 401. Had the test overridden nothing, every origin would have been the
gateway, goingToGateway would have been true, and the developer's API key would have gone out on
a request generated by npx vitest.

It is not a hang, and not a platform thing

I had previously recorded these two as an unexplained hang that survived raising the timeout to
120s. That was wrong, and the mechanism is ordinary: under a full suite they reach the 30s timeout
because everything is slow at once; run alone they fail immediately with the 401 above. Both pass
with an isolated config home, on Windows and anywhere else.

The repo already knew

compare.test.ts had met this and worked around it inside a single describe:

compareCommand falls back to readConfig(), which reads the real ~/.config/orca. On a
machine where anyone has run orca setup that config has models in it, so this test never
reached the branch it names […] Nothing to do with the platform; CI passes only because CI has
no config.

That is the complete diagnosis, written down, and the fix it got covered one block. The same
fallback was meanwhile calling a real gateway from two other test files.

The change

vitest.setup.ts gives every worker an empty XDG_CONFIG_HOME, so no future test has to remember.
A real empty directory rather than a missing path, so that a test which writes a config stays out
of the developer's own.

• the local workaround in compare.test.ts retires into it, its history kept in the comment
• config.test.ts gains the guard that makes a silent failure of the setup file loud:
  readConfig() with no argument must resolve to {}

Tests that legitimately exercise config all pass an explicit env object already, so none of them
are affected.

Measured

Windows, same machine and build: 10 failures before, 8 after — the two gone are exactly these,
and nothing was skipped to get there. prettier --check, tsc --build --force,
scripts/conformance.mjs (63 events, 0 failures) and scripts/fidelity.mjs --check
(0 regressions) are clean.

CI cannot demonstrate this fix — it has no config, which is the point.

🤖 Generated with Claude Code

[orcacode-review]

🐳 OrcaCode Review

✅ No findings — nothing to flag in this PR. Great work!

_{OrcaCode Review — Route Smarter. Ship Safer. Spend Less.
Engine-reported: 1021 calls · 118.7M tokens · 99% cached}

❤️ Share · Install OrcaCode Review

Free on GitHub — the review runs on your own OrcaRouter key. If it helped, a shout-out goes a long way.

Share: X · Reddit · LinkedIn
Follow: X · Discord · LinkedIn · OrcaRouter