Skip to content

ci: test Node 18 with a smoke test of the built artifact, not with a runner that refuses it - #239

Closed
Rome-1 wants to merge 1 commit into
mainfrom
ci/node18-smoke-not-vitest
Closed

ci: test Node 18 with a smoke test of the built artifact, not with a runner that refuses it#239
Rome-1 wants to merge 1 commit into
mainfrom
ci/node18-smoke-not-vitest

Conversation

@Rome-1

@Rome-1 Rome-1 commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

Unblocks #237's last failure. The shipped code is unchanged — this is a CI-matrix fix.

The question, and the answer

cross-platform (ubuntu-latest, 18) failed on verify-hook-livenessexpected null to be 'deny'. That probe is rf-fuwy, so it mattered whether the probe fails on Node 18 or only the test. Measured on a real Node 18.20.8:

Node 18.20.8 Node 24.14.0
shipped dist, exported runConfiguredHook, 200 iterations 200/200 read deny 200/200
raw spawnSync call, no vitest, no TS, 350 each 0 failures 0 failures
built CLI node ./dist/index.js --version 0.10.1
the failing test file, in isolation 4/4 pass

The probe does not fail on Node 18. The cause is structural:

vitest 4.1.0   engines.node = ^20.0.0 || ^22.0.0 || >=24.0.0
matrix         node: ["18", "20", "22"]

vitest 4 does not run on Node 18. That leg ran the full 2194-test suite on a runner that declares it will not run there — which is why 18 alone failed, and why it failed intermittently (89 of 90 files passed).

What changes

engines stays >=18 — the shipped code works there, measured. What was dishonest is a matrix leg claiming to test Node 18 while running a runner that refuses it. So 18 leaves the vitest matrix and is replaced by node18-smoke: build on 20, run on 18, which is the user's situation since they install a prebuilt package rather than compiling one.

The smoke asserts the rf-fuwy probe both ways — a live gate yields its decision, an inert one yields none.

Mutation-checked, including my own first attempt

  • blank the live fixture → FAIL: probe read null, expected "deny"
  • make the inert fixture work → FAIL: an inert gate was not reported inert

My first mutation attempt was itself vacuous: the mutants exited 1 on a module-not-found rather than on the assertion. These run in-tree so the import resolves and the failure is the real one.

Gated on run_core rather than the run that gated cross-platform — run_core is true on every PR, and a skipped job satisfies a required check.

Not reproduced, stated rather than glossed

I could not make the probe miss once in 900 attempts at load 12–16, so I can't name the exact CI mechanism. It's consistent with all three spawn attempts exhausting on a 2-core runner under a full-suite fork storm. The residual's direction is the safe one: a miss reports a live gate as dead, never a dead gate as live.

…runner that refuses it

The cross-platform (ubuntu-latest, 18) leg failed #237 on
verify-hook-liveness — expected null to be 'deny'. That probe IS rf-fuwy,
so the question mattered: does the PROBE fail on Node 18, or only the
test? Opposite answers.

MEASURED, on a real Node 18.20.8:

    the SHIPPED dist, exported runConfiguredHook, 200 iterations
        v18.20.8  200/200 read "deny"
        v24.14.0  200/200 read "deny"
    the raw spawnSync call, isolated from vitest and TS, 700 iterations
        0 failures on either version, byte-identical results
    the built CLI          node ./dist/index.js --version -> 0.10.1
    the failing test file, in isolation on 18   4/4 PASS

So the probe does not fail on Node 18 and users are not affected. The
cause is structural:

    vitest 4.1.0  engines.node = ^20.0.0 || ^22.0.0 || >=24.0.0
    matrix        node: ["18", "20", "22"]

vitest 4 does not run on Node 18. That leg ran the full 2194-test suite
on a runner that declares it will not run there, which is why 18 alone
failed and why it failed intermittently — 89 of 90 files passed.

WHAT THIS CHANGES, AND WHAT IT DOES NOT. engines stays >=18: the shipped
code works there, measured. What was dishonest was a matrix leg claiming
to test Node 18 while running a runner that refuses it, so 18 comes out
of the vitest matrix and is replaced by node18-smoke — build on 20, RUN
on 18, which is the user's situation since they install a prebuilt
package rather than compiling one.

The smoke asserts the rf-fuwy probe BOTH ways: a live gate yields its
decision, an inert one yields none. Both directions mutation-checked —
blanking the live fixture gives `FAIL: probe read null, expected "deny"`,
and making the inert fixture work gives `FAIL: an inert gate was not
reported inert`. My first mutation attempt was itself vacuous (the
mutants exited 1 on a module-not-found, not on the assertion); these run
in-tree so the import resolves and the failure is the real one.

Gated on run_core, not the `run` that gated cross-platform: run_core is
true on every PR, and a skipped job satisfies a required check.

NOT REPRODUCED, stated rather than glossed: I could not make the probe
miss once in 900 attempts at load 12-16, so I cannot name the exact CI
mechanism. Consistent with all three spawn attempts exhausting on a
2-core runner under a full-suite fork storm. The residual's DIRECTION is
the safe one — a miss reports a live gate as dead, never a dead gate as
live. poe is chasing a separate finding about the fixture never reading
stdin; that is a real question about the fixture and is beaded on its own.
@Rome-1

Rome-1 commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator Author

Superseded by #240. This branch could not be force-pushed after 5409a84 landed the matrix change directly on main, so it was reopened as a fresh rebased branch rather than rewritten. #240 carries the same two files — the node18-smoke job and the probe script — with the now-redundant matrix edit dropped. Verified there on a real v18.20.8 runtime, asserting both directions (live gate -> deny, inert gate -> no decision, status 127).

@Rome-1 Rome-1 closed this Sep 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant