ci: test Node 18 with a smoke test of the built artifact, not with a runner that refuses it - #239
Closed
Rome-1 wants to merge 1 commit into
Closed
ci: test Node 18 with a smoke test of the built artifact, not with a runner that refuses it#239Rome-1 wants to merge 1 commit into
Rome-1 wants to merge 1 commit into
Conversation
…runner that refuses it The cross-platform (ubuntu-latest, 18) leg failed #237 on verify-hook-liveness — expected null to be 'deny'. That probe IS rf-fuwy, so the question mattered: does the PROBE fail on Node 18, or only the test? Opposite answers. MEASURED, on a real Node 18.20.8: the SHIPPED dist, exported runConfiguredHook, 200 iterations v18.20.8 200/200 read "deny" v24.14.0 200/200 read "deny" the raw spawnSync call, isolated from vitest and TS, 700 iterations 0 failures on either version, byte-identical results the built CLI node ./dist/index.js --version -> 0.10.1 the failing test file, in isolation on 18 4/4 PASS So the probe does not fail on Node 18 and users are not affected. The cause is structural: vitest 4.1.0 engines.node = ^20.0.0 || ^22.0.0 || >=24.0.0 matrix node: ["18", "20", "22"] vitest 4 does not run on Node 18. That leg ran the full 2194-test suite on a runner that declares it will not run there, which is why 18 alone failed and why it failed intermittently — 89 of 90 files passed. WHAT THIS CHANGES, AND WHAT IT DOES NOT. engines stays >=18: the shipped code works there, measured. What was dishonest was a matrix leg claiming to test Node 18 while running a runner that refuses it, so 18 comes out of the vitest matrix and is replaced by node18-smoke — build on 20, RUN on 18, which is the user's situation since they install a prebuilt package rather than compiling one. The smoke asserts the rf-fuwy probe BOTH ways: a live gate yields its decision, an inert one yields none. Both directions mutation-checked — blanking the live fixture gives `FAIL: probe read null, expected "deny"`, and making the inert fixture work gives `FAIL: an inert gate was not reported inert`. My first mutation attempt was itself vacuous (the mutants exited 1 on a module-not-found, not on the assertion); these run in-tree so the import resolves and the failure is the real one. Gated on run_core, not the `run` that gated cross-platform: run_core is true on every PR, and a skipped job satisfies a required check. NOT REPRODUCED, stated rather than glossed: I could not make the probe miss once in 900 attempts at load 12-16, so I cannot name the exact CI mechanism. Consistent with all three spawn attempts exhausting on a 2-core runner under a full-suite fork storm. The residual's DIRECTION is the safe one — a miss reports a live gate as dead, never a dead gate as live. poe is chasing a separate finding about the fixture never reading stdin; that is a real question about the fixture and is beaded on its own.
Collaborator
Author
|
Superseded by #240. This branch could not be force-pushed after |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Unblocks #237's last failure. The shipped code is unchanged — this is a CI-matrix fix.
The question, and the answer
cross-platform (ubuntu-latest, 18)failed onverify-hook-liveness—expected null to be 'deny'. That probe is rf-fuwy, so it mattered whether the probe fails on Node 18 or only the test. Measured on a real Node 18.20.8:runConfiguredHook, 200 iterationsdenyspawnSynccall, no vitest, no TS, 350 eachnode ./dist/index.js --version0.10.1The probe does not fail on Node 18. The cause is structural:
vitest 4 does not run on Node 18. That leg ran the full 2194-test suite on a runner that declares it will not run there — which is why 18 alone failed, and why it failed intermittently (89 of 90 files passed).
What changes
enginesstays>=18— the shipped code works there, measured. What was dishonest is a matrix leg claiming to test Node 18 while running a runner that refuses it. So 18 leaves the vitest matrix and is replaced bynode18-smoke: build on 20, run on 18, which is the user's situation since they install a prebuilt package rather than compiling one.The smoke asserts the rf-fuwy probe both ways — a live gate yields its decision, an inert one yields none.
Mutation-checked, including my own first attempt
FAIL: probe read null, expected "deny"FAIL: an inert gate was not reported inertMy first mutation attempt was itself vacuous: the mutants exited 1 on a module-not-found rather than on the assertion. These run in-tree so the import resolves and the failure is the real one.
Gated on
run_corerather than therunthat gated cross-platform —run_coreis true on every PR, and a skipped job satisfies a required check.Not reproduced, stated rather than glossed
I could not make the probe miss once in 900 attempts at load 12–16, so I can't name the exact CI mechanism. It's consistent with all three spawn attempts exhausting on a 2-core runner under a full-suite fork storm. The residual's direction is the safe one: a miss reports a live gate as dead, never a dead gate as live.