make test refuses a missing pinned toolchain by name, and names a skip that is asked for (#599) - #660
Merged
Merged
Conversation
`make test` passed over the JS, Dart, Elixir and Java legs when dist/ was absent, and a red inside one of them rode a green run. The gate: - every make/<lang>.mk registers TOOLCHAIN_LEGS, a TOOLCHAIN_PIN_<lang> and its own toolchain-<lang> target, one $(call toolchain_probe) per pin; the Makefile lists no language, as with every other registry here - toolchain runs before the chain and refuses by name, printing the leg, the pin and the path the pin looked in - SCHEMA_SKIP_LEGS names a skip on purpose, and every named leg says so - SCHEMA_SKIP_LEGS naming something that is not a registered leg is refused before any pin is probed - toolchain-negative-control points every registered pin at a path that does not exist and requires each leg to refuse by its own name, then names every leg and requires the same gate green with each skip printed - the C# leg gets the DOTNET pin it never had, and every dotnet invocation in the tree reads it Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
#599) A skip that stops at the leg loop is not a skip: `make test` still built the skipped leg's conformance driver, still ran it, and still ran the C# half of every block gate in the Makefile, on the same toolchain that is not there. - harness run --skip <langs>: the Makefile fills it from SCHEMA_SKIP_LEGS, the harness prints each leg it passes over by name, and the reference leg is refused because every other leg compares against it - each leg registers its conformance prerequisites through unless_skipped, so a named skip does not build the driver either - the C# half of the block gates and the C#-only block controls print the skip and stand down, rather than dying on a dotnet the gate already named Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The building section says `make test` refuses a missing pinned toolchain by name and runs without a leg only when SCHEMA_SKIP_LEGS names the skip, with the reason it is a rule rather than a preference. The registry section says what a port with a pinned toolchain registers, beside every other list it already registers. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
errcheck: a write to the harness's own stdout is not a verdict this gate can improve on, which is what the matrix write below already says. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The repo's lint holds a range over a split to the sequence form, as readDrivers beside it already does. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…599) A leg registers TOOLCHAIN_PINS_<lang>, the whole list of pins its toolchain-<lang> target probes, and the control walks the list: each pin in turn at a path that does not exist, the leg's other pins at one that resolves, and the refusal must name that pin. Pointing one pin per leg left the JAVAC, ELIXIRC and MIX probes watching nothing; deleting all three kept the control green, which is a gate with no blade in the place the gate is about. The Makefile header, the registry comment and CONTRIBUTING said "every pin" already, so they are now true rather than aspirational. The other pins point at a path that resolves while one is absent because the bench this gate exists for has none of the toolchains installed: with the leg's own defaults left alone, an earlier probe answers first and the refusal names the wrong pin. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
skipSet and the "conformance SKIPS the <leg> leg" line had no test: the harness grew a --skip argument and absence_test.go grew an empty one beside it. Delete skipSet and every test in this package stayed green. skip_test.go drives run through absence_test.go's fake corpus with the skip filled in. The skipped leg's driver exits 1 and says so on stderr, which the matrix folds into its failure list, so each test reads the skip's EFFECT and not only its line: TestUnskippedLegRuns is the control that the leg would be red if reached, the named skip turns it green and takes its matrix column with it, a name no leg carries skips nothing, blanks and empty entries are not names, and --skip cpp is refused before any driver runs because every other leg compares against the reference. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Nothing under .github runs `make test`, and `make test` is the only thing that runs the toolchain gate, so the gate this branch adds had no watcher on the diff that changes it. The go-test job already runs three negative controls by hand for exactly that reason, and this one belongs beside them: it needs no toolchain of its own, since it points each registered pin at a path that does not exist and reads the refusal, and it takes about a second. Issue #660's sibling, #659, puts every negative control on pull-request CI from one manifest; when it lands this step folds into that manifest's base group and comes back out of here. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The legs were the gate's own prerequisites, so make stopped at the first one whose pin did not resolve: on this bench, with none of the five toolchains installed, `make toolchain` said cs and nothing else, and a fresh clone became a queue of install-one-run-again. The legs now go through sub-makes instead, each leg's refusal passed through exactly as it printed it, and the summary names them together with a SCHEMA_SKIP_LEGS line that carries the skips already in force plus the newly missing legs, ready to paste. The green line also stops claiming what it did not look at: with every leg named in SCHEMA_SKIP_LEGS it said "0 of 5 registered legs probed, every pinned toolchain resolves", and now says that none were probed because every registered leg is named. A leg that fails without printing the gate's own refusal has its stderr shown; a leg that refuses by name does not also carry the sub-make's error notice. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Comment reflow only, no rule change: the previous commit left a half line mid-sentence in the file header and opened the aggregate's comment with a tautology. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The building section said the refusal prints "the leg it would have skipped", singular, which was true of a gate that stopped at the first missing pin and is not true of one that probes every registered leg. It also did not say where the negative control runs, which is the thing a contributor wants to know before pushing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ew C# recipes read the DOTNET pin (#599) The five language makefiles took both sides: the toolchain gate's per-leg probes and the packet defaults, text and wide targets the ports added; the C# recipes those ports wrote with a bare dotnet now read the pin the gate holds. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This was referenced Sep 7, 2026
rowan-claude
added a commit
that referenced
this pull request
Sep 7, 2026
The rule (Glenn, 2026-09-07): "if we have any CI that runs per-commit it
needs to finish 1 minute, 2 minutes max. Anything longer than that needs to
be nightly or manually triggered as needed." It is a rule about a JOB, and a
matrix row is a job, so every row of this leg is cut to fit it rather than
the leg as a whole being cut to fit it.
MEASURED on this branch's own pull-request run (34102397965), before this
change: base 8:57, wire-fuzz 7:13, cs 2:46, message-form 2:36, conformance
2:31, block-fuzz 1:59. Five of the eleven rows were over the rule and a
sixth was one second under it.
WHAT MOVED. The plan now carries 34 pull-request groups instead of 11, cut
where the measurement cuts them and not where the family names do:
- base splits ten ways. The map gate is 17 controls at 4 s each on the
author's machine, so it runs as base-maps and base-maps-keys; the
conformance driver's two sabotage rebuilds are 45 s and 47 s of runner
time each, so they are base-conformance and base-conformance-absence;
the rest are the packet families, the list gate, the arm gate, the
cook, the keyed controls and what is left.
- wire-fuzz splits eleven ways, one control per job for ten of them.
Each rebuilds the compiler under a source overlay and then fuzzes the
sabotaged wire, 55 s to 75 s on the runner, so no two of them fit
together; the four pinned-vector oracles are cheap and share the
eleventh.
- cs splits four ways. The C# layout check is the instrument for three
block controls at 39 s, 31 s and 29 s.
- block-fuzz splits in two, at 48 s and 33 s over two SDK installs.
- conformance is gone as a group. Its one control had the bare
`conformance` target as its prerequisite, which builds and runs all
nine legs and needed all seven SDKs on one runner. The prerequisite is
now `conformance-rust`, the rust leg and the derived manifest the
control actually reads, and the control rides one SDK in rust-alloc.
The soak and the audit keep `conformance`, deliberately: a NUMBER
measured over a corpus whose matrix is red is a number about a defect,
and this control is not a number.
ONE CONTROL DOES NOT FIT AT ANY GROUP SIZE, so it does not run on the pull
request at all: tables-message-form-negative-control drives 49 sabotage
rows one submake each and takes 124 s where the rule is 120. It is the
`nightly` group, and certify.yml runs that tier on the schedule it already
carries. Its four blades stay on the pull request in `message-form`.
So a group now names its tier. `when` is `pull-request` or `nightly`,
`matrix` renders one tier, and the enumerator REFUSES a group that names
neither: a control cannot leave the pull request without landing on the
nightly, and the package's test reads both workflows to prove each runs the
plan rather than a typed list.
The 35 controls main gained since this branch last merged it are in the
plan too, toolchain-negative-control from #660 among them, and the three
family umbrellas that would re-run a split family's other half join the
exclusion list with the reason. 194 controls, all of them in a group or in
an explained exclusion.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rowan-claude
added a commit
that referenced
this pull request
Sep 7, 2026
… Makefile (#563) (#659) * ci: enumerate every negative control from the Makefile (#563) A negative control proves a gate is watching by breaking what the gate watches and requiring the gate to go red. Each one in this tree already refuses when its sabotage patches nothing, so a control whose pattern has drifted off the line it aims at says so. Until now the only reader of that refusal was `make test`, which runs after a merge. This is the half that makes "every control" mean what it says. tools/negativecontrols reads the Makefile and every file the Makefile includes, collects each explicit target whose name carries `negative-control`, and holds that set against make/negative-controls.json, the plan the pull-request leg's matrix comes from. The test fails on a difference in either direction: a control the makefiles define and the plan does not carry, and a control the plan names and no makefile defines. An exclusion needs a non-empty reason, so nothing leaves the leg silently. The reader follows the tree's own include lines rather than a glob typed here, and it reads the rule heads a makefile actually writes: several targets on one head, a head continued over a backslash, `.PHONY` lines, and `define` blocks and `:=` assignments that only look like rules. Both readings of this tree agree at 159 controls. The plan groups those 159 by what a runner has to install. Eight groups are toolchain families; two more, wire-fuzz and message-form, are base-toolchain families split out on measured cost. Closes #563 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * ci: every negative control runs on pull requests (#563) Twice on 2026-09-05 main's certification went red on a merge whose pull-request run was 20 of 20 green, and one of the two was a stale sabotage pattern: #562 rewrote the line the maps fit control aimed at, the sabotage patched nothing, and the control refused where only certification could see it. The controls were doing their job; nobody was reading them until after the merge. The leg is two jobs. negative-controls-matrix runs the enumeration gate and prints the plan as a matrix. negative-controls fans out over it, builds bin/schema and the generated tree once, and runs its group's controls in one `make -k` invocation so a control that refuses does not hide the ones behind it. Red comes from two places: make's exit status, and a scan of the log for the controls' own refusal lines, which puts the sentence a reader needs into the run's annotations instead of leaving it a thousand compiler lines deep. The scan was held against the 159 real control logs and matches no passing one. The per-port groups install their SDK exactly as the conformance jobs above do, keyed on the row's field rather than on a language name, so a group is a row in make/negative-controls.json and no edit here. Also a prerequisite this leg found by running a control on its own for the first time: conformance-negative-control-absent's second half runs the harness over a substituted registry naming the Go driver, which execs build/conformance-go, and the target did not name it. Inside `make test` the Go leg is already built, which is why the omission stayed invisible. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * ci: the first stale control the leg found, and lint on the reader (#563) tables-block-layout-model-negative-control refused on the leg's first run with "NEGATIVE CONTROL FAILED: C# went red, but not on the layout check", and it was right: the C# build had failed on `CS0246: The type or namespace name 'Blockhome' could not be found`, not on the layout check the control exists to watch. The C# half overrides BlockGeneratedDir alone, so the project's BlockHomeGeneratedDir keeps its default and the blockhome sources have to be on disk, and the target named only bin/schema. The padding and pitch controls beside it already name build/tables-generated-cs/.stamp; this one now does too. Inside `make test` an earlier leg had generated those sources already, which is why nothing said so until every control ran on its own. The reader's four golangci-lint findings, all mechanical: SplitSeq and FieldsSeq for the range loops, CutSuffix for the line continuation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * ci: every negative-control job fits the owner's two-minute rule (#563) The rule (Glenn, 2026-09-07): "if we have any CI that runs per-commit it needs to finish 1 minute, 2 minutes max. Anything longer than that needs to be nightly or manually triggered as needed." It is a rule about a JOB, and a matrix row is a job, so every row of this leg is cut to fit it rather than the leg as a whole being cut to fit it. MEASURED on this branch's own pull-request run (34102397965), before this change: base 8:57, wire-fuzz 7:13, cs 2:46, message-form 2:36, conformance 2:31, block-fuzz 1:59. Five of the eleven rows were over the rule and a sixth was one second under it. WHAT MOVED. The plan now carries 34 pull-request groups instead of 11, cut where the measurement cuts them and not where the family names do: - base splits ten ways. The map gate is 17 controls at 4 s each on the author's machine, so it runs as base-maps and base-maps-keys; the conformance driver's two sabotage rebuilds are 45 s and 47 s of runner time each, so they are base-conformance and base-conformance-absence; the rest are the packet families, the list gate, the arm gate, the cook, the keyed controls and what is left. - wire-fuzz splits eleven ways, one control per job for ten of them. Each rebuilds the compiler under a source overlay and then fuzzes the sabotaged wire, 55 s to 75 s on the runner, so no two of them fit together; the four pinned-vector oracles are cheap and share the eleventh. - cs splits four ways. The C# layout check is the instrument for three block controls at 39 s, 31 s and 29 s. - block-fuzz splits in two, at 48 s and 33 s over two SDK installs. - conformance is gone as a group. Its one control had the bare `conformance` target as its prerequisite, which builds and runs all nine legs and needed all seven SDKs on one runner. The prerequisite is now `conformance-rust`, the rust leg and the derived manifest the control actually reads, and the control rides one SDK in rust-alloc. The soak and the audit keep `conformance`, deliberately: a NUMBER measured over a corpus whose matrix is red is a number about a defect, and this control is not a number. ONE CONTROL DOES NOT FIT AT ANY GROUP SIZE, so it does not run on the pull request at all: tables-message-form-negative-control drives 49 sabotage rows one submake each and takes 124 s where the rule is 120. It is the `nightly` group, and certify.yml runs that tier on the schedule it already carries. Its four blades stay on the pull request in `message-form`. So a group now names its tier. `when` is `pull-request` or `nightly`, `matrix` renders one tier, and the enumerator REFUSES a group that names neither: a control cannot leave the pull request without landing on the nightly, and the package's test reads both workflows to prove each runs the plan rather than a typed list. The 35 controls main gained since this branch last merged it are in the plan too, toolchain-negative-control from #660 among them, and the three family umbrellas that would re-run a split family's other half join the exclusion list with the reason. 194 controls, all of them in a group or in an explained exclusion. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * ci: the two controls main gained while this branch was measuring (#563) tables-block-const-negative-control and tables-json-list-walk-negative-controls landed on main mid-run, and the enumerator did what it is for: the leg went red naming both rather than running 194 of 196. Both are cheap, 1.6 s and 3.7 s on the author's machine, and both belong beside the block and JSON refusers already in `base`, which measures 24 s as a group with them in it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * ci: cut again where the first cut was inside the rule by luck (#563) MEASURED on run 34115675206, the first run of the split leg: 52 of the 54 jobs were inside the owner's two-minute rule, and the two that were not are tables-wire-fuzz-retain-negative-control at 140 s of control time and `big-endian`, which is not this leg's job and is named in this file's header. The retain control is over the rule ON ITS OWN, so grouping cannot save it: it joins tables-message-form-negative-control in the `nightly` group, which certify.yml runs. Its two siblings, the retain-class control and the retain oracle, are inside the rule and stay on the pull request, so what moves is that one pass. Eight groups came in between 100 s and 111 s, which is a rule met by luck rather than by design: `big-endian` was 1:49 when this file's header measured it and is 4:19 today, on a job nobody changed. So base-maps, base-maps-keys, base-arms, base-keyed and the oracle half of wire-fuzz are each two groups now, cut where the measurement puts the weight rather than where the family name does. 38 pull-request groups, 196 controls. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * ci: three controls the naming convention hid, and an umbrella off the nightly (#563) The marker `tools/negativecontrols` reads is `negative-control`, and three controls the Makefile introduces as "Its NEGATIVE CONTROL" spell themselves `-negative`: tables-big-endian-negative, tables-hostile-negative and tables-pack-negative. The enumerator cannot see a name it does not match, so "every negative control runs on every pull request" passed over three of them, and two ran only inside `make test`. They are renamed into the convention and placed. tables-hostile-negative-control and tables-pack-negative-control go in a new base-pack group: one pack corpus, one JSON engine and a `go build -overlay` each, 19 s over the two in one invocation on a cold Go build cache with bin/schema and the generated tree already built. tables-big-endian-negative-control goes in the exclusion list. It is the body of ci.yml's big-endian job, which runs it on every pull request already, and issue #684 owns moving that job; a second s390x cross-compile on this leg proves nothing the first does not. tables-wire-fuzz-retain-negative-control leaves the nightly group for the same list. Makefile:4428 gives it no recipe: it is an umbrella over the retain oracle and retain class leaves, which wire-fuzz-message-oracle and wire-fuzz-retain-class already run. The nightly tier is one control now, and stays a group, because it is where the next control that outgrows the rule lands. Also: the negative-control leg's timeout drops from 25 minutes to 5, which is the rule enforcing itself on a leg whose rows are cut to 120 s; ci.yml no longer says certify.yml runs on every push to main (#669); and the past-tense narration on this branch's own lines is present tense. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * ci: the negative-control tests hold three joins a substring scan cannot (#563) The leg's target list is only the plan while the leg EXPANDS the plan. TestTheLegRunsTheManifestAndNotATypedList greps three substrings, and a leg that typed its own include list and left the old expression in a comment satisfies every one of them. So the workflows are parsed. tools/negativecontrols carries a reader for the block-YAML subset these files are written in, and the new test asserts what each leg actually expands: `strategy.matrix` IS `${{ fromJSON(needs.<matrix-job>.outputs.matrix) }}`, the leg `needs:` that job, and that job's matrix output comes from the step that runs this tool. Both tiers, ci.yml and certify.yml. Proven red first. With ci.yml's matrix expression replaced by a hand-typed three-row include list, TestTheLegRunsTheManifestAndNotATypedList still passes, and the new test says: .github/workflows/ci.yml: the negative-controls job expands map[string]interface {}{"include":[]interface {}{...}} as its matrix, want "${{ fromJSON(needs.negative-controls-matrix.outputs.matrix) }}": a matrix written any other way is a target list this package did not enumerate A marked rule head spelled through a variable or as a pattern rule is refused by name rather than dropped. A dropped head is a control in no group, in no exclusion and in no job, with every test in the package green, which is the one outcome the package exists to prevent. And the plan's toolchain versions are held against test/conformance/*/ci.json. node 20, dart 3.13.2, java 21, otp 29.0.5 and elixir 1.20.4 are written in both files and nothing joined them; a bump in one alone runs a negative control against a runtime its own conformance leg no longer uses. dotnet is exempt by construction: the SDK version lives in .github/dotnet-version and the test refuses a group that writes one here. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * ci: the four controls main gained while this branch was reading (#563) The enumeration gate on the merge ref named all four rather than running 199 of 203, which is the gate doing its job. Measured off one warm build on the author's machine and placed in the group whose corpus each already reads: tables-float-nan-negative-control 1.6 s base tables-maps-cook-check-negative-control 1.6 s base-maps tables-lists-tool-cook-negative-control 0.3 s base-lists tables-message-form-retain-negative-control 2.4 s message-form No group changes tier and none goes near the rule: base 28 s over 24, base-maps 20 s over 5, base-lists 35 s over 14, message-form 16 s over 5. base-pack's own line is restated in the same convention the rest of the file uses, off one warm build rather than off a cold Go build cache: 20 s over the two, most of it the sanitized build of the hostile-value driver. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * ci: cut base-pack, the tightest job the last run measured (#563) On run 34120798793 the base-pack row was 81 s of control time in a 113 s job, which is the band this branch already cut once: a rule met by luck rather than by design. The two controls share no compile that matters. tables-pack's floor is a `go build -overlay` and a byte comparison; tables-hostile's floor is the SANITIZED build of the hostile-value driver, which nothing else in the leg builds and which is most of the 81 s. So they run as two rows, base-pack at 7 s and base-hostile at 14 s off one warm build. The map gate's count in docs/CONTRIBUTING.md is four groups, not two, which it has been since the second cut. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * ci: the message form has five blades on the leg, not four (#563) tables-message-form-retain-negative-control joined the message-form group when main landed it, so the two places that count the blades say "other blades" rather than a number that moves whenever the family grows. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #599
make testpassed over a leg whose pinned toolchain was not there. This makesthat a refusal, by name, and makes the skip something you have to ask for.
From the cold read
A cold read of this branch found three gaps, and they are closed here. The
negative control pointed ONE pin per leg at the absent path, so the
JAVAC,ELIXIRCandMIXprobes had never gone red: deleting all three kept thecontrol green, while the Makefile's comment and CONTRIBUTING both said "every
pin". Each leg now registers
TOOLCHAIN_PINS_<lang>and the control walks alleight pins one at a time, with the leg's other pins pointed at a path that
does resolve so the refusal is attributable to the pin under test.
skipSetand the
conformance SKIPS the <leg> legline had no test at all, sotest/conformance/harness/skip_test.gogates them through the same fakecorpus the absence tests drive, with the skipped leg's driver rigged to turn
the run red if it is ever reached: delete
skipSetand three of its fourtests fail. And nothing on a pull request ran the gate, because nothing on a
pull request runs
make test, somake toolchain-negative-controljoins thethree controls the
go-testjob already runs by hand for that reason. #659puts every negative control on pull-request CI from one manifest; when it
lands, this step folds into that manifest's base group and comes back out of
ci.yml. The read also noted that the gate stopped at the first missing leg,which on this bench named
csand nothing else, and that its green lineclaimed "every pinned toolchain resolves" when it had probed none; both are
fixed below.
Red first, on a bench with no node, dart, java, elixir or dotnet and no
dist/At
bb5f3bd4, before this branch, every one of the five toolchain legs diesdeep inside itself with an anonymous shell error that names neither the leg,
nor the pin, nor the path it looked in, after minutes of generation:
The JS leg is the issue's own class, verbatim: a negative control that goes red
and blames the emitter for a node that is not installed.
One correction to the issue's account, measured rather than assumed: on today's
mainthese legs no longer report exit 0.make/checks/packet-arm-defaults.mklanded on 2026-09-06, one day after the issue, and its per-leg controls turn
four of the silent skips into the anonymous 127s above. What has not changed is
the defect the issue names: nothing states which leg is unmeasurable or why,
nothing distinguishes "the toolchain is absent" from "the port is broken", and
make testreaches none of it until it has spent an hour on C++. A run thatmeasured nine legs and a run that measured four still read the same at the top.
The gate
toolchainis the first prerequisite oftest, so a pin that does not resolvestops the chain before it starts:
Every registered leg is probed and every missing one is named in the same run,
with a skip line ready to paste, rather than the gate stopping at the first
and turning a fresh clone into a queue of install-one-run-again.
Nothing in the Makefile lists a language. Each
make/<lang>.mkregistersTOOLCHAIN_LEGS, aTOOLCHAIN_PINS_<lang>listing every pin it probes, andits own
toolchain-<lang>target carrying one$(call toolchain_probe,...)per pin, beside the lists it already registers;
make registryprints the newone. The C# leg gets the
DOTNETpin it never had, and everydotnetinvocation in the tree reads it.SCHEMA_SKIP_LEGSnames a skip on purpose, and the skip reaches everywhere theleg does: the toolchain gate, the conformance matrix (
harness run --skip,which prints each leg it passes over and refuses to skip the reference leg),
the C# halves of the block gates in the Makefile, and the leg loop. A name that
is not a registered leg is refused before any pin is probed.
Neither
.github/workflows/ci.ymlnor.github/workflows/certify.ymlsetsSCHEMA_SKIP_LEGS. Certification installs each toolchain and overrides thepins (
DART=dart JAVA=java JAVAC=javac ELIXIR=elixir MIX=mix NODE=node), whichresolve and pass the gate.
The negative control, and its positive half
The red half points EVERY PIN of every registered leg at a path that does not
exist, one pin at a time, with the leg's other pins pointed at one that
resolves, and requires each to refuse by the leg's own name and that pin's.
The positive half leaves every pin pointed at that same absent path, names
every leg in
SCHEMA_SKIP_LEGS, and requires the same gate green with eachskip printed. It runs inside
make test, and now inside thego-testjob ofci.ymlbeside the projection's three controls, so the gate is held to therule every other gate here is held to on the diff that changes it.
The bench acceptance
make test SCHEMA_SKIP_LEGS=js,dart,java,elixir,csruns whole to its end onthis bench, which has go, clang, cmake and cargo and none of the other five
toolchains and no
dist/. It opens on the gate:the conformance matrix names the same five and runs the rest against the
reference:
the C# halves of the block gates say which half stood down:
and the leg loop runs C, Go and Rust and names the five it did not:
The tail, and the exit code, re-run whole after the cold read's three fixes:
Also
docs/CONTRIBUTING.mdstates the rule in its building section and says what aport with a pinned toolchain registers, including that every pin the leg
probes goes on
TOOLCHAIN_PINS_<lang>and what happens to one that does not.test/conformance/harness/skip_test.gois the skip's own gate, beside theabsence tests it borrows its fake corpus from.
.github/workflows/ci.ymlrunsmake toolchain-negative-controlin thego-testjob, beside the projection's three controls.The state of CI on this pull request
The
generatedred is GONE:e52f464b(#664) re-pinnedtestdata/golden/tables/maps/TextTable.cpponmain, and the merge alreadyon this branch carries it. CI ran green end to end on that merge commit
(
e18ad846, run 34102399601).What is red now is not a check but the ABSENCE of them. This branch conflicts
with
main, which has since landed the packet wide-string work across allfive
make/<lang>.mkfiles, and GitHub creates nopull_requestrun while apull request has a merge conflict with its base. That is the exact case
ci.yml's own header documents, and it predates the commits added here: thebranch at
e18ad846, before any of them, conflicts with today'smainin thesame five files. Only
cla.ymlfires, becausepull_request_targetneeds nomerge ref, which is why
gh pr checks 660shows one row.Until
mainis merged in, CI on this branch has to come from the documentedfallback,
gh workflow run ci.yml --ref fix-599-toolchain-refusal. Mergingmainhere is a conflict resolution across five language makefiles and isleft to the owner rather than taken unasked.
🤖 Generated with Claude Code