Repository navigation
fix(b1-gate): refuse --held-out outright; the ADR-0028 set is spent - #312
Merged
Merged
Conversation
The held-out set 100-124 was evaluated once (ADR-0036). Maintainer decision, 2026-11-07: `navkit gate b1` takes development seeds only. The flag stays so the refusal names the reason and points at `navkit gate candidate`. run_gate and check_seeds lose their held_out mode. The seed-flag inventory moves b1_gate.py from the gate list to the guarded list. ADR-0028 gets a dated note; its decision section is unchanged. Test-first. On the old code the new run_gate test got past the guard on 100-124 and was stopped by the monkeypatched sentinel, so no held-out seed ran in the red phase. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…a removed keyword CodeQL py/call/wrong-named-argument flagged the deliberate held_out=True call. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #309. Maintainer decision (2026-11-07): the held-out set of ADR-0028 (seeds 100 to 124) was evaluated once (ADR-0036) and is spent, so
navkit gate b1no longer runs it.Changes
navkit gate b1 --held-outexits 2 with the reason and a pointer tonavkit gate candidate(ADR-0030). The flag is kept only so the refusal is explicit.run_gateandcheck_seedshave no held-out mode: development seeds only.tests/test_seed_policy.py:b1_gate.pymoves from the gate list to the guarded list. The candidate gate is now the one procedure that may use held-out seeds.seed_policydocstring,navkit gatehelp and a CI comment are updated to match.docs/data/b1_gate_heldout.csv(scripts/render_b1_heldout.py, which runs no case).Evidence
run_gate(100-124, held_out=True)got past the guard and hit the sentinel, so the red phase ran no held-out seed.-n auto). ruff, format, mypy andmake docs-fastare clean.🤖 Generated with Claude Code