Repository navigation
Adapter Node suite in CI; Item 4: null cash admitted only with the estimate-only marker; adapter writes the markers - #99
Merged
Conversation
…orkflow What changed: .github/workflows/regression-gate.yml gains the job adapter-tests (display name claude-code-adapter-tests, ubuntu-24.04): the same pinned actions/checkout step as the other jobs, actions/setup-node pinned by sha (v7.0.0, the pin Research-Kit's workflows use) with Node 24, then `node --test hosts/claude-code/route.test.mjs`. The header comment says it is not a required status context. hosts/claude-code/README.md says CI runs the suite; the artifact test's docstring and CHANGELOG (Unreleased) say the same. Why: PR #98's remaining limits: the adapter's 91-test suite was run by no gate, local or CI. What you verified: yaml.safe_load parses the workflow and the job has the name, runner and three steps above (verified). The job's command under Node v24.21.0 locally: 91 tests, 91 pass, 0 fail (verified). The job itself has not run on GitHub Actions (untested). test_claude_code_host_artifacts.py OK, validate_skills --strict exit 0, check_secrets clean (verified); the regression gate ran in the pre-commit hook. What you got wrong: the first header comment said the job runs "on push and pull_request"; with no `if:` it runs on every trigger the gate job runs on (dispatch and schedule too). Reworded before committing; yaml re-parsed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VdJ51CTnHT4Zet6eScsBBh
…er (Item 4)
What changed: tools/conform_spec.py. check_record: counters.cash-spent and
counters.cash-cap may be null only when counters["cash-basis"] ==
"estimate-only". check_outcome: evidence.actual-spend may be null only when
evidence["actual-spend-basis"] == "estimate-only"; every other Outcome field
still goes through outcome_ledger.validate (on a copy with a placeholder
spend; the document is not modified). An unmarked null keeps today's text; a
marker with any other value is a new violation ('LedgerRecord
counters.cash-basis must be "estimate-only" when present', and 'Outcome
evidence.actual-spend-basis ...' likewise); a marker beside a finite number
is allowed. tools/test_conform_spec.py: K10, nine tests. The artifact test's
docstring no longer says the record check is never relaxed.
Why: docs/proposals/usage-windows-gate4.md Item 4, accepted by the
maintainer on 2026-10-18: a subscription host cannot write a conformant
record without inventing 0.0 cash, which USAGE_TELEMETRY forbids.
What you verified: K10 red before the change (21 failures, 2 errors: marked
null refused, wrong marker accepted, CLI exit 1) and 29/29 OK after
(verified). Guard removal, each restored after: marker-value check removed
-> 12 wrong-marker record subtests red; record null admission removed ->
marked-null record test and CLI test red; null admitted without marker ->
unmarked-record pin red; outcome marker check removed -> 8 subtests red;
outcome null admission removed -> marked-null outcome and CLI red; outcome
null admitted without marker -> unmarked-outcome pin red (verified). A
committed adapter ledger line and outcome: VIOLATION unmarked, CONFORMANT
with the marker added (verified). tools/outcome_ledger.py (the store) is
unchanged and still refuses a null spend (expected, out of scope).
What you got wrong: nothing to report.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdJ51CTnHT4Zet6eScsBBh
…LEMETRY, the proposal and CHANGELOG What changed: skills/smart-router/references/spec-v2.md §10.3 Outcome gains evidence.actual-spend-basis and §10.5 LedgerRecord gains counters.cash-basis, one line each (optional, additive §13, only "estimate-only", the one case where the cash field may be null). docs/USAGE_TELEMETRY.md (beside the "stays null, never zero" line): a record or outcome with unknown cash carries the marker. docs/proposals/usage-windows-gate4.md: the review record carries the 2026-10-18 decision (Item 4 accepted as proposed and implemented with this change; Items 1-3 remain proposals; no independent non-author verdict recorded), and the status line no longer says nothing in skills/ is amended. CHANGELOG.md: the Unreleased entry for Item 4. Why: the maintainer accepted Item 4 on 2026-10-18; the checker change (previous commit) needs the contract to name the fields it admits. What you verified: validate_skills --strict exit 0, check_secrets clean, tests/docs_drift.py pinned (30 skills docs) (verified); nothing in tools/ or tests/ parses the spec's JSON blocks (git grep, verified); the regression gate ran in the pre-commit hook. What you got wrong: the brief said the §10.3/§10.5 definitions are under docs/; they are in skills/smart-router/references/spec-v2.md (a normative file). Edited there, and flagged in the report. The first status-line edit left one over-long line and "the items below" still covering Item 4; reflowed to name Items 1-3. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VdJ51CTnHT4Zet6eScsBBh
… and refuses any other value
What changed: tools/test_claude_code_host_artifacts.py. The per-line ledger
checks move into check_ledger_line (same assertions, seq order stays with
the caller), which now also requires the optional counters.cash-basis, when
present, to be "estimate-only". Two fixture tests: a line with null cash and
the marker passes the key check and conform_spec.check_record; a line with
a wrong marker ("estimate", "", None, 0) fails the key check.
Why: Item 4 (accepted 2026-10-18). The next adapter refresh writes
counters.cash-basis; the gate's artifact check must accept that and must not
accept a different value that conform_spec now refuses.
What you verified: after the pure refactor the file was OK (5 tests before
the fixtures were added: 3, OK) (verified). With the fixtures and no marker
check: the marked fixture passed and the wrong-marker test failed 4 subtests
(verified). With the check: 5 tests OK; check line replaced by `pass`: 4
failures, restored: OK (verified). Every one of the 31 committed ledger
lines with cash-basis added passes check_ledger_line and
conform_spec.check_record (verified). validate_skills --strict exit 0,
check_secrets clean (verified); the regression gate ran in the pre-commit
hook.
What you got wrong: the brief's "red-first: a fixture record with the marker
must pass the key check" could not go red: the key check was already a
subset test (LEDGER_KEYS <= keys, COUNTER_KEYS <= counters), so an extra
marker key always passed. The red half is the wrong-marker fixture. The
file has no outcome-evidence check at all (it never reads trace/outcomes/),
so there was no actual-spend-basis key check to relax; none was added.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdJ51CTnHT4Zet6eScsBBh
…ll actual-spend What changed: CHANGELOG.md (the Unreleased Item 4 entry) and the review record in docs/proposals/usage-windows-gate4.md each gain a "known limit" note: the reference §10.3 store, tools/outcome_ledger.py, is not amended by Item 4; validate and record still refuse an Outcome whose evidence.actual-spend is null (marked or not), and read_all fails closed on a ledger holding such a line. Bringing the store in line is a follow-up. Why: review finding R2 (S3). conform_spec now accepts a marked-null Outcome that the repository's own store refuses. The brief scoped Item 4 to the checker and named its files; outcome_ledger.py is not among them and the brief's stop trigger covers a change outside them, so this commit takes the finding's documented alternative (record the gap) and leaves the code change to a follow-up unit. Set aside: changing outcome_ledger.validate here. What you verified: reproduced the gap on an Outcome with actual-spend null and actual-spend-basis "estimate-only": conform_spec --outcome prints CONFORMANT (exit 0); outcome_ledger.validate and record raise FailClosed "evidence actual-spend must be a finite non-negative number"; read_all on a one-line ledger fails closed "ledger line 1 invalid" (verified). validate_skills --strict exit 0, check_secrets clean, docs_drift pinned (verified); the regression gate ran in the pre-commit hook. What you got wrong: nothing to report. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VdJ51CTnHT4Zet6eScsBBh
…ine width What changed: tools/test_claude_code_host_artifacts.py module docstring, lines 6-8 (now 6-9) rewrapped to about 105 columns; same words, no code change. Why: review finding R3 (S4). The Item 4 reflow left line 6 at 174 characters, wider than any line in the file at base (143). What you verified: before, line 6 was 174 characters (awk, verified red); after, the docstring lines are 101-108 and the file's maximum is 143, the base value (awk, verified green); the docstring's words are unchanged (whitespace-normalised diff against HEAD, verified); the file's 5 tests pass (verified); the regression gate ran in the pre-commit hook. What you got wrong: nothing to report. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VdJ51CTnHT4Zet6eScsBBh
…arries the estimate-only cash markers (Item 4), and its newest ledger line and outcome pass conform_spec The host adapter writes counters.cash-basis "estimate-only" on every ledger record and actual-spend-basis "estimate-only" on an outcome whose actual-spend is null (a supplied basis is validated: null is no basis, any other value is refused). Older committed lines stay unmarked and conform_spec keeps refusing them, as the README says. The trace gains the sr-item4-host and sr-ci-item4 units (decisions, dispatches, accepted outcomes) and the fix-x2-1, fix-x2-2 and fix-u7-r3 decisions and dispatches (the first MoonAliza units routed by the adapter). The copy carries no build briefs.
…te-only marker (Item 4), as conform_spec does
what changed: tools/outcome_ledger.py validate(): a null spend is valid beside actual-spend-basis "estimate-only"; a present marker with any other value is refused ('evidence actual-spend-basis must be "estimate-only" when present'); the docstring names it. tools/test_outcome_ledger.py: one test (marked null accepted; unmarked null keeps today's refusal; six wrong markers refused; a marker beside a number allowed).
why: the R2 follow-up recorded in this branch: the canonical outcome store still refused the records conform_spec now accepts.
what you verified: red first on the unchanged module (the new test errors with 'finite non-negative number' on the marked null); green after the change (12 tests OK); guard removal (the marked-null clause replaced by False) makes it red again and the module was restored from a saved copy; the regression gate runs in the pre-commit hook (verified by the commit landing).
what you got wrong: the first draft of the test used names the module does not export (GOOD, validate, FailClosed bare); it errored on both sides of the change and was rewritten on the file's own helpers (good(), ol.validate, ol.FailClosed) before the red run was taken.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VdJ51CTnHT4Zet6eScsBBh
StepenkoAnatoli
marked this pull request as ready for review
October 18, 2026 05:11
This was referenced Oct 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem and resulting behavior
Two of the "Remaining limits" of #98, solved:
.github/workflows/regression-gate.ymlgains a jobadapter-tests(display nameclaude-code-adapter-tests): the same pinned checkout,actions/setup-nodepinned by sha (v7.0.0, the pin Research-Kit's workflows use) with Node 24, thennode --test hosts/claude-code/route.test.mjs. It runs on every trigger the gate job runs on and is not a required status context (branch protection is the maintainer's setting). Its first run on this PR passed (93 tests). The adapter README says CI runs the suite.tools/conform_spec.pyrefused every record a subscription host writes because its cash fields arenull(proposaldocs/proposals/usage-windows-gate4.md, Item 4). Accepted as proposed on 2026-10-18:counters.cash-spentandcash-capmay benullonly besidecounters.cash-basis: "estimate-only", andevidence.actual-spendonly besideevidence.actual-spend-basis: "estimate-only"; an unmarkednullkeeps today's violation text; any other marker value is a violation; a marker beside a finite number is allowed. The additive fields are written into §10.3 and §10.5 ofskills/smart-router/references/spec-v2.md(the normative definitions live there, not underdocs/; the maintainer's decision covers that amendment),docs/USAGE_TELEMETRY.md, the proposal's review record and the CHANGELOG. The host-artifact check accepts the marker and refuses any other value.tools/outcome_ledger.py, the canonical outcome store, applies the same rule.hosts/claude-code/, refreshed from the host): every ledger record carriescash-basis, an outcome with a null spend getsactual-spend-basis(a supplied basis is validated:nullis no basis, any value butestimate-onlyrefused). Older committed lines stay unmarked and are still refused, as the adapter README says. The trace also gains the five units the adapter routed since Claude Code host adapter, usage-window affordability proposal, calibration evidence, two test-isolation and two link fixes #98 (two of them this change's own build units, three MoonAliza fix units).Related issue
#98 "Remaining limits or risks" (items 1 and 5); proposal Item 4 — no issue number.
Validation
Run from the repo root on this branch (
50e5107), local fixtures, no live provider:python3 tools/run_regression.py→ "REGRESSION GATE: all checks PASS" (through the pre-commit hook on every commit)python3 tools/test_conform_spec.py→ 29 OK (9 new K10 tests, red-first with 7 guard-removal proofs);python3 tools/test_claude_code_host_artifacts.py→ 5 OK;python3 tools/test_outcome_ledger.py→ 11 OK (one new test, red-first, guard proof)python3 tools/validate_skills.py --strictexit 0;python3 tools/check_secrets.pyclean (0 hits)node --test hosts/claude-code/route.test.mjs(Node 24) → 93 pass, 0 fail, locally and in the new CI jobpython3 tools/conform_spec.py --record <the newest ledger line>→ CONFORMANT;--outcome hosts/claude-code/trace/outcomes/sr-ci-item4-3.json→ CONFORMANT;--record <the first, unmarked ledger line>→ VIOLATION (counters.cash-cap must be a finite non-negative number), as intendedf2cd97c: regression-gate, claude-code-adapter-tests and watcher-once-proof all successRemaining limits or risks
50e5107says "12 tests OK"; the file runs 11 (the count in the message is off by one, the result is not).🤖 Generated with Claude Code
https://claude.ai/code/session_01VdJ51CTnHT4Zet6eScsBBh