Skip to content

feat: add a read-only local qualification reporter - #160

Open
abruption wants to merge 7 commits into
mainfrom
feat/158-static-reporter
Open

abruption wants to merge 7 commits into
mainfrom
feat/158-static-reporter

Conversation

@abruption

@abruption abruption commented Oct 21, 2026 •

Copy link
Copy Markdown
Owner

Summary

Refs #158 and #23. Implement the owner's explicitly requested extension of the checklist design: a repository-only, read-only local static reporter. This replaces the design-only handoff in #159 while preserving its reviewed history; it does not revive #155.

Run node scripts/remote-qualification.mjs < assessment.json from a source checkout. There is no installed session-peer subcommand or runtime SDK export.

  • Inspect only operator-selected local package JSON name/version text and selected regular-file SHA-256 measurements/labelled baseline comparisons.
  • Preserve independent not_checked/pass/fail/unknown/unsupported predicates, explicit operator-attested context and observation recording times. No aggregate readiness/qualification or action token.
  • Never discover/execute programs, invoke diagnostics, connect to SSH/registries, create persistent state, infer ownership/current callers, or authorize installation/activation/cleanup/recovery.
  • Bound stdin to 64 KiB plus one overflow byte, selected files to their observed size plus one byte (1 MiB maximum file size) with <=64 KiB read requests and one buffer per input/file, and explicit files to 16. Reject ambiguous decoded JSON keys; suppress raw input/error details and unselected metadata fields.
  • Use a conservative local POSIX regular-file reader with nofollow/nonblock, exact bigint descriptor/path identity and observable stability checks, and close attempts in finally. Windows/special/unsupported paths remain explicit unsupported evidence; actual Windows stdin/CLI behavior has not been qualified and the new reporter tests are not in the existing Windows subset; byte bounds do not promise deadlines, snapshots or authentication.
  • Keep actual runtime execution, diagnostics, native qualification, storage ownership and caller use not_checked. Matching digest labels or metadata never prove actual installed/running identity or continuous preservation.

Scope and design extension

The user explicitly selected reporter code after #159 chose checklist-only initially. Update that decision and document the limited input/report contract. Practical rationale is reducing transcription errors in local static comparisons; no empirical pilot/benefit claim is made.

This PR includes the #159 baseline, supersedes its design-only PR, and targets main. #155 remains deferred; #23's real destination qualification and historical recovery debt remain open. No production runtime/transport/wire/profile/version/pin/dependency changes, new live-host/agent/ACK experiments, merge or publication.

The script/tests remain repository-only. Both review/design documents are excluded with a narrow nested npm ignore; existing package file selection/allowlist is preserved. The installed ADR links to issue158, not an absent source-only script/document.

Work separation

A fresh author subagent owned only the script; a separate subagent owned focused tests. Codex integrated the explicit scope, documentation, stdin consumption bound, bigint file identity and verification. The old Claude TUI was not used.

Validation

  • npm run build, npm run test:types, node scripts/check-repository.mjs, local document links and git diff --check: pass.
  • Targeted reporter tests: 14/14 pass locally, including actual POSIX regular-file/CLI behavior and deterministic mock special/race/FD/budget cases. Windows file-reading support is not claimed; synthetic POSIX mocks do not qualify Windows or destination/native behavior.
  • Existing npm run test:package: pass. Its actual npm pack --json result must exactly match the existing 77-member allowlist; the reporter and two source-only design documents are absent. No release allowlist changes.
  • Input, duplicate/UTF8, baseline uncertainty/healthy siblings, no content execution/state writes, safe imports, generic CLI errors, exact stdin/file read caps and large-inode identity are covered. Short-read buffer identity/total allocation, huge-inode positive/negative controls, sanitized stdin I/O failures, and symlink CLI entry with both default and preserved resolution also pass. These checks are not a hostile-host or arbitrary-I/O deadline guarantee.
  • Node v24.16.0 / npm 11.15.0 on macOS arm64. The final raw-input/validation change reran the 14 targeted tests, types and repository checks; package smoke ran at 22244e4 and the subsequent change touched only the excluded reporter/test files. Full suites will run on current-head CI.
  • Current-head CI will be recorded below. No historical green run is substituted.

Fresh noninteractive Opus 5.5 review

Used independent-cli-review, with fresh noninteractive Claude Code 2.1.305 / actual canonical model claude-opus-5-5. Existing Claude TUI/session context was not reused. All three calls completed subtype=success, is_error=false, one turn; tools, MCP, Chrome and slash commands disabled, no session persistence, OAuth account previously approved by the owner. Model responses were suggestions; Codex edited and verified.

  • Initial: source 5a5145826ab19bfeb047733627f4761397d69958, frozen input SHA-256 b44db8fc5f9dad0fb0dd11a77839c3e7811da8de6351a274535d5b379dcc67f6 (68,170 bytes); exit 0; input/source unchanged checks passed.
  • Correction: source 22244e40eb6cb9b5e066dc8af2a351517d9b9b48, frozen input SHA-256 984bf33b1ecb51c68dc4f0c32558778bef63a2e51083bd1687a928c9ee1a583b (42,550 bytes); exit 0; input/source unchanged checks passed.
  • Final: source 4396c78b6dc17e82c1cb12b3323ff5955a6a4577, frozen input SHA-256 854af9728358e1dcebf19fadf4d628edabd09536f9a96617c6cdc144ce350ea7 (44,561 bytes); exit 0; input/source unchanged checks passed.

Input scope: initial complete script, tests, expanded reporter contract, checklist, current #158 scope, ADR delta, npm ignore and package manifest; subsequent calls independently read the complete script/tests/reporter contract with focused correction questions. No secrets, native transcripts or credentials were supplied.

Findings addressed: short reads retaining oversized buffer backing, silent symlink entry no-op, raw input reserialization causing rejection/recursion, and negative tests that did not distinguish intended refusal from caught exceptions. The final model read found all three code defects resolved and no code regressions; it still requested two exact-error test assertions. Codex added precisely those assertions in 4f1b8f2 and reran 14/14 tests/types; reporter source is byte-identical to the final-reviewed 4396c78. This test-only delta was not sent through a fourth model call. Optional recommendations are not represented as mandatory gates.

Reviewers performed static analysis only: no commands/tests/installations/native/SSH/ACK verification. Package exclusion and TypeScript resolution were checked by Codex's actual smoke/types runs; they were not independently verified by the final model input. These reviews provide no operational approval.

Human PR review remains the merge gate. Do not treat report processing, CI or model review as qualification, ownership or operational approval.

Current-head GitHub checks

Head 4f1b8f2d0841326395c375b699fabe1c601e6fa9: automatic Code Review and Security Review both completed with has_any_findings=false, has_unresolved_findings=false; no manual mention was posted.

Initial CI run 39528507956 has two failures in unchanged tests: Ubuntu/Node22 app-server.test.ts original-scope fixture (deadline), and macOS/Node24 handoff-delivery.test.ts unmatched-injection deadline (unsupported instead of timed_out_unknown). The reporter tests passed in those runs; no src or those existing test files changed. The cause is not established merely by that diff. Only failed jobs were rerun, without a source change. Attempt 2 passed both failed contract jobs and the release gate; current checks are 12/12 SUCCESS on the same head. The first-attempt failures remain recorded here; a passing rerun does not establish their root cause or destination qualification.

Define assessment-only states, evidence limits and the remaining qualification work without introducing a reporter, installer or authority/recovery engine. Preserve the historical ADR and defer implementation until human design review.

Refs #158
Require explicit selected command paths for diagnostic attribution, scope current digest comparisons without inheriting historical pass results, and clarify human decisions for open qualification debt. Keep every F1–F6 follow-up outside this checklist-only proposal.

Refs #158
Exclude only the unapproved checklist proposal with a nested npm ignore rule and use its stable PR URL in the installed ADR. A dry-run file-list comparison confirms no other package selection changes. The reviewed checklist contract text is unchanged.

Refs #158
Extend the reviewed checklist with bounded local JSON field and digest observations. Keep execution, networking, ownership, recovery and readiness outside the reporter; preserve per-predicate uncertainty and test input/read budgets and descriptor cleanup.

Refs #158
Refs #158. Address independent Opus review and strengthen regression assertions.
Refs #158. Keep malformed baseline evidence predicate-local and assert exact reader refusal reasons.
Refs #158. Apply the final Opus test-quality findings without changing reporter behavior.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-21T01:40:55.562001Z 4f1b8f2 PR opened
🔒 Security Review ✅ Completed 2026-10-21T01:41:22.779667Z 4f1b8f2 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant