Skip to content

Docs: record the repo-sentinel dogfood findings; C1f is on by default - #266

Draft
b-macker wants to merge 1 commit into
masterfrom
claude/naab-inadmissible-action-prevention-4cmn1m
Draft

b-macker wants to merge 1 commit into
masterfrom
claude/naab-inadmissible-action-prevention-4cmn1m

Conversation

@b-macker

Copy link
Copy Markdown
Owner

Summary

This is a documentation-only PR, following up #263, #264 and #265.

  • docs/governance-campaign-findings.md: a new section records rounds 3–5 of the repo-sentinel dogfood, run on live Gemini. It uses the file's existing live-status labels, and every fix is labelled honestly: only coherence_adjustments has actually been seen working live.
    • coherence_adjustments (CDD: reconcile coherence from telemetry; analyze the response after a failed call #263): confirmed. 171 analyzed rows across 8 runs, none that fail to reconcile. That count comes from the dogfood's own reconciler; the reviewer rows of one run were re-derived by hand and match.
    • Analysis of the response after a failed call (CDD: reconcile coherence from telemetry; analyze the response after a failed call #263): stub-only. Round 5 did retry on 503s (73 retry or skip events), but no send was shown to exhaust its retries, which is the case the fix covers.
    • C1f: the defect is confirmed live (4 of 4 runs, identical to six decimals). The fix is now on by default (Default entropy_baseline_adaptive to true (C1f) #265), but no live run has used it yet, so it stays stub-only.
    • A result that did not reproduce: round 4 found the adversarial reviewer was caught earlier. In round 5 there were only two failed validations across eight runs, both in one run, so the fixtures can't yet tell injection from no injection.
    • Evidence destroyed before it was read: the project's run.sh deleted round 3's telemetry, and its findings cited extract files that were never written.
  • docs/open-investigations.md C1f: status changes from "partially addressed; ships off" to "fixed; on by default since Default entropy_baseline_adaptive to true (C1f) #265".

Test Plan

No code changes, so no tests were run. The numbers above come from the round 5 handoff file, and each claim was checked against the raw telemetry in it.

🤖 Generated with Claude Code

https://claude.ai/code/session_01ELUfjXZvx8kzXo1UJjrAhC


Generated by Claude Code

- docs/governance-campaign-findings.md: new section for rounds 3-5 of the
  repo-sentinel dogfood (live Gemini). coherence_adjustments (#263) is
  confirmed live: 171 analyzed rows across 8 runs, 0 unreconciled. The
  post-failure analysis fix (#263) stays stub-only: round 5 retried on 503s
  but no send was shown to exhaust its retries. C1f's defect is confirmed
  live; its new default (#265) is stub-only until a round runs on it. Also
  records that round 4's "adversarial caught earlier" did not reproduce
  (two failed validations in eight runs), and that round 3's evidence was
  deleted by the project's run.sh before it could be checked.
- docs/open-investigations.md C1f: status "partially addressed; ships off"
  -> "fixed; on by default since #265".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELUfjXZvx8kzXo1UJjrAhC
@github-actions

Copy link
Copy Markdown

NAAb Governance Report

Metric Count
Files checked 16
Passed 16
Failed 0

All governance checks passed!

Generated by NAAb Governance Engine v4.0

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants