Skip to content

feat: add github pull request monitor probe - #5174

Closed
kyleseaman wants to merge 1 commit into
token-monitors-accountingfrom
codex/token-monitors-github-probe
Closed

feat: add github pull request monitor probe#5174
kyleseaman wants to merge 1 commit into
token-monitors-accountingfrom
codex/token-monitors-github-probe

Conversation

@kyleseaman

@kyleseaman kyleseaman commented Aug 23, 2026

Copy link
Copy Markdown
Collaborator

Stack: #5171#5172#5173#5174#5175#5176#5177

Position: 4 of 7. Base: #5173. Next: #5175.

Adds a bounded public GitHub pull-request probe with canonical facts, stable actionable fingerprints, and no-wake shadow execution.

Verification: provider, shadow, security posture, and validation suites.

@github-actions

github-actions Bot commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of 9efece8ab3d4c0bdaaf16ae57d0cff27b11d3d98 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 9efece8

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 9efece8ab3d4c0bdaaf16ae57d0cff27b11d3d98: <one-sentence reason>

@kyleseaman kyleseaman closed this Aug 23, 2026
@kyleseaman
kyleseaman deleted the codex/token-monitors-github-probe branch August 23, 2026 05:18
@github-actions github-actions Bot removed the readiness: checking Automated validation is still running label Aug 23, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — 🟡 CONCERNS

Design-level review of 9efece8ab3d4c0bdaaf16ae57d0cff27b11d3d98 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: CONCERNS

Head-change wakes fire on any open-PR status, so routine pushes — not actionable events — will drain the 8-turn wake budget.

Watch

  • decision.py now reads if observation.head_changed or observation.status is ACTIONABLE — but the spec text justifies head-change precedence only "over an otherwise-green success". After a push, the very next probe is almost always PENDING (checks_pending), and head_changed still forces WAKE_ACTIONABLE; each push therefore buys one wake/agent turn, and ~8 pushes exhaust max_agent_turns and stop the monitor via STOP_BUDGET before the PR ever reaches review-ready. If waking on every push is intended, say so in the spec; if not, gate head_changed to the SUCCESS case the doc actually describes. Shadow mode masks this today, but the decision policy lands here and later stack slices inherit it.

Suggestions

  • run_shadow_probe mutates MonitorState in place before await persist(state); a persist failure leaves an in-memory state ahead of durable state — mutate-after-persist (or persist a copy) removes the divergence before real callers arrive in the next slices.

[DESIGN-REVIEWED] 9efece8

@kyleseaman

Copy link
Copy Markdown
Collaborator Author

Replaced by #5183 after renaming the branch to remove the prefix.

@github-actions

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — 🟡 CONCERNS

Premise-level review of 9efece8ab3d4c0bdaaf16ae57d0cff27b11d3d98 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Contract read, patch read, repository greps run. Producing the review.

First-Principles-Verdict: CONCERNS

Five new persisted MonitorState fields ship with zero readers, and the probe re-spells three GitHub-reading mechanisms the dashboard already owns.

What this change ships

Intent: let a background monitor observe a public GitHub PR's review-readiness without spending model turns — an ADDITION (slice 4 of a declared 7-PR stack).

  1. Probe reads PR state via gh, exact public github.com PR URLs only — justified
  2. Canonical allowlisted observation + stable SHA-256 fingerprint — justified
  3. Actionable fingerprint ignores unrelated pending-check churn — justified
  4. A pushed head now wakes even when the new state is green (decision.py) — justified, spec'd same commit
  5. Shadow runner persists decisions, never wakes — justified (declared shadow slice; consumer arrives in feat: expose session monitors to agents #5175)
  6. wake_delivery parameter that exists only to be refused — zero consumers
  7. Five new persisted metrics fields on MonitorState — undeclared, zero consumers
  8. New terminal SETUP provider-error kind — justified
  9. Spec/security-posture doc updates — justified (AGENTS.md same-commit rule)
  10. Second spelling of URL-parse / check-collapse / threads-query vs dashboard — duplicate of source_providers.py

Watch

  • Zero-reader persisted fields. Grepped probe_count|provider_error_count|last_probe_at|last_decision|last_provider_error across src/: written in monitoring/shadow.py:66-79, validated in models.py, read nowhere. consecutive_provider_errors is the only counter any decision consumes. Persisted schema acquired before any reader is surface the next slice must honor.
  • Three counted second spellings of dashboard mechanisms: PR-URL validation (parse_source_url, source_providers.py:509), latest-run check collapse — including the identical "\uffff" queued-run sentinel (source_providers.py:1076 vs _normalize_check), and the review-threads GraphQL query (source_providers.py:1135 vs _REVIEW_THREADS_QUERY). The security spec declares the separation deliberately, but the collapse logic in particular will diverge silently; extracting one collapse helper is the general fix, accepted-and-deferred.

Subtractions

  • Drop the wake_delivery parameter and ShadowWakeDeliveryRefused from run_shadow_probe (shadow.py:59-63) — 0 callers can pass it; a parameter that does not exist refuses more strongly than one that raises.
  • Drop last_decision, last_provider_error, and last_probe_at from MonitorState (models.py:200-202) until the slice that reads them lands — 0 readers counted; probe_count/provider_error_count can follow the same rule.
  • Drop the response field from GitHubPullRequestProbeResult (github_pull_request.py:215) — shadow.py consumes only canonical and observation; 0 non-test readers.

[FIRST-PRINCIPLES-REVIEWED] 9efece8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant