pr-debug: drop a landed-fix record for external knowledge intake - #110
Open
tzhouam wants to merge 4 commits into
Open
pr-debug: drop a landed-fix record for external knowledge intake#110tzhouam wants to merge 4 commits into
tzhouam wants to merge 4 commits into
Conversation
New `pr.harvest_debug_knowledge` step (deterministic, knowledge risk) at the end of the pr-debug playbook: after a real, non-dry-run push it writes one JSON record per run — failure signatures with their verified root causes and fixes, keyed by the repo's full GitHub identity — into `knowledge_intake_dir` (KNOWLEDGE_INTAKE_DIR), where the reviewbot's knowledge-intake scanner batches it into a daily knowledge PR for human promotion. Deliberately fail-open as a no-op: unset directory, dry-run push, or zero verified fixes return ok, and a write failure is traced and swallowed — closing the learning loop never fails the landed fix. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwmmhpCK9pfEsWWzUBx3GB
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3a5b473136
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| "(knowledge_intake_dir unset)") | ||
| outputs_map = ctx.state.get("outputs") or {} | ||
| push_outputs = outputs_map.get("push") | ||
| if push_outputs is None or push_outputs.get("dry_run"): |
There was a problem hiding this comment.
When the debug agent reports status=success with root-cause and verification text but fails to create its required commit, review.patch_gate accepts the resulting empty diff and ci.push returns success with empty outputs even though Git reports everything up to date. This condition therefore treats that no-op push as a landed fix and emits unverified model output into the knowledge intake; require an actual pushed commit or confirmed remote-ref advancement rather than merely the absence of dry_run.
Useful? React with 👍 / 👎.
Two executor-driven tests replace hand-built state with the engine's own outputs map: a full debug->push->harvest playbook run drops the record end to end, and a crash-before-harvest followed by a resume over the same run dir proves the checkpoint restore path feeds the harvest step (completed steps replay from progress.json without re-executing — asserted by invocation count, since the cached summary is replayed verbatim). The e2e boundary is the process edge: the agent LLM and git push are simulated as registered steps; engine, checkpointing, and the harvest step itself are real. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwmmhpCK9pfEsWWzUBx3GB
RFC-knowledge-intake.md records the design behind this PR and its reviewbot half (omni-reviewbot #21/#22): why the promotion gate is a reviewed PR rather than any direct write, the three decoupled stages (recording, daily distillation, fork-based publication) with their fail-closed and fail-open choices, the cross-repo drop-record contract and where it is pinned by tests, the fit with this repo's invariants, the alternatives that were rejected, and the flag-by-flag rollout. Indexed as a features row in doc/README.md (the single directory); both doc validators pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwmmhpCK9pfEsWWzUBx3GB
CI's spec-freshness gate flagged config, engine/steps/pr, and playbooks/PLAYBOOKS as stale. engine/steps/pr now documents pr.harvest_debug_knowledge (12th step, deterministic/knowledge) with its invariant — real-push-only, verified-fixes-only, deliberately fail-open, outputs-map consumption restored on resume; config lists knowledge_intake_dir; GUIDE §5's pr-debug chain gains the harvest hop; PLAYBOOKS re-verified unchanged. All three doc gates pass locally. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EwmmhpCK9pfEsWWzUBx3GB
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Copilot half of the knowledge-update hook (consumed by zuiho-kai/omni-reviewbot#21/#22): after a pr_debug run whose fixes actually landed, the run's verified learnings flow into the reviewbot's daily knowledge batch, which raises the rules PR back against this repo for human promotion.
pr.harvest_debug_knowledgestep (deterministic,knowledgerisk) appended to thepr-debugplaybook afterci.push: it writes one JSON record per run — failure signatures with their verified root causes, fix summaries, and verification — intoknowledge_intake_dir(KNOWLEDGE_INTAKE_DIR).knowledge_intake_write_failed) and swallowed — closing the learning loop never fails the landed fix. The record keys the repo by its full GitHub identity fromrepo_full_nameswhen configured.🤖 Generated with Claude Code
https://claude.ai/code/session_01EwmmhpCK9pfEsWWzUBx3GB