Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion doc/GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -426,7 +426,8 @@ pr.fetch_diff → pr.gate_check → agent.review_diff → [post] pr.post_review
```
workspace.guard_clean → pr.checkout_branch → pr.fetch_ci_failures → pr.group_failures
→ [not report_only] agent.debug_group (foreach failure_groups)
→ [not report_only] review.patch_gate → [not report_only] ci.push → report.final_summary
→ [not report_only] review.patch_gate → [not report_only] ci.push
→ [not report_only] pr.harvest_debug_knowledge → report.final_summary
```

**`pr-rebase`** — push 只针对 PR head 分支,force 仅 with-lease。
Expand Down
1 change: 1 addition & 0 deletions doc/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
| [`features/strict-review-deep-engine.md`](features/strict-review-deep-engine.md) | 已实现(默认开) |
| [`features/review-recall.md`](features/review-recall.md) | 已实现(默认开)——v14/v15 召回攻坚 |
| [`features/auto-run.md`](features/auto-run.md) | **draft,未实现**——GitHub 事件触发 |
| [`RFC-knowledge-intake.md`](RFC-knowledge-intake.md) | 已实现(默认关,shadow 起步)——合并 PR 与 bugfix 学习经 reviewbot 蒸馏,以 fork PR 回流知识库,人审合并即晋升 |

## architecture/ —— 维护者

Expand Down
167 changes: 167 additions & 0 deletions doc/RFC-knowledge-intake.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
# RFC — Knowledge intake: merged-PR and bugfix learnings return as reviewed knowledge PRs

- Status: implemented, shadow-first and default-off — copilot side in
PR #110 (`pr.harvest_debug_knowledge`), reviewbot side in
zuiho-kai/omni-reviewbot#21 (intake ledger) and #22 (daily distiller)
- Owner: knowledge plane (`knowledge/`, `AGENTS.md` contract) on this side;
`omni_reviewbot.knowledge_intake` / `knowledge_distiller` on the bot side
- Evidence: `test/test_knowledge_harvest.py` (step + executor/resume e2e),
reviewbot `tests/test_knowledge_intake.py`, `tests/test_knowledge_distiller.py`,
`tests/test_e2e_knowledge_flow.py` (cross-repo contract pinned)

## Motivation

The knowledge tree has a written intake contract — PR-learning produces only
executable rules in the nearest owner's `rules.md`, raw material is deleted,
two validators plus the release audit gate every batch — but no automated
feeder. Learnings evaporate at exactly the two moments they are freshest:

1. **A vllm-omni PR merges.** The review discussion and final diff often
carry a generalizable lesson (a contract the reviewer had to rediscover,
a failure class the diff fixed), and today nobody distills it.
2. **A copilot `pr_debug` run lands a fix.** The agent produced a verified
root cause and fix; `record_debug_memory` keeps it in the run-scoped
debug store, but nothing carries it into the human-curated tree.

The constraint that shapes everything below is the knowledge plane's safety
model: **agents propose, humans promote**. Any automation that writes the
tree directly would break it. A pull request against this repo *is* the
promotion gate — review and merge are the human act — so the design's job is
to deliver well-formed, validator-clean proposals as PRs, and nothing else.

## Design

Three stages, deliberately decoupled so each can fail without losing events.

### 1. Recording (cheap, every watch cycle)

- **Every merged PR counts — literally.** The reviewbot scans GitHub's
closed pulls directly (drafts and never-tracked PRs included) with a
durable cursor in GitHub's own timestamp format; rows are unique per
source, so overlapping windows and crashes re-read but never drop or
duplicate. It does not trust its own `tracked_pulls`.
- **Bugfix runs arrive as drop files.** This repo's
`pr.harvest_debug_knowledge` step (end of the `pr-debug` playbook, risk
`knowledge`) writes one JSON record per run into
`knowledge_intake_dir` (`KNOWLEDGE_INTAKE_DIR`) — but only when the fix
actually landed: a real, non-dry-run `ci.push` and at least one fix with
both root cause and verification (the same bar as debug memory). The
step is fail-open by design: unset directory, dry-run, zero verified
fixes, or a write error each no-op with an honest summary — closing the
learning loop must never fail the landed fix.
- Records land in a `knowledge_intake` SQLite table, `pending` until a
batch consumes them.

### 2. Distillation (expensive, at most once a day)

- The model **only proposes**: it reads a dedicated work clone of this repo
(never the live checkout the Direct bridge imports) and returns
catalog-constrained structured output — target `rules.md` page from the
scanned catalog, a new auditable rule id, one complete section in the
page's own language and format, cited sources. Source events are fenced
as `<untrusted_data>`; instructions never come from them.
- Code **applies mechanically and append-only**: new sections plus an
`updated:` frontmatter bump. Existing sections are never rewritten —
merging and pruning stay human/curator work. Proposals that miss the
catalog, collide with an existing rule id, or fail shape checks are
dropped and counted.
- The **validators are the gate**: `check_knowledge_tree.py` and
`check_wiki_lint.py` run in the clone; a failure — or a missing
validator — fails the batch closed. Rows stay `pending` and retry on the
next daily batch, bounded by an attempt cap that parks poison rows
visibly as `error`. The vllm-omni release audit runs in CI on the PR, as
it does for any knowledge edit.

### 3. Publication (fork-based, double-gated)

- The date-scoped branch (`knowledge/intake-YYYY-MM-DD`) pushes to the
**bot's fork**; the PR targets this repo. The bot holds no write access
here — merging the PR is the human promotion gate, and the PR body lists
every source event plus the validator transcript.
- Shadow is the default: the batch stays a local commit plus an artifact
under the bot's `state/artifacts/knowledge-intake/`. Posting requires
`POST_MODE=review` **and** `KNOWLEDGE_PR_ENABLED` with a configured
fork — the same double-gate shape as every outward write on both sides.
- The automatic cadence yields at most one PR per day. The distiller
reuses the day-branch's **open** PR when one exists; if the day's PR was
already merged or closed, a manual same-day rerun with new rules opens a
follow-up PR on the same branch (showing only the new commits) — a
deliberate consequence of trusting GitHub's open-PR state over local
bookkeeping. Same-day reruns build on the day's branch (never
reset it) and re-attempt publication of anything committed but
unpublished — so a manual `knowledge-batch` retry after a failed push or
PR creation recovers the same branch, and a no-rules retry can never
mark rows done while their committed batch sits unpublished on the same
branch. **The recovery boundary is the day**: the automatic cadence
retries ~24h later under a fresh date branch, re-distilling the still-
pending rows from upstream state. Rows are never lost, but a prior
day's committed-yet-unpublished branch is not resumed — at worst it
survives as a dead branch on the fork (no PR references it). Cross-day
branch recovery was considered and skipped: it trades real complexity
for saving one day of latency on an already-daily loop.

## The cross-repo contract

The drop record is the only coupling between the two repos:

```json
{"run_id": "...", "repo": "owner/repo", "pr": 123, "kind": "bugfix_run",
"title": "...", "groups": [{"signature": "...", "jobs": [],
"root_cause": "...", "fix_summary": "...", "verification": "...",
"files": []}], "created_at": "..."}
```

- `repo` carries the full GitHub identity from `repo_full_names` when
configured; the reviewbot normalizes alias-form values to its watched
slug so rows can never strand under an unqueryable key.
- Each side pins the shape in its own tests: the producer here in
`test_knowledge_harvest.py` (field-level assertions on the written
record), the consumer in the reviewbot's `tests/test_e2e_knowledge_flow.py`
via a **deliberately duplicated** fixture — the repos share no code, so
there is no single fixture both import. A change on one side breaks that
side's test but not the other's; the contract therefore changes only in
lockstep, with this section as the reference.

## Fit with this repo's invariants

- **Repo neutrality**: the harvest step contains no repo literal; the drop
directory is plain settings, empty by default.
- **State via `state_updates` / executor restore**: the step consumes the
executor-maintained `outputs` map, which the resume path restores from
`progress.json` — pinned by the crash-then-resume e2e test.
- **Read-wide/write-narrow**: the tree is still written only by humans
merging PRs; the automation's write surface is a drop directory and a
fork.

## Alternatives considered

- **Per-event PRs** — cleanest provenance, but several PRs a day on an
active repo turns the promotion gate into review spam. Daily batch with
skip-if-empty bounds the load; provenance lives in the PR body instead.
- **Direct writes to the tree (or pushing to this repo)** — rejected
outright: it deletes the human promotion step the knowledge plane is
built on, and would require granting the bot write access here.
- **Copilot-hosted distiller** — this repo has no daemon; the reviewbot
already runs a poll loop with maintenance gating, budgets, and crash
recovery, so the scheduled half lives there and this repo stays a
library plus one playbook step.
- **Stopping at skill candidates / debug memory** — that is where learnings
already go today; both are run-scoped proposals with no path into the
human-curated tree. This design is that missing path, not a replacement
for them.

## Rollout

1. Reviewbot: `KNOWLEDGE_INTAKE_ENABLED=true` — merged-PR recording plus
daily shadow batches; inspect artifacts and `knowledge-batch` output.
2. Wire the bugfix channel: point this repo's `KNOWLEDGE_INTAKE_DIR` and
the reviewbot's `COPILOT_INTAKE_DIR` at the **same directory** (v1
assumes a shared host). Without this pair, everything else works but
`pr_debug` learnings are silently absent from batches — both values
default to empty/off.
3. Create the bot's fork of this repo; set `KNOWLEDGE_FORK_SLUG`.
4. `KNOWLEDGE_PR_ENABLED=true` under `POST_MODE=review` — PRs start; every
one is reviewed like any other knowledge edit.

Rollback at any stage is turning the flag off; nothing in the tree changes
without a merged PR.
4 changes: 3 additions & 1 deletion doc/architecture/SPEC/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,9 @@ metrics 与升级,提供带类型字段和安全默认值;外加 PR4d 知识
MCP `repo_path` 授权面与 idempotency 保留期:
`mcp_allowed_repo_roots`(`MCP_ALLOWED_REPO_ROOTS`,空 = 只有已配置的
checkout —— 最小权限;放宽是**运维**决定,不是调用方的)与
`idem_retention_days`(默认 30,圈住 `.idem/` 索引)。
`idem_retention_days`(默认 30,圈住 `.idem/` 索引);以及
知识回流落盘目录 `knowledge_intake_dir`(`KNOWLEDGE_INTAKE_DIR`,默认空 = 关闭;
消费方见 `doc/RFC-knowledge-intake.md`)。

## 公开契约
带全部可调项的 `Settings`;`reviewer` / `intent`(回退到 `agent_model`);
Expand Down
16 changes: 13 additions & 3 deletions doc/architecture/SPEC/engine/steps/pr.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

<!-- verified-against: 2026-08-28 -->

`LOC ~1300(6 个文件) · step 库(PR) · refactor-status: ok`
`LOC ~1390(6 个文件) · step 库(PR) · refactor-status: ok`

## 职责
受守卫的推送、只读的 PR 抓取/门禁、PR rebase、PR debug、受门禁的评审发布。
Expand All @@ -15,13 +15,14 @@
- `rebase.py` —— `pr.checkout_branch`、`pr.rebase_onto_base`、`pr.analyze_diff`、
`agent.verify_module`。
- `debug.py` —— `pr.fetch_ci_failures`(+ `_enrich_ci_logs`)、`pr.group_failures`、
`agent.debug_group`。
`agent.debug_group`、`pr.harvest_debug_knowledge`
- `publish.py` —— 对外写入(risk=push):`ci.push`、`pr.post_review`。
- `utils.py` —— 纯函数 `extract_signature`(及其正则)。

## Steps(11 个)
## Steps(12 个)
`ci.push`(script/push);`pr.fetch_diff`、`pr.gate_check`、`pr.checkout_branch`、
`pr.analyze_diff`、`pr.fetch_ci_failures`、`pr.group_failures`(deterministic/read);
`pr.harvest_debug_knowledge`(deterministic/knowledge);
`pr.rebase_onto_base`、`agent.debug_group`(agent/write_workspace);
`agent.verify_module`(validation/read);`pr.post_review`(script/push)。

Expand Down Expand Up @@ -58,6 +59,14 @@
- `pr.fetch_ci_failures` 经 profile 选定的 CI provider 富化日志,否则记一条
`capability_gap`(**E2**);`pr.group_failures` 按**归一化后**的签名分组。
- `pr.post_review` 是**双闸**的(**C5**)。
- `pr.harvest_debug_knowledge` **只在真实推送后落盘**(push 输出存在且非 dry-run,
且至少一组修复同时有 root_cause 与 verification —— 与 debug memory 同一门槛),
写入 `settings.knowledge_intake_dir`(空 = 关闭,默认)。它是**刻意 fail-open**
的:目录未配、dry-run、无已验证修复、写失败(trace
`knowledge_intake_write_failed` 后吞掉)都返回 ok —— 关闭学习回路
**绝不能**让已落地的修复失败。它消费 executor 维护的 `state["outputs"]`,
该表在 resume 时由 checkpoint 恢复(见 `test_knowledge_harvest.py` 的
crash-then-resume 用例)。设计记录:`doc/RFC-knowledge-intake.md`。

## 边界 —— 不属于这里
不含推送授权逻辑(那是 `push`);不含 CI 日志抓取机制(那是 `ci/providers`);
Expand All @@ -70,6 +79,7 @@
## 测试
`test_pr_steps.py`(含钉 ref run 域隔离、head 移动检测、stale expected_head
BLOCK、worktree 分键/拒外来树)、`test_push_and_steps.py`、
`test_knowledge_harvest.py`(harvest step + executor crash-then-resume)、
`test_ci_and_repo_map.py`(注意:`test_ci_and_repo_map` monkeypatch 的是
`pr.debug._gh`,即 `pr.fetch_ci_failures` 绑定 `gh` 的那个子模块);
端到端:`test_thin_mcp_server.py`(评审跑在钉住的 worktree 上、head 移动
Expand Down
2 changes: 1 addition & 1 deletion doc/architecture/SPEC/playbooks/PLAYBOOKS.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# playbooks/*.yaml —— 规范

<!-- verified-against: 2026-08-25 -->
<!-- verified-against: 2026-08-28 -->

`8 个文件 · 声明式编排数据 · refactor-status: ok`

Expand Down
4 changes: 4 additions & 0 deletions playbooks/pr-debug.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -20,5 +20,9 @@ steps:
- {id: debug, step: agent.debug_group, foreach: failure_groups, when: not report_only}
- {id: gate, step: review.patch_gate, params: {pre_push: true}, when: not report_only}
- {id: push, step: ci.push, when: not report_only}
# Landed-fix learning loop: drops a bugfix-run record for external
# knowledge intake; a no-op unless knowledge_intake_dir is configured
# and the push above was real (not dry-run).
- {id: harvest, step: pr.harvest_debug_knowledge, when: not report_only}
- {id: report, step: report.final_summary}
success: each failure group fixed at root cause or escalated; additive push only
6 changes: 6 additions & 0 deletions src/infermatrix_copilot/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,12 @@ class Settings(BaseSettings):
# doc is reachable on demand via doc_search/doc_read.
knowledge_dir: Path = _resource_dir("knowledge")
knowledge_general_docs: list[str] = ["general/_index.md"]
# Drop directory for bugfix-run intake records (KNOWLEDGE_INTAKE_DIR):
# after a pr_debug run lands a real push, `pr.harvest_debug_knowledge`
# writes one JSON record here for an external knowledge-intake consumer
# (e.g. the reviewbot's daily distillation batch). Empty disables the
# drop; the step then no-ops without failing the run.
knowledge_intake_dir: str = ""

# Engine
run_root: Path = Path.home() / ".infermatrix-copilot" / "runs"
Expand Down
80 changes: 80 additions & 0 deletions src/infermatrix_copilot/engine/steps/pr/debug.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
from __future__ import annotations

import json
from datetime import datetime, timezone
from pathlib import Path

from ....scopes import post_plan_scope
Expand Down Expand Up @@ -182,3 +183,82 @@ async def _pr_debug_group(ctx: StepContext) -> StepResult:
else:
result.summary = f"'{sig}': {result.summary[:250]}"
return result


@step("pr.harvest_debug_knowledge", "deterministic", "knowledge",
"Drop a bugfix-run record for external knowledge intake.")
async def _pr_harvest_debug_knowledge(ctx: StepContext) -> StepResult:
"""After a pr_debug run whose fixes actually landed (a real, non-dry-run
push), write one JSON record — failure signatures with their verified root
causes and fixes — into `settings.knowledge_intake_dir`, where an external
consumer (the reviewbot's knowledge-intake scanner) batches it into a
knowledge PR for human promotion.

Deliberately fail-open as a no-op: unset directory, dry-run push, or zero
verified fixes each return ok with an honest summary, and a write failure
is traced and swallowed (`knowledge_intake_write_failed`) — closing the
learning loop must never fail the landed fix."""
intake_dir = str(ctx.settings.knowledge_intake_dir or "").strip()
if not intake_dir:
return StepResult(True, summary="knowledge intake disabled "
"(knowledge_intake_dir unset)")
outputs_map = ctx.state.get("outputs") or {}
push_outputs = outputs_map.get("push")
if push_outputs is None or push_outputs.get("dry_run"):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Require proof that the push advanced the PR branch

When the debug agent reports status=success with root-cause and verification text but fails to create its required commit, review.patch_gate accepts the resulting empty diff and ci.push returns success with empty outputs even though Git reports everything up to date. This condition therefore treats that no-op push as a landed fix and emits unverified model output into the knowledge intake; require an actual pushed commit or confirmed remote-ref advancement rather than merely the absence of dry_run.

Useful? React with 👍 / 👎.

return StepResult(True, summary="no landed fix (push missing or "
"dry-run) — nothing to harvest")
groups = ctx.state.get("failure_groups") or []
debug_outputs = outputs_map.get("debug") or {}
# foreach with one item returns the agent's outputs unmerged; more items
# arrive keyed by index (executor._merge).
indexed = ({"0": debug_outputs} if "root_cause" in debug_outputs
else {k: v for k, v in debug_outputs.items() if k.isdigit()})
fixes = []
for index, group in enumerate(groups):
out = indexed.get(str(index)) or {}
if not (out.get("root_cause") and out.get("verification")):
continue # same bar as debug memory: unverified fixes don't teach
fixes.append({
"signature": str(group.get("signature", ""))[:300],
"jobs": [str(j)[:120] for j in (group.get("jobs") or [])[:10]],
"root_cause": str(out.get("root_cause", ""))[:2_000],
"fix_summary": str(out.get("fix_summary", ""))[:2_000],
"verification": str(out.get("verification", ""))[:1_000],
"files": [str(f)[:300]
for f in (out.get("files_modified") or [])[:20]],
})
if not fixes:
return StepResult(True, summary="no verified fixes to harvest")
spec = ctx.state.get("task_spec") or {}
alias = str(spec.get("repo", ""))
record = {
"run_id": ctx.run_dir.name,
# Full GitHub identity when configured, so the consumer keys the
# event correctly; the local alias otherwise.
"repo": ctx.settings.repo_full_names.get(alias, alias),
"pr": spec.get("pr"),
"kind": "bugfix_run",
"title": f"pr_debug fixes for PR #{spec.get('pr')}",
"groups": fixes,
"created_at": datetime.now(timezone.utc).isoformat(),
}
try:
directory = Path(intake_dir).expanduser()
directory.mkdir(parents=True, exist_ok=True)
final = directory / f"{ctx.run_dir.name}.json"
tmp = final.with_suffix(".tmp")
tmp.write_text(json.dumps(record, ensure_ascii=False, indent=1),
encoding="utf-8")
tmp.replace(final)
except OSError as exc:
ctx.trace.record("knowledge_intake_write_failed",
error=str(exc)[:300])
return StepResult(True, summary=f"intake drop failed (traced): {exc}",
outputs={"intake_error": str(exc)[:300]})
ctx.trace.record("knowledge_intake_dropped", path=str(final),
fixes=len(fixes))
return StepResult(True,
summary=f"dropped {len(fixes)} verified fix record(s) "
"for knowledge intake",
outputs={"intake_path": str(final),
"fixes": len(fixes)})
Loading
Loading