Skip to content

feat(pr_knowledge): probe whether an upstream PR is in a local git tree - #1847

Merged
chaojhou merged 10 commits into
AMD-AGI:mainfrom
okorzh-amd:okorzh/pr-inclusion-probe
Oct 22, 2026
Merged

chaojhou merged 10 commits into
AMD-AGI:mainfrom
okorzh-amd:okorzh/pr-inclusion-probe

Conversation

@okorzh-amd

@okorzh-amd okorzh-amd commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor
  • Description: what and why

    There was no mechanical way to answer "is upstream PR N already in this tree?". An agent working on a framework or kernel tree could see the PR's merge state and version labels. It could not tell whether a fork or a pinned checkout already carried the PR, so it could pick up a PR the tree already had.

    This adds kernelforge.knowledge.pr_knowledge.inclusion, a probe that reads local git only. It gives one of two answers (Inclusion, a (str, Enum) with exactly two members, inclusion.py:63-67):

    • included, with the git evidence;
    • unknown, with the reason nothing showed the PR.

    There is no "not included" answer. Local history can show that a PR is present, but never that it is absent. A backport that was edited, had a conflict resolved, or was folded into a larger commit looks the same as no backport. An answer of absence would therefore be wrong exactly where it matters, so the probe never gives one.

    The answer's method (InclusionMethod, also (str, Enum), inclusion.py:70-75) is subject, merge_commit or none. none always goes with unknown. The evidence is read in this order:

    1. Subject (method: subject). git log --fixed-strings --grep finds a commit reachable from the checked commit whose subject names the PR. Either of two forms counts:

      • a trailing (#N), alone or in a run such as (#N) (#M). A squash merge writes it this way, and so do a fork's cherry-pick of that merge and a fork re-merging the cherry-pick through its own PR;
      • a subject that starts with Merge pull request #N from . A merge commit writes it this way, and so does a git cherry-pick -m 1 of it.

      #12 never matches #123 or #112. These do not count: a mention in the middle of the subject, a mention only in the body, and Revert "Title (#12)" (#200).

    2. Merge commit (method: merge_commit). This step runs only when the caller gives the PR's merge commit sha, meaning its merge or squash commit upstream. It is one call (inclusion.py:143-145): git rev-list --missing=print --no-merges --cherry-pick --right-only <head>...<merge> ^<merge>^1 --. That call lists each non-merge commit the merge brought in that <head> has neither by sha nor by patch-id.

      • An empty list is included. That covers a merge commit that is an ancestor, and also a cherry-pick with an identical diff under any subject.
      • A non-empty list is unknown. The evidence counts the missing commits and names one of them.

      There is no merge-base --is-ancestor call and no shallow-repository check. An ancestor merge commit gives an empty list from this same call. A shallow clone just has less history to read: in a clone cut through a two-commit PR, the probe answered unknown.

    Every other outcome is unknown, with the reason in the evidence. That covers three cases: no subject matched and no sha was given; some commits are not carried; a git call failed.

    Nothing is fetched. The checked commit and the merge commit are read through rev-list --missing=print, which turns lazy fetching off.

    • In a partial clone, a merge sha that was never fetched makes the call fail, and the answer is unknown.
    • A patch-id also needs the blobs of both sides. A blob-filtered clone that lacks them answers unknown from the merge-commit step instead of fetching them.

    The object store is left as it was.

    Error handling:

    • Every git call goes through the shared kernelforge.llm.git.git_bytes runner, with safe_directory_args, and its exit status is checked.
    • A non-zero exit, a timeout, or a git that cannot start is caught in one place, probe_inclusion. The answer is then unknown with the error as the evidence. For a non-zero exit, that is git's own message.
    • At most three calls run: resolve, subject search, and merge commit. Each is bounded at 15 s (_GIT_TIMEOUT_SEC = 15.0, inclusion.py:57), so a probe ends inside the 60 s the Forge agent gives one pr_monitor tool call.
    • The revision to check reaches git after --end-of-options. A value starting with - is therefore a bad revision, never an option. Without this, rev-list would accept --output=<file> and create the file.
    • Output is read as bytes and decoded with replacement. A subject in a non-UTF-8 log encoding therefore neither crashes the probe nor hides its suffix.

    Two surfaces:

    • MCP tool. The PR stdio MCP server, kernelforge.knowledge.pr_knowledge.mcp_server, gets a fourth tool, pr_check_inclusion. Its input schema is {number: integer, tree: string, commit?: string}, with number and tree required (mcp_server.py:104-123). It answers from subjects alone: _check_inclusion always passes merge_commit_sha=None (mcp_server.py:272). That is because nothing the Forge Implementer can call returns a merge commit sha: pr_get_reference carries the state and merge time, not the merge commit.
      • The schema has no merge_commit_sha. The server does not reject an extra merge_commit_sha argument; it ignores it and gives the subject-only answer.
      • The name is added to agent_tools.TOOL_NAMES, which the Forge agent also reads to register mcp__pr_monitor__pr_check_inclusion.
    • Command line. python -m kernelforge.knowledge.pr_knowledge.inclusion --tree PATH --commit SHA --pr N [--merge-commit-sha SHA].
      • It prints one JSON object with inclusion, method and evidence.
      • It exits 0 for included, 3 for unknown and 2 for a usage error (inclusion.py:163,176). There is no other verdict exit code.
      • It is the only surface that takes the merge sha. Its consumer is the check-prior-art contributor skill in feat(skills): add check-prior-art, a contributor skill that finds a planned change upstream #1850, which is stacked on this PR. That skill reads the PR's merge commit (mergeCommit) with gh pr view and passes it as --merge-commit-sha.

    The new tool reads the PR number through a shared _pr_number parser. The two existing numbered tools now use it too.

    Known limits, stated in the module docstring:

    • included means the PR's change is in the history. The probe does not look for a later revert.
    • A subject carries only the number, so a fork that numbers its own PRs the same way can collide. The evidence names the matching commit and its subject, so a caller can compare titles.
    • A patch-id matches only a diff that is identical apart from whitespace and line numbers.
    • For a rebase merge, the merge commit sha is the PR's last rebased commit, so only that commit is compared.

    Cost was measured on a full clone of the vLLM repository (23,448 commits on main), with the probe at this head. Each time covers the whole python -m run, as the median of three runs:

    • tag v0.12.0 checked for #29893, which its release branch cherry-picked, by subject: included (subject) in 0.13 s;
    • the same tag with a PR number that no subject names and no sha, which searches every subject: unknown in 0.13 s;
    • the same tag with the main squash commit of #29893 and the number withheld: included (merge_commit) in 0.14 s;
    • main checked against the release branch's cherry-pick of #29893, with the number withheld. This walks 11,610 diverged commits and gives included (merge_commit, by patch-id) in 0.65 s;
    • the tag checked against a later main commit: unknown in 0.15 s.

    The derive_kernel_names move that earlier revisions of this PR were stacked on has landed on main as refactor(kb_store): move derive_kernel_names into kb_store.identity.kernel_names #2028. This PR no longer carries it and does not touch kb_store or .importlinter.

  • Linked issue(s): none.

  • Tests: added/updated? commands run? (what to test; if a test was replaced, where its assertions live now):

    The new src/kernelforge/tests/knowledge/pr_knowledge/test_inclusion.py has 17 test functions that collect as 33 cases. They build temporary git repos and pin:

    • subjects:
      • these are included: (#N), (#N) (#M), Merge pull request #12 from ..., and a subject with a line separator before the suffix;
      • these are each unknown: #123 and #112 against 12, Merge pull request #123 from ... against 12, a mid-subject mention, a revert, and a body-only mention;
    • a subject in a non-UTF-8 i18n.logOutputEncoding still matches;
    • merge commit:
      • an ancestor merge commit is included (merge_commit);
      • a cherry-pick under a rewritten subject is included by patch-id;
      • under an upstream two-parent merge of a two-commit PR, all commits carried is included. One carried or none carried is unknown, with the count in the evidence;
    • a merge sha git cannot find: git exits 128, the answer is unknown, and the evidence carries git's message;
    • a git call that hangs: unknown, with the timeout in the evidence;
    • a bad revision, and a leading-dash revision (--output=<file>): unknown, and no file is created;
    • a directory outside git: unknown;
    • a --filter=blob:none clone of a file:// upstream that serves any sha:
      • a merge sha the clone lacks is unknown, and a present one is included;
      • a patch-id that needs blobs the clone lacks is unknown;
      • in both tests, the object store is unchanged;
    • the command line:
      • JSON keys, and exit 0 or 3 with --merge-commit-sha;
      • exit 2 for a missing --pr or a --tree that is not a directory;
      • exit 3 for a --tree git cannot enter (mode 000, where git cannot start);
      • one real python -m run;
    • the MCP tool: the happy path (commit defaults to HEAD, and neither the service nor PR_KB_REPO is needed), and invalid arguments.

    Updated pins:

    • test_mcp_server.py: tool names, and a tool count of 4.
    • test_pr_kb_injection_point.py: the Forge agent registers mcp__pr_monitor__pr_check_inclusion.

    test_stdio_tool_schemas.py is not changed. It parametrizes over every server's TOOL_DEFINITIONS, so it now checks the new tool's schema without an edit.

    Commands, at this head with a clean environment:

    • pytest src/kernelforge/tests/knowledge src/kernelforge/tests/test_stdio_tool_schemas.py src/kernelforge/tests/test_forge_campaign_config.py: 710 passed.

    • pytest scripts/tests/test_check_cli_references.py scripts/tests/test_code_metrics_design.py: 115 passed.

    • pytest -n 16 src/kernelforge/tests: 5970 passed, 19 skipped, 5 failed. The same 5 fail identically on main (5936 passed) in this environment. The 34 extra passes are the 33 new cases and the new tool's case in test_stdio_tool_schemas.py. The 5 failures are:

      • two test_rocpc_profile_discovery tests that find a host ROCm install;
      • three tests that assume git's default branch is master: two in test_worktree.py, and test_knowledge_integration::test_git_checkout_branch_creates_and_switches_branch.

      test_opportunity_agent::test_hookless_resume_republication_clears_a_corrected_refusal fails intermittently on both main and this head in this environment, so a run can show it as a sixth failure on either.

    • Each of these edits to the probe makes at least one test_inclusion.py case fail:

      • ignoring git's exit status;
      • dropping OSError from the one except;
      • dropping the Merge pull request match;
      • splitting with splitlines();
      • dropping --no-merges, ^<merge>^1, --cherry-pick or --end-of-options;
      • decoding strictly.

    Gates:

    • ruff check and ruff format --check on the changed files: clean.
    • lint-imports: 10 kept, 0 broken.
    • python scripts/code_metrics.py --base-ref origin/main: PASSED, 7 files judged.
    • python scripts/check_cli_references.py: exit 0.
    • python scripts/check_broad_suppress.py <changed .py>: exit 0.
  • Size/complexity triggers crossed: split, or why the shape is right: none.

    • The diff is 7 files, +595/-20. pr_hygiene.py size counts 253 production Python lines (size/L).
    • inclusion.py is 180 lines, including a 38-line module docstring. test_inclusion.py is 351 lines.
    • Every new function has a cyclomatic complexity of at most 3 (_subject_naming, _merge_commit_carried) and a cognitive complexity of at most 5 (_subject_naming).
    • The shared _pr_number parser changes two existing functions:
      • _get_reference: cyclomatic 3 to 2, cognitive 4 to 3;
      • _get_file_patch: cyclomatic 5 to 4, cognitive 5 to 4.
  • If this simplifies or refactors: which mechanism was removed, which contract was preserved, and the tests that show it: the two existing numbered tools, pr_get_reference and pr_get_file_patch, now read number through the shared _pr_number parser. Their error message ("number must be an integer") and the number schema ({"type": "integer"}) are unchanged. test_mcp_server.py passes with only the tool-name and tool-count pins changed.

  • Observable effect: what an operator will now see, and what the old behaviour cost them (if none, why the change is unobservable):

    • The Implementer gets a fourth tool, mcp__pr_monitor__pr_check_inclusion{number, tree, commit?}, in a Forge run that has --pr-kb, a resolved repository and an MCP-capable provider.
      • It returns {number, tree, commit, inclusion, method, evidence} from local git, without the PR Monitor service.
      • inclusion is included or unknown. No answer is ever not_included.
      • method is subject for an included answer and none for unknown. The tool never gives merge_commit, because it takes no merge sha.
      • A caller that sends merge_commit_sha gets the subject-only answer; the argument is ignored.
      • Its description tells the agent that unknown never means absent, and to look for the change itself before treating the PR as missing.
    • There is a new command, python -m kernelforge.knowledge.pr_knowledge.inclusion --tree PATH --commit SHA --pr N [--merge-commit-sha SHA]. It prints the same inclusion / method / evidence object as JSON.
      • With a merge sha, method can also be merge_commit.
      • It exits 0 for included, 3 for unknown and 2 for a usage error. A script should treat 3 as "not shown", never as "absent".
      • No verdict uses exit 1, the status of an uncaught Python error, so a crash never reads as a verdict.
    • A blob-filtered partial clone answers from the objects it has. A merge-commit check that needs missing blobs is unknown, and nothing is fetched.
    • docs/kernelforge/reference/cli.md (--pr-kb) names the four tools, states the two-valued answer, and names the command line's consumer.
    • Before this change, no agent or operator had a mechanical answer. A PR already in the tree could be applied again and benchmarked again as if it were new.
  • Breaking changes: no. The three existing MCP tools keep their names, schemas and errors.

  • PR addresses single concern: yes. The change is the inclusion probe and its two surfaces, the MCP tool and the command line. The derive_kernel_names move it used to depend on is already on main as refactor(kb_store): move derive_kernel_names into kb_store.identity.kernel_names #2028.

  • Root cause is upstream (Magpie/TraceLens/GEAK/IntelliKit/AgentKernelArena), ticket filed: no; not applicable.

@okorzh-amd
okorzh-amd requested a review from a team as a code owner October 13, 2026 05:25
@rpoornac
rpoornac requested a review from chaojhou October 13, 2026 19:04
@okorzh-amd
okorzh-amd force-pushed the okorzh/pr-inclusion-probe branch from b6b9633 to 05c2e84 Compare October 13, 2026 19:38

@chaojhou chaojhou left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @okorzh-amd. These parts hold:

  • The git calls are safe: argument lists, ^{commit} --, a refused leading -, and 30 s timeouts.
  • A shallow clone or a missing sha ends as unknown.
  • The three .importlinter lines removed are cycle-breaker ignores whose imports still exist, so removing them tightens the contract.

One must-fix:

not_included is reported for a PR that is in the tree when it arrived through a commit without the (#N) suffix.

  • probe_inclusion returns not_included when the merge sha is present but not an ancestor and no subject ends in (#N) (inclusion.py:119-120).
  • Two cases get a confident not_included (both reproduced on scratch repos):
    • a fork cherry-picks the upstream squash commit and rewrites the subject ([ROCm] Port ...);
    • the upstream uses merge commits, and the fork cherry-picks the PR's commit.
  • The tool description says not_included is withheld only when the merge sha is missing, so the Implementer will trust it and re-apply a change the tree already has. ROCm/vllm → vllm-project/vllm is already a target in FORK_UPSTREAM_MAP.
  • Please confirm absence with a patch-id check (git cherry or git log --cherry-pick, without fetching) before answering not_included, and answer unknown otherwise. At minimum, state the limit in the tool description and the module docstring.

pr-hygiene will warn at 452 production lines (cumulative with #1846). CI has not run yet; the fork workflows are awaiting approval.

@github-actions

github-actions Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Gate job conclusion: success (run) on 005924d3a5a926a7c8dea470f0eb6d74f26dc727, as reported by GitHub.

The report below was uploaded by that run; the conclusion above is the result.


Code metrics gate: PASSED

Judged: the 7 files this change touches. Files it does not touch never fail it; anything found there is listed under Outside this change for information.

Dimension Limit New Worse Baseline out of date Baseline entries vs base
Cyclomatic complexity fails > 20 0 0 0 59 +0
Cognitive complexity fails > 30 0 0 0 245 +0
Function length (lines) fails > 80 0 0 0 390 +0
Nested block depth fails > 5 0 0 0 24 +0
Parameter count fails > 5 0 0 0 95 +0
Module length (lines) warns > 800, fails > 1200 0 0 0 50 +0
Duplicated lines per file fails > 0 0 0 0 96 +0
Dead code (vulture) fails > 0 0 0 0 2 +0
Check Findings Effect
Comments 0
English only 0
Production code imports test code 0
Repeated literals 0
A1 Private name of another package 0
A8 Flag argument 0
A9 Tuple return 0
A10 Raw vocabulary comparison 0
A11 Hidden global write 0
A12 Pure forwarder 0
A12 Single-implementation abstraction 0
116 findings in files this change does not touch

Outside this change (informational): 116

Unit Dimension Baseline Now
src/hyperloom/orchestrator/state/shared_state.py resolve_graded_comparison Cognitive complexity absent 34
src/hyperloom_kb/collect/mapping.py _experience Cognitive complexity absent 31
src/hyperloom_kb/schema.py FieldDeclaration.__post_init__ Cognitive complexity absent 32
src/kernelforge/cli/forge_loop_command.py _prepare_measurement_driver Cognitive complexity absent 36
src/kernelforge/cli/forge_loop_command.py _read_kb_warm_start Cognitive complexity absent 40
src/kernelforge/cli/forge_loop_command.py forge_loop Cognitive complexity absent 31
src/kernelforge/loop/runner.py IterationLoop._collect_session_candidate Cognitive complexity absent 44
src/kernelforge/loop/runner.py IterationLoop._rule_on_attempt Cognitive complexity absent 38
src/hyperloom/inference_optimizer/cli/__init__.py _reexport_session_environment Function length (lines) absent 111
src/hyperloom/inference_optimizer/cli/__init__.py _resolve_agent_lineup Function length (lines) absent 89
src/hyperloom/inference_optimizer/cli/__init__.py _resolve_run_topology Function length (lines) absent 92
src/hyperloom/inference_optimizer/cli/__init__.py _resume_session Function length (lines) absent 170
src/hyperloom/inference_optimizer/cli/__init__.py _seed_fresh_session Function length (lines) absent 170
src/hyperloom/inference_optimizer/multi_node/cli.py _restart_multinode Function length (lines) absent 107
src/hyperloom/inference_optimizer/multi_node/scripts/launch_multinode.py _build_parser Function length (lines) absent 84
src/hyperloom/inference_optimizer/multi_node/scripts/launch_multinode.py _spawn_rank_actors Function length (lines) absent 84
src/hyperloom_kb/sync.py GlobalSync.pull Function length (lines) absent 84
src/kernelforge/cli/forge_loop_command.py _RunReporter.build_result Function length (lines) absent 92
src/kernelforge/cli/forge_loop_command.py _prepare_measurement_driver Function length (lines) absent 136
src/kernelforge/cli/forge_loop_command.py _read_kb_warm_start Function length (lines) absent 149
src/kernelforge/loop/runner.py IterationLoop._close_iteration Function length (lines) absent 117
src/kernelforge/loop/runner.py IterationLoop._collect_session_candidate Function length (lines) absent 185
src/kernelforge/loop/runner.py IterationLoop._implementer_history Function length (lines) absent 112
src/kernelforge/loop/runner.py IterationLoop._rule_on_attempt Function length (lines) absent 102
src/kernelforge/loop/runner.py IterationLoop._source_attempt Function length (lines) absent 175

...and 91 more; the job log lists every row.

Verdict computed by the gate scripts in /home/runner/work/_temp/code-metrics-gate, not by this tree's copy.

Thresholds and sources: [tool.hyperloom.code_metrics] in pyproject.toml. Tools: complexipy 8.0.1, jscpd 5.4.1, ruff 0.16.2, vulture 2.16.

Comment thread src/kernelforge/tests/knowledge/pr_knowledge/test_inclusion.py Fixed
@okorzh-amd
okorzh-amd requested a review from chaojhou October 14, 2026 18:31

@chaojhou chaojhou left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR #1847 -- feat(pr_knowledge): probe whether an upstream PR is in a local git tree

What it does: Adds a local-git probe that answers whether upstream PR N is already in a tree at a given commit (merge-commit ancestry, then a trailing (#N) subject suffix, then patch-id against the commits the merge brought in), exposed as the MCP tool pr_check_inclusion and as python -m kernelforge.knowledge.pr_knowledge.inclusion; it also moves derive_kernel_names into kb_store.identity.kernel_names to break the knowledge / mcp_server import cycle.

Blocking issues: 1

  1. [X2] The PR description does not describe the patch-id stage present at the current head [verified]
    Problem: The body says not_included needs only a not-ancestor answer and no subject match, and lists the methods as merge_ancestry / subject_suffix / none. At head the probe adds a patch-id stage that can return included or unknown where the description promises not_included, and adds the method value patch_id. The body also gives 237 lines and 43 inclusion tests, the counts at 05c2e84; at head the module is 291 lines and test_inclusion.py holds 58 cases. src/kernelforge/knowledge/pr_knowledge/inclusion.py:236
    Impact: Operators read the wrong decision rule for an operator-visible verdict (CLI exit code 4 vs 3, MCP inclusion and method fields).
    Action: Update the description: add the patch-id stage, the patch_id method, the multi-commit unknown case and the remaining not_included limit, and correct the line and test counts.

SKIPPED: V4 -- PARTIAL overlap for #1032, #1154, #1276, #1675 (each cut at its first 100 files), overlap there unchecked

Checked: inclusion.py (probe_inclusion, _merge_ancestry, _subject_suffix, _patch_equivalence, main), pr_stdio_server.py (_check_inclusion, _pr_number, pr_check_inclusion schema and description), kernel_names.py rename, implementation.py, source_resolution.py, campaign_config.py, .importlinter, cli.md, test_inclusion.py, ci.txt | Ran: python ..\pr_tools\hl_fetch.py 1847; python ..\pr_tools\c901.py 1a444f6 186ff29 inclusion.py pr_stdio_server.py implementation.py source_resolution.py (max 9); probe_inclusion at head on scratch repos (squash with rewritten subject -> included/patch_id, merge-commit pick -> included/patch_id, edited backport -> not_included/patch_id); gh run list --commit 186ff29 after approving the fork runs (9 workflows success, the 2 label-gated E2E skipped) | Base: 1a444f6 | Head: 186ff29

@chaojhou chaojhou left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR #1847 -- feat(pr_knowledge): probe whether an upstream PR is in a local git tree

What it does: Adds a local-git probe (kernelforge/knowledge/pr_knowledge/inclusion.py) that answers whether upstream PR N is already in a tree at a given commit -- merge-commit ancestry, then a trailing (#N) subject suffix, then patch-id against the commits the merge brought in -- exposed as the MCP tool pr_check_inclusion and as a CLI. It also moves derive_kernel_names into kb_store.identity.kernel_names to break an import cycle, and carries #1846's PR-state fields.

Blocking issues: 1

  1. [X2] The description still describes the probe before its patch-id stage [verified]
    Problem: unchanged since the last round at this head. The body gives the not_included rule as "a definite not-ancestor answer and no subject matches", lists the methods as merge_ancestry / subject_suffix / none, and says "43 tests" and "237 lines". At head the probe adds a patch-id stage that can answer included or unknown where the description promises not_included, adds the patch_id method, and the module is 291 lines with 58 cases in test_inclusion.py. src/kernelforge/knowledge/pr_knowledge/inclusion.py:236
    Impact: operators read the wrong decision rule for an operator-visible verdict (CLI exit 4 vs 3, MCP inclusion and method).
    Action: rewrite the body to the rule that finally lands -- preferably the two-valued one below.

Design review (requested by the maintainer; beyond the blocking bar)

The aim of this section is a smaller PR, not more guards: each point says what to delete or merge.

Purpose, route, files. Purpose: let the Forge Implementer tell mechanically whether an upstream PR is already in the tree it optimizes, instead of re-applying it or guessing from version strings. Route: probe_inclusion (inclusion.py:120-143) over kernelforge.llm.git.git, the MCP tool in pr_stdio_server.py, a CLI in inclusion.py:266-287. Files: inclusion.py (new), pr_stdio_server.py (the tool and a shared _pr_number), the kernel_names move and its three importers, .importlinter (three cycle-breaker ignores removed), cli.md, test_inclusion.py (58 cases), tool-name and schema pins, and #1846's lifecycle fields.

1. Completeness. Nothing in the tree produces merge_commit_sha: it appears only in inclusion.py and as a pass-through argument of pr_check_inclusion (pr_stdio_server.py:128, :304), and pr_get_reference does not return it. So for the only consumer -- the Implementer through MCP -- merge ancestry, patch-id and not_included never run; it gets included by subject suffix or unknown. An upstream that records the PR as Merge pull request #N from ... also gets unknown, because the suffix rule ignores that subject.

2. Simpler / smaller. Local history can prove presence (ancestry, patch-id, a squash suffix) but never absence: an edited backport, a folded commit or a conflict-resolved pick all look absent. The right tool is therefore two-valued -- included with evidence, or unknown with a reason. That removes not_included, _is_shallow, the --count call, the one-commit and multi-commit branches, exit code 4 and the tests that pin a known-wrong not_included. The patch-id stage exists because my earlier must-fix asked to confirm absence before answering not_included; with not_included gone it is not needed. If a merge sha is ever supplied, one git rev-list --missing=print --no-merges --cherry-pick --right-only <head>...<merge> ^<merge>^1 -- reproduces the probe's included set and fetches nothing for a missing sha, which also makes _merge_ancestry and _merge_unavailable redundant. The CLI has no in-repo caller. The StrEnum shim copies agents/quantization/driver/outcomes.py although to_record reads .value, and #1846's own PRState is a plain (str, Enum). The kernel_names move is independent and can land on its own.

3. Redundancy and silent failure. Git failure is handled in six places, four of them as separate "no answer" and "non-zero exit" branches (inclusion.py:169-170, 186-193, 203-204, 210-213, 239-242, 248-251); one error raised by the git wrapper and caught once in probe_inclusion covers them all. No failure becomes a definite verdict (timeouts, exit 128 and shallow history end unknown), but the evidence for a timed-out resolve reads "is not in the tree". Seven 30 s git calls can outlast the 60 s tool timeout.

4. Docs and comments. The docstrings, the tool description and cli.md match the code; the PR body does not (item 1, plus "read only commits": --cherry needs blobs, so a blob-filtered clone answers unknown).

5. Tests. About 20 of the 58 cases carry the behaviour: the (#N) and (#N) (#M) hits; misses for a prefix, a suffix, mid-subject, a revert and body-only; non-UTF-8 output; no match gives unknown; a bad and a leading-dash revision; a directory outside git; one git failure; the MCP happy path and invalid arguments. Redundant: the ten-case git-failure matrix (one hang and one exit-128 case suffice; it picks the failing call by argv token, so merging two git calls breaks it with no behaviour change), the cases that pin a known-wrong not_included (test_inclusion.py:172-199), the schema test that restates the schema literal, and the CLI tests if the CLI goes.

Starting from zero. About 70-90 lines: probe_inclusion, a git wrapper that raises one error, the existing _resolve, and a subject matcher that also recognises merge-commit subjects; add the single cherry-pick call only once something supplies a merge sha.

Change list, in order of impact:

  1. Drop not_included: delete _is_shallow, the --count call, the one-commit and multi-commit branches, exit code 4 and the tests pinning it. #1850's Steps 5 and 6 then attack every answer that is not included.
  2. Drop the merge-sha path until a producer exists, or keep it as the one cherry-pick call; either way delete _merge_ancestry and _merge_unavailable.
  3. One git error path; cut the failure matrix to two cases.
  4. Delete the CLI and its cli.md sentence, or name the consumer that needs it.
  5. Recognise Merge pull request #N from ... subjects, with one test.
  6. Replace the StrEnum shim with (str, Enum).
  7. Trim test_inclusion.py to the cases listed in point 5; drop the schema restatement.
  8. Land the kernel_names move as its own PR.
  9. Rewrite the body, the tool description, the module docstring and cli.md to the final rule (item 1).

Checked: inclusion.py (probe_inclusion, _merge_ancestry, _merge_unavailable, _subject_suffix, _patch_equivalence, main), pr_stdio_server.py (pr_check_inclusion schema, _check_inclusion, pr_get_reference), the description against the head; tree-wide git grep for every non-test writer of merge_commit_sha (none besides the probe and the tool's pass-through) | Ran: git grep sweeps; no test suite | Base: 1a444f6 | Head: 186ff29

@chaojhou chaojhou left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR #1847 -- feat(pr_knowledge): probe whether an upstream PR is in a local git tree

What it does: The PR adds kernelforge.knowledge.pr_knowledge.inclusion, which uses local git only to check whether upstream PR N is already in a tree. The answer is two-valued: included when a reachable commit subject names the PR (a trailing (#N) run or "Merge pull request #N from"), or when a supplied merge sha's non-merge commits are carried by sha or patch-id; otherwise unknown. It is exposed as a fourth pr_monitor MCP tool (pr_check_inclusion, subjects only) and as a python -m CLI (exit 0 or 3). The PR also moves derive_kernel_names into kb_store.identity.kernel_names, which removes the knowledge/mcp_server import cycle and its three .importlinter ignore lines.

Blocking issues: 1

  1. [X2] The PR description still describes the superseded three-valued probe, not the code at head bab0be7 [verified]
    Problem: The live PR body (re-fetched with gh at head bab0be7; identical to body.txt) describes a probe that no longer exists. (1) It says the probe answers not_included when complete history gives a definite not-ancestor and no subject matches, but Inclusion at inclusion.py:63-67 has only INCLUDED and UNKNOWN. (2) It names the methods merge_ancestry / subject_suffix / none, but InclusionMethod at :70-75 is subject / merge_commit / none. (3) It gives CLI exit codes 0 / 4 / 3, but main returns only 0 or 3 (:176, epilog :163). (4) It says the MCP tool takes {number, tree, commit?, merge_commit_sha?}, but the pr_check_inclusion schema in pr_stdio_server.py has no merge_commit_sha, and _check_inclusion always passes merge_commit_sha=None. (5) It describes merge-base --is-ancestor plus rev-parse --is-shallow-repository, but the code makes one rev-list --cherry-pick --right-only call (:143-145) with no shallow check. (6) It says the timeout is 30 s; :57 sets 15.0. (7) It says the enums use the StrEnum shim; they are (str, Enum). (8) It gives 237 lines and 43 tests; the module has 180 lines and test_inclusion.py 17 test functions. (9) It lists test_stdio_tool_schemas.py as updated; this PR does not touch that file. (10) It never states that Merge pull request #N from subjects are recognised. Two earlier review-pr cards (2026-10-14 and 2026-10-15) raised this X2 point. The code was reworked along the design review, but the body was never rewritten. src/kernelforge/knowledge/pr_knowledge/inclusion.py:63
    Impact: Operators would learn a wrong contract for an operator-visible verdict. They would script against exit 4 / not_included, which can never occur. They would expect the MCP tool to take merge_commit_sha, which it rejects, and would trust method names it never emits. The release notes would carry the wrong behaviour.
    Action: Rewrite the description to match the head. Cover the two-valued included/unknown rule and why absence is never reported, the subject rule including Merge pull request #N from, the merge-sha path via rev-list --cherry-pick by sha or patch-id, method values subject/merge_commit/none, exit codes 0/3/2, the MCP schema without merge_commit_sha, and the 15 s timeout. Correct the file and test counts. Drop the test_stdio_tool_schemas claim. State that the kernel_names move is also open as #2028.

Checked: Read the review-pr SKILL.md. I did not reread the rules.md index or references/tiers.md, so no rule-by-rule verdict file was written. Read at head: inclusion.py (all of it), the full diff of every other changed file (pr_stdio_server.py, kernel_names rename, implementation.py, source_resolution.py, campaign_config.py, .importlinter, cli.md, test_writer_helpers.py, the injection-point and stdio-server pins), and the test names in test_inclusion.py. Also read the call sites in agent.py where pr_monitor is registered (tool_timeout_sec=60), and llm/git.py and git_safety.py at origin/main and at base b7004d1. Re-fetched the live PR body with gh and confirmed it is the same as body.txt. Ran git merge-tree origin/main against origin/pr-1847 (clean), checked the base..main log on the touched paths (8 commits, none touching them), ran gh pr view 1850 to look up the check-prior-art skill, and read the earlier review cards in comments.txt. Exported the head tree with git archive to a local script and ran a local script (C:/Users/chaojhou/AppData/Local/Temp/hl-ok-1847/scripts/probe.py) with the a local script Python. It checks 15 scratch-repo scenarios plus the CLI, and all came out as the co | Base: b7004d1 | Head: bab0be7

okorzh-amd and others added 10 commits October 20, 2026 12:39
Nothing answered "is upstream PR N already in this tree or commit?", so an
agent could take up a PR that its framework tree already carries.

probe_inclusion reads local git only. Revisions resolve through rev-list
with a --missing action, so a sha that a partial clone lacks is not
fetched. It checks ancestry of the PR's merge commit when that commit is
present, then looks for a reachable commit whose subject ends in a (#N)
suffix run, which is how squash merges and fork cherry-picks record the
PR. It answers not_included only when a complete history gives a definite
not-ancestor answer and no subject matches. Every other case, including a
git call that fails, cannot start or times out, is unknown, with the
reason in the evidence. Each git call is bounded by a timeout.

The verdict is available as python -m kernelforge.knowledge.pr_knowledge.inclusion
(JSON on stdout; exit 0, 4 or 3 for included, not included or unknown, 2 on
a usage error; 1 stays the status of an uncaught error) and as the
pr_check_inclusion tool of the PR stdio MCP server, whose commit defaults
to the tree's HEAD.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The stub that fails one git command was a lambda wrapping a call, which
CodeQL reports as an unnecessary lambda. A nested function does the same
job without the wrapper.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ncluded

probe_inclusion answered not_included whenever the merge commit was
present but not an ancestor and no subject ended in (#N). A PR the tree
does carry read as absent when a fork cherry-picked the squash commit
under a rewritten subject, or, under an upstream that merges with merge
commits, cherry-picked the PR's own commits.

Before answering not_included, the probe now compares the non-merge
commits the merge commit brought in (merge^1..merge: the squash commit
itself, or the PR's commits under a merge commit) with the commits
reachable from the checked commit, using git rev-list --cherry. A commit
counts as carried when the tree has it by sha or by patch-id. All carried
is included, through the new method patch_id; none is not_included; some
is unknown, and so is a call that fails or times out.

Both calls run as rev-list with --missing=print, which turns lazy fetching
off, so a partial clone that lacks a blob a patch-id needs answers unknown
and fetches nothing. A backport without the suffix whose diff was edited
still reads as not_included; the module docstring and the MCP tool
description now say so, and a test pins it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… is carried

Under an upstream that merges with merge commits, the patch-id step
compares each of the PR's commits on its own. A fork that carries a
multi-commit PR folded into one commit, with git cherry-pick -m 1 of the
merge or by squashing the PR's commits, matches none of their patch-ids,
and the default cherry-pick -m 1 subject "Merge pull request #N from ..."
has no (#N) suffix. Such a tree read as not_included, exit 4.

When the merge brought in several commits and none is carried one by
one, the probe now answers unknown, and the evidence names the fold. A
squash or rebase merge brings in one commit and still answers
not_included when that commit has no patch-id match.

A patch-id of the merge's first-parent diff would match the fold, but
computing it needs git diff-tree -p or git log -p. Neither takes
--missing, and only git 2.44 and later can stop them lazily fetching
blobs in a partial clone (GIT_NO_LAZY_FETCH), while the probe must
fetch nothing.

Tests: the none-carried case of the two-commit merge test is now
unknown, and git cherry-pick -m 1 of the merge is included for a
one-commit PR and unknown for a two-commit PR.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The module docstring said the patch-id step finds a cherry-pick under
any subject, and the pr_check_inclusion description named only an
edited backport as a miss. A patch-id hashes the diff with its context
lines and compares one commit at a time, so a backport with no (#N)
suffix that git applied cleanly still reads as not_included when the
fork had changed a line next to a hunk, and so does one that carries a
squash-merged PR as the PR's own commits picked one by one.

The docstring now says a patch-id matches only a commit whose diff,
whitespace and line numbers aside, is identical to the commit the merge
brought in, and names the misses: an edit or a resolved conflict, a
change landed next to fork changes, a squash-merged PR carried as its
commits, a fold into a larger commit, and a fork that squashed a
rebase-merged PR. The MCP tool description, which is what an agent
reads, says the same in short. rev-list --cherry cannot compare with
less context: it rejects -U0 and ignores diff.context.

A new test pins the clean backport next to a fork change as
not_included.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Inclusion and InclusionMethod used a StrEnum with a Python 3.10 shim
copied from the quantization driver, but nothing formats them with
str(): to_record reads .value, and the PR-state enum next to them is a
plain (str, Enum). They are now the same, and the shim is gone.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Local history can show that a PR is in a tree, never that it is absent:
a backport that was edited, had a conflict resolved or was folded into
a larger commit looks the same as no carry. The probe still answered
not_included (exit 4) for such trees, so a caller could take up a PR the
tree already carries.

The probe now answers included, with the git evidence, or unknown, with
the reason. not_included, exit status 4, the shallow-history check and
the one-commit and several-commit branches are gone.

With a merge commit sha, one call,

    git rev-list --missing=print --no-merges --cherry-pick --right-only \
        <head>...<merge> ^<merge>^1 --

replaces the ancestry check, the commit count and the --cherry pass. It
lists the non-merge commits the merge brought in that head has neither
by sha nor by patch-id, so an empty list is included: for a merge
commit that is an ancestor, and for a cherry-pick under another
subject. A sha the tree lacks fails the call, and nothing is fetched.
Subjects are now searched first, because they need no sha.

Every git call goes through the shared wrapper with its exit status
checked. A non-zero exit, a timeout or a git that cannot start is caught
once in probe_inclusion and answered unknown with the error as the
evidence (git's own message for a non-zero exit), so a timed-out
resolve no longer reads "is not in the tree". At most three
calls run, each bounded at 15 s, inside the 60 s the Forge agent gives a
pr_monitor tool call. The revision to check reaches git after
--end-of-options: rev-list accepts --output=<file> and creates that
file, so a leading-dash value is now a bad revision, not an option.

The methods are now subject, merge_commit and none. The module
docstring, the pr_check_inclusion description and cli.md state the rule,
and cli.md names the check-prior-art contributor skill as the consumer
that passes the merge commit sha on the command line.

Tests keep the subject hits and misses, non-UTF-8 output, the
merge-commit cases, the partial-clone no-fetch cases, one hang, one
exit 128, a bad and a leading-dash revision, a directory outside git,
the command line and the MCP tool. The not_included cases, the
ten-case failure matrix and the schema restatement test are gone.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
An upstream that lands PRs with merge commits records the PR as
"Merge pull request #N from <branch>", and so does a fork's
git cherry-pick -m 1 of that merge. The subject search only knew the
trailing (#N) of a squash merge, so without a merge commit sha such a
tree answered unknown.

The search now also greps for that subject and counts a commit whose
subject starts with it for exactly N; #N followed by another digit, or
the phrase inside a revert subject, does not count. Two cases pin it:
the subject for AMD-AGI#12 is included, and the one for AMD-AGI#123 is not taken for
AMD-AGI#12.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…gent cannot supply

pr_check_inclusion took merge_commit_sha and passed it through, but no
tool the Forge Implementer can call returns one: pr_get_reference
carries the PR's state and merge time, not its merge commit. The
argument only ever arrived from tests, so the merge-commit check never
ran for the one consumer of the tool.

The tool now answers from commit subjects alone, and its schema and
description say so. The command line keeps --merge-commit-sha for the
check-prior-art contributor skill, which reads the sha from the GitHub
API.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…he probe

The module docstring said the check-prior-art skill passes the sha it
"reads from the GitHub API". The skill reads the PR's mergeCommit with
gh pr view, and avoids REST's merge_commit_sha, which names a test merge
for an open PR. Say which field it passes.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@okorzh-amd
okorzh-amd force-pushed the okorzh/pr-inclusion-probe branch from bab0be7 to 005924d Compare October 20, 2026 17:40
@okorzh-amd okorzh-amd changed the title feat(pr_knowledge): probe whether an upstream PR is in a local git tree feat(pr_knowledge): probe whether an upstream PR is in a local git tree Oct 20, 2026
@okorzh-amd

Copy link
Copy Markdown
Contributor Author

@chaojhou thanks for review. The description is now rewritten against 005924d, and it goes through your list item by item. It covers: the two-valued included/unknown answer and why absence is never reported (1); methods subject/merge_commit/none (2); exit codes 0/3/2 (3); the MCP schema {number, tree, commit?} without merge_commit_sha (4); the single rev-list --cherry-pick --right-only call, matching by sha or patch-id, with no ancestry or shallow check (5); the 15 s timeout (6); the (str, Enum) types (7); and the real counts (8): inclusion.py is 180 lines, and test_inclusion.py has 17 test functions that collect as 33 cases.
For (9), test_stdio_tool_schemas.py is now named only as a file this PR does not change; it picks up the new tool through its parametrization. For (10), the "Merge pull request #N from" subjects are now stated.
I rebased the branch onto main 7ac8c12. inclusion.py is byte-identical to bab0be7. #1907 moved the server to knowledge/pr_knowledge/mcp_server.py and TOOL_NAMES into agent_tools.py, so the new tool name is now added there. The derive_kernel_names commit is gone because it landed on main as #2028, and the PR no longer touches kb_store or .importlinter. That makes it 7 files, +595/-20, 253 production lines.
One correction to the review: the server does not reject an extra merge_commit_sha. The schema does not declare it, and _check_inclusion ignores it and gives the subject-only answer. The description now says exactly that.

@chaojhou
chaojhou dismissed stale reviews from themself October 22, 2026 07:38

Resolved at the current head; see the follow-up review comment.

@chaojhou
chaojhou dismissed stale reviews from themself October 22, 2026 07:38

Resolved at the current head; see the follow-up review comment.

@chaojhou

Copy link
Copy Markdown
Collaborator

PR #1847 -- feat(pr_knowledge): probe whether an upstream PR is in a local git tree

What it does: adds a probe that answers whether an upstream PR's change is already in a local git tree. It reports one of two answers, Inclusion.INCLUDED or Inclusion.UNKNOWN (inclusion.py:63-67), together with the method that decided it. The probe is exposed through the PR-knowledge MCP server and agent tools.

Blocking issues: none

The earlier blocker ([X2] at bab0be7: the description described the superseded three-valued probe) is resolved. The description now matches the two-valued Inclusion and the methods at head.

Checked: inclusion.py, mcp_server.py, agent_tools.py, cli.md and the three test files at 005924d. Every git call goes through git_bytes with safe_directory_args and a 15 s timeout, and is read-only (rev-list, log). The revision goes after --end-of-options, and --missing=print keeps the probe from fetching | Ran: git merge-tree --write-tree origin/main 005924d3a (clean); test_inclusion.py and test_mcp_server.py on the merged tree (79 passed, 1 skipped, 1 failed; the failure is a Windows-host encoding error in Path.write_text for the line-separator case, not the PR); CI approved and running at this head | Head: 005924d

LGTM

@chaojhou chaojhou left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving per the review comment above (resolved at 005924d; CI green).

@chaojhou
chaojhou merged commit 7db84af into AMD-AGI:main Oct 22, 2026
38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants