[okorzh-amd]

• Description: what and why

  Before someone builds an optimization for an upstream framework or kernel library, nothing in this repo checks whether the same change is already in an open or merged upstream PR, or already in the tree they measure. review-pr's overlap check starts from a PR that already exists and covers this repository's own open PRs. Duplicate work therefore shows up late: in upstream review, or as a "gain" that the baseline already had.

  This adds .claude/skills/check-prior-art/, a contributor skill in the style of review-pr. It is written for an agent to run top to bottom. Every step writes an artifact into a work directory, and a step that cannot run is recorded as SKIPPED. It reads GitHub through gh and jq and never posts to it.

  Everything below describes head 75d77c943, rebased onto main at 7ac8c1229 (through feat(pr_knowledge): probe whether an upstream PR is in a local git tree #1847's head 005924d3a).

  Inputs: a one-sentence mechanism, the upstream repositories, the search phrases, the planned paths in each repository's layout (a path ending in / covers a directory), and optionally a local tree and commit.

  Steps 0 and 1 write those inputs. Then:

  • Step 2, keyword search. search.sh sends one gh api search/issues --jq call per repository, state (is:open, is:merged) and phrase, with a fixed sleep 2 after each, which keeps under the search API's 30 queries a minute. It does not read rate-limit headers.

    • It appends one queries.tsv row per query as that query is answered (there is no pre-written or pending row): its status, how many PRs it listed and how many matched. The status is OK, TRUNCATED (more matched than the 100 listed), INCOMPLETE (GitHub timed out searching) or FAILED (with gh's error). A failed query does not stop the others.
    • Once, after the last query, it writes candidates.tsv: one row per PR, ranked by how many queries found it, then by most recent update.

  • Step 3, open PRs on the planned paths. overlap.sh compares every open PR of each repository that has a planned path.

    • It pages GraphQL 100 PRs at a time, up to 100 pages (10,000 PRs). The order is by creation, so a PR updated mid-scan cannot jump ahead of the cursor.
    • GraphQL lists at most 100 files per PR, and a renamed file by its new path only. A PR that this cannot settle is compared again on its REST file list, which names old paths and runs to 3,000 files. A PR that renames a planned file away shows as old->new.
    • overlap.tsv has one row per overlapping PR, with the planned paths it changes.
    • scan.txt gets a PARTIAL line for each PR whose overlap still cannot be decided. A repository gets its one line only when its scan ends, never before: COMPLETE; PARTIAL (the page cap, or a failed page, keeping what was already compared); or SKIPPED (no planned path there). An empty overlap.tsv means "none" only where scan.txt says COMPLETE.
    • With an upstream clone, git log --first-parent <commit>..<default branch> -- <paths> adds the merged PRs on those paths that the tree's commit lacks: each (#N) squash suffix and each Merge pull request #N from subject in it. --first-parent keeps a merge commit, where a path-limited log would list the PR's own commits, which name no PR.

  • Step 4, deep read. The shortlist takes every overlap row, every path-log PR and every keyword candidate. At most 12 are read; the rest stay in the shortlist as # cut: lines. read.sh makes four gh calls:

    • gh pr view --json into pr.json: state, draft flag, mergeCommit (null until the PR merges), body, comments and reviews;
    • the REST file list into files.txt;
    • gh pr diff into diff.txt;
    • the timeline's PR cross-references into xrefs.tsv.

    A diff that cannot be read, such as one GitHub refuses past 300 files, becomes a SKIPPED: diff line that says how many of the PR's changed files files.txt lists. Any other failure exits non-zero.

  • Step 4, classes. Each read PR takes the first row of one table that applies:

    • SUPERSEDED: closed, reverted, or a replacement is named;
    • DUPLICATE: its diff does the mechanism in the same code path;
    • OVERLAPPING: it changes a planned file or function, or carries part of the mechanism in any file, a sibling backend's included;
    • UNRELATED.

  • Step 5, inclusion. Each DUPLICATE, open or merged, is checked against the tree with python -m kernelforge.knowledge.pr_knowledge.inclusion. For a merged PR, the call passes mergeCommit.oid from pr.json; an open PR has none and is probed by subject alone. The probe prints one JSON object (inclusion, method, evidence) and answers only included (exit 0) or unknown (exit 3); exit 2 is a usage error and exit 1 a crash with no answer. There is no not_included: local git can show that a PR is in a tree, never that it is absent.

    • A subject answer can be a fork's own PR with the same number, so the commit the evidence names is checked against the PR's title: in its subject for a (#N) match, and in its body (git log -1 --format=%b), where GitHub writes the title, for a Merge pull request #N from match. A (#N) commit with another title still counts when it changes the PR's files, as a squash of a single commit keeps that commit's subject. Any other mismatch is recorded as unknown.
    • With no tree, or a probe that cannot run, the answer is a SKIPPED: inclusion line.

  • Step 6, refutation. Each DUPLICATE and OVERLAPPING class is attacked on mechanism, failure mode, operating point and revert, and ends as the first row of the class table it still meets. Each DUPLICATE's inclusion answer that is not included, an open PR's too, is attacked in the tree's own history with git log -S<symbol> and --grep=<title words>; so is a probe that could not run on a tree and commit. A hit whose diff is the PR's change makes the answer included. A SKIPPED: inclusion line with no tree or no commit stands.

  • Step 7, report. report.md holds one verdict, an evidence row per classified PR, Coverage, and the next step. Before writing it, the agent checks that each repository has a scan.txt line and two queries.tsv rows per phrase; a missing one did not run and goes on the report as SKIPPED.

  references/gotchas.md holds eight generic traps (G1-G8), each named from the steps that need it. Among them: PR and issue numbers share one counter per repository, and where GitHub writes a PR's title in a squash or a merge commit; a merged PR is not in an image built before it; and the inclusion probe matches a cherry-pick by subject, or by patch-id given the merge commit, but not an edited or folded backport, which is why Step 6 searches the log. AGENTS.md gains a row for the skill next to review-pr's.

  Stacked on "feat(pr_knowledge): probe whether an upstream PR is in a local git tree" (feat(pr_knowledge): probe whether an upstream PR is in a local git tree #1847), which provides the probe Step 5 runs. feat(pr_knowledge): probe whether an upstream PR is in a local git tree #1847 no longer stacks on anything: the derive_kernel_names move it used to carry landed on main as refactor(kb_store): move derive_kernel_names into kb_store.identity.kernel_names #2028, so this diff no longer has it. Until feat(pr_knowledge): probe whether an upstream PR is in a local git tree #1847 merges, this PR's diff also carries feat(pr_knowledge): probe whether an upstream PR is in a local git tree #1847's commits (7 files, 253 production Python lines). In short, as they stand at 005924d3a:

  • kernelforge/knowledge/pr_knowledge/inclusion.py answers included or unknown, with method subject, merge_commit or none. subject: a commit reachable from the checked commit whose subject ends in (#N) (alone or in a run such as (#N) (#M)) or starts with Merge pull request #N from ; (#N) anywhere else in the subject, or only in the body, does not count. merge_commit, only when a merge commit sha is given: rev-list --cherry-pick finds every non-merge commit that merge commit brought in already in the tree, by sha or by patch-id. Each git call has a 15 s timeout, and a git call that fails, cannot start or times out answers unknown with the error as the evidence. Nothing is fetched.
  • The pr_monitor MCP server (mcp_server.py, its name since refactor(kernelforge): reorganize evaluation, shared tools, MCP servers and the test root #1907) gains a fourth tool, pr_check_inclusion. It takes number (integer) and tree (string), both required, and an optional commit that defaults to HEAD. It answers from subjects alone, because nothing the agent can call returns a merge commit sha. It returns number, tree, commit, inclusion, method and evidence. docs/kernelforge/reference/cli.md describes it under --pr-kb.
  • Tests: test_inclusion.py (33 tests, new), test_mcp_server.py (the tool count goes from 3 to 4) and test_pr_kb_injection_point.py (the registered tool set gains mcp__pr_monitor__pr_check_inclusion). test_stdio_tool_schemas.py is not changed: it is parametrized over every tool in the server's TOOL_DEFINITIONS, so it covers the new tool's schema as it stands.

  Follow-up, not in this PR: review-pr's fetch.sh compares a PR against the newest 100 open PRs on their first 100 files each, and names the rest PARTIAL. overlap.sh pages every open PR and re-lists the cut and renamed ones. One open-PR overlap script, shared by both skills, would serve both. Also not in this PR: overlap.sh stops a repository at the first failed page, and on a large repository a 100-PR page that also lists 100 files per PR can fail with HTTP 502 (all three smoke runs below hit one). Retrying a failed page, or retrying it at a smaller size, would let the scan get further.

• Linked issue(s): none.

• Tests: added/updated? commands run? (what to test; if a test was replaced, where its assertions live now):

  New scripts/tests/test_check_prior_art.py (10 tests). It runs the three scripts end to end with bash, against a stub gh on PATH. The stub answers from a fake GitHub, applies --jq with the real jq, and answers gh pr view with only the --json fields asked for, as gh does. A stub sleep returns at once. The module is skipped when jq is missing.

  The tests pin these contracts:

  • several queries that find one PR give one candidate row, ranked by hits and then by recency; merged comes from merged_at, and a tab in a title cannot break a column;
  • every query lands in queries.tsv as TRUNCATED, INCOMPLETE, FAILED (with gh's error) or OK, with its counts; a failed query does not stop the rest, and comment and blank lines in repos.txt are skipped;
  • the open scan matches exact files and / directories, and each repository ends COMPLETE with its counts, or SKIPPED;
  • a repository past the page cap reads PARTIAL, 10,000 of 10,050;
  • a PR with a rename or more than 100 files is compared on its REST list: a planned file renamed away is a row as old->new, a planned path at file 101 is found, and a rename elsewhere is no row;
  • a PR whose REST list cannot be read, or is cut at 300 of 3,500 files, gets its own PARTIAL line;
  • a failed page keeps the rows already compared, reads PARTIAL 100 of 250, and the next repository is still scanned;
  • a planned path for a repository missing from repos.txt stops the scan and names the line;
  • a deep read whose diff GitHub refuses writes SKIPPED: diff with gh's error and "2 of 2" files, while files.txt keeps every path and xrefs.tsv keeps only the PR cross-references; pr.json holds the mergeCommit that read.sh asks gh for, null for an open PR, and the diff line's count comes from the changedFiles it asks for;
  • a deep read that cannot read the timeline exits non-zero.

  The scan.txt assertions compare each line's status, its repository or PR, and the numbers it states, not its wording. The inclusion logic is not re-tested here; its tests come with feat(pr_knowledge): probe whether an upstream PR is in a local git tree #1847.

  No test pins a quota wait, a pre-written PENDING row or a merge sha that is read only for a merged PR: the scripts no longer do any of these, so the tests and mutation checks that covered them were removed with that code. pr.json's mergeCommit is gh's own field, null until the PR merges, and the deep-read test pins that it is requested and null for an open PR.

  Mutation checks, re-run against the scripts at 75d77c943. Each of these 14 made at least one of the 10 tests fail:

  • stopping the search at the first failed query;
  • dropping the TRUNCATED status;
  • reading a PR's state without merged_at;
  • dropping directory-prefix matching;
  • not re-listing undecided PRs;
  • ignoring a renamed file's old path;
  • writing COMPLETE at the page cap;
  • aborting the whole run at a failed page;
  • dropping the PARTIAL line for a cut REST list;
  • letting a diff failure abort read.sh;
  • dropping -e from read.sh's set;
  • keeping non-PR cross-references;
  • dropping mergeCommit, or changedFiles, from read.sh's --json fields.

  Commands:

  • pytest scripts/tests/test_check_prior_art.py scripts/tests/test_check_cli_references.py src/kernelforge/tests/knowledge/pr_knowledge/test_inclusion.py: 47 passed (10 + 4 + 33).
  • pytest src/kernelforge/tests/knowledge/pr_knowledge/ src/kernelforge/tests/test_stdio_tool_schemas.py: 387 passed. The schema test collects test_tool_schema_has_no_top_level_combinator[pr_check_inclusion].
  • pytest -n 16 scripts/tests: 738 passed, 4 failed. Main at 7ac8c1229 gives 728 passed and the same 4 failed: Node-harness tests in test_code_metrics.py, which fail with ReferenceError: require is not defined under this environment's Node 12.
  • ruff check and ruff format --check on the test file and the Python files feat(pr_knowledge): probe whether an upstream PR is in a local git tree #1847 changes: clean.
  • pre-commit run --files on this PR's 8 files: shellcheck, reuse, gitleaks, codespell, check-broad-suppress, check-cli-references and ruff-format pass. The ruff hook wrapper rejects its own duplicated --force-exclude for any Python file in this checkout, the same as on main, so ruff was run directly.
  • lint-imports: 10 kept, 0 broken.
  • python scripts/code_metrics.py --base-ref origin/main: PASSED, 15 files judged. Against feat(pr_knowledge): probe whether an upstream PR is in a local git tree #1847's head: PASSED, 8 files judged.
  • python scripts/check_cli_references.py: exit 0 (the skill's python -m kernelforge.knowledge.pr_knowledge.inclusion resolves).
  • python scripts/check_broad_suppress.py on the test file, inclusion.py and mcp_server.py: exit 0.
  • python scripts/pr_hygiene.py size: 253 production Python lines against main, 0 against feat(pr_knowledge): probe whether an upstream PR is in a local git tree #1847's head.

  Live read-only smoke run of the scripts at 75d77c943 against a large public upstream repository:

  • search.sh with two phrases sent 4 queries in 15 s: 2 OK and 2 TRUNCATED (100 of 105, 100 of 202). It found 247 candidates.
  • overlap.sh, with one planned file and one planned directory, ran three times. Each run hit a GraphQL page that GitHub answered with HTTP 502, after 700, 3,200 and 700 of 6,487 open PRs. Each run exited 0, kept the rows already compared (72, 288 and 72 overlapping PRs), and wrote PARTIAL: <repo>: the scan stopped after N of 6487 open PRs -- gh: HTTP 502. That line makes the report -- incomplete, see Coverage. None of the three runs reached COMPLETE, and a fourth, run while checking this description, stopped the same way after 2,700 of 6,490 open PRs (23 overlapping rows kept, exit 0).
  • read.sh on two merged PRs wrote a pr.json with state MERGED and a mergeCommit.oid. One of them had 2 linking PRs in xrefs.tsv. On an open PR, mergeCommit was null. All three diffs were read.
  • Step 5's snippet, on a local clone of a fork of that repository:
    • a merged PR the fork carries: included (exit 0, method subject), on a commit whose subject is the PR's title followed by (#N);
    • two merged PRs that merged after the clone's commit, with the merge sha read from pr.json: unknown (exit 3, method none), the evidence saying that the 1 non-merge commit each merge commit brought in is not in the tree by sha or by patch-id;
    • an open PR, probed by subject alone: unknown (exit 3), "no merge commit sha was given".

  Step 3's path log and Step 5's title check, on this repository, which lands PRs both as squash and as merge commits:

  • the path log of src/kernelforge/knowledge/pr_knowledge/references.py from 5b4ac6fae~1 to a56028d11 lists Merge pull request #1846 from ... with --first-parent, and without it only the PR's own commit, which names no PR;
  • the probe answers included by subject for feat(pr_knowledge): carry PR state, draft, merge time and base branch #1846 at a56028d11, naming the merge commit fc25a6d73, and the first line of that merge commit's body is the PR's title;
  • Re-Order Tasks by TraceLens Sorting Order #1763 is a single-commit squash whose subject is not its PR's title; git show --name-only on it lists the same 4 files as its PR.

  On a scratch upstream whose merge commit is Merge pull request #5 from dev/topic with the PR's title in its body, the probe answers included by subject and Step 5's check finds the title in that body. On a scratch fork, the probe also answers included by subject for a commit Fork's own work (#5). Its subject is not the PR's title and git show --name-only lists none of the PR's files, so Step 5 records it as a collision, unknown. The probe matches numbers; the title check is what tells a fork's own PR apart.

• Size/complexity triggers crossed: split, or why the shape is right: none.

  • No Python outside the test file. pr-hygiene counts 0 production lines for this PR's own commits. Both PRs in the stack target main, so this PR's CI count, against main, is 253, all from feat(pr_knowledge): probe whether an upstream PR is in a local git tree #1847.
  • The 8 files add 1,000 lines:
    • SKILL.md is 239 lines, under the 240-line budget it states for itself, as review-pr's SKILL.md states one of 350. Material that only some checks need goes in references/gotchas.md (90 lines), and collection goes in the scripts.
    • The scripts are search.sh 70 lines, overlap.sh 148, read.sh 46 and lib.sh 31. lib.sh holds what the three share: die, the tool check, the input-line reader, the repository check, and the PR-state jq that search.sh and read.sh both apply. overlap.sh defines its path match and row format once, as jq functions that its GraphQL pass and its REST pass both use.
    • The test file is 375 lines, 10 tests, about 70 of them the stub.

• If this simplifies or refactors: which mechanism was removed, which contract was preserved, and the tests that show it: not a refactor; a new skill.

• Observable effect: what an operator will now see, and what the old behaviour cost them (if none, why the change is unobservable):

  • This PR's own commits do not change the runtime, the CLI or the shipped package. What the diff changes there comes from feat(pr_knowledge): probe whether an upstream PR is in a local git tree #1847 beneath it: with --pr-kb, when the repository resolves and the Agent provider supports MCP, the Implementer also gets mcp__pr_monitor__pr_check_inclusion, and python -m kernelforge.knowledge.pr_knowledge.inclusion becomes runnable.
  • A contributor working in this repository with Claude Code now has a check-prior-art skill, listed in AGENTS.md. Given a planned change, it writes a work directory that ends in report.md. The report's verdict is one of six lines:
    • EXISTS-MERGED -- in your tree and EXISTS-OPEN -- in your tree: a DUPLICATE that the probe or the tree's log found in the tree. The next step is to drop the plan, and, for an open PR, to carry results to it.
    • EXISTS-MERGED -- inclusion unchecked and EXISTS-OPEN -- inclusion unchecked: a DUPLICATE that nothing found in the tree. Local git can show that a PR is present, never that it is absent. The next step is to take the upstream commit, or apply the open PR's diff, and measure it rather than write it again.
    • PARTIAL-OVERLAP and NOT-FOUND. Each reads -- incomplete, see Coverage when a query is not OK, a scan.txt line is not COMPLETE, a shortlisted PR was cut from the deep read, or a step wrote SKIPPED.
  • The work directory holds plan.txt, repos.txt, terms.txt, paths.txt, queries.tsv, candidates.tsv, overlap.tsv, scan.txt, overlap.log, pathlog.txt, shortlist.txt, prs/<owner>/<name>/<N>/ (pr.json, files.txt, diff.txt, xrefs.tsv), classes.txt, inclusion.txt, refutations.txt and report.md. A PR's body, comments and reviews are fields of its pr.json; there are no separate files for them. queries.tsv has a row for each query that ran, a failed one included, and none for a query a killed run never reached; a repository with no scan.txt line is one whose scan did not end. Step 7 reports either gap as SKIPPED.
  • A fork that carries an open upstream PR under its own commit subject is reported EXISTS-OPEN -- in your tree once the log search finds the change. It is not told to apply the PR and measure it again.
  • On an upstream that lands PRs as merge commits, the merged PRs on the planned paths reach the shortlist from their Merge pull request #N from subjects, and a probe match on such a subject counts as included when the merge commit's body holds the PR's title. A fork's own commit that only shares the PR's number is reported unknown, not in your tree.
  • When the probe cannot run on a given tree, for example because the package cannot be imported, the tree's log is still searched, and a carry that search finds is reported in your tree rather than inclusion unchecked.
  • An upstream PR that does the planned trick in a sibling backend's file is OVERLAPPING, so the verdict is PARTIAL-OVERLAP, not NOT-FOUND.
  • The report's Coverage part states:
    • the queries, by status;
    • the path scan, per repository;
    • "Read N of M shortlisted; cut: ..." for the deep read;
    • every SKIPPED line;
    • what no check searches: closed-unmerged PRs (reached only through cross-references), issues and RFCs, repositories outside repos.txt, and planned paths that do not exist upstream, which match nothing while the scan says COMPLETE.
  • Before this change, that check was manual and its coverage unrecorded. An empty search result could not be told apart from "not searched", so effort went into changes that already existed upstream or were already in the measured tree.
  • The scripts need gh, authenticated because the open-PR scan uses GraphQL, and jq.

• Breaking changes: no.

• PR addresses single concern: yes. The skill is one capability; the other commits in the diff are feat(pr_knowledge): probe whether an upstream PR is in a local git tree #1847's, the PR beneath it.

• Root cause is upstream (Magpie/TraceLens/GEAK/IntelliKit/AgentKernelArena), ticket filed: no; not applicable.

[chaojhou]

Thanks @okorzh-amd. This PR's own changes (the check-prior-art skill) have no blockers:

• GitHub data reaches files only through jq --arg or @tsv, and there is no eval.
• Temp dirs are cleaned up by a trap, and every call is a read.
• The tests run offline.
• The skill stays repo-local; it is not in the package's data-files.

The PR contains #1847, so I'll approve it once #1847's must-fix is in.

[github-actions]

Gate job conclusion: success (run) on 56702fb22398a88b9c36050d38e560bad2791ff9, as reported by GitHub.

The report below was uploaded by that run; the conclusion above is the result.

---

Code metrics gate: PASSED

Judged: the 21 files this change touches. Files it does not touch never fail it; anything found there is listed under Outside this change for information.

Dimension	Limit	New	Worse	Baseline out of date	Baseline entries	vs base
Cyclomatic complexity	fails > 20	0	0	0	68	+0
Cognitive complexity	fails > 30	0	0	0	274	+0
Function length (lines)	fails > 80	0	0	0	412	+0
Nested block depth	fails > 5	0	0	0	25	+0
Parameter count	fails > 5	0	0	0	95	+0
Module length (lines)	warns > 800, fails > 1200	0	0	0	51	+0
Duplicated lines per file	fails > 0	0	0	0	97	+0
Dead code (vulture)	fails > 0	0	0	0	2	+0

Check	Findings	Effect
Comments	0
English only	0
Production code imports test code	0
Repeated literals	0
A1 Private name of another package	0
A8 Flag argument	0
A9 Tuple return	0
A10 Raw vocabulary comparison	0
A11 Hidden global write	0
A12 Pure forwarder	0
A12 Single-implementation abstraction	0

106 findings in files this change does not touch

Outside this change (informational): 106

Unit	Dimension	Baseline	Now
src/hyperloom/orchestrator/enablement/recipe/projections.py project_launch_evidence	Cognitive complexity	absent	36
src/hyperloom/orchestrator/state/shared_state.py resolve_graded_comparison	Cognitive complexity	absent	34
src/hyperloom_kb/collect/mapping.py _experience	Cognitive complexity	absent	31
src/hyperloom_kb/schema.py FieldDeclaration.__post_init__	Cognitive complexity	absent	32
src/kernelforge/cli/forge_loop_command.py _prepare_measurement_driver	Cognitive complexity	absent	36
src/kernelforge/cli/forge_loop_command.py _read_kb_warm_start	Cognitive complexity	absent	40
src/kernelforge/loop/runner.py IterationLoop._collect_session_candidate	Cognitive complexity	absent	44
src/kernelforge/loop/runner.py IterationLoop._rule_on_attempt	Cognitive complexity	absent	38
src/hyperloom/inference_optimizer/cli/__init__.py _reexport_session_environment	Function length (lines)	absent	111
src/hyperloom/inference_optimizer/cli/__init__.py _resolve_agent_lineup	Function length (lines)	absent	89
src/hyperloom/inference_optimizer/cli/__init__.py _resolve_run_topology	Function length (lines)	absent	92
src/hyperloom/inference_optimizer/cli/__init__.py _resume_session	Function length (lines)	absent	170
src/hyperloom/inference_optimizer/cli/__init__.py _seed_fresh_session	Function length (lines)	absent	170
src/hyperloom/inference_optimizer/multi_node/cli.py _restart_multinode	Function length (lines)	absent	107
src/hyperloom/inference_optimizer/multi_node/scripts/launch_multinode.py _build_parser	Function length (lines)	absent	84
src/hyperloom/inference_optimizer/multi_node/scripts/launch_multinode.py _spawn_rank_actors	Function length (lines)	absent	84
src/hyperloom_kb/sync.py GlobalSync.pull	Function length (lines)	absent	84
src/kernelforge/cli/forge_loop_command.py _RunReporter.build_result	Function length (lines)	absent	91
src/kernelforge/cli/forge_loop_command.py _prepare_measurement_driver	Function length (lines)	absent	136
src/kernelforge/cli/forge_loop_command.py _read_kb_warm_start	Function length (lines)	absent	149
src/kernelforge/loop/runner.py IterationLoop._close_iteration	Function length (lines)	absent	117
src/kernelforge/loop/runner.py IterationLoop._collect_session_candidate	Function length (lines)	absent	184
src/kernelforge/loop/runner.py IterationLoop._implementer_history	Function length (lines)	absent	112
src/kernelforge/loop/runner.py IterationLoop._rule_on_attempt	Function length (lines)	absent	102
src/kernelforge/loop/runner.py IterationLoop._source_attempt	Function length (lines)	absent	173

...and 81 more; the job log lists every row.

Verdict computed by the gate scripts in /home/runner/work/_temp/code-metrics-gate, not by this tree's copy.

_{Thresholds and sources: [tool.hyperloom.code_metrics] in pyproject.toml. Tools: complexipy 8.0.1, jscpd 5.4.1, ruff 0.16.2, vulture 2.16.}

[chaojhou]

PR #1850 -- feat(skills): add check-prior-art, a contributor skill that finds a planned change upstream

What it does: Before this PR, nothing in the repo let a contributor check whether a planned optimization was already in an open or merged upstream PR, or already in the tree being measured. It adds the repo-local .claude/skills/check-prior-art/ skill: search.sh runs keyword search, overlap.sh runs a paged open-PR path scan with explicit coverage, and read.sh deep-reads a PR; every query and scan records how far it got, and SKILL.md turns the results into a single verdict. The tree check comes from the stacked #1847 (python -m kernelforge.knowledge.pr_knowledge.inclusion and the pr_check_inclusion MCP tool), whose commits this PR's diff also carries.

Blocking issues: 1

1. [X2] The description's test run predates the patch-id stage this PR carries [verified]
   Problem: The body says pytest scripts/tests/test_check_prior_art.py scripts/tests/test_check_cli_references.py src/kernelforge/tests/knowledge/pr_knowledge/test_inclusion.py: 67 passed. At head 07cff09 those files hold 82 cases (20 + 4 + 58). 67 is 20 + 4 + 43, the test_inclusion.py count before e4325c7, 6e5359b and 186ff29, all in this PR's diff, added the patch-id, multi-commit unknown and backport cases. src/kernelforge/tests/knowledge/pr_knowledge/test_inclusion.py:110
   Impact: The validation the description reports does not cover the head that would merge; the probe that the skill's Step 5 runs changed after that run.
   Action: Rerun the listed commands at the current head and update the counts.

SKIPPED: V4 -- PARTIAL overlap for #1032, #1154, #1276, #1675 (each cut at its first 100 files), overlap there unchecked
SKIPPED: local test run -- no pytest, bash or jq on this host; the shell scripts were reviewed statically and the inclusion cases traced by reading _patch_equivalence

Checked: .claude/skills/check-prior-art/{SKILL.md,lib.sh,search.sh,overlap.sh,read.sh,references/gotchas.md}, scripts/tests/test_check_prior_art.py, #1847 files inclusion.py probe_inclusion/_merge_ancestry/_subject_suffix/_patch_equivalence/main, search.py PRLifecycle/PRState, references.py, pr_stdio_server.py, kernel_names.py rename and its importers; tree-wide sweep at head for every non-test reader of is_merged, PRReference, reference_to_entry/entry_to_reference, tools.pmc, pr_check_inclusion and TOOL_NAMES (orchestrator/agent.py:268); body.txt against the head tree | Ran: python ..\pr_tools\hl_fetch.py 1850; python ..\pr_tools\c901.py 1a444f6 07cff09 <8 src paths> (max 10, none over 20); git grep at head; AST test-case count of the three listed test files at 07cff09 and 05c2e84; git blame on test_inclusion.py:110; gh run list --commit 07cff09 after approving the fork runs (9 workflows success, the 2 label-gated E2E skipped) | Base: 1a444f6 | Head: 07cff09

[chaojhou]

PR #1850 -- feat(skills): add check-prior-art, a contributor skill that finds a planned change upstream

What it does: Adds the repo-local .claude/skills/check-prior-art/ skill so a contributor can check, before building an optimization, whether it already exists in an open or merged upstream PR or in the tree being measured: search.sh (keyword search with a status per query), overlap.sh (paged open-PR path scan), read.sh (deep read of one PR), and SKILL.md (shortlist, classify, probe with #1847, refute, one verdict with its coverage). The diff also carries the stacked #1847.

Blocking issues: 4

1. [free:open-pr-carried-by-fork] A fork that already carries an open upstream PR is told to apply and measure it again [verified]
   Problem: for an open PR read.sh writes merge_commit_sha: - (read.sh:45), so Step 5 runs the probe without --merge-commit-sha (SKILL.md:174), and without a sha the probe answers included only for a (#N) subject, otherwise unknown. Step 6 attacks only not_included answers with the fork's git log -S / --grep search (SKILL.md:182, :200-201), so for an open PR that search never runs. .claude/skills/check-prior-art/SKILL.md:182
   Impact: a fork that cherry-picked an open PR under its own subject -- the G5 case -- gets EXISTS-OPEN and the next step "apply the open PR's diff to your tree and measure it" (SKILL.md:46) for a change its baseline already runs, although references/gotchas.md:55-58 promises -- in your tree for exactly this case.
   Action: in Step 6, attack every DUPLICATE inclusion answer that is not included, open PRs included, with the log search; give EXISTS-OPEN an -- inclusion unchecked qualifier when nothing settles it.

2. [free:overlapping-definition] OVERLAPPING is defined two ways, so a PR that carries part of the mechanism in another file ends UNRELATED [verified]
   Problem: Step 4 classes a PR that "changes the same files or functions, or carries part of the mechanism" as OVERLAPPING (SKILL.md:159); Step 6 drops an OVERLAPPING PR to UNRELATED unless "files.txt or the diff shows a planned path or function". .claude/skills/check-prior-art/SKILL.md:198
   Impact: an upstream PR doing the same trick in a sibling backend's file is dropped, and the verdict can be NOT-FOUND with "proceed" (SKILL.md:30, :48).
   Action: keep one definition: drop "or carries part of the mechanism" from :159, or let Step 6 keep such a PR as OVERLAPPING.

3. [free:read-cap-coverage] The 12-PR read budget never reaches the report's coverage [verified]
   Problem: Step 4 reads at most 12 candidates and keeps the rest as # cut: lines in shortlist.txt, but the Coverage part lists only non-OK queries, non-COMPLETE scan lines and SKIPPED: lines (SKILL.md:35-37), and the -- incomplete qualifier has the same triggers (SKILL.md:29). .claude/skills/check-prior-art/SKILL.md:141
   Impact: a bare NOT-FOUND can rest on 12 of 76 candidates picked by title (the live run in the description found 76 for one phrase), against the skill's own rule that "nothing found" must not read as "not searched".
   Action: add "read N of M; cut: ..." to Coverage, and make any # cut: line a trigger for -- incomplete.

4. [X2] The description's test run predates the head [verified]
   Problem: unchanged since the last round at this head: it says the three listed test files ran "67 passed"; at head they hold 82 cases (20 + 4 + 58). src/kernelforge/tests/knowledge/pr_knowledge/test_inclusion.py:110
   Impact: the validation the description reports does not cover the head that would merge.
   Action: rerun the listed commands at the head and update the counts.

---

Design review (requested by the maintainer; beyond the blocking bar)

The aim of this section is a smaller PR, not more guards: each point says what to delete or merge.

Purpose, route, files. Skill files: SKILL.md (240 lines), references/gotchas.md (G1-G8); scripts search.sh, overlap.sh, read.sh, lib.sh; tests scripts/tests/test_check_prior_art.py (20 cases against a stub gh); one AGENTS.md row. Everything is a read through authenticated gh (REST and GraphQL) plus jq.

1. Completeness. Items 1-3. By design it cannot see closed-unmerged attempts (queries are open and merged only, although SUPERSEDED covers "is closed"), issues and RFCs (is:pr only), a duplicate in a second repository, or a planned path that does not exist upstream (the scan reports COMPLETE with an empty overlap and nothing records that). Say these in Coverage or gotchas.

2. Simpler / smaller.

• Keyword search: gh api search/issues --jq is the right call (it returns total_count and incomplete_results, which gh search prs does not), but the machinery around it is not needed: the -i header split and reset arithmetic (search.sh:56-82) can be a fixed sleep 2 from the documented 30/min search limit, and the pre-written PENDING rows, full-file rewrites and replace_file (search.sh:51-54, 84-99; lib.sh:29-34) can be appends -- Step 7 already counts two rows per term.
• Open-PR scan: the GraphQL query and the REST re-listing for more than 100 files or a rename are warranted (gh pr list --json files stops at 100 files and has no change type); the killed-run bookkeeping on top of it (overlap.sh:76, 152-153, 162) is not, since SKILL.md already runs the scan in the background and checks the counts at Step 7.
• Deep read: gh pr view N -R R --json number,title,state,isDraft,mergedAt,mergeCommit,changedFiles,body,comments,reviews,url, plus gh pr diff and the timeline call, replaces most of read.sh: mergeCommit is null until merge and state already says MERGED.
• Probing merged OVERLAPPING PRs (SKILL.md:165) feeds no verdict.
• Follow-up: review-pr's fetch.sh answers the same open-PR overlap question with the 100-file and rename blind spots overlap.sh fixes; one shared script would serve both.

3. Redundancy and silent failure. A killed run is handled three times (script pre-writes, background run, Step 7 count check); keep one. The merged-state jq is written three times (search.sh:105; read.sh:42, :83), pulls/N/files is fetched by two scripts, and every gh call gets < /dev/null although nothing reads stdin. Failures become FAILED or PARTIAL rows, so no failed query reads as "nothing found"; two silent negatives remain: the planned path that does not exist upstream, and files_listed, which read.sh writes and SKILL.md never compares.

4. Docs and comments. Besides items 1-4: gotchas.md:51-53 says a folded backport "still reads as not_included", but for a multi-commit merge the probe answers unknown, which Step 6 never attacks; read.sh:10-11 says it "exits non-zero, except a diff GitHub refuses to render", but :62-66 turn every diff failure into SKIPPED with exit 0; SKILL.md:123 promises overlap-path columns overlap.tsv does not have.

5. Tests. Keep: dedupe and ranking (test_check_prior_art.py:169), the query-status vocabulary (:189), file and directory matching (:274), the page cap (:296), REST re-listing for a rename and file 101 (:305), an undecidable PR becomes PARTIAL (:326), a failed page (:343), an unknown repo (:378), the diff-refusal half of :456. Redundant or freezing the current shape: the rate-limit pacing test with wall-clock bounds (:211), the two killed-run tests (:228, :361) and the stub's SIGTERM path, the merge-sha test whose open and closed cases take the same branch (:449), three parameters of one pipefail mechanism (:471), the trivial input check (:243), and exact-English equality on scan.txt (assert status, repo and counts instead).

Starting from zero. SKILL.md of about 150 lines, search.sh of about 40 (loop, --jq, fixed pace, appends), overlap.sh of about 100 (paging, re-listing, one line per repository at the end), the deep read as four gh commands, and about 8 tests -- roughly 500 lines instead of 1,221. The skill is worth having: the full open-PR path scan and the tree probe are what a contributor cannot do reliably by hand.

Change list, in order of impact:

1. Item 1: Step 6 attacks every DUPLICATE answer that is not included; -- inclusion unchecked for EXISTS-OPEN. If #1847 becomes two-valued, this is simply "every answer that is not included".
2. Item 2: one OVERLAPPING definition.
3. Item 3: "read N of M" in Coverage; a cut triggers -- incomplete.
4. Delete the killed-run bookkeeping and append instead (search.sh:51-54, 84-99; overlap.sh:76, 152-153, 162; lib.sh:29-34; tests :228, :361).
5. Replace the header pacing with --jq plus sleep 2; drop test :211.
6. Deep read via gh pr view --json ...,mergeCommit; shrink read.sh; drop test :449 and two parameters of :471.
7. Fix the doc mismatches in point 4, and use or drop files_listed.
8. Loosen the exact-prose assertions; update the description (item 4).
9. Follow-up PR: one open-PR overlap script shared with review-pr's fetch.sh.

Checked: SKILL.md (verdict and next-step tables, Steps 4-6), references/gotchas.md G5, read.sh (meta fields), search.sh, overlap.sh, lib.sh, test_check_prior_art.py, the #1847 probe's behaviour without a merge sha, the description against the head | Ran: static review only (no bash or jq on this host); no test suite | Base: 1a444f6 | Head: 07cff09

[chaojhou]

PR #1850 -- feat(skills): add check-prior-art, a contributor skill that finds a planned change upstream

What it does: The PR adds a repo-local contributor skill, .claude/skills/check-prior-art/, which checks whether a planned optimization already exists upstream, either in an open or merged PR or in the tree being measured. It does this with search.sh (keyword search), overlap.sh (a paged GraphQL scan of open PRs plus a REST relist of their files), read.sh (a deep read of each candidate through gh pr view/gh pr diff) and the inclusion probe from the stacked #1847, which this diff also carries along with the pr_check_inclusion MCP tool and #2028's move of derive_kernel_names.

Blocking issues: 1

1. [X2] The PR description describes an earlier shape of the scripts, the verdicts and the tests, not the code at head 56702fb [verified]
   Problem: Later commits in this PR (a9bf0ff append instead of pre-write, 527c562 --jq plus a fixed sleep, 10084a9 gh pr view, 4e9d705, 7a6f72e never not_included) rewrote the skill, but the body was never updated. Each of these claims is false at head. (1) search.sh 'paces itself from each response's own rate-limit headers': search.sh:51-52 is a fixed sleep 2. (2) queries.tsv 'lists every planned query as PENDING from the start, and both files are rewritten after each answer': search.sh:40-50 appends rows as each query is answered, has no PENDING status, and writes candidates.tsv once at the end (:59-65). (3) scan.txt 'names every repository before the first page and is rewritten after each page', with 'did not finish -- N of M' and 'not scanned': overlap.sh:136-144 writes a repository's line only once its scan ends, and neither string exists. (4) read.sh writes 'meta.txt (merge_commit_sha)', 'body.txt and comments.txt, each headed as untrusted data': read.sh:29-31 writes pr.json with mergeCommit instead. (5) The verdicts EXISTS-MERGED-INCLUDED / EXISTS-MERGED-NOT-IN-TREE: SKILL.md:24-31 uses 'EXISTS-MERGED -- in your tree' / '-- inclusion unchecked' and 'EXISTS-OPEN -- inclusion unchecked'. (6) 'a not_included answer is dropped': inclusion.py:63-67 only answers included or unknown. (7) lib.sh 'one atomic file replace': lib.sh:1-31 has no such helper. (8) Tests: the body claims '20 tests' and '67 passed', and says the quota-wait, killed-run PENDING and merge-sha-only-when-merged contracts are tested and mutation-checked. At head, test_check_prior_art.py has 10 tests and the three listed files collect 10+4+33=47, and none of those three contracts has a test left. The previous card's X2 ('67 passed') is therefore still open, and it has grown. .claude/skills/check-prior-art/search.sh:52
   Impact: The release notes would tell contributors about artifacts and statuses (meta.txt, PENDING rows, 'did not finish', header-based pacing) and verdict names that the skill never produces. The test and mutation evidence in the description covers code that is no longer in the diff, so the head as it would merge has no stated validation.
   Action: Rewrite the Steps, Tests and Observable-effect sections of the body against the head: fixed 2 s pacing, appended queries.tsv rows, per-repository scan lines written only at the end, pr.json, the four EXISTS-* verdict lines, included/unknown only, no atomic replace in lib.sh. Then rerun the listed pytest commands and quote the real counts (10/4/33) and the mutation checks that still apply.

Checked: I read the head versions of .claude/skills/check-prior-art/{SKILL.md, lib.sh, search.sh, overlap.sh, read.sh, references/gotchas.md}, scripts/tests/test_check_prior_art.py, src/kernelforge/knowledge/pr_knowledge/inclusion.py, the pr_stdio_server.py diff, the kernel_names move (implementation.py, source_resolution.py, campaign_config.py), .importlinter (the acyclic contract), the consumer at kernelforge/orchestrator/agent.py:25/541 (PR_TOOL_NAMES), the hyperloom gate.py PR_MONITOR_TOOL_NAMES list (a different tool set, not a twin), kernelforge/llm/git.py git/git_bytes at head and at main (identical), and the body against the head tree, using the previous review cards in comments.txt. I ran git archive of head, main and the merge-tree result (bd600b12c) into a local script, then pytest --noconftest: on head, test_inclusion plus test_pr_stdio_server gave 84 passed, 1 skipped and one cp1252 failure that passes with PYTHONUTF8=1; on the merged tree, the same plus writer_helpers and injection_point gave 125 passed, with the remaining failures all fcntl or Windows permission errors; the shell tests were skipped because jq is not on this host. I also ran per-file collect-only counts (10/4/ | Base: b7004d1 | Head: 56702fb

[okorzh-amd]

@chaojhou Thanks. The description is now rewritten against 75d77c9, and it covers all eight points: the fixed sleep 2 pacing with no header reads, queries.tsv rows appended per answered query with no PENDING rows, candidates.tsv written once at the end, a scan.txt repository line only when its scan ends, pr.json (no meta/body/comments files), the four EXISTS-* lines plus PARTIAL-OVERLAP/NOT-FOUND, included/unknown only, and lib.sh's real contents with no atomic replace.
Tests now quote the real counts: 10 + 4 + 33 = 47 passed. It says plainly that the quota-wait, PENDING and merge-sha-only-when-merged contracts have no tests, because that code is gone. The 14 mutation checks were re-run against the head scripts, and each one still makes a test fail.
The branch is rebased onto main 7ac8c12 through #1847 (005924d). The rebase applied cleanly and changed none of this PR's files. The one substantive change is that the derive_kernel_names move landed as #2028, so this diff no longer carries it. The MCP server it extends is now mcp_server.py (#1907). The description also summarizes what #1847 contributes to the diff.
I agree with the review. I re-ran the live smoke test, and overlap.sh no longer finishes on a large upstream: all three runs hit a GraphQL HTTP 502 and ended PARTIAL, naming where they stopped. The description now reports that rather than repeating the old COMPLETE run, and lists page retry as a follow-up.