[MaxGhenis]

Fixes #712

Draft until CI and the independent review pass.

What and why

Subfleet build jobs keep their state in a .job/ folder at the root of their worktree: HANDOFF.md, PR_BODY.md, env, a HEAD file, git bundles and sometimes a whole repo.git. On 23 October 2026, #704 committed .job/HANDOFF.md, and #699 committed .job/HANDOFF.md and .job/PR_BODY.md. Both were removed by hand. Nothing in CI would have stopped them. pr_code_changes.yaml has a paths: filter, so a pull request that adds only such files runs no workflow at all.

This PR adds an Agent notes workflow (.github/workflows/pr_agent_notes.yaml). It runs .github/check-agent-notes.sh on every pull request and fails when the PR's head commit contains:

• a .job/, .job-git/ or .diag/ folder at any depth, or a file of one of those names. .job-git/ is a Subfleet job's git directory. A local clone of policyengine-uk has 674 .job-git/ files and 8 .diag/ files in its branch history, none ever on main.
• a root-level HANDOFF.md, handoffs.jsonl, PR_BODY.md, PR-BODY.md, REPORT.md or PROGRESS.md, in any ASCII capitalisation, so pr-body.md and pr_body.md count too. Four of these (REPORT.md, PROGRESS.md, HANDOFF.md, handoffs.jsonl) are policyengine-uk's list.

It also ignores the three folders in .gitignore, so git add -A leaves them out, and documents the check in docs/engineering/skills/github-prs.md.

Design

policyengine-uk's "No agent working notes at the repository root" step (pr_code_changes.yaml on its main) is the model, and its pieces carry over:

• Pathspecs. git ls-files with :(top,icase) pathspecs, plus :(glob,icase)**/ for the folders at any depth.
• Failure handling. shell: bash adds pipefail, and the script also sets set -euo pipefail, so a git failure fails the step instead of passing it. A failure is reported with ::error:: and exit 1.
• Cancellation. A new push to a pull request cancels the run on its previous head. Runs on main are never cancelled.

It departs from policyengine-uk's guard where pe.py differs, or where review found a gap:

• Its own workflow, with no path or branch filter, and edited handled. policyengine-uk's guard is a step in Lint, which runs on every pull request there. pe.py's code workflow has a path filter, so the guard needs its own workflow. types: [opened, synchronize, reopened, edited]: retargeting emits edited, so a stacked PR whose old base had no guard is checked when it is retargeted to main. Every edit runs the full check on the head, with no condition, so a title edit cannot replace a failure with a skipped run.
• It runs the default branch's guard. On a pull request, the job fetches the default branch along with the head. It copies the default branch's .github/check-agent-notes.sh into $RUNNER_TEMP and runs that copy, so a PR cannot pass by editing the script. Round 3 reproduced a false pass with a pass-always script. Only while the default branch has no guard, on this PR, does the checked-out copy run. A missing default-branch ref fails the step rather than falling back. Push, schedule and dispatch runs check out the commit they check. A PR that edits the workflow file itself is out of reach without pull_request_target or a required-workflow setting; that is visible in review.
• It checks the PR's head commit, not the merge with the base. The job fetches github.event.pull_request.head.sha by exact SHA. The script reads <commit>^{tree} into a temporary GIT_INDEX_FILE and lists it with the same pathspecs. The result does not depend on the base, so retargeting a stacked pull request cannot change it. Main has never had any of these paths (git log origin/main for them is empty), so a PR based on main fails exactly when it adds one.
• It also checks main on every push and once a day. A skip instruction in a pushed commit's message skips pull_request and push runs. GitHub's docs: "Skip instructions only apply to the push and pull_request events." A merge that carries such a commit to main can skip the push run too. This PR showed it by accident: two of its own head commits quoted the instruction while describing this gap, and no pull request workflow ran on them. No commit message can skip the daily schedule run, so a note that reaches main that way is found by the next scheduled run, if it is still there. GitHub can delay or drop a scheduled run. The push run catches an ordinary direct push at once. GitHub disables schedules in public repositories after 60 days without activity; this one is active.
• pull_request, not pull_request_target. Round 2 used pull_request_target. That made skip instructions powerless, ran on conflicted PRs, and ran main's copy of the guard. Two things ruled it out. GitHub blocks pull_request_target in public repositories without an Actions event policy from 2 December 2026, and neither this repository nor the org has one. And pe.py's PR verification never uses it (docs/release-bundles.md). Allowing it is a settings decision (queued as d1933, with the five other PolicyEngine workflows that rely on it).
• Output that can't forge workflow commands. A path name comes from the pull request. The runner reads ::name:: at the start of a line, and the legacy ##[name] anywhere in a line. So the paths are printed between ::stop-commands::<random 32-hex token> and ::<token>:: (the runner masks the token in the log), and % is percent-encoded in the annotation data. The log lists every path; the annotation gives the count and the first path. This handles a tracked repo.git.
• A script, not inline, like .github/check-changelog.sh, so pytest can drive it. It runs under the macOS bash 3.2 as well as Ubuntu's bash 5.
• permissions: contents: read, no secrets, ubuntu-latest. Let CI test and check jobs use another runner pool, set by repository variables #706's runner-variable test (invariant 2: every other job keeps ubuntu-latest) still holds.

Invariants

1. The guard fails exactly when the tree holds a note, as defined above. It reports every such path and no other.
2. Given a commit, it judges that commit's tree, whatever the index holds, and leaves the index as it was.
3. A git failure fails the guard instead of passing it.
4. The runner reads no path the guard prints as a workflow command. The only commands it emits are its own stop-commands, the matching resume and one error.
5. The workflow checks every pull request's head, with no path or branch filter, on every push, retarget or other edit. It also checks main on every push and on a schedule, which no commit message can skip. It has a read-only token and no pull_request_target.
6. On a pull request, it runs the default branch's guard, so editing the script cannot pass the PR. Only while the default branch has no guard does the checked-out copy run.

Test plan

• tests/test_check_agent_notes.py, 55 tests:

  • the sample .job/HANDOFF.md fails, and a normal change passes;
  • 21 notes fail, and 13 near misses pass (docs/HANDOFF.md, .jobs/, x.job/, my.job, REPORTS.md and others);
  • a tracked repo.git is reported in full;
  • commit mode judges the commit's tree, not the index, and leaves the index unchanged;
  • command-shaped names (##[stop-commands]…, ##[add-mask]…, ::warning::…, an indented ::error::…) and newline names, run through an emulation of the runner's parsing written from actions/runner source (ActionCommand.cs TryParseV2/TryParse, ActionCommandManager.cs TryProcessCommand);
  • a % in the annotation;
  • git failures: outside a repository, and an unknown commit;
  • this repository's index and HEAD, and the .gitignore rules;
  • the workflow's triggers, steps and token;
  • the workflow's own guard step, read from the YAML and run as GitHub runs a bash step, in scratch repositories laid out as the job leaves them. A PR with a pass-always script and .job/HANDOFF.md still fails. Before main has a guard, the checked-out copy runs. A missing default-branch ref fails closed. Push, schedule and dispatch events check their own commit;
  • a Hypothesis differential test on 150 generated trees. It checks the index mode against a Python reference, and the commit mode against the index mode. The trees mix glob characters, colons, non-ASCII letters and random case.

• Mutation check. Fourteen mutants each fail the suite:

  • case folding dropped for .diag;
  • .job matched at the root only;
  • PR-BODY.md dropped;
  • set -e and pipefail removed;
  • the line prefix removed;
  • note names matched at any depth;
  • stop-commands removed;
  • the commit read into the real index;
  • the commit argument ignored;
  • the % encoding removed;
  • main's copy never used;
  • the fail-closed default-branch check removed;
  • edited dropped;
  • the PR's copy preferred over main's.

• The guard's steps, simulated against the head of every open PR. A fresh git clone --depth=1 of main, then git fetch --no-tags --depth=1 origin <head sha> and the script on that SHA. Only Add behavior input binding contract #511 and Build and verify isolated wrapper release candidates #520 fail (both PROGRESS.md); the other 27, Allow a UK local-area release on an accepted model mismatch (hold: needs María's routing call) #699, Support bundle-pinned local data release manifests #704 and this PR among them, pass.

• actionlint 1.7.12 on pr_agent_notes.yaml: no findings. On all workflows, its one finding is the existing actions/configure-pages@v3 in push.yaml, which this PR does not touch.

• shellcheck on the script, and ruff format --check . and ruff check .: clean.

• This version on GitHub:

  • it passes on 693d012 (run 40015348571);
  • it fails on a demonstration commit that adds .job/HANDOFF.md (run 40015409561), which the next commit reverts. Its log shows ::stop-commands::*** (the token is masked), then Agent or job note: .job/HANDOFF.md, then the annotation "1 agent or job note path(s) in the tree, starting with .job/HANDOFF.md". The .gitignore made the demonstration need git add -f.

  The round-1 version did the same (pass, fail). The PR is squash-merged, so no demonstration commit reaches main.

• CI on af2c21024: Agent notes (run 40015473244) and Docs passed. Code changes was superseded by the next push.

• CI on the head 8656ee484.

• After merging, with a squash message free of skip instructions (the repository builds squash messages from commit messages, and 0fe7a2d's quotes one):

  • dispatch the workflow on main and check that it passes;
  • open a throwaway draft PR whose head adds .job/HANDOFF.md and replaces the script with a pass-always one. The check must fail, which shows main's copy runs. Then close the PR and delete its branch.

Review

GPT-6.1 Sol (subfleet run --task review --tier hard):

• Round 1 (pull_request, merge tree): REQUEST_CHANGES.

  1. A base retarget did not re-run the check, so a stacked PR checked against a cleaned parent could pass.
  2. A skip instruction in the head commit skipped it.
  3. A path containing ##[stop-commands]token suppressed the error annotation.

  Fixed: the check now reads the head tree, and output goes through stop-commands. Finding 2 is narrowed by the push and schedule runs on main.

• Round 2 (pull_request_target): REQUEST_CHANGES. It found no bypass, and reproduced the retarget and stop-commands fixes, including with a compiled copy of the runner's parser. The one finding was the 2 December pull_request_target block. Fixed by returning to pull_request as above.

• Round 3 (pull_request, head tree, push and schedule on main): REQUEST_CHANGES. It found no path-matching or output bypass, and verified the three GitHub runs. Its findings:

  1. A PR could pass by replacing the script; it reproduced a false pass.
  2. A retargeted old stacked PR got no first run, because edited was not handled.
  3. Nit: the docs said push runs cannot be skipped, but they can.

  Fixed in 8656ee4: the job runs the default branch's guard, handles edited, and the wording is corrected.

• Round 4 (8656ee4): APPROVE. No blocker or should-fix. It reproduced the closure of both round-3 findings in scratch repositories: main's copy runs over a PR's pass-always script, and an old stacked head retargeted to main is checked. It verified the shallow two-refspec fetch, fork-object fetch and concurrency, and it caught three workflow-step mutations. One nit: "found within a day" was too strong, because GitHub can delay or drop scheduled runs. The wording is fixed in the next commit, along with the concurrency comment it flagged.

Sweep of open pull requests

At 2026-10-23 18:00 UTC, the changed-file lists of all 28 open PRs (gh api repos/PolicyEngine/policyengine.py/pulls/N/files) were matched against every path this guard rejects. The search also flagged any root-level .md, .jsonl or .txt file and any path naming a hand-off, PR body, progress note, report or review. Shortly after 19:00 UTC the guard itself was run on every open PR's head commit, as in the test plan.

PR	Branch	Finding	When the guard sees it
#511	feat/be-behavior-input-contract → main	adds root PROGRESS.md	fails on its next push, reopen or re-run
#520	max/wrapper-release-candidates-20260914 → max/spm-canonical-wrapper-release-20260910	root PROGRESS.md, which its stacked base tracks	fails on its next push, reopen or re-run

No open PR has a .job/, .job-git/ or .diag/ path, or a hand-off or PR-body file. #699 and #704 are clean at their current heads (c669bbd98, 463c3a953), though earlier commits in their branches add .job/HANDOFF.md and .job/PR_BODY.md. A squash merge keeps those commits off main. The other root-level matches are edits to README.md (#581, #596, #709, #710). Nothing was pushed to any of these branches.

Not in this PR

• policyengine-uk's guard rejects root notes only. It does not reject .job/, .job-git/ or .diag/, and nothing checks its main on a schedule. That is a separate change to policyengine-uk, after Budget week.
• pe.py's main has no branch protection or ruleset, so the check is not required for merging. Agents' merge gates (gh pr checks must pass) already block on it. If it were required, a skipped run would stay pending and block the merge (GitHub's skip docs). That needs a ruleset that lets the release bot's version commits through, which is a settings decision.

axiom: n/a: CI infrastructure only

🤖 Generated with Claude Code