[anth-volk]

Fixes #861

Same-Repository Draft PR Check

• This PR is a draft.
• This PR is from a branch in PolicyEngine/policyengine-sim-api, not a personal fork.

Summary

Step 2.8 of the code-quality plan: enforce rule sim.process.pr-tracking-issue, which says the first line of every pull request description is Fixes #ISSUE_NUMBER.

• New PR description job (.github/workflows/pr-description.yml). It runs .github/scripts/check_pr_description.py, a standard-library script, on pull_request events of type opened, edited, reopened and synchronize.
• What the script checks. It reads the description from the event payload and requires the first line to be Fixes #N. It then runs gh api repos/<repo>/issues/<N> and fails unless #N is an open issue, not a pull request, in this repository.
• Daily dependency-update PRs used to start with "Automated daily dependency updates", so they would have failed. They now open or reuse an "Update dependencies" tracking issue and put Fixes #N first.
• Registry. The rule moves from doc-only to enforced, with enforcer kind workflow-job and a registered proof. github-prs.md gains a section on what passes and how to fix a failure, and the rendered views are regenerated.

Of the last 300 PRs here, about half would fail this check, for example Fixes #615. (#616), Refs #434 (#543) and a ## Summary heading.

Not in this PR: making the job a required check, which is a ruleset change; see "After merge".

Commits

1. Read the tracking issue number from a PR description's first line. Adds the parser, issue_number_from_description().
2. Confirm the issue a PR description names is open and in this repository. Adds the gh api lookup and the script's entry point.
3. Run the PR description check on every pull request event that can change it. Adds the workflow.
4. Name a tracking issue on the first line of dependency-update PRs.
5. Enforce sim.process.pr-tracking-issue with the PR description job. Adds the registry entry, proof fixtures, docs and rendered views.
6. Skip the PR description check on closed pull requests. Fixes a self-review finding.
7. Say every PR needs a tracking issue, and test two more lookup paths. Fixes self-review findings.
8. Say the PR description check runs on open pull requests. Corrects wording.

Design decisions

A separate workflow, not a pr.yml job.

• pr.yml runs full CI and doesn't trigger on edited. Adding edited there would rerun all of CI on every description edit.
• test_ci_complete_gate.py requires every pr.yml job to be needed by CI complete and to carry the "lowest PR in the stack" condition. Neither fits a check that must run on every PR.
• The new workflow is listed in the registry's [discovery].workflows, so the rules guard requires every job in it to be registered, as it does for pr.yml.

Runs on every open PR, including stacked ones.

• No branches or paths filter, and no stack condition.
• The job's only condition is github.event.pull_request.state == 'open'. GitHub also sends edited for closed PRs, and merging a PR closes the issue it names, so editing a merged PR's description would otherwise show a failing check on it.
• A newer event for the same PR cancels a run still in progress (concurrency, keyed on the PR number).

Permissions.

• The workflow's token has contents: read, issues: read and pull-requests: read, and nothing else.
• The check step gets the token as GH_TOKEN: ${{ github.token }}.
• The checkout is a sparse checkout of the script only, and the job runs no PR code beyond that script.
• PRs from forks get a read-only token, which is all the check needs.

How the first line is read (each choice has a test):

Input	Result	Why
CRLF line endings (Fixes #404 \r\n…, as in #405)	pass	The web editor writes CRLF. A lone \r also ends a line, as in Markdown.
Trailing spaces or tabs	pass	They don't show, and GitHub still reads the closing keyword. 4 of the last 300 PRs had Fixes #N \r.
Leading whitespace	fail	Four spaces make the line a Markdown code block, where GitHub doesn't read closing keywords.
Blank first line, or Fixes #N further down	fail	The message gives the line number where Fixes #N was found.
Empty or null description	fail
Fixes #0, Fixes #0861, non-ASCII digits	fail	The number must be ASCII digits without a leading zero.
Fixes #615., fixes #1, Closes #1, Fixes owner/repo#1, an issue URL	fail	The line must be exactly Fixes #N.

Which numbers count.

• A pull request fails. The issues API also answers for PRs; the answer then has a pull_request field.
• An issue moved to another repository fails. The script compares the answer's repository_url (ignoring case, as GitHub does) and number with the request. gh api follows GitHub's redirect for a transferred issue. For example, #154 here answers as PolicyEngine/policyengine.py#115, which is the case the test uses.
• A closed issue fails. The message says to reopen the issue and re-run the job.
• HTTP 404 or 410 fails with "There is no issue #N in …".
• Any other gh failure fails with gh's own message.

The description comes from the event payload, as the plan specifies. Re-running a run checks the description as it was when that run started. github-prs.md says so: after editing, rely on the run the edit starts; after reopening an issue, re-run the job.

Where the tests live.

• The tests are in projects/policyengine-simulation-entry/tests/, beside the other .github/scripts tests (test_workflow_scripts.py).
• They use the same FakeCommands helper, which puts a fake gh first on PATH, so no test makes a network call.
• pr.yml's test job runs them, and the proof is registered against that job.
• They are pytest tests, not stdlib unittest: that is the convention for this directory, and ruff's PT rules apply there.

Dependency-update automation

• The policyengine package update (update-policyengine-package.sh) already opens an issue per version and starts its PR with Fixes #N, as chore(deps): update policyengine to 6.9.0 #857 and chore(deps): update policyengine to 6.8.0 #832 do. Unchanged.
• The daily dependency update (dependency-update-create-pr.sh) now runs whenever the update branch was pushed, with or without an open update PR:
  • It uses the oldest open issue titled "Update dependencies", skipping pull requests, or opens one. Fixes #N closes it when the PR merges, so the next PR gets a new issue.
  • Without an open update PR, it opens one whose description starts with Fixes #N.
  • With one, it makes Fixes #N the first line: it replaces an earlier Fixes # line, or adds the line above a description that has none.
  • Open PR Update dependencies #583, created in June, has none. Its first push after this merges fails the check once, and the edited event from the description update then passes it.
  • A failed issue lookup stops the script before it can open a duplicate issue. Bash turns off set -e inside $(...), so the lookup sets a variable rather than printing.
• Token permissions.
  • update-dependencies.yml creates its GitHub App token with no permission-* inputs, so the token has every permission of the App's installation.
  • That installation can write issues: check-policyengine-updates.yml requests permission-issues: write from the same organization secrets (APP_ID, APP_PRIVATE_KEY, visible to all repositories). Its runs succeed, and policyengine[bot] opened Update policyengine to 6.8.0 #831 and Update policyengine to 6.9.0 #856.
  • I did not add permission-* inputs. Naming any permission limits the token to the ones named, and this job also pushes merges from main, which can carry workflow-file changes that need further permissions.
• The daily workflow currently fails at "Generate API clients" every day, which is unrelated to this PR, so it opens no PRs until that is fixed.

After merge

Make PR description a required status check in the main ruleset, next to CI complete, with GitHub Actions as its source.

The check is reported only after an event on a PR, so open PRs need a push, a description edit, or a close and reopen before it appears. #583 gets one on the daily job's next successful push.

Environment variables

Variable	Read by	Provider	How it reaches the step	Verified
GITHUB_EVENT_PATH	check_pr_description.py	GitHub Actions runner	Set by the runner for every step	Yes, on this PR's runs
GITHUB_REPOSITORY	check_pr_description.py, dependency-update-create-pr.sh	GitHub Actions runner	Set by the runner for every step	Yes, on this PR's runs
GH_TOKEN (check)	gh in check_pr_description.py	The workflow's github.token	Step env: in pr-description.yml, limited by the workflow permissions: block	Yes, on this PR's runs
GH_TOKEN (dependency update)	gh in dependency-update-create-pr.sh	GitHub App token from actions/create-github-app-token, using organization secrets APP_ID and APP_PRIVATE_KEY	Step env: from steps.app-token.outputs.token, unchanged	Issue-write permission inferred from the other workflow (above). The new issue and edit calls have not run in production yet.

Suppressions

None added.

Red and green record

Commit	Red	Green
1, parser	26 errors: FileNotFoundError, the script didn't exist	26 passed
2, lookup	18 failed: no entry point, so the script exited 0 printing nothing. The two "passes" cases passed trivially for the same reason.	46 passed
3, workflow	4 errors: FileNotFoundError for pr-description.yml	50 passed; actionlint clean
4, automation	11 failed: no issue lookup, a fixed --body, and the step still required has_open_pr == 'false'	35 passed, after the test helper was changed to read bytes, because read_text turned the CRLF under test into LF
5, registry	The proof test failed because its fixtures were missing. With the workflow added to discovery alone, the guard reported CI job 'pr-description' … enforces no registered rule.	51 passed; guard passed
6, closed PRs	KeyError: 'if'	51 passed
7, extra tests	These cover existing behaviour, so they passed when added	52 and 36 passed

Self-review

This is a self-review, so it assigns no review disposition.

Finding	Resolution
Editing a closed or merged PR's description would run the check and fail, because merging closed the issue	Fixed in commit 6 (red and green above)
README said to open an issue only "for non-trivial work"; AGENTS.md listed PR CI without the new check	Fixed in commit 7
No tests for HTTP 410, or for a Fixes #N line that is an open update PR's whole description	Added in commit 7
github-prs.md said "every pull request" after commit 6; a fixture docstring was inaccurate	Fixed in commit 8

A further pass found nothing more to act on.

Known limitation, not changed. pull_request workflows run the PR's own copy of the workflow and script, so a PR could weaken the check that runs on itself. The plan's §9.1.4 accepts this for workflow files, and reviewers read workflow diffs. pull_request_target would run main's copy instead, but it wouldn't run on the PR that adds it.

Testing

• uv run pytest tests/ in projects/policyengine-simulation-entry: all pass. That includes test_pr_description_check.py (52) and test_workflow_scripts.py (36).
• python -m unittest discover -s scripts/rules/tests -t .: OK.
• python scripts/run_quality_guards.py: rules-registry and ci-scripts pass.
• python -m scripts.rules guard --base-ref origin/main: passes.
• ruff 0.15.22: ruff check . and ruff format --check . are clean from the repository root.
• shellcheck and actionlint: clean on the changed script and workflows.
• Live, on this PR:
  • Pass: run 39616984737, from the edited event with this description, passed. The job token had Contents, Issues, Metadata and PullRequests read only. The step printed "The first line is Fixes Check that every pull request description starts with Fixes #N for an open issue in this repository #861, and Check that every pull request description starts with Fixes #N for an open issue in this repository #861 is an open issue in PolicyEngine/policyengine-sim-api."
  • Fail: with the first line temporarily changed to Refs #861, run 39619516824 failed. Its annotation, titled "PR description", read: The first line is "Refs #861". Edit the description so its first line is exactly Fixes #ISSUE_NUMBER, …. The description was then restored, which starts another run.
• Live lookups, run locally with my own gh credentials (not the job token):
  • Check that every pull request description starts with Fixes #N for an open issue in this repository #861 (open issue, CRLF body): passed.
  • chore(deps): update policyengine to 6.9.0 #857: failed, "is a pull request".
  • #154: failed, "moved to Create reform serializer and deserializer policyengine.py#115".
  • Refactor folder structure into shared monorepo #1: failed, "is closed".
  • #999999: failed, "There is no issue".
  • Refs #861: failed, wrong first line.

🤖 Generated with Claude Code