pr_agent_job reports pass on every pull request and has never posted a
single comment or review. Checked #209, #217, #218, #219, #221, #222, #223 and
#227 - zero comments, zero reviews, all green.
The job log says why:
.github/workflows/pr-agent.yml triggers on pull_request, and GitHub does not
pass repository secrets to workflows triggered from a fork. So
PR_AGENT_API_KEY and PR_AGENT_API_BASE resolve to empty strings. The action
still starts, applies repo settings, prunes the diff, and exits 0 having
generated nothing.
Every one of the PRs above came from a fork, which is to say the reviewer has
never run on a contributor PR - the only kind where a second opinion is worth
most. The green check reads as coverage and is not.
Two separate things to fix
1. Fail loudly when the key is missing. A reviewer that silently no-ops is
worse than no reviewer, because the check tells you it ran. A guard step that
exits non-zero (or at minimum annotates the run) when OPENAI_KEY is empty
would have surfaced this the first time it happened.
2. Decide how it should run on forks at all. pull_request_target would
give it secrets, but it runs against the base repo with write access, so it
needs the usual care about not checking out or executing fork code. If that
tradeoff is not wanted, then the job should be skipped explicitly on forks
rather than pretending to pass.
Also worth raising while it is open
.pr_agent.toml sets max_model_tokens = 32000. On #227 the diff was 44,103
tokens, so the action pruned files out of the review entirely. That run dropped:
test/vo2max_calorie_wiring_test.dart
test/workout_calorie_anchors_test.dart
lib/compute/manual_session.dart
lib/compute/profile.dart
lib/compute/crossday_pipeline.dart
lib/live/live_activity.dart
ios/OpenStrapWidget/OpenStrapWidgetLiveActivity.swift
ios/LiveActivityBridge.swift
Both Swift files, on a PR that changed the Live Activity ContentState shape -
which is exactly the kind of change nobody else was going to catch by reading.
So even with a working key, the largest PRs get the thinnest review, silently.
The pruning is at least logged; nothing surfaces it on the PR itself.
The instructions in .pr_agent.toml are good and specific to this codebase.
They have just never been used.
pr_agent_jobreports pass on every pull request and has never posted asingle comment or review. Checked #209, #217, #218, #219, #221, #222, #223 and
#227 - zero comments, zero reviews, all green.
The job log says why:
.github/workflows/pr-agent.ymltriggers onpull_request, and GitHub does notpass repository secrets to workflows triggered from a fork. So
PR_AGENT_API_KEYandPR_AGENT_API_BASEresolve to empty strings. The actionstill starts, applies repo settings, prunes the diff, and exits 0 having
generated nothing.
Every one of the PRs above came from a fork, which is to say the reviewer has
never run on a contributor PR - the only kind where a second opinion is worth
most. The green check reads as coverage and is not.
Two separate things to fix
1. Fail loudly when the key is missing. A reviewer that silently no-ops is
worse than no reviewer, because the check tells you it ran. A guard step that
exits non-zero (or at minimum annotates the run) when
OPENAI_KEYis emptywould have surfaced this the first time it happened.
2. Decide how it should run on forks at all.
pull_request_targetwouldgive it secrets, but it runs against the base repo with write access, so it
needs the usual care about not checking out or executing fork code. If that
tradeoff is not wanted, then the job should be skipped explicitly on forks
rather than pretending to pass.
Also worth raising while it is open
.pr_agent.tomlsetsmax_model_tokens = 32000. On #227 the diff was 44,103tokens, so the action pruned files out of the review entirely. That run dropped:
Both Swift files, on a PR that changed the Live Activity
ContentStateshape -which is exactly the kind of change nobody else was going to catch by reading.
So even with a working key, the largest PRs get the thinnest review, silently.
The pruning is at least logged; nothing surfaces it on the PR itself.
The instructions in
.pr_agent.tomlare good and specific to this codebase.They have just never been used.