Skip to content

fix(ci): render the review lanes' fork check name instead of its expression source - #8216

Merged
bolichen97 merged 1 commit into
mainfrom
fix/fork-lane-skipped-name-renders-raw-expression
Sep 3, 2026
Merged

fix(ci): render the review lanes' fork check name instead of its expression source#8216
bolichen97 merged 1 commit into
mainfrom
fix/fork-lane-skipped-name-renders-raw-expression

Conversation

@iamwhatever

Copy link
Copy Markdown
Collaborator

Problem / Motivation

Every fork PR's check list carries five rows whose names are raw GitHub Actions
expression source rather than a check name, e.g.

github.event.pull_request.head.repo.full_name == github.repository && 'GPT 5.6 Review' || 'GPT 5.6 Review (same-repo lane, not applicable to forks)'

One per same-repo AI review lane (GPT, Opus, Design, UX, First Principles).
Observed on #7518; same-repo
PRs render correctly.

Why it matters

Every external contributor sees five unreadable rows on their PR, and a
maintainer scanning a fork PR's checks cannot tell at a glance which lane each
row belongs to. It also makes the collision fix itself look broken: the intended
alias name (... (same-repo lane, not applicable to forks)) is present only as a
fragment inside the expression text.

What changed (motivation → approach → change)

Symptom: the fork PR check-run name is the name: expression's source text.

Root cause: GitHub does not evaluate a skipped job's name: expression. The
fork guard was a job-level if:, so on a fork PR the job never started, the
name was never evaluated, and the check-run was published under the raw
expression body.

Change: the guard moves off the job and onto every step, in all five lanes. On a
fork PR the job now starts, evaluates its name, skips every step, and completes
as an empty success under the intended alias. The unavoidable cost is that a
runner slot is taken for a few seconds per lane per fork PR event; the
alternative (a static job name) is what created the branch-protection collision
that #7982 fixed, so it is not available.

Semantics are preserved rather than re-derived: each existing step condition is
wrapped, so if: always() becomes if: <guard> && ( always() ). The expression
still contains a status function, so GitHub's implicit success() is still not
applied and the post/gate steps still run after a failed review step. Steps with
no status function keep their implicit success().

Removing the single job-level guard turns one fail-closed boundary into 49
per-step ones, in lanes that hold pull-requests: write and id-token: write.
That regression is closed by test, not by convention: the guard is now asserted
per step, and asserted not to be at job level (which would silently restore
the rendering bug).

Also in the diff: the job-header rationale comments in all five lanes, the
docs/ci/ci-and-reviews.md security-posture bullet, and one stale
pr-readiness.yml comment claiming the same-repo lane posts a skipped
check-run under the protected name on forks — untrue since #7982 renamed it.

No behaviour change for same-repo PRs: the job ran before and runs now, and every
step's effective condition is unchanged (true && (<original>)).

Tests

test/test_ai_review_workflows.py, in
TestProtectedCheckNameHasOnePublisherPerPrType:

  • test_every_step_carries_the_fork_guard (new, parametrized over all five
    lanes) — asserts every step's if: contains the fork guard, so a step added
    later without it fails CI instead of executing against fork content.
  • test_same_repo_lane_keeps_the_protected_name_only_for_same_repo_prs
    (amended) — the old assert GUARD in job["if"] is replaced by
    assert "if" not in job, locking out the placement that stops the name from
    rendering.

Both are mutation-verified: dropping the guard from one step fails the first,
and re-adding a job-level if: fails the second.

Manual verification

Confirmed the root cause against live check-run data rather than by reading the
YAML: fork PR #7518 publishes all five lanes under raw-expression names with
status skipping, while same-repo PRs #8199 / #8194 / #8189 publish the exact
protected names. The change is not observable locally — GitHub's own scheduler
decides whether a job is skipped — so the rendered fork name will be verified on
this PR's own CI if it is pushed from a fork, and otherwise on the next fork PR
after merge.

Local gates green: 302 workflow/profile tests, full backend suite matching the
origin/main baseline exactly (89 pre-existing environment failures on both
sides), black / isort / flake8, brand, docs-lint, changelog, feature-map,
harness-parity, focus-cue, testpaths, scrub-lint.

Related Issues

no linked issue: follow-up to #7982, which introduced the conditional name this
PR makes render.

Pattern harvest

Rule candidate: review-prompt
Pattern: a workflow expression that is only evaluated on some paths (a skipped
job's name:) silently ships its own source text as the value.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

…me renders

The five same-repo AI review lanes name their job conditionally so a fork PR
gets an alias instead of the protected check name. GitHub never evaluates a
SKIPPED job's `name:`, and the fork guard was a job-level `if:` -- so on every
fork PR the check-run was published under the raw expression source, e.g.
`github.event.pull_request.head.repo.full_name == github.repository && 'GPT
5.6 Review' || 'GPT 5.6 Review (same-repo lane, not applicable to forks)'`.

The guard now sits on every step instead. The job starts on a fork, evaluates
its name, runs no step, and completes as an empty success under the intended
alias. Same-repo PRs are unchanged.

Because the one job-level guard became a per-step guard, a step added later
without it would execute against fork content in a lane holding
`pull-requests: write` and `id-token: write`. `test_ai_review_workflows.py`
now asserts the guard per step, and asserts it never returns to the job level.
@iamwhatever
iamwhatever requested a review from a team as a code owner September 3, 2026 17:28
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Sep 3, 2026
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — ✅ PASS

Design-level review of 7ab9727b7f02b051c80202828cc0a6c17865998a — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Root cause correctly named (skipped jobs never evaluate name:), the only viable shape given #7982's collision, and the diluted fork boundary is re-pinned by mutation-verified tests.

Suggestions

  • The step-guard test accepts any if: containing the guard substring, so always() || <guard> would pass while running on forks; assert the guard is the leading && conjunct instead, since this test is now the sole enforcement of a lane holding id-token: write.

[DESIGN-REVIEWED] 7ab9727

@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — ✅ PASS

Premise-level review of 7ab9727b7f02b051c80202828cc0a6c17865998a — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Both grep sweeps confirm the conditional-name pattern (name: "${{ ... && ... || ... }}") exists in exactly the five lanes this PR touches — no unfixed siblings — and no job-level fork guard remains anywhere in .github/workflows/. The root cause (GitHub never evaluates a skipped job's name: expression) is a platform rule, and the static-name alternative is foreclosed by the #7982 collision decision already recorded in the repo. Every inventory item is declared in the description, including the status change and the runner-cost trade-off.

First-Principles-Verdict: PASS

A platform rule (skipped jobs never evaluate name:) forces the per-step guard; every item is declared, tested, and sibling-complete.

What this change ships

Intent: make fork PRs' five review-lane check rows show readable names instead of raw expression source — a FIX.

  1. Fork PR check rows render the intended alias name, not expression source — justified (the fix).
  2. Five lane jobs on a fork now finish as empty green instead of skipped — declared consequence.
  3. Fork PR events now spend seconds of runner time per lane — declared cost; static name recreates the fix(ci): give the same-repo review lanes a distinct check name on fork PRs #7982 collision, so no smaller option.
  4. Fork boundary becomes 49 per-step guards instead of one job guard — declared risk, pinned by the new parametrized test.
  5. New test failing any lane step missing the guard — justified regression lock.
  6. Amended test forbidding a job-level guard — justified; that placement silently restores the bug.
  7. Rewritten lane comments + docs/ci/ci-and-reviews.md bullet — mandated same-commit doc sync.
  8. Corrected stale pr-readiness.yml comment about skipped runs — declared, trues up the same fact.

Grepped name: "${{ ... && ... || across .github/workflows/: 5 hits, all fixed here — zero unfixed siblings. The conditional-name design itself is a recorded decision (#7982), not relitigated.

[FIRST-PRINCIPLES-REVIEWED] 7ab9727

@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Opus 4.8 Review — ✅ no blocking findings

Reviewed 7ab9727b7f02b051c80202828cc0a6c17865998a — this comment is updated in place on each push.

Review details

All five lanes carry the fork guard on every step; I verified design (6/6), codex (14/14), claude (10/10), and first-principles (9/9) directly, and the ux hunks add it to all 7. The composed conditions (<repo> && ( … )) skip on forks and reduce to the original on same-repo, and always()-bearing steps still run regardless of prior failure only when same-repo — no same-repo behavioral regression, and no ungated privileged step. The pr-readiness, docs, and test changes are consistent and non-executing.

No findings.

[OPUS-REVIEWED] 7ab9727

Verdict parsed from the review's SHA-scoped output markers for commit 7ab9727b7f02b051c80202828cc0a6c17865998a.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 7ab9727b7f02b051c80202828cc0a6c17865998a: <one-sentence reason>

@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of 7ab9727b7f02b051c80202828cc0a6c17865998a and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 7ab9727

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 7ab9727b7f02b051c80202828cc0a6c17865998a: <one-sentence reason>

@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Sep 3, 2026
@bolichen97
bolichen97 merged commit a9469df into main Sep 3, 2026
64 checks passed
@bolichen97
bolichen97 deleted the fix/fork-lane-skipped-name-renders-raw-expression branch September 3, 2026 21:12
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Sep 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants