Skip to content

fix(ci): add workflow-level permissions for PR comments#663

Open
morluto wants to merge 1 commit intogarrytan:mainfrom
morluto:fix/workflow-permissions
Open

fix(ci): add workflow-level permissions for PR comments#663
morluto wants to merge 1 commit intogarrytan:mainfrom
morluto:fix/workflow-permissions

Conversation

@morluto
Copy link
Copy Markdown
Contributor

@morluto morluto commented Mar 30, 2026

Summary

Fix the report job in the E2E Evals workflow failing with:

GraphQL: Resource not accessible by integration (addComment)

The Problem

When a PR is opened, the workflow runs 12 parallel eval suites, then the report job attempts to post a summary comment on the PR with results.

The report job fails because the GitHub token lacks pull-requests: write permission, even though the permission was declared at the job level.

Why It Matters

  • The PR author has no visibility into eval results — they must check the Actions tab manually
  • Failed PRs show ✅ green checkmarks in the PR UI despite having failures
  • The eval summary (pass/fail, costs, per-suite breakdown) never gets posted
  • Prevents proper CI feedback loops

Root Cause

GitHub Actions does not reliably propagate job-level permissions: blocks when:

  1. The workflow is triggered by pull_request (which grants a read-only token by default)
  2. There are needs: dependencies between jobs
  3. Docker container jobs exist in the dependency chain

The runner logs showed effective permissions as only Contents: read despite the job-level declaration.

The Fix

Move permissions: from the report job to workflow level. This ensures permissions are granted before any job executes, regardless of dependencies.

permissions:
  contents: read
  pull-requests: write
  packages: write

Testing

To verify the fix:

  1. Trigger the workflow: Push a commit or go to the Actions tab and run the workflow manually
  2. Watch the report job: It should complete with status success (not failure)
  3. Check for the PR comment: Look for a new comment on this PR with the eval summary table showing pass/fail counts per suite

Before fix:

  • report job: ❌ failed with exit code 1
  • PR comment: Not posted

After fix:

  • report job: ✅ success
  • PR comment: Posted with eval results table

@morluto morluto force-pushed the fix/workflow-permissions branch from 867ad38 to 12055d1 Compare March 30, 2026 10:48
@morluto morluto marked this pull request as ready for review March 30, 2026 10:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant