diff --git a/.github/dependabot.yml b/.github/dependabot.yml index 500d40c..b295f0b 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -56,3 +56,15 @@ updates: applies-to: security-updates patterns: - "*" + ignore: + # Bumped by hand, not weekly by Dependabot. Each release of the Claude action + # bundles a fixed Claude Code version, and that CLI is what confines the + # automated reviewer (tool list, permission mode, deny rules). A bump needs a + # check of what changed: the action's diff between the two commits and the + # Claude Code changelog for the bundled versions in between. + # To bump: update the commit SHA and the version comment of the pin in + # .github/workflows/claude-review.yml, and the Claude Code version named in + # the comment above the pin. See .github/scripts/claude-review/README.md. + # Ignore rules also apply to Dependabot security updates, so no security PR + # is raised for this action either; watch its releases and advisories. + - dependency-name: "anthropics/claude-code-action" diff --git a/.github/scripts/claude-review/README.md b/.github/scripts/claude-review/README.md index c51a945..a5955b4 100644 --- a/.github/scripts/claude-review/README.md +++ b/.github/scripts/claude-review/README.md @@ -9,7 +9,7 @@ change the pinned versions. | Job | Step | Script | Holds | |-----|------|--------|-------| | `review` | Fetch the pull request diff | `fetch-diff.sh` | read-only token | -| `review` | Review with Claude | (action) | Claude credential, read-only token | +| `review` | Review with Claude | (action) | Claude credential (from the `claude` environment), read-only token | | `verdict` | Validate and render | `report.sh`, `validate.jq`, `render.jq` | nothing sensitive | | `verdict` | Post or update the comment | `post-comment.sh` | token with `pull-requests: write` | | `verdict` | Enforce the verdict | (inline) | nothing | @@ -69,8 +69,10 @@ comment, and the runner image is `ubuntu-24.04` rather than `ubuntu-latest`. The commit and its version live together on each `uses:` line, for example `actions/checkout@ # v7.0.1`, so there is no second list to keep in -step. Dependabot updates both parts. To bump one by hand, resolve the tag and -replace the commit and the comment wherever the action is used: +step. Dependabot updates both parts for every action except the Claude action, +which `.github/dependabot.yml` ignores so that it is bumped by hand. To bump one +by hand, resolve the tag and replace the commit and the comment wherever the +action is used: ```bash gh api repos/OWNER/REPO/commits/TAG --jq .sha @@ -78,10 +80,58 @@ gh api repos/OWNER/REPO/commits/TAG --jq .sha The Claude action is not an ordinary bump. Each release hard-codes a CLI version, and the CLI is what enforces the tool list, the permission mode and -the deny rules that keep the reviewer away from its own credential. The -reviewer itself reports a change of that pin as `high`, so such a pull request -does not get a green verdict on its own. Before merging one, re-check the -confinement described below. +the deny rules that keep the reviewer away from its own credential. Before +bumping it, check what changed: the action's diff between the two commits and +the Claude Code changelog for the bundled versions in between. Then update the +commit and the version comment in `claude-review.yml`, along with the Claude +Code version named in the comment above the pin. The reviewer itself reports a +change of that pin as `high`, so such a pull request does not get a green +verdict on its own. Before merging one, re-check the confinement described +below. + +Checks done for past bumps of the Claude action: + +- **v1.0.244 to v1.0.248** (Claude Code 2.1.292 to 2.1.296, checked + 2026-10-22). The action's diff between the two commits is four commits that + only raise the bundled Claude Code and Agent SDK versions; 2.1.296 was read + from `src/entrypoints/run.ts` and `base-action/action.yml` at the new commit. + In the Claude Code changelog for 2.1.293 to 2.1.296, the one entry touching + the reviewer's limits is a 2.1.295 fix to `--tools` not applying to built-in + tools that register after launch, which tightens them. 2.1.296 adds an + `allow_large` option to `Read` for files over the usual size limit; it + concerns size, not which paths may be read. Nothing found widens the paths + the reviewer can read. Confinement on Linux remains unobserved, as below. + +## Credential scope + +`CLAUDE_CODE_OAUTH_TOKEN` is used by one job: `review` here. It runs in the +`claude` environment with `deployment: false`, which uses the environment for +its secret and its branch rule and creates no deployment records. + +The environment is meant to be set up as follows (repository settings, not +code): deployment branches limited to the default branch, no required +reviewers, no wait timer, and the token stored there. A workflow defined on +any other branch then cannot read the token, because GitHub matches the branch +rule against the ref a run executes on, and `pull_request_target` executes on +the default branch (as would `issue_comment`, should a mention workflow be +added). + +Two things to keep in mind: + +- **The scope is only real once the repository-level secret of the same name + is deleted.** While it exists, any workflow on any branch can read it, and + this job silently falls back to it. +- If neither copy exists, the review ends as `credential_unavailable` (red). It + does not pass. +- Runs started by Dependabot have been seen to receive repository-level + Actions secrets here. That they also receive the environment's secret is + expected, since the same runs are treated as ordinary Actions runs, but it + can only be observed once the repository-level copy is gone: the first + review Dependabot starts after that either produces a verdict or ends as + `credential_unavailable`. + +`test.sh` fails if any workflow uses the credential outside that environment or +on an event whose definition a pull request controls. ## Read confinement: what has and has not been checked @@ -93,12 +143,23 @@ refused), and the deny list in `settings`. **This was probed with a local CLI (2.1.284) on macOS only.** There, reads, greps and globs outside the two locations were refused, the deny list held even with reads deliberately pre-approved, and pre-approving `Read` was shown to -open every path. **It has not been exercised on a Linux runner**, and not with -the CLI version the pinned action installs (2.1.292). That matters because on -Linux the credential is also present in the reviewer's process environment -under `/proc`, which macOS does not have. `/proc` is covered both by the -working-directory limit and by an explicit deny rule, but neither has been -observed to hold there. +open every path. + +The first runs on the default branch (from 2026-10-14, started by Dependabot +on its grouped pull requests and by a maintainer, `ubuntu-24.04`, CLI 2.1.292) +showed that the workflow works end to end on Linux with the pinned CLI: the +arguments were accepted, every run received the credential (`Secret source: +Actions`), valid verdicts came back and the comments were posted. **They say +nothing about confinement**: the reviewer made no attempt to read outside its +two locations (every run logged zero permission denials), so nothing was there +to be refused. The pin has since moved to a release that installs 2.1.296, +which has not run here yet. + +**Confinement on a Linux runner therefore remains unobserved.** That matters +because on Linux the credential is also present in the reviewer's process +environment under `/proc`, which macOS does not have. `/proc` is covered both +by the working-directory limit and by an explicit deny rule, but neither has +been seen to hold there. Re-check whenever the Claude action pin, `claude_args` or `settings` change: in a private scratch repository with a throwaway credential, run the same action @@ -135,7 +196,7 @@ tool list, schema, pins). Needs `jq`; the checks that parse YAML also need `ruby`. `.github/workflows/claude-review-scripts.yml` runs the same tests on a hosted -runner for every pull request that touches these files. +runner for every pull request that touches these files or any workflow file. The review workflow itself runs only from the default branch, so a pull request that changes it or these scripts is still reviewed by the version already diff --git a/.github/scripts/claude-review/test.sh b/.github/scripts/claude-review/test.sh index 8ad25f1..5c792e2 100644 --- a/.github/scripts/claude-review/test.sh +++ b/.github/scripts/claude-review/test.sh @@ -440,8 +440,11 @@ if command -v ruby > /dev/null 2>&1; then check "workflow: the verdict job tells report.sh whether Dependabot opened the pull request" is "$(wf '.jobs.verdict.steps[] | select(.id == "report") | .env.OPENED_BY_DEPENDABOT')" "\${{ github.event.pull_request.user.login == 'dependabot[bot]' }}" check "workflow: nothing is uploaded" is "$(wf '[.jobs[].steps[] | .uses // empty | select(test("upload-artifact"))] | length')" 0 check "workflow: header no longer suggests reusing the credential as a Dependabot secret" test -z "$(grep -F 'makes this workflow work unchanged' "$workflow")" - check "workflow: header warns against pull-request-defined uses of the credential" grep -qF 'pull_request_review_comment' "$workflow" - check "workflow: header states that confinement is untested on Linux" grep -qF 'been exercised on a Linux runner' "$workflow" + check "workflow: header says the first run does not test the read confinement" grep -qF 'does not test the reviewer'"'"'s read confinement' "$workflow" + check "workflow: header says confinement on a Linux runner remains unobserved" grep -qF 'remains unobserved' "$workflow" + check "workflow: header says the scope depends on deleting the repository-level secret" grep -qF 'repository-level secret of the same name has been' "$workflow" + check "workflow: review job takes the credential from the claude environment, without deployments" is "$(wf '.jobs.review.environment | tojson')" '{"name":"claude","deployment":false}' + check "workflow: verdict job references no environment" is "$(wf '.jobs.verdict | has("environment")')" false # The last step turns the verdict into the job conclusion: only "pass" exits 0. enforce="$(wf '.jobs.verdict.steps[-1].run')" @@ -454,17 +457,24 @@ if command -v ruby > /dev/null 2>&1; then check "workflow: the verdict step is the last step and is not conditional" is "$(wf '.jobs.verdict.steps[-1] | [.name, has("if"), has("continue-on-error")] | join(",")')" 'Enforce the verdict,false,false' check "workflow: only the comment step may fail without failing the job" is "$(wf '[.jobs[].steps[] | select(.["continue-on-error"] == true) | .name] | join(",")')" 'Post or update the pull request comment' - # No workflow defined by a pull request (plain pull_request trigger) may use the credential. - leak="" + # Every workflow that uses the Claude credential, whichever file it is in: + # * may only be started by events that run the default branch's definition + # (a pull request cannot redefine it), and + # * must take the credential, in every job that references it, from the + # `claude` environment, so that the environment's branch rule applies. + wrong_trigger=""; wrong_scope=""; users="" for f in "$here"/../../workflows/*.yml "$here"/../../workflows/*.yaml; do [ -f "$f" ] || continue - if ruby -ryaml -rjson -e 'puts JSON.generate(YAML.load_file(ARGV[0]))' "$f" 2> /dev/null | - jq -e '((.on // .["true"]) | if type == "object" then has("pull_request") elif type == "array" then index("pull_request") != null else . == "pull_request" end) - and (tostring | test("CLAUDE_CODE_OAUTH_TOKEN"))' > /dev/null 2>&1; then - leak="$leak $(basename "$f")" - fi + ruby -ryaml -rjson -e 'puts JSON.generate(YAML.load_file(ARGV[0]))' "$f" > "$T/any.json" 2> /dev/null || { wrong_scope="$wrong_scope $(basename "$f")(unparsed)"; continue; } + jq -e 'tostring | test("CLAUDE_CODE_OAUTH_TOKEN")' "$T/any.json" > /dev/null 2>&1 || continue + users="$users $(basename "$f")" + jq -e '((.on // .["true"]) | if type == "object" then keys elif type == "array" then . else [.] end) + - ["pull_request_target", "issue_comment"] == []' "$T/any.json" > /dev/null 2>&1 || wrong_trigger="$wrong_trigger $(basename "$f")" + jq -e '[.jobs[] | select(tostring | test("CLAUDE_CODE_OAUTH_TOKEN")) | .environment == {"name": "claude", "deployment": false}] | length > 0 and all' "$T/any.json" > /dev/null 2>&1 || wrong_scope="$wrong_scope $(basename "$f")" done - check "workflows: none triggered by pull_request references the Claude credential" test -z "$leak" + check "workflows: the credential is used only by the review workflow" is "$users" ' claude-review.yml' + check "workflows: none that uses the credential can be redefined by a pull request" test -z "$wrong_trigger" + check "workflows: every job that uses the credential takes it from the claude environment" test -z "$wrong_scope" elif [ "${REQUIRE_WORKFLOW_CHECKS:-0}" = "1" ]; then bad "workflow: checks could not run (ruby not available)" else diff --git a/.github/workflows/claude-review-scripts.yml b/.github/workflows/claude-review-scripts.yml index d607695..3f440c1 100644 --- a/.github/workflows/claude-review-scripts.yml +++ b/.github/workflows/claude-review-scripts.yml @@ -1,23 +1,22 @@ name: Claude Review Scripts # Offline tests for the scripts behind claude-review.yml, run on the same -# runner image those scripts run on. Uses no secrets and a read-only token, so -# it is safe as an ordinary pull_request workflow. The review workflow itself -# cannot be exercised from a pull request: it always runs from the default -# branch. +# runner image those scripts run on. They also check every workflow file for +# how it uses the Claude credential, hence the path filter on all workflows. +# Uses no secrets and a read-only token, so it is safe as an ordinary +# pull_request workflow. The review workflow itself cannot be exercised from a +# pull request: it always runs from the default branch. on: pull_request: paths: - .github/scripts/claude-review/** - - .github/workflows/claude-review.yml - - .github/workflows/claude-review-scripts.yml + - .github/workflows/** push: branches: [main] paths: - .github/scripts/claude-review/** - - .github/workflows/claude-review.yml - - .github/workflows/claude-review-scripts.yml + - .github/workflows/** permissions: contents: read diff --git a/.github/workflows/claude-review.yml b/.github/workflows/claude-review.yml index eaab218..fd5bdc2 100644 --- a/.github/workflows/claude-review.yml +++ b/.github/workflows/claude-review.yml @@ -13,7 +13,8 @@ name: Claude Code Review # Title, description, commit messages and release notes are not passed on. # * The `review` job holds the Claude credential and a read-only token. The # reviewer has file-reading tools only: no shell, no network, no GitHub -# write access. +# write access. The job takes the credential from the `claude` environment +# (see "Credential scope" below). # * The `verdict` job holds a token that can comment on pull requests and no # Claude credential. It treats the reviewer's output as untrusted data: # strict validation, then rendering that cannot produce mentions, links or @@ -26,38 +27,53 @@ name: Claude Code Review # permission mode and the deny rules below are enforced. See the README next # to the scripts before changing a pin. # -# Not yet verified on a runner -# ---------------------------- -# The reviewer's read confinement (no --allowedTools, the permission mode, the -# settings deny list) was probed with a local CLI on macOS only. It has not -# been exercised on a Linux runner, where the credential is also present in -# the reviewer's process environment under /proc. Re-check it whenever the -# Claude action pin, claude_args or settings change. +# Observed on a runner, and not +# ----------------------------- +# The first runs on the default branch (from 2026-10-14, started by Dependabot +# on its grouped pull requests and by a maintainer, ubuntu-24.04, Claude Code +# 2.1.292) showed that the workflow runs end to end on Linux with the pinned +# CLI: the arguments were accepted, the credential reached the job, +# schema-valid verdicts came back and the comments were posted. The pin has +# since moved to a release that installs Claude Code 2.1.296. # -# Other uses of the credential -# ---------------------------- -# This is the only workflow in this repository that uses the credential. Do -# not add one that uses it on an event whose definition comes from the pull -# request (`pull_request`, `pull_request_review`, -# `pull_request_review_comment`): for a branch in this repository that -# definition, and its use of the credential, would be under the pull request's -# control. The credential is a repository-level secret, so a workflow pushed -# to any branch by someone with write access can still read it; keeping it in -# an environment restricted to the default branch would close that. +# What they show does not test the reviewer's read confinement (no +# --allowedTools, the permission mode, the settings deny list): the reviewer +# attempted nothing outside its two locations, so nothing was there to be +# refused. Confinement has been probed with a local CLI on macOS only. Whether +# it holds on a Linux runner, where the credential is also present in the +# reviewer's process environment under /proc, remains unobserved. Re-check it +# whenever the Claude action pin, claude_args or settings change. # -# The workflow this one replaces (claude-code-review.yml) also started Claude +# Credential scope +# ---------------- +# The `review` job, the only job in this repository that uses the credential, +# runs in the `claude` environment. When CLAUDE_CODE_OAUTH_TOKEN is stored in +# that environment only, and the environment's deployment-branch rule admits +# only the default branch, a workflow defined on any other branch cannot read +# it: the rule is matched against the ref a run executes on, which for +# pull_request_target is the default branch. `deployment: false` uses the +# environment for its secret and branch rule without creating deployment +# records. The environment must have no required reviewers and no wait timer. +# +# This holds only once the repository-level secret of the same name has been +# deleted. While that copy exists, every workflow in the repository can still +# read it, and this job falls back to it. Any other workflow that needs the +# credential must use the same environment and an event that runs the default +# branch's definition; test.sh checks both. +# +# The workflow this one replaced (claude-code-review.yml) also started Claude # on issue and pull request comments that mention @claude. This one does not. # # Dependabot # ---------- -# Runs started by Dependabot are subject to GitHub's restrictions on secrets. -# If such a run does not receive CLAUDE_CODE_OAUTH_TOKEN, it ends as "did not -# complete (credential_unavailable)"; the "Set up job" log of the run shows -# `Secret source:`. Should that turn out to be the case, the options are a -# dedicated, spend-capped key stored as a Dependabot secret, or a different -# trigger design. Do not put a personal subscription token into Dependabot -# secrets: they are handed to every workflow run that Dependabot triggers, -# including `pull_request` runs whose definition comes from the branch. +# The runs above that Dependabot started received Actions secrets ("Secret +# source: Actions" in their "Set up job" logs), so no Dependabot secret is +# needed. If a later run does not receive the credential, it ends as "did not +# complete (credential_unavailable)". The options then are a dedicated, +# spend-capped key stored as a Dependabot secret, or a different trigger +# design. Do not put a personal subscription token into Dependabot secrets: +# they are handed to every workflow run that Dependabot triggers, including +# `pull_request` runs whose definition comes from the branch. # # Scripts and their offline tests: .github/scripts/claude-review/ @@ -94,6 +110,10 @@ jobs: contains(fromJSON('["dependabot[bot]", "rbrenton", "schenksj", "chrisfauerbach"]'), github.event.pull_request.user.login) runs-on: ubuntu-24.04 timeout-minutes: 30 + # Source of CLAUDE_CODE_OAUTH_TOKEN; see "Credential scope" in the header. + environment: + name: claude + deployment: false permissions: contents: read # check out the default branch pull-requests: read # read the pull request and its diff @@ -126,10 +146,10 @@ jobs: # A step timeout fails the step, so the verdict job still runs and # reports "did not complete". timeout-minutes: 20 - # This release installs Claude Code 2.1.292, which is what enforces + # This release installs Claude Code 2.1.296, which is what enforces # the tool and path limits below. Re-check them on every change of # this pin (see the README next to the scripts). - uses: anthropics/claude-code-action@58985842b834ed26087302ba27d07bc24ca8697a # v1.0.244 + uses: anthropics/claude-code-action@1d6de8cb0c237e7c15e9e1bdf973826ebae490cc # v1.0.248 with: claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }} # The job's own read-only token. Passing it also keeps the action @@ -236,10 +256,11 @@ jobs: or action whose version changes, work out the old and new version from the diff, then: - - .github/dependabot.yml has no ignore list here. What is held on - purpose is visible in native/Cargo.toml instead; read the comments - next to the changed line. A change to any of the following is `high`, - even when the step is small: + - .github/dependabot.yml ignores only anthropics/claude-code-action, + which is bumped by hand (see the next rules). For dependencies, what + is held on purpose is visible in native/Cargo.toml instead; read the + comments next to the changed line. A change to any of the following + is `high`, even when the step is small: - the `git` and `rev` of the tantivy and quickwit crates, which come from the project's own forks at a fixed revision, including tantivy-sstable, tantivy-common and the [patch] section. These