[WIP][experimental] multi turn chat benchmark

[cquil11]

No description provided.

In general, to fix this class of issue you add an explicit permissions: block to the workflow (or to each job) that grants only the minimal scopes needed. If the jobs only need to read repository contents and use artifacts, a typical baseline is permissions: contents: read at the workflow level. Additional fine-grained permissions are added only if some step needs them.

For this specific workflow, nothing in the provided snippet requires write access via GITHUB_TOKEN: repository writes are done via a separate secrets.REPO_PAT, and artifact operations work with the default contents: read (artifact permissions are managed separately and are allowed with minimal contents scope). Therefore, the best change that does not alter existing functionality is to add a workflow-level permissions block directly under the name/run-name section, before on:, setting contents: read. This will apply to all jobs that don’t override permissions, including generate-matrix, sweep, and collect, and will ensure the token is limited to read-only access to repository contents.

Concretely: in .github/workflows/multiturn-sweep.yml, insert:

permissions:
  contents: read

after line 2 (right after run-name:) and before the on: block. No additional imports or methods are required.

In general, you fix this by explicitly defining a permissions: block either at the top level of the workflow (to apply to all jobs) or individually per job, granting only the minimal scopes required (typically contents: read for reading the repo and actions: read for artifact access; the artifact actions don’t require additional special scopes).

For this specific workflow, the minimal, safe change without altering functionality is:

• Add a top-level permissions: block after the run-name: and before on: that sets contents: read. This allows jobs to read the repository with GITHUB_TOKEN if needed, while not granting write.
• No per-job overrides are necessary because none of the shown jobs need to write to the repository or interact with issues/PRs; they only check out code (already handled by PAT) and handle artifacts (which work with default GITHUB_TOKEN + actions implied). GitHub recommends at least setting contents: read explicitly to mirror a read-only default.

Concretely, in .github/workflows/multiturn-sweep.yml, insert:

permissions:
  contents: read

between line 2 (run-name: ...) and line 4 (on:).

In general, the fix is to explicitly declare a permissions: block that limits the default GITHUB_TOKEN to the minimal privileges required, typically at the workflow root so it applies to all jobs unless overridden. For this workflow, none of the jobs need to write to the repository; they only check out code, run benchmarks, and upload artifacts. The artifact actions do not require repository write permissions, and code checkout only needs contents: read. Therefore the best fix is to add a root-level permissions: block with contents: read near the top of .github/workflows/multiturn-sweep.yml.

Concretely, add:

permissions:
  contents: read

right after the run-name: (line 2) and before the on: block (line 4). This will apply the restricted permissions to all jobs (generate-matrix, sweep, collect) without changing any existing job logic or secrets usage. No additional methods, imports, or definitions are required, since this is purely a YAML configuration change.