Skip to content

feat(ci): a configuration change is exercised in a job that holds nothing, or refused by name #430

Description

@tpouyer

Split out of #421, which reports that CI did not exercise a configuration change but does not run
it. Reporting is the cheap half and it lands first; this is the half that would have turned #418
red before it merged instead of after.

What #421 leaves standing

config_ref.py argues, correctly, that configuration must not come from the ref under review:

Under review, that root is the pull request's — which would mean the change being reviewed
supplies the module defining every binding, every policy contribution, the egress mode, and the
protected-path tiers that are supposed to constrain reviewing it.

So every job on a pull request loads base's .lockstep/lockstep.py. #418 changed the Validate
binding, its container job passed against main's binding in 2m39s, and main was red within the
hour. #420 reverted it and its container job failed, against the same stale configuration. Both
jobs did exactly what they were designed to do. After #421 both would say so; neither would have run
the change.

The shape

Run selfcheck a second time with the change's own configuration, in a job that holds no credential
and no write token — the shape the sandbox job already has. A maintainer's same-repo pull request
is a case where the configuration is human-authored by construction, which is the sentence
config_ref.py itself uses. A fork's is exactly what the rule refuses, and stays on base.

Not the fix, and this stays written down: loading the change's configuration in the privileged
jobs. That is the control #421 is about not weakening.

What the design work on #421 turned up

A second entry point, not a bypass parameter. read_config refuses trusted=False
unconditionally today. Giving it an allow_untrusted=True argument makes the refusal a control with
a switch. A second, explicitly-named function is a second call site a test can enumerate and a
reader can grep for, and the refusal keeps no parameter that turns it off.

ConfigRef.under_review comes back here. #421 deletes it, because it is constructed by no caller
but the test asserting the refusal. This is the path that takes it, and the config the ref under review line it produces is half of what makes the job legible in a log.

CiEnvironment has no fork field. It carries host, repo, ref, base_ref, pr_number,
actor, run_id, oidc_available, event. Whether the head repository is a fork is not in it, and
the refusal needs it rather than reading github.event... in a workflow if:.

The authorization is a Decision, not a boolean. platform/actors.py::authorize already returns
one carrying everything considered, so a refusal can be read without re-running it. The same shape
here: which credential was found in the environment, whether the head is a fork, what the actor's
association was, and which of those declined.

A question worth settling before building. ci.yml's check job already runs a fork's own test
code on a credential-free runner — make ci executes whatever the pull request wrote. So executing
fork code is not new here, and the marginal risk of executing a fork's lockstep.py in a job that
holds nothing is narrower than it first reads: what configuration additionally decides is bindings,
egress mode and tool grants, none of which bite in a job that calls no model. Whether the fork
refusal is therefore load-bearing or kept for legibility is a decision to make deliberately, not one
to inherit from the sentence in config_ref.py.

It needs a workflow edit, which a model cannot make. .github/ and .lockstep/ are both in
DENY_ALWAYS (core/changes.py), and no grant lifts either. Whatever this issue asks for in
.github/workflows/ is a person's act and has to be written up as one — an acceptance criterion the
framework forbids a model from satisfying is how the second /implement on #410 was set up to fail.

Acceptance

  • Whatever runs the change's own configuration holds no credential and no write token, and refuses by
    name where the actor or the fork makes it inappropriate — the refusal quoting which rule declined,
    not a bare skip.
  • A fork's pull request never loads its own configuration in any job, asserted with a negative
    control rather than by reading the code.
  • The refusal is checked at run time as well as asserted over the workflow files, so adding the flag
    to a job that holds a credential is refused by the process rather than only by a test.
  • ConfigRef.under_review is the constructor the new path uses.
  • The config line the run prints says the configuration came from the ref under review, so a log
    cannot be mistaken for an ordinary run.

Objectives

O3 — the same process at a terminal and in CI, which is what does not hold for the one file that
decides what the process is. O10 — this was found by the framework breaking its own default
branch, and #421 only makes that visible; this is what stops it happening.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions