Skip to content

sec(ci): deployment environments have no protection rules, so environment-bound credentialed jobs run unapproved #141

Closed

Description

@cristim

Found while fixing LeanerCloud/cloud-commitments-cli#1542 (PR LeanerCloud/cloud-commitments-cli#1641) and confirmed against the live API. This is the missing half of LeanerCloud/cloud-commitments-cli#1542's control LeanerCloud/cloud-commitments-cli#2. PR LeanerCloud/cloud-commitments-cli#1641 eliminated the injection and narrowed id-token: write onto four environment-bound jobs — but the environments those jobs bind to do not exist and have no protection rules, so the approval gate the narrowing was meant to enforce is not actually enforcing anything.

Evidence

$ gh api repos/LeanerCloud/CUDly/environments
total_count: 5
aws-fargate-dev:     protection_rules=[]  can_admins_bypass=true
aws-fargate-staging: protection_rules=[]  can_admins_bypass=true
azure-:              protection_rules=[]  can_admins_bypass=true
dev:                 protection_rules=[]  can_admins_bypass=true
gcp-:                protection_rules=[]  can_admins_bypass=true

Three separate problems:

  1. No environment in this repo has any protection rules. environment: currently only scopes secrets/vars and sets the OIDC sub claim. It is not a reviewer gate anywhere, in any workflow — not for rollback, not for database migrations, not for deploys.
  2. None of the 12 <cloud>-<env>-rollback environments exists, nor do staging / prod. GitHub auto-creates a referenced environment bare on first use, so the first rollback dispatch would silently create e.g. aws-lambda-prod-rollback with no reviewers and run straight through.
  3. azure- and gcp- are proof the auto-create path has already fired here. Those names are the residue of an environment: expression rendering empty — a workflow ran with an unset environment input, GitHub minted the junk environment, and nobody noticed.

Why it matters

The jobs binding to these environments hold id-token: write and run terraform apply -auto-approve against production. Auto-created-bare means a credentialed, production-mutating job with no approval step.

Aggravating factor (GCP): terraform/environments/gcp/ci-cd-permissions/github_oidc.tf:37 sets

attribute_condition = "assertion.repository == '${var.github_repo}' && assertion.ref == '${var.deploy_ref}'"

That constrains repository and ref but never sub. So unlike AWS and Azure — whose sub allowlists would reject an unrecognised environment:<name> (see LeanerCloud/cloud-commitments-cli#1648) — GCP would mint a token for a job bound to any environment name, including one auto-created bare. The environment name is itself built from a dispatch input (gcp-${{ inputs.environment }}-rollback), so if type: choice server-side enforcement were ever absent or bypassed, a dispatcher could name an arbitrary ruleless environment and still authenticate to GCP. Unverifiable without dispatching a real rollback, so recorded as a risk rather than a demonstrated exploit.

Fix direction

  1. Declare the environments in IaC, not by auto-creation — github_repository_environment plus github_repository_environment_deployment_policy, with reviewers set. Auto-creation is what produced both the missing rules and the junk azure- / gcp- names; declaring them is what stops it recurring.
  2. Set required reviewers on at least every prod and *-prod-rollback environment. Consider can_admins_bypass = false for prod.
  3. Delete the junk azure- and gcp- environments once the workflow that created them is identified and fixed (an environment: name must never be allowed to render empty — validate the input before it reaches the binding, as PR fix(ci): stop interpolating the release tag into a run block cloud-commitments-cli#1657 now does in deploy-aws-lambda.yml).
  4. Add sub to the GCP attribute_condition, or bind by attribute.environment, so GCP is no more permissive than AWS/Azure.
  5. Keep in lockstep with LeanerCloud/cloud-commitments-cli#1648: the environments must both exist with reviewers (this issue) and be listed in each cloud's trust policy (LeanerCloud/cloud-commitments-cli#1648). Either one alone leaves the rollback path broken or ungated.

Scope note

PR LeanerCloud/cloud-commitments-cli#1641 does not claim to fix this, and its workflow header comment says the binding is necessary but not sufficient. The p0 injection in LeanerCloud/cloud-commitments-cli#1542 is genuinely dead; this is the separate, previously-unnoticed fact that the control it was narrowed onto was never provisioned.

Related: LeanerCloud/cloud-commitments-cli#1542 / PR LeanerCloud/cloud-commitments-cli#1641, LeanerCloud/cloud-commitments-cli#1649 / PR LeanerCloud/cloud-commitments-cli#1657, LeanerCloud/cloud-commitments-cli#1648 (trust-policy sub allowlists), LeanerCloud/cloud-commitments-cli#1591, #90.


Addendum: the ref:refs/heads/main trust subject is not a restriction either

Added after the same finding surfaced under LeanerCloud/cloud-commitments-cli#1649 / PR LeanerCloud/cloud-commitments-cli#1657. Filing it here rather than as a fourth issue in this family, because it is the same defect species this issue is about: a control that reads as a restriction and restricts nothing.

Every cloud's trust policy accepts a main-branch subject:

The comment above the AWS list says the intent is to "restrict to protected branches and named deployment environments", and rules out an any-branch wildcard because it "would let a developer push to an unprotected feature branch and mint valid deploy credentials".

main is not protected:

$ gh api repos/LeanerCloud/CUDly/branches/main/protection
{"message":"Branch not protected","status":"404"}

So the stated premise does not hold. Anyone with repo write access can push directly to main and mint credentials for the full production deploy role — no review, no environment, no approval. The subject narrows which ref can assume the role, but since that ref is writable by every collaborator, it does not narrow who.

Consequence: repo write access is currently equivalent to production deploy credentials on all three clouds. That is the ceiling on every workflow-level hardening in this family — LeanerCloud/cloud-commitments-cli#1542/#1641, LeanerCloud/cloud-commitments-cli#1649/#1657, LeanerCloud/cloud-commitments-cli#1591, LeanerCloud/cloud-commitments-cli#1659 all reduce the number of paths to that capability, but none of them can reduce the capability itself while this holds.

It is also why the severity caveat on LeanerCloud/cloud-commitments-cli#1649 is honest rather than deflecting: an attacker who can cut a release can already push to main, so an injection bug that yields the same credentials adds little marginal capability.

Fix direction (either, ideally both):

  1. Protect main — require PR review, dismiss stale approvals, disallow force-push and deletion. This makes the existing trust-policy comment true rather than aspirational, and costs nothing in IaC.
  2. Drop the ref:refs/heads/main subject so only environment:* subjects can assume the role. This cannot be done until the ungated credentialed jobs in sec(ci): destroy and rollback workflows have no environment binding, so no reviewer gate applies cloud-commitments-cli#1591 and LeanerCloud/cloud-commitments-cli#1659 are given environment bindings — they depend on that subject today. Sequencing: sec(ci): destroy and rollback workflows have no environment binding, so no reviewer gate applies cloud-commitments-cli#1591 + LeanerCloud/cloud-commitments-cli#1659 first, then drop the subject, then this family is actually closed.

Item 1 is a settings/API change a privileged human makes; item 2 is IaC. Neither is code in a workflow file.

Findings from the 2026-09-02 codebase audit

Added by an automated audit of 3c0f8ac94048a2c36fce5ccddee54e6c4849a5cd (tip of origin/main). Each item below was reported by one reviewer and independently confirmed by a second that did not write it. Full report: docs/audits/codebase-audit-2026-09-02.md.

A14-007 (medium)

There is a documentation half to this that will misdirect whoever implements fix direction 1. .github/workflows/README.md:521-524 instructs the operator to create aws-lambda-dev/staging/prod, aws-fargate-*, gcp-* and azure-*. Only the aws-fargate-* family matches reality. deploy-aws-lambda.yml:231,377 bind plain dev/staging/prod; rollback.yml:192,296,376,458 bind the <cloud>-<env>-rollback family this issue enumerates; database-migration.yml:265,376,470 bind aws-db-<env> / gcp-db-<env> / azure-db-<env>, which appear nowhere in the README; and deploy-gcp.yml and deploy-azure.yml carry no job-level environment: at all. An operator following the README configures required reviewers on environments nothing binds to while the credentialed jobs keep auto-creating theirs bare, which is precisely the state this issue describes. Regenerating the README list from the actual environment: expressions belongs in the same change that declares them in IaC. Audit finding A14-007.

Activity

  1. cristim commented on Jul 28, 2026

    @cristim
    MemberAuthor

    Appended an addendum to the issue body: the ref:refs/heads/main trust subject is not a restriction either, because main is unprotected (gh api .../branches/main/protection → 404) on a repo whose trust policies accept that subject on all three clouds. Same defect species as this issue — a control that reads as a restriction and restricts nothing — so recording it here rather than opening a fourth issue in this family.

    Net effect: repo write access is currently equivalent to production deploy credentials. That is the ceiling on every workflow-level fix in this family (#1542/#1641, LeanerCloud/cloud-commitments-cli#1649/#1657, LeanerCloud/cloud-commitments-cli#1591, LeanerCloud/cloud-commitments-cli#1659): they reduce the number of paths to that capability but cannot reduce the capability while this holds.

    Includes a sequencing note — the ref:refs/heads/main subject cannot be dropped until LeanerCloud/cloud-commitments-cli#1591 and LeanerCloud/cloud-commitments-cli#1659 give environment bindings to the jobs that currently depend on it.

  2. cristim commented on Jul 29, 2026

    @cristim
    MemberAuthor

    Root-cause identification for the azure-/gcp- junk environments (this issue's fix-direction item 3) is done: filed as #147, cross-linked here. Short version: deploy-azure.yml/deploy-gcp.yml at commit cb386249e bound a job directly to environment: {name: azure-${{ inputs.environment }}} (and gcp-...), which rendered empty on the workflow's push trigger since inputs.environment only populates on workflow_dispatch/workflow_call. Confirmed via the Deployments API that only 3 deployment records exist against each, all from that commit and its two immediate follow-ups, all on feat/multicloud-web-frontend, none since 2026-03-25. The vulnerable pattern was refactored away by fecd8c637/626d96651 and current main has no environment: job binding at all in either file, so this is dead code, safe to clean up per #147's ask (delete both environments) rather than an active risk.

    Also confirmed while auditing #139/#1683: neither environment is reachable by any job that assumes the AWS cudly_deploy role, so this has no bearing on that allowlist.

  3. cristim commented on Sep 28, 2026

    @cristim
    MemberAuthor

    Verified against the split repo (LeanerCloud/cloud-commitments-platform)

    $ gh api repos/LeanerCloud/cloud-commitments-platform/environments
    {"total_count":1,"environments":[{"name":"dev","protection_rules":[],
     "can_admins_bypass":true,"deployment_branch_policy":null}]}
    

    One environment exists (dev, auto-created, presumably by whatever CI run
    first referenced it after the split), with no protection rules and no
    branch policy -- confirming this issue's core finding on the new repo, not
    just the pre-split one. staging and prod don't exist at all yet. The
    azure-/gcp- junk-environment cleanup this issue's addendum asked for
    (root-caused as PR #147 in the pre-split repo, confirmed dead code) is moot
    here: neither junk environment exists on the new repo either.

    What's landing in code now: #369

    PR #369 regenerates .github/workflows/README.md's "Set Up Environments"
    section from the actual environment: bindings across every workflow
    (previously stale -- audit finding A14-007 quoted in this issue's body). It
    documents the full correct environment list once #139 and #140 also merge:
    dev/staging/prod (shared by every deploy workflow, rollback.yml, and
    the two destroy workflows), aws-fargate-{dev,staging,prod}, and
    aws-db-*/gcp-db-*/azure-db-* for database-migration.yml.

    What's still blocked on a repo-admin / bootstrap action

    Two pieces of this issue's fix direction are outside what a PR to this
    repo's tracked files can apply:

    1. Creating the environments with protection rules. Per this repo's
    CLAUDE.md: "Repo settings (branch rulesets, environment protection rules,
    secrets) are admin actions. Don't try to change them via the API." A
    privileged human needs to, for each environment name PR #369 lists:

    # Example for `prod` -- repeat per environment, adjusting reviewers/policy
    # per the dev/staging/prod tiers PR #369's README section describes.
    gh api -X PUT repos/LeanerCloud/cloud-commitments-platform/environments/prod \
      -f 'deployment_branch_policy[protected_branches]=true' \
      -f 'deployment_branch_policy[custom_branch_policies]=false' \
      -f 'reviewers[][type]=User' -f 'reviewers[][id]=<reviewer-user-id>'

    The issue's own fix direction suggests doing this via IaC
    (github_repository_environment + github_repository_environment_deployment_policy)
    instead of the API directly, which would make it reviewable. I'm not adding
    that Terraform module speculatively in this pass: it needs a github
    provider with a repo-admin-scoped token this codebase does not currently
    configure anywhere (grep -rl 'provider "github"' finds nothing but a
    stray README mention), and there is no local way to verify a first cut of
    that module against the real repo before a human applies it. Standing this
    up is a reasonable follow-up but is itself the kind of decision (where does
    the token live, who administers the module) that needs a human call, not
    something to invent unasked in this pass.

    2. Hardening GCP's OIDC attribute_condition. Per this issue's
    addendum, terraform/environments/gcp/ci-cd-permissions/github_oidc.tf:37
    only checks assertion.repository and assertion.ref, never sub, so
    GCP is more permissive than AWS/Azure: any job bound to any environment name
    (including one that does not exist, per GitHub's bare-auto-create behavior)
    still authenticates as long as repository+ref match. I did not attempt this
    change: it is the same bootstrap-only, manually-applied module as the
    AWS/Azure siblings, and getting the CEL attribute_mapping/
    attribute_condition syntax wrong has no local way to be caught before a
    human applies it against the real Workload Identity Pool -- the failure
    mode is silently locking out every legitimate GCP deploy. This needs to be
    written and reviewed by someone who can validate it against GCP's WIF
    condition syntax, then applied and smoke-tested by the person with access,
    not shipped as an unverified PR.

    Suggested order

    1. sec(iac): rename-proof OIDC deploy trust + branch-restricted environments for the split platform repo #2's bootstrap re-apply (rename-proof trust, correct repo name) --
      without it nothing in this family authenticates against a live account
      regardless of environment protection.
    2. Create dev/staging/prod and the aws-fargate-*/*-db-* families
      as real Environments with protection rules (this comment, item 1).
    3. Harden the GCP attribute_condition (this comment, item 2) so it's no
      more permissive than AWS/Azure once the environments above exist.
    4. sec(ci): main is unprotected while ref:refs/heads/main is a trusted OIDC subject for the AWS deploy role #154's branch protection on main, so the ref:refs/heads/main subject
      every cloud still trusts is at least gated by required review.

    Labeling status/blocked for the two items above; #369 is the part of this
    issue's fix that's safe to land now.

  4. cristim commented on Oct 4, 2026

    @cristim
    MemberAuthor

    This is marked closed, but the control it describes was not in place on this repo: on 2026-10-04 only dev existed, with no branch policy. Applied now.

    Created 15 environments, each with a deployment-branch policy of exactly main: dev, staging, prod, aws-fargate-{dev,staging,prod}, aws-db-{dev,staging,prod}, gcp-db-{dev,staging,prod}, azure-db-{dev,staging,prod}. The first nine are what the live AWS trust names; the last six are bound by database-migration.yml and would otherwise auto-create bare. Verified per environment against gh api .../environments/<name>/deployment-branch-policies.

    Not done: no required reviewers or wait timers, and not tested by dispatching from a feature branch (that would start a real deploy). Known consequence: release: created deploys to prod run from the tag ref, which a main-only policy refuses, so the first release deploy to prod and aws-fargate-prod will fail until a tag policy (e.g. v*) is added to those two. No releases or tags exist yet, so nothing is broken today. Left as a deliberate decision rather than widening every environment. Rollback and the log are in data/platform-hardening-2026-10-04/ in the split workspace.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions