Skip to content

fix(ci): gate nightly publication on the production dependency audit - #8397

Merged
iamwhatever merged 1 commit into
mainfrom
fix/nightly-dep-audit-gate-8362
Sep 4, 2026
Merged

fix(ci): gate nightly publication on the production dependency audit#8397
iamwhatever merged 1 commit into
mainfrom
fix/nightly-dep-audit-gate-8362

Conversation

@iamwhatever

Copy link
Copy Markdown
Collaborator

Problem / Motivation

.github/workflows/nightly.yml publishes to the nightly channel with no production dependency-vulnerability gate anywhere in its graph. On current main:

  • nightly.yml has no audit job at all — lines ~121-125 were a comment stating the audit "runs before tagged releases only (release.yml)".
  • publish-cli needed [version, build-wheel]; the six publish-linux-* publishers needed [version, build-desktop]; publish-windows-x64, publish-docker and sign-and-notarize likewise reached their S3/GHCR/feed writes through build-only needs.
  • The gate existed only in release.yml as dependency-vulnerability-gate (release.yml:~312), where build-wheel / build-desktop / build-windows depend on it.

There is also no PR-level dependency audit (deliberately — see below), so nothing between a dependency bump and a nightly publish looks at the dependency tree.

Why it matters

A high or critical production dependency vulnerability that lands on main shipped to nightly-channel users unaudited — across the CLI wheel, the PEP 503 index, all six Linux packages, the Windows installer, the GHCR image and the notarized macOS DMG — and stayed shipped until the next tagged release happened to run the gate.

What changed (motivation → approach → change)

Root cause: the audit was removed from nightly.yml outright because it reaches the npm registry, and the registry's slow hours failed the nightly for hours at a stretch by blocking every build behind the gate. That diagnosis was right; the remedy went one step too far and dropped the control rather than moving it.

The fix moves it rather than restoring the old shape: the gate hangs off the PUBLISH jobs, never the build jobs.

  • New dependency-vulnerability-gate job in nightly.yml, byte-consistent with the release caller — same uses: ./.github/workflows/dependency-vulnerability.yml, no with:/secrets: (the reusable workflow declares no inputs), and unconditional, because the gate script emits the exception-expiry warning itself and an if: would silence it.
  • Added to the needs: of every job that ships bytes to a nightly user: publish-cli, publish-linux-{appimage,deb,rpm}-{x64,arm64}, publish-windows-x64, publish-docker, sign-and-notarize.
  • No build job depends on it. build-wheel, build-desktop and build-windows keep exactly the needs: they had.

That placement is what keeps the original outage from coming back: a slow or flaky registry now delays publication of an already-built nightly instead of failing the build, and the artifacts stay in the run so a re-run publishes them once the audit answers. No reusable workflow needed a parameter change.

Deviation to flag for review: the finding named publish-cli and the publish-linux-* jobs. I also gated publish-windows-x64, publish-docker and sign-and-notarize, because they are publish jobs by the same test and leaving them out would have left the same vulnerability shipping to Windows, Docker and macOS nightly users. Say so if you want the scope narrowed to the two families the finding enumerated.

Tests

test/test_dependency_vulnerability_gate.py previously pinned the opposite contract in test_only_the_release_workflow_calls_the_gate — it asserted no nightly.yml job calls the gate and no nightly.yml job lists it in needs. That assertion is what the old design was worth; it is replaced by tests that pin the new one:

  • test_the_gate_never_runs_per_pull_request — keeps the part of the old contract that still holds: no caller and no needs: edge in code-review.yml, so the registry flake cannot come back to every PR.
  • test_release_gates_the_builds — unchanged release assertions, split out.
  • test_nightly_gates_publication_but_never_the_builds — the gate job exists with the right uses: and no if:; the set of jobs listing it in needs is exactly the ten publish jobs; the set not listing it is exactly version + the three build jobs. A build job growing this edge goes red.
  • test_nightly_needs_graph_is_acyclic_and_fully_resolved — every needs: entry names a real job, and a Kahn drain proves no cycle.

Revert-verified: adding dependency-vulnerability-gate to build-wheel's needs fails test_nightly_gates_publication_but_never_the_builds, so the split is load-bearing rather than incidentally true.

Manual verification

N/A — CI-config change with no runtime surface; the contract is fully covered by the workflow-parsing tests above. Validated locally:

  • pytest test/test_dependency_vulnerability_gate.py test/test_workflow_permissions.py test/test_workflow_secret_and_cache_scope.py test/test_nightly_version_contract.py — 108 passed.
  • pytest test/test_github_workflow_security.py test/test_workflow_security.py test/test_workflow_cache_setup_uniqueness.py test/test_workflow_checkout_credentials.py test/test_release_channel.py test/test_release_macos_fail_closed.py test/test_release_promotion_contract.py test/test_stable_release_gate.py test/test_ci_surface_tests.py — 159 passed.
  • YAML parses; the needs: graph is acyclic (15 jobs, full topological drain) and every referenced job exists. actionlint is not installed on this machine, so validation was a YAML parse plus the graph checks, not a linter run.
  • scripts/docs-lint.sh (260 files), scripts/check_black_formatting.py, flake8, isort, and the four diff-scoped gates (check_brand_name, check_focus_cue, check_changelog_history, check_harness_parity) run with their *_BASE_REF exported — all clean.

Screenshots / video

N/A — no user-visible UI change.

Related Issues

Follow-up to the nightly-gate removal in #8362.

Pattern harvest

Pattern: a gate removed to stop it blocking the wrong stage, instead of being re-hung on the stage it was actually protecting — leaving the protected artifact ungated.

Rule candidate: review-prompt. A diff that deletes a needs: edge onto a security/audit gate, or deletes the gate's only caller in a workflow that still has publish jobs, should have to say which job now carries the control. This is hard to express as a semgrep pattern (it is a graph property of the workflow, not a token pattern), but it is exactly what test_nightly_gates_publication_but_never_the_builds now enforces for this one workflow — the generalizable form is a test that asserts every publishing job in a release-shaped workflow is transitively behind the audit gate, which would have caught this at #8362 time.

Checklist

  • At most two commits (one is the norm), with a Conventional Commits title (feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)
  • Existing tests pass and new tests added for new functionality
  • Self-review completed; code follows project style guidelines
  • Documentation updated (if applicable)
  • No secrets, credentials, or internal references in the diff

main has no dependency-vulnerability gate, so a high/critical production
vulnerability landing there shipped to nightly-channel users unaudited until the
next tagged release: nightly.yml carried no audit job at all, and every publish
job reached its S3/GHCR write through build-only needs.

Re-add dependency-vulnerability.yml to nightly.yml as a needs: of the PUBLISH
jobs only -- publish-cli, the six publish-linux-* callers, publish-windows-x64,
publish-docker and sign-and-notarize. No build job depends on it: gating the
builds is what once failed the nightly for hours behind a slow npm registry, and
hanging the gate off publication means a stalled registry delays publishing an
already-built nightly instead of failing the build, with the artifacts still in
the run for a re-run to publish.

Same job shape and reusable-workflow ref as the release caller, unconditional so
the gate script's exception-expiry warning still reaches a run.

test_dependency_vulnerability_gate.py previously pinned the opposite contract
(no nightly caller at all); it now pins both halves of the new one -- every
nightly publish job gated, no nightly build job gated -- plus a needs-graph
acyclicity/resolution check. Docs corrected in three places that stated the
audit never runs on a nightly.
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Sep 4, 2026
@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — ✅ PASS

Design-level review of 137ab4c2a074cb744c92f094b807683d1ce12887 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

Re-hangs the control on the stage it actually protects (publication, not build), with exact-set tests that force any new nightly job to declare its side of the gate.

[DESIGN-REVIEWED] 137ab4c

@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Opus 4.8 Review — ✅ no blocking findings

Reviewed 137ab4c2a074cb744c92f094b807683d1ce12887 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] 137ab4c

Verdict parsed from the review's SHA-scoped output markers for commit 137ab4c2a074cb744c92f094b807683d1ce12887.

False positive or not applicable? A repository writer can comment:
/ai-review override fable 137ab4c2a074cb744c92f094b807683d1ce12887: <one-sentence reason>

@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

First Principles Review (Fable 5) — ✅ PASS

Premise-level review of 137ab4c2a074cb744c92f094b807683d1ce12887 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

All verification is complete. The nightly file at HEAD matches the diff, the pinned job partition is exhaustive (15 jobs: 1 gate, 4 ungated, 10 publishers), the gate is a second caller of the existing reusable workflow rather than a duplicate, no other workflow publishes user-facing artifacts ungated, and no existing test or linter already validates the needs: graph. Here is the review:

First-Principles-Verdict: PASS

Restores a deleted security control at the layer it actually protects — publication, not build — with the original outage's cause designed out.

What this change ships

Intent: stop unaudited high/critical production dependency vulnerabilities shipping to nightly-channel users. This is a FIX (follow-up to the #8362 removal).

  1. Nightly runs the production dependency audit again — justified; reuses the existing dependency-vulnerability.yml, correct second caller, not a duplicate.
  2. All ten byte-shipping nightly jobs now wait for the audit — the fix; partition verified exhaustive (15 jobs total).
  3. A slow registry or found vulnerability now delays nightly publication — declared timing change; the point of the shape.
  4. Build jobs keep their exact needs: — justified; removes the cause of the original outage recurring.
  5. Scope widened past the finding to Windows/Docker/macOS — declared deviation; same harm, same test, justified.
  6. Old release-only contract test split into three — justified; the old test pinned the defect.
  7. Exact-set test: gated == the ten publishers, ungated == version + builds — justified regression pin.
  8. New acyclicity/resolved-needs test for nightly.yml — rides along; harm nameable (an invalid graph in a schedule-only workflow fails silently at the next 06:00 run; no actionlint in CI, counted: 0 existing graph tests).
  9. Three docs updated in the same commit — mandated by AGENTS.md, derived.

Watch

Item 8 is the one non-fix item: declared, and it guards the ten edges this diff edits, but it covers nightly.yml only — release.yml's graph has the same silent-failure property and stays unguarded, so this pin is a point instance if that harm is why it exists.

[FIRST-PRINCIPLES-REVIEWED] 137ab4c

@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of 137ab4c2a074cb744c92f094b807683d1ce12887 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] 137ab4c

False positive or not applicable? A repository writer can comment:
/ai-review override gpt 137ab4c2a074cb744c92f094b807683d1ce12887: <one-sentence reason>

@iamwhatever
iamwhatever marked this pull request as ready for review September 4, 2026 15:57
@iamwhatever
iamwhatever requested a review from a team as a code owner September 4, 2026 15:57
@iamwhatever
iamwhatever requested a review from cixuuz September 4, 2026 15:57
@github-actions github-actions Bot added readiness: passed Eligible automated validation passed for the current revision and removed readiness: checking Automated validation is still running labels Sep 4, 2026
@iamwhatever
iamwhatever enabled auto-merge (squash) September 4, 2026 17:26
@iamwhatever
iamwhatever merged commit fe47cde into main Sep 4, 2026
65 checks passed
@iamwhatever
iamwhatever deleted the fix/nightly-dep-audit-gate-8362 branch September 4, 2026 17:27
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants