fix(ci): gate nightly publication on the production dependency audit - #8397
Conversation
main has no dependency-vulnerability gate, so a high/critical production vulnerability landing there shipped to nightly-channel users unaudited until the next tagged release: nightly.yml carried no audit job at all, and every publish job reached its S3/GHCR write through build-only needs. Re-add dependency-vulnerability.yml to nightly.yml as a needs: of the PUBLISH jobs only -- publish-cli, the six publish-linux-* callers, publish-windows-x64, publish-docker and sign-and-notarize. No build job depends on it: gating the builds is what once failed the nightly for hours behind a slow npm registry, and hanging the gate off publication means a stalled registry delays publishing an already-built nightly instead of failing the build, with the artifacts still in the run for a re-run to publish. Same job shape and reusable-workflow ref as the release caller, unconditional so the gate script's exception-expiry warning still reaches a run. test_dependency_vulnerability_gate.py previously pinned the opposite contract (no nightly caller at all); it now pins both halves of the new one -- every nightly publish job gated, no nightly build job gated -- plus a needs-graph acyclicity/resolution check. Docs corrected in three places that stated the audit never runs on a nightly.
Design Review (Fable 5) — ✅ PASSDesign-level review of Design-Verdict: PASS Re-hangs the control on the stage it actually protects (publication, not build), with exact-set tests that force any new nightly job to declare its side of the gate. [DESIGN-REVIEWED] 137ab4c |
Opus 4.8 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
First Principles Review (Fable 5) — ✅ PASSPremise-level review of All verification is complete. The nightly file at HEAD matches the diff, the pinned job partition is exhaustive (15 jobs: 1 gate, 4 ungated, 10 publishers), the gate is a second caller of the existing reusable workflow rather than a duplicate, no other workflow publishes user-facing artifacts ungated, and no existing test or linter already validates the First-Principles-Verdict: PASS Restores a deleted security control at the layer it actually protects — publication, not build — with the original outage's cause designed out. What this change shipsIntent: stop unaudited high/critical production dependency vulnerabilities shipping to nightly-channel users. This is a FIX (follow-up to the #8362 removal).
WatchItem 8 is the one non-fix item: declared, and it guards the ten edges this diff edits, but it covers [FIRST-PRINCIPLES-REVIEWED] 137ab4c |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Problem / Motivation
.github/workflows/nightly.ymlpublishes to the nightly channel with no production dependency-vulnerability gate anywhere in its graph. On currentmain:nightly.ymlhas no audit job at all — lines ~121-125 were a comment stating the audit "runs before tagged releases only (release.yml)".publish-clineeded[version, build-wheel]; the sixpublish-linux-*publishers needed[version, build-desktop];publish-windows-x64,publish-dockerandsign-and-notarizelikewise reached their S3/GHCR/feed writes through build-onlyneeds.release.ymlasdependency-vulnerability-gate(release.yml:~312), wherebuild-wheel/build-desktop/build-windowsdepend on it.There is also no PR-level dependency audit (deliberately — see below), so nothing between a dependency bump and a nightly publish looks at the dependency tree.
Why it matters
A high or critical production dependency vulnerability that lands on
mainshipped to nightly-channel users unaudited — across the CLI wheel, the PEP 503 index, all six Linux packages, the Windows installer, the GHCR image and the notarized macOS DMG — and stayed shipped until the next tagged release happened to run the gate.What changed (motivation → approach → change)
Root cause: the audit was removed from
nightly.ymloutright because it reaches the npm registry, and the registry's slow hours failed the nightly for hours at a stretch by blocking every build behind the gate. That diagnosis was right; the remedy went one step too far and dropped the control rather than moving it.The fix moves it rather than restoring the old shape: the gate hangs off the PUBLISH jobs, never the build jobs.
dependency-vulnerability-gatejob innightly.yml, byte-consistent with the release caller — sameuses: ./.github/workflows/dependency-vulnerability.yml, nowith:/secrets:(the reusable workflow declares no inputs), and unconditional, because the gate script emits the exception-expiry warning itself and anif:would silence it.needs:of every job that ships bytes to a nightly user:publish-cli,publish-linux-{appimage,deb,rpm}-{x64,arm64},publish-windows-x64,publish-docker,sign-and-notarize.build-wheel,build-desktopandbuild-windowskeep exactly theneeds:they had.That placement is what keeps the original outage from coming back: a slow or flaky registry now delays publication of an already-built nightly instead of failing the build, and the artifacts stay in the run so a re-run publishes them once the audit answers. No reusable workflow needed a parameter change.
Deviation to flag for review: the finding named
publish-cliand thepublish-linux-*jobs. I also gatedpublish-windows-x64,publish-dockerandsign-and-notarize, because they are publish jobs by the same test and leaving them out would have left the same vulnerability shipping to Windows, Docker and macOS nightly users. Say so if you want the scope narrowed to the two families the finding enumerated.Tests
test/test_dependency_vulnerability_gate.pypreviously pinned the opposite contract intest_only_the_release_workflow_calls_the_gate— it asserted nonightly.ymljob calls the gate and nonightly.ymljob lists it inneeds. That assertion is what the old design was worth; it is replaced by tests that pin the new one:test_the_gate_never_runs_per_pull_request— keeps the part of the old contract that still holds: no caller and noneeds:edge incode-review.yml, so the registry flake cannot come back to every PR.test_release_gates_the_builds— unchanged release assertions, split out.test_nightly_gates_publication_but_never_the_builds— the gate job exists with the rightuses:and noif:; the set of jobs listing it inneedsis exactly the ten publish jobs; the set not listing it is exactlyversion+ the three build jobs. A build job growing this edge goes red.test_nightly_needs_graph_is_acyclic_and_fully_resolved— everyneeds:entry names a real job, and a Kahn drain proves no cycle.Revert-verified: adding
dependency-vulnerability-gatetobuild-wheel'sneedsfailstest_nightly_gates_publication_but_never_the_builds, so the split is load-bearing rather than incidentally true.Manual verification
N/A — CI-config change with no runtime surface; the contract is fully covered by the workflow-parsing tests above. Validated locally:
pytest test/test_dependency_vulnerability_gate.py test/test_workflow_permissions.py test/test_workflow_secret_and_cache_scope.py test/test_nightly_version_contract.py— 108 passed.pytest test/test_github_workflow_security.py test/test_workflow_security.py test/test_workflow_cache_setup_uniqueness.py test/test_workflow_checkout_credentials.py test/test_release_channel.py test/test_release_macos_fail_closed.py test/test_release_promotion_contract.py test/test_stable_release_gate.py test/test_ci_surface_tests.py— 159 passed.needs:graph is acyclic (15 jobs, full topological drain) and every referenced job exists.actionlintis not installed on this machine, so validation was a YAML parse plus the graph checks, not a linter run.scripts/docs-lint.sh(260 files),scripts/check_black_formatting.py,flake8,isort, and the four diff-scoped gates (check_brand_name,check_focus_cue,check_changelog_history,check_harness_parity) run with their*_BASE_REFexported — all clean.Screenshots / video
N/A — no user-visible UI change.
Related Issues
Follow-up to the nightly-gate removal in #8362.
Pattern harvest
Pattern: a gate removed to stop it blocking the wrong stage, instead of being re-hung on the stage it was actually protecting — leaving the protected artifact ungated.
Rule candidate:
review-prompt. A diff that deletes aneeds:edge onto a security/audit gate, or deletes the gate's only caller in a workflow that still has publish jobs, should have to say which job now carries the control. This is hard to express as a semgrep pattern (it is a graph property of the workflow, not a token pattern), but it is exactly whattest_nightly_gates_publication_but_never_the_buildsnow enforces for this one workflow — the generalizable form is a test that asserts every publishing job in a release-shaped workflow is transitively behind the audit gate, which would have caught this at #8362 time.Checklist
feat|fix|docs|refactor|perf|test|chore|ci|build|revert: ...)