From ca0b5a9606a097465a8e33efc7777b736005c397 Mon Sep 17 00:00:00 2001 From: Arnel Robles Date: Tue, 18 Aug 2026 22:50:44 +0800 Subject: [PATCH] record the failure class nobody is waiting on Mapsicle's audit job was red for three days on the org's most-downloaded package and nothing surfaced it. The gate worked; the escalation was missing. Also records the wrong conclusion it produced, since a red build says something failed rather than who is exposed. --- DELIVERY.md | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/DELIVERY.md b/DELIVERY.md index deb2715..519fb5c 100644 --- a/DELIVERY.md +++ b/DELIVERY.md @@ -264,6 +264,38 @@ non-expiring option is a deliberate setting. Worth a periodic link check in CI on the files that onboard people. Note that a bare `curl` is not enough for this class: the expiring invite returned `200` right up until it did not. +### A failure with no commit behind it needs somewhere to go + +A gate that only fails on a pull request is watched, because somebody is waiting on that pull +request. A gate that can fail on its own, with no commit behind it, is not watched by anybody. + +Dependency auditing is the obvious one. `NuGetAuditMode=all` with `TreatWarningsAsErrors=true` +turns any newly published advisory against any transitive dependency into a red build, days or +weeks after the last commit. So does an npm advisory, a base image moving, or a runtime reaching +end of life. + +**Caught:** Mapsicle's audit job failed for at least three days on five advisories. Every other job +was green: build and test passed on both operating systems, formatting passed, the +core-stays-dependency-free check passed. Nothing announced the failure, because nothing was +waiting on it. It surfaced only when the org's repositories were listed by download count for an +unrelated reason, and Mapsicle turned out to have more downloads than every other package in the +org combined. + +Two things this cost, and both are worth naming: + +- **Three days of a red default branch** on the most-used package, which is what a stranger sees. +- **A wrong conclusion, briefly.** The red build was read as "users are on a broken package". They + were not: all five advisories reached test projects only and no shipped package was affected. + A red build says something failed, not what is exposed. Check which projects before saying who + is at risk. + +The same week, two of three other red repositories in the org had no commit behind them either. +The world moved; the code did not. + +**What to do about it:** give these jobs their own schedule and their own notification, rather than +leaving them to be noticed on the next pull request. If a job can fail without anyone pushing, it +needs a route to a human that does not depend on anyone pushing. + ### Publishing is not releasing Pushing a package is one step. If the repository does not also record what shipped, the project