Record the failure class nobody is waiting on - #9
Conversation
Mapsicle's audit job was red for three days on the org's most-downloaded package and nothing surfaced it. The gate worked; the escalation was missing. Also records the wrong conclusion it produced, since a red build says something failed rather than who is exposed.
|
Warning Review limit reached
Next review available in: 11 minutes Limit details: You’ve used all 3 included reviews currently available under your plan. Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Comment |
1 participant
Adds one section to
DELIVERY.md, with a real incident behind it, per the document's own rule that a gate with nothing to cite is ceremony.What happened
Mapsicle's
dependency auditjob failed for at least three days. Every other job was green: build and test on both Windows and Ubuntu, formatting, and the core-stays-dependency-free check. Nothing surfaced the failure, because nothing was waiting on it.It came to light only because the org's repositories were listed by download count for an unrelated reason. Mapsicle has 66,043 NuGet downloads, more than every other BaryoDev package combined, and its default branch had been red for days.
Why the existing sections do not cover it
The document is thorough on gates that fail a change. This is a gate that fails on its own, with no commit behind it: a newly published advisory against a transitive dependency, a base image moving, a runtime reaching end of life. Nobody is waiting on the result, so nobody reads it.
Read a signal before trusting itis the nearest section, and it is about not over-reading an alert count. This is about not reading it at all.The second thing it cost
Worth recording separately, because it is the kind of mistake the document exists to catch. The red build was read as "users are on a broken package". They were not. All five advisories reached test projects only:
A red build says something failed. It does not say who is exposed. The new section says so plainly.
Not claimed
No new process, no new required job. The recommendation is one line: a job that can fail without anyone pushing needs a route to a human that does not depend on anyone pushing.
The fix for the incident itself is BaryoDev/Mapsicle#13.