Add a criterion: a release becomes public only after its smoke kit has passed

[trsdn]

Proposal, made through an issue as "Changing This Standard" asks.

The gap

R05 asks that the artifact a consumer receives has been smoke-tested, and R03/R07 describe how a tag becomes a release. None of them says when the check runs relative to publication. A repository can pass R05 with a kit that runs after the release is public, so a build that cannot be installed or launched is already downloadable, and, for an application with an in-app updater, already offered to every installed copy.

A repository the change would score differently

trsdn/OpenWritr. Its v1.6.5 release ran the smoke kit only after the assets were published (release.yml created the release, then a second job ran smoke-test.yml). It passes R05 (the kit ran and passed), but a failure would have shipped. It has since changed the order to draft, smoke test, publish (trsdn/OpenWritr#56), which is the behavior this criterion would ask for.

Proposed criterion

New identifier (identifiers are append-only, so not a change to R05): R09, section Package And Release Repositories.

• Requirement: A release is made available to consumers only after its smoke kit has passed against the release's own assets.
• Evidence: The release workflow's order (a draft, a pre-release, or an unpublished or staging channel that the smoke kit reads, and a publish step that depends on it), or, for a manual release, the documented steps that put the check before the publication.

Suggested results, in the style of 1.15.0

Pass	Partial	Fail	Not applicable
The kit runs against the assets before they are public, and publication depends on it passing	The kit runs before publication but nothing stops publication when it fails, or it runs against a build that is not the file that is published	The kit runs only after publication, or not at all	The repository publishes no artifact, or the ecosystem offers no way to hold a release back and no staging channel (state which)

Readings to state

• Where a draft exists (GitHub Releases): a draft release is the natural gate. A draft is invisible to consumers and updaters, and its assets can be downloaded by a token with write access.
• Where publication cannot be undone or held back (a package registry that forbids re-publishing a version): a staging channel counts (a pre-release tag, next dist-tag, TestPyPI); where there is none, the criterion is Not applicable with that recorded.
• The tested bytes must be the published bytes. A check of the published files' checksum after publication closes the gap between what was tested and what is served.

Versioning

Adding a criterion is minor: no recorded result can weaken, and repositories are due for reassessment on it. Because it is new, a repository that publishes before testing would record Fail at its next assessment, which is the intended signal.

Notes

• This is independent of R05 and does not narrow it.
• R03's automation-agnostic wording carries over: a manual release satisfies this criterion when the documented steps put the check first.

🤖 Generated with Claude Code

https://claude.ai/code/session_011TFQ786kVfVMuuvgu7ppdA