ci(build): the release images exist before the release can be merged - #1329
Merged
Merged
Conversation
The version bump landed on main and the images were built minutes later, so Flux applied a version whose images did not exist yet and the new pod waited in ImagePullBackOff while the old one served. A push to the release branch now builds and publishes the full set under the release tag, taken from the manifest release-please has already bumped there. One step resolves the tag for the whole run, so no later job needs to know which trigger fired. promote stays gated on the workflow input rather than the resolved tag: the branch build publishes the tag early, but until that branch merges there is no release, and moving :latest would deploy one that does not exist.
Contributor
Changes
No production or test lines added. Coverage
No baseline is cached from Patch coverage: this pull request changes no line that coverage measures. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1291. Part of #1289.
The version bump lands on
mainand the images are built minutes later, so Flux applies a version whose images do not exist yet and the new pod sits inImagePullBackOffwhile the old one serves. That happens on every release.It is not hypothetical right now: #1066 is open as
chore(main): release 1.8.0, andghcr.io/esa-blueshell/api:v1.8.0does not exist. Merge that today and production is pointed at a tag nobody has built.What this achieves
A push to the release branch builds and publishes the full set of images under the release tag, so they are on ghcr before the release pull request can be merged. Nothing is deployed and no
:latestmoves — this only makes the artefacts exist ahead of the commit that names them.How
build.ymlgainsrelease-please--branches--mainas a push trigger, and one step indetect-changesresolves the tag for the whole run: the workflow input when set, otherwise the version from.release-please-manifest.json— which release-please has already bumped on that branch — and empty on an ordinarymainpush. Every later job reads that resolved value, so none of them needs to know which trigger fired.The rest is unchanged. A release still rebuilds the complete set rather than the changed subset, images are still published by digest and assembled into a manifest list, and
mainstill tags:sha-<short>only.release.ymlstill calls this workflow once the release is actually cut. That republishes the same tag over the same layers, which is cheap, and it means the release tag exists even if the branch build never ran.Not in scope
The digest pin and the required check that verifies it are #1293, and the app identity that will own the release branch is #1290. Until those land the deployed version is still a tag edited by hand, so this publishes images without deploying anything.
Worth a reviewer's attention
promoteis deliberately gated on the workflow input, not the resolved tag. A release-branch push publishes the tag early, but until that branch merges there is no release, and moving:latestwould deploy one that does not exist. So the branch build publishes images and moves nothing; only the real release promotes.packages: write. Today anyone with push access can write to that branch. A single app identity owns the release branch, and nothing else may write to it #1290 restricts it to a single app identity, which is what closes this properly — worth knowing that the two tickets are related in more than ordering.release.yml.mainmoves. That is harmless here, since each push rebuilds, but it is the behaviour The deployed version is a digest, written by the pipeline and checked before merge #1293's pin check has to survive.Verification
The tag resolver was exercised for all three triggers against the real tree: a
mainpush resolves to empty, a release-branch push tov1.8.0, an explicit input passes through unchanged. Thev1.8.0came from the liverelease-please--branches--mainmanifest, not from a fixture.build.ymlparses, and the manifest-check job now covers it.What cannot be shown until this is on
main: the last acceptance criterion, that the tag resolves on ghcr from an open release pull request. A push trigger only fires from the default branch, so the next push to #1066's branch is what confirms it end to end.