Skip to content

Commit afe8d4a

Browse files
myftijamyftija
committed
fix(base-images): publish gates, atomic tag pairs, and a pullability check
Authenticate to Docker Hub before any image pull (and on PR runs) so pulls stop counting against anonymous rate limits, build both targets before pushing either so a mid-run failure can't leave the runtime and -build tags on different snapshots, and verify pushed digests resolve anonymously so an accidentally-private repo can't publish green. Only publish from main (branch dispatches build without pushing), reject future snapshot timestamps that would silently disable timestamp normalization, upgrade with --with-new-pkgs so updates that add dependencies aren't held back, validate images.json entries, drop the per-run GHA cache which could never hit across runs, don't fail a publish on attestation hiccups, and fix the README verify recipe's label inspection to a form that works against multi-platform indexes.
1 parent 2da3a4c commit afe8d4a

3 files changed

Lines changed: 84 additions & 28 deletions

File tree

‎.github/workflows/base-images.yml‎

Lines changed: 73 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
name: 🐳 Deploy base images
22

33
# Publishes the base images that deployed task containers build on
4-
# (triggerdotdev/node:<major>-bookworm and triggerdotdev/bun:<line>-bookworm,
4+
# (triggerdotdev/node:<major>-bookworm and triggerdotdev/bun:<line>-node<major>-bookworm,
55
# each with a -build toolchain variant) to Docker Hub. Tags are mutable and
66
# rebuilt in place; the CLI pins images by digest, so consumers are unaffected
77
# until a digest bump ships in a release.
@@ -58,15 +58,18 @@ jobs:
5858
env:
5959
SNAPSHOT_INPUT: ${{ inputs.debian_snapshot }}
6060
EVENT_NAME: ${{ github.event_name }}
61+
REF: ${{ github.ref }}
6162
run: |
62-
PACKAGES="$(jq -r '.packages' base-images/images.json)"
63-
BUILD_PACKAGES="$(jq -r '.buildPackages' base-images/images.json)"
64-
SUITE="$(jq -r '.suite' base-images/images.json)"
63+
PACKAGES="$(jq -er '.packages' base-images/images.json)"
64+
BUILD_PACKAGES="$(jq -er '.buildPackages' base-images/images.json)"
65+
SUITE="$(jq -er '.suite' base-images/images.json)"
6566
6667
# Values land in build args and shell lines; keep them boring
67-
echo "$PACKAGES" | grep -qE '^[a-z0-9 .+-]+$' || { echo "invalid packages value"; exit 1; }
68-
echo "$BUILD_PACKAGES" | grep -qE '^[a-z0-9 .+-]+$' || { echo "invalid buildPackages value"; exit 1; }
68+
echo "$PACKAGES" | grep -qE '^[a-z0-9][a-z0-9 .+:=~-]*$' || { echo "invalid packages value"; exit 1; }
69+
echo "$BUILD_PACKAGES" | grep -qE '^[a-z0-9][a-z0-9 .+:=~-]*$' || { echo "invalid buildPackages value"; exit 1; }
6970
echo "$SUITE" | grep -qE '^[a-z]+$' || { echo "invalid suite value"; exit 1; }
71+
jq -e '.images | length > 0 and all((.repo | test("^[a-z0-9-]+$")) and (.tag | test("^[a-z0-9.-]+$")) and (.base | test("^[a-zA-Z0-9./:@-]+$")))' base-images/images.json > /dev/null \
72+
|| { echo "invalid images entries"; exit 1; }
7073
7174
SNAPSHOT="$SNAPSHOT_INPUT"
7275
if [ -z "$SNAPSHOT" ]; then
@@ -77,9 +80,13 @@ jobs:
7780
# Layer and config timestamps come from the snapshot, so images stay
7881
# reproducible while the registry shows a meaningful created date
7982
EPOCH="$(date -u -d "${SNAPSHOT:0:4}-${SNAPSHOT:4:2}-${SNAPSHOT:6:2} ${SNAPSHOT:9:2}:${SNAPSHOT:11:2}:${SNAPSHOT:13:2}Z" +%s)"
83+
# A future snapshot resolves to "latest" server-side and a future
84+
# epoch stops rewrite-timestamp normalizing mtimes; reject both
85+
[ "$EPOCH" -le "$(date -u +%s)" ] || { echo "debian_snapshot is in the future: $SNAPSHOT"; exit 1; }
8086
81-
# Only publish from main; pull requests validate the build without pushing
82-
if [ "$EVENT_NAME" = "pull_request" ]; then
87+
# Only publish from main; pull requests and branch dispatches
88+
# validate the build without pushing
89+
if [ "$EVENT_NAME" = "pull_request" ] || [ "$REF" != "refs/heads/main" ]; then
8390
PUSH=false
8491
else
8592
PUSH=true
@@ -107,11 +114,22 @@ jobs:
107114
fail-fast: false
108115
matrix:
109116
image: ${{ fromJSON(needs.setup.outputs.images) }}
117+
env:
118+
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
110119
steps:
111120
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
112121
with:
113122
persist-credentials: false
114123

124+
# Before any image pull so tooling and base pulls count against the
125+
# authenticated rate limit; skipped on fork PRs, which have no secrets
126+
- name: 🐳 Login to Docker Hub
127+
if: env.DOCKERHUB_USERNAME != ''
128+
uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
129+
with:
130+
username: ${{ secrets.DOCKERHUB_USERNAME }}
131+
password: ${{ secrets.DOCKERHUB_TOKEN }}
132+
115133
- name: 🐳 Set up QEMU
116134
uses: docker/setup-qemu-action@96fe6ef7f33517b61c61be40b68a1882f3264fb8 # v4.2.0
117135
with:
@@ -120,27 +138,44 @@ jobs:
120138
- name: 🐳 Set up Docker Buildx
121139
uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4.2.0
122140

123-
- name: 🐳 Login to Docker Hub
124-
if: needs.setup.outputs.push == 'true'
125-
uses: docker/login-action@650006c6eb7dba73a995cc03b0b2d7f5ca915bee # v4.2.0
141+
# Build everything before pushing anything so a build failure can't
142+
# leave the runtime and -build tags pointing at different snapshots
143+
- name: 🐳 Build both targets (no push)
144+
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
126145
with:
127-
username: ${{ secrets.DOCKERHUB_USERNAME }}
128-
password: ${{ secrets.DOCKERHUB_TOKEN }}
146+
context: base-images
147+
file: base-images/Dockerfile
148+
target: build
149+
platforms: linux/amd64,linux/arm64
150+
provenance: false
151+
outputs: type=image,push=false,rewrite-timestamp=true
152+
tags: triggerdotdev/${{ matrix.image.repo }}:${{ matrix.image.tag }}-build
153+
build-args: |
154+
BASE_IMAGE=${{ matrix.image.base }}
155+
DEBIAN_SNAPSHOT=${{ needs.setup.outputs.snapshot }}
156+
DEBIAN_SUITE=${{ needs.setup.outputs.suite }}
157+
PACKAGES=${{ needs.setup.outputs.packages }}
158+
BUILD_PACKAGES=${{ needs.setup.outputs.build_packages }}
159+
SOURCE_DATE_EPOCH=${{ needs.setup.outputs.source_date_epoch }}
160+
labels: |
161+
org.opencontainers.image.source=https://github.com/${{ github.repository }}
162+
org.opencontainers.image.revision=${{ github.sha }}
163+
dev.trigger.debian-snapshot=${{ needs.setup.outputs.snapshot }}
129164
130-
- name: 🐳 Build and push runtime image
165+
# rewrite-timestamp + SOURCE_DATE_EPOCH make layer blobs a pure
166+
# function of the build inputs, so published layers are verifiable
167+
- name: 🐳 Push runtime image
131168
id: build_runtime
169+
if: needs.setup.outputs.push == 'true'
132170
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
133171
with:
134172
context: base-images
135173
file: base-images/Dockerfile
136174
target: runtime
137175
platforms: linux/amd64,linux/arm64
138176
provenance: false
139-
# rewrite-timestamp + SOURCE_DATE_EPOCH make layer blobs a pure
140-
# function of the build inputs, so published layers are verifiable
141-
outputs: type=image,push=${{ needs.setup.outputs.push }},rewrite-timestamp=true
177+
outputs: type=image,push=true,rewrite-timestamp=true
142178
tags: triggerdotdev/${{ matrix.image.repo }}:${{ matrix.image.tag }}
143-
cache-from: type=gha,scope=base-${{ matrix.image.repo }}-${{ matrix.image.tag }}
144179
build-args: |
145180
BASE_IMAGE=${{ matrix.image.base }}
146181
DEBIAN_SNAPSHOT=${{ needs.setup.outputs.snapshot }}
@@ -152,19 +187,18 @@ jobs:
152187
org.opencontainers.image.revision=${{ github.sha }}
153188
dev.trigger.debian-snapshot=${{ needs.setup.outputs.snapshot }}
154189
155-
- name: 🐳 Build and push build-variant image
190+
- name: 🐳 Push build-variant image
156191
id: build_toolchain
192+
if: needs.setup.outputs.push == 'true'
157193
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
158194
with:
159195
context: base-images
160196
file: base-images/Dockerfile
161197
target: build
162198
platforms: linux/amd64,linux/arm64
163199
provenance: false
164-
outputs: type=image,push=${{ needs.setup.outputs.push }},rewrite-timestamp=true
200+
outputs: type=image,push=true,rewrite-timestamp=true
165201
tags: triggerdotdev/${{ matrix.image.repo }}:${{ matrix.image.tag }}-build
166-
cache-from: type=gha,scope=base-${{ matrix.image.repo }}-${{ matrix.image.tag }}
167-
cache-to: type=gha,scope=base-${{ matrix.image.repo }}-${{ matrix.image.tag }},mode=max
168202
build-args: |
169203
BASE_IMAGE=${{ matrix.image.base }}
170204
DEBIAN_SNAPSHOT=${{ needs.setup.outputs.snapshot }}
@@ -177,8 +211,24 @@ jobs:
177211
org.opencontainers.image.revision=${{ github.sha }}
178212
dev.trigger.debian-snapshot=${{ needs.setup.outputs.snapshot }}
179213
214+
# A repo auto-created private would let the publish go green while every
215+
# customer pull fails; require anonymous pullability before declaring success
216+
- name: 🔎 Verify anonymous pullability
217+
if: needs.setup.outputs.push == 'true'
218+
env:
219+
IMAGE_REPO: ${{ matrix.image.repo }}
220+
RUNTIME_DIGEST: ${{ steps.build_runtime.outputs.digest }}
221+
BUILD_DIGEST: ${{ steps.build_toolchain.outputs.digest }}
222+
run: |
223+
for digest in "$RUNTIME_DIGEST" "$BUILD_DIGEST"; do
224+
TOKEN="$(curl -fsS "https://auth.docker.io/token?service=registry.docker.io&scope=repository:triggerdotdev/$IMAGE_REPO:pull" | jq -r .token)"
225+
curl -fsS -o /dev/null -H "Authorization: Bearer $TOKEN" -H "Accept: application/vnd.oci.image.index.v1+json" "https://registry-1.docker.io/v2/triggerdotdev/$IMAGE_REPO/manifests/$digest" || { echo "triggerdotdev/$IMAGE_REPO@$digest is not anonymously pullable; is the repo private?"; exit 1; }
226+
done
227+
228+
# Image is already pushed by this point; don't fail publishes on a Sigstore hiccup
180229
- name: 🔏 Attest runtime image provenance
181230
if: needs.setup.outputs.push == 'true'
231+
continue-on-error: true
182232
uses: actions/attest-build-provenance@a2bbfa25375fe432b6a289bc6b6cd05ecd0c4c32 # v4.1.0
183233
with:
184234
subject-name: index.docker.io/triggerdotdev/${{ matrix.image.repo }}
@@ -187,6 +237,7 @@ jobs:
187237

188238
- name: 🔏 Attest build-variant image provenance
189239
if: needs.setup.outputs.push == 'true'
240+
continue-on-error: true
190241
uses: actions/attest-build-provenance@a2bbfa25375fe432b6a289bc6b6cd05ecd0c4c32 # v4.1.0
191242
with:
192243
subject-name: index.docker.io/triggerdotdev/${{ matrix.image.repo }}

‎base-images/Dockerfile‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -29,9 +29,9 @@ RUN . /etc/os-release && [ "$VERSION_CODENAME" = "${DEBIAN_SUITE}" ] || { echo "
2929
"deb [check-valid-until=no signed-by=/usr/share/keyrings/debian-archive-keyring.gpg] http://snapshot.debian.org/archive/debian-security/${DEBIAN_SNAPSHOT} ${DEBIAN_SUITE}-security main" \
3030
"deb [check-valid-until=no signed-by=/usr/share/keyrings/debian-archive-keyring.gpg] http://snapshot.debian.org/archive/debian/${DEBIAN_SNAPSHOT} ${DEBIAN_SUITE}-updates main" \
3131
> /etc/apt/sources.list && \
32-
printf 'Acquire::Retries "3";\n' > /etc/apt/apt.conf.d/99-snapshot-retries && \
32+
printf 'Acquire::Retries "5";\nAcquire::http::Timeout "30";\n' > /etc/apt/apt.conf.d/99-snapshot-retries && \
3333
apt-get update && \
34-
apt-get upgrade -y && \
34+
apt-get upgrade -y --with-new-pkgs && \
3535
apt-get install -y --no-install-recommends ${PACKAGES} && \
3636
apt-get clean && \
3737
rm /etc/apt/sources.list /etc/apt/apt.conf.d/99-snapshot-retries && \
@@ -52,7 +52,7 @@ RUN [ -n "${DEBIAN_SNAPSHOT}" ] && [ -n "${BUILD_PACKAGES}" ] || { echo "DEBIAN_
5252
"deb [check-valid-until=no signed-by=/usr/share/keyrings/debian-archive-keyring.gpg] http://snapshot.debian.org/archive/debian-security/${DEBIAN_SNAPSHOT} ${DEBIAN_SUITE}-security main" \
5353
"deb [check-valid-until=no signed-by=/usr/share/keyrings/debian-archive-keyring.gpg] http://snapshot.debian.org/archive/debian/${DEBIAN_SNAPSHOT} ${DEBIAN_SUITE}-updates main" \
5454
> /etc/apt/sources.list && \
55-
printf 'Acquire::Retries "3";\n' > /etc/apt/apt.conf.d/99-snapshot-retries && \
55+
printf 'Acquire::Retries "5";\nAcquire::http::Timeout "30";\n' > /etc/apt/apt.conf.d/99-snapshot-retries && \
5656
apt-get update && \
5757
apt-get install -y --no-install-recommends ${BUILD_PACKAGES} && \
5858
apt-get clean && \

‎base-images/README.md‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,9 @@ images derived from these behave like their upstream bases.
1616
Tags are mutable and rebuilt in place on demand; each rebuild picks up Debian
1717
security updates published up to its snapshot date. The
1818
runtime itself (the node or bun binaries from the upstream base) only moves
19-
when the base digests in `images.json` are bumped. Consumers pin digests: the CLI's generated
19+
when the base digests in `images.json` are bumped. When bumping a base
20+
digest, keep the snapshot at least as new as the upstream image's own archive
21+
state, or the upgrade step silently becomes a no-op. Consumers pin digests: the CLI's generated
2022
Containerfile references these images as `triggerdotdev/node:22-bookworm@sha256:...`,
2123
and digests only move when a CLI release updates its pins.
2224

@@ -33,8 +35,9 @@ digests differ because they carry build metadata labels like the source
3335
revision):
3436

3537
```bash
38+
# needs a docker-container builder (docker buildx create --use)
3639
SNAPSHOT=$(docker buildx imagetools inspect triggerdotdev/node:22-bookworm \
37-
--format '{{index .Image.config.Labels "dev.trigger.debian-snapshot"}}')
40+
--format '{{index (index .Image "linux/amd64").Config.Labels "dev.trigger.debian-snapshot"}}')
3841
# GNU date; the epoch must match the one the workflow derived from the snapshot
3942
EPOCH=$(date -u -d "${SNAPSHOT:0:4}-${SNAPSHOT:4:2}-${SNAPSHOT:6:2} ${SNAPSHOT:9:2}:${SNAPSHOT:11:2}:${SNAPSHOT:13:2}Z" +%s)
4043
docker buildx build base-images --target runtime \
@@ -47,7 +50,9 @@ docker buildx build base-images --target runtime \
4750
--output type=oci,dest=rebuilt.tar,rewrite-timestamp=true
4851
# then compare .layers[].digest of the rebuilt per-platform manifests against
4952
# the published ones (imagetools inspect --raw returns the index; fetch each
50-
# platform manifest it references to see its layers)
53+
# platform manifest it references to see its layers). For the -build variant,
54+
# use --target build and additionally pass --build-arg BUILD_PACKAGES. Layer
55+
# digests are stable for a given BuildKit version and compression settings.
5156
```
5257

5358
Every published digest also carries a GitHub build provenance attestation:

0 commit comments

Comments
 (0)