Publish a signed loreserver image from the released binaries by aleksanderllada · Pull Request #180 · EpicGames/lore

aleksanderllada · 2026-08-28T21:51:26Z

Summary

Fills in publish-loreserver-image.yml. It packages the loreserver binaries from a published GitHub release into signed images at ghcr.io/epicgames/lore/loreserver, so the community Helm chart can reference a verifiable tag. Nothing is compiled — the release assets are the artifacts.

Trigger

release: published, not the tag push: releases here are drafted, have assets uploaded, and are published by hand hours or days later, so a tag-push trigger would race the upload and publish an image for a draft nobody had announced.

The draft is created by automation but published by hand, so the event fires from a person rather than from GITHUB_TOKEN — which would have triggered nothing at all.

workflow_dispatch takes a required release and an optional tag, for backfilling a release that predates this workflow and for proving the workflow from a branch.

Shape

Job
`meta`	validate inputs, decide which moving tags this release may take, resolve tags/labels/annotations
`build`	fetch the assets, take `default.toml` from the release tag, build both variants, push each by digest
`smoke-arm64`	pull by digest on a native arm64 runner, run `--version`
`publish`	per variant: assert the shape, sign, verify, rehearse the copy, promote, re-resolve

push-by-digest pushes each variant untagged, so it is signed and verified before it is ever named. Promotion re-resolves every tag and fails if one is off the signed digest. arm64 is smoke tested natively rather than under emulation, because the binary is tuned past baseline — though that runner has SVE, so it proves nothing about older arm64, which is what the tag says. No secrets: GITHUB_TOKEN plus OIDC to Fulcio.

Safeguards

Mostly added in review, and the reason for most of the workflow's complexity:

latest goes only to the newest release, and X.Y only to the newest of its series, so a backfill or an older hotfix cannot drag either backwards.
A release GitHub flags as a prerelease takes neither, whatever its tag looks like.
The tag input cannot take a release-tag shape unless release tags are open, so a dispatch from a branch cannot publish latest from unreviewed code.
The staging tag carries the release as well as the commit — every backfill runs from the default branch and shares its SHA.
default.toml comes from the release tag, not the branch: config keys move between releases and can fail an older binary at startup. Only the packaging comes from the workflow's ref.

Each tarball's SHA-256 is recorded in the image labels and the run summary, alongside the cosign verify invocation — the releases carry no checksums of their own.

Test evidence

Run 34513691611: green on all five jobs against the real v0.9.0 assets. Both moving-tag gates resolved true and the branch gate still withheld every release tag, so the two layers are independent.
Run 34487676120: dispatched with tag: latest and correctly refused before anything was built.
From an M3 Max: a plain docker pull of the unsuffixed tag fetches linux/amd64 and runs under emulation; -graviton refuses to pull as amd64, and its arm64 binary exits 132 (SIGILL) on non-Graviton hardware, as the tag says. The baked config is byte-identical to git show v0.9.0:lore-server/config/default.toml.
The latest regression was reproduced against the pinned metadata-action before being fixed.

duncangrist

Can you ensure you run a /code-review in CC as my review brought up a few things.

Also, two CI jobs are failing currently.

aleksanderllada · 2026-09-02T15:25:23Z

@duncangrist done, and I've updated the workflow to build both portable arm64 and arm64-graviton.

duncangrist · 2026-09-03T14:42:23Z

CI sill failing.

duncangrist

Looking good!

epic-lore-bot · 2026-09-04T16:00:31Z

Imported as Lore CR-551.

## Summary Fills in the publish workflow: each architecture builds on its own native runner and is pushed to GHCR by digest, a merge job stitches the digests into one manifest list, and cosign signs that list keylessly through the job's OIDC token. No secrets are involved. Also drops the Graviton3+ target-cpu from the arm64 container build. The repo tunes aarch64-unknown-linux-gnu for Epic's own deployment, which emits instructions that fault on older arm64 parts and, because the build passes no --target, reaches the proc macros cargo runs on the build host too. A published image has to run anywhere armv8-a runs. ## Test Plan 1. actionlint passes on the workflow. 2. Dispatch from this branch and confirm the manifest list carries linux/amd64 and linux/arm64, and that cosign verify succeeds. 3. Pull the arm64 image on a non-Graviton arm64 host and confirm the server starts. Signed-off-by: Aleksander Arruda <aleksander.arruda@xa.epicgames.com>

The first full run measured 145G on the runner root with 108G still free after both release builds, so deleting unrelated preinstalled toolchains buys nothing and only adds a step that can fail on its own. Signed-off-by: Aleksander Arruda <aleksander.arruda@xa.epicgames.com>

## Summary The previous attempt set CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_RUSTFLAGS to drop the Graviton3+ target-cpu. That does not work: the variable is another source for the same config key, and cargo joins config arrays rather than replacing them, so -C target-cpu=neoverse-512tvb survived and the published arm64 image died with SIGILL on any older arm64 part. Use RUSTFLAGS instead, which is a mutually exclusive source that suppresses [build] and [target.*] rustflags outright, and repeat the flags the build actually needs. Also stop overriding the release strip setting. The workspace asks for line-tables-only debug info on purpose, and the override was discarding the line numbers that make a panic backtrace useful. Finally, run the freshly pushed image on its native runner. Nothing in the workflow executed the binary, which is exactly why a SIGILL shipped past a fully green run. ## Test Plan 1. actionlint passes. 2. The new smoke step fails the arm64 job when the Graviton flag is present and passes once it is gone. 3. Pull the arm64 image on Apple Silicon, which has no SVE, and confirm loreserver --version prints. Signed-off-by: Aleksander Arruda <aleksander.arruda@xa.epicgames.com>

## Summary Review of the publish workflow turned up six problems, three of them able to publish or sign the wrong thing. Tags and labels are now resolved once, in a new `meta` job that runs before anything is built. That addresses three of them together: - `latest` was gated only on the ref starting with refs/tags/v, so v1.0.0-rc.1 moved and signed `latest`, and a v-prefixed tag that is not semver at all left `latest` as the only tag the run published. The default `latest=auto` flavor applies it only to a tag that parses as a stable semver release. - A workflow_dispatch from a branch with a blank tag input resolved to no tags at all, and the job then died on `imagetools inspect ghcr.io/...:`, after both arch images had been pushed. A type=ref,event=branch entry gives that path a tag from the ref alone, as the input description already promised, and a gate in the meta job fails the run before any layer reaches GHCR if the list is still empty. - The build job ran its own metadata-action with no tags config, so the labels baked into the image carried the ref-derived version (v1.2.3, or a branch name) while the published tag was 1.2.3. And three smaller ones: - The tag input was interpolated straight into the tags list, where an embedded newline would have smuggled in a further directive. It is held to Docker's tag grammar before it gets there. - imagetools create never received the annotations, so the published index carried none. Building its argument list as an array also removes the word splitting the old command relied on, which would have broken on the spaces in image.description. - The cosign identity regexp was anchored on the repository alone, so it accepted a signature from any workflow here that can ask for an OIDC token. It is anchored on this workflow's path now, and defined once, since the summary hands the same expression to downstream users. Separately, the image was built with --release, which Cargo.toml labels a fast build suitable for local development: debug-assertions, and with them overflow-checks, stay on and there is no LTO. Acceptable for a local convenience image, not for the artifact the Helm chart points at. Build release-lto, "the actual release build", and strip the full DWARF it asks for back out, keeping the symbol table so backtraces still name frames. ## Test Plan 1. actionlint, with the repo config, reports no errors. 2. docker buildx build --check on the Dockerfile is clean. 3. The tag validator accepts edge, 1.2.3 and v1.0.0-rc.1, and rejects an embedded newline, a leading dash, a leading dot, a space, a slash and a 129-character input. 4. The imagetools argument construction, dry-run against a representative metadata-action JSON, keeps image.description as one argument. 5. Still to confirm on the first run: fat LTO links inside the memory and time a 4-core runner has, a prerelease tag does not move latest, and a branch dispatch publishes under the branch name. Signed-off-by: Aleksander Arruda <aleksander.arruda@xa.epicgames.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

## Summary Lore is tuned for and deployed on Graviton3+, so a tuned arm64 image now ships as well — but as an opt-in `-graviton` tag rather than as the default, because `-C target-cpu=neoverse-512tvb` faults on any older arm64 part and the default tag is what a community chart pulls. X.Y.Z, X.Y, latest arm64 baseline armv8-a X.Y.Z-graviton, latest-graviton arm64 tuned for Graviton3+ The Dockerfile now assembles RUSTFLAGS itself from an ARM64_TARGET_CPU build arg, defaulting to empty. amd64 is baseline in both variants, so it is built once and both manifest lists reference that single digest — the extra arm64 leg runs in parallel and leaves the critical path where it was. OCI cannot express this: platform.variant covers v6/v7/v8, not microarchitecture, so a client cannot be steered to the right arm64 build automatically. Separate tags are the only honest mechanism. ## Also corrects the smoke test's comment The comment claimed the step catches codegen aimed at the wrong CPU. It does not. GitHub's arm64 runner is Neoverse-N2 and reports SVE, so it executes the Graviton-tuned build without complaint — measured, not assumed. The step is still worth keeping for a broken entrypoint or a missing shared library, and the comment now says only that. ## Test Plan 1. actionlint passes. 2. Dispatch from a branch and confirm two manifest lists publish, each carrying linux/amd64 and linux/arm64, each signed and verified. 3. Confirm both lists reference the same amd64 digest. 4. Run the default arm64 image on Apple Silicon, which has no SVE, and confirm loreserver starts; confirm the -graviton arm64 image does not. Signed-off-by: Aleksander Arruda <aleksander.arruda@xa.epicgames.com>

release-lto sets debug = 2, and a fat-LTO link holds the whole dependency graph's DWARF in memory at once. On a 16 GB runner rustc is SIGKILLed part way through linking loreserver, so both the amd64 and the Graviton arm64 builds died. Build with -C debuginfo=0. strip --strip-debug discarded that DWARF a line later regardless, and tracing's file and line fields come from compile-time macros rather than debug info, so nothing observable is lost. Signed-off-by: Aleksander Arruda <aleksander.arruda@xa.epicgames.com>

## Summary Four ways this workflow could have shipped the wrong thing without saying so. ### A failed graviton leg no longer withholds the default image `merge` declared `needs: [meta, build]`, which waits on the whole matrix *and* requires all of it to pass. So the opt-in graviton leg was a hard dependency of the portable default image — the one a community chart pulls — exactly inverting what this workflow is for. It also made the partial-manifest guard unreachable, since `merge` could never run with a leg missing. The smoke test makes this concrete rather than theoretical: it passes only because GitHub's arm64 pool is Neoverse-N2 and reports SVE. Rotate that pool to an N1 part and the graviton leg SIGILLs, taking the default release with it. `if: !cancelled() && needs.meta.result == 'success'` depends on the legs having finished rather than on all of them having passed. `meta` stays a hard requirement — a variant with no resolved tags has nothing to publish — and the leg-existence guard now decides which variant can actually ship. ### No release tag points at an unsigned digest `imagetools create` moved every tag, `latest` included, and `cosign sign` ran seven steps later. A Fulcio or OIDC failure in between left `latest` published and unverifiable, which undercuts the verifiable tag this workflow exists to produce. The index is now created under a `sha-<commit>` staging tag alone, signed, verified, and only then promoted onto the release tags. The staging tag never moves and nothing consumes it, so it is the one thing exposed to that window. Promotion depends on `imagetools create` copying a lone index source through byte for byte. That is proven first, by copying the index onto the staging tag it already occupies: a no-op while the digest is preserved, and if some later buildx stops preserving it, the tag that moves out from under the signature is the staging tag rather than `latest`. Each release tag is re-resolved afterwards as well. `primary-tag` and `graviton-primary-tag` go with it — the staging tag addresses the index now, so those outputs had no remaining reader. The gate step they came from stays, since an empty tag list still has to fail here. ### The -graviton image says so in its own labels The graviton leg was handed the default variant's labels, so its arm64 manifest reported `image.version=X.Y.Z` while the index annotation said `X.Y.Z-graviton`. `docker inspect` on a pulled image could not tell the two arm64 builds apart, which is the first thing anyone debugging a SIGILL on older arm64 hardware would reach for. amd64 is shared between both manifest lists and can only carry one set, which remains the default's. ### One place to retune Graviton `GRAVITON_TARGET_CPU` was never read; the matrix hard-coded the literal instead, so editing the env var changed nothing about what was built. The build arg now derives from that env var via the variant, and the matrix column is gone. `env` is unavailable in `strategy.matrix`, hence resolving it at the step. An untuned build is a perfectly successful one, so a graviton leg that resolved no target CPU would have published a baseline image under the `-graviton` tag with every step green. It now fails before building. ## Test Plan 1. actionlint passes. 2. Dispatch from a branch; confirm both manifest lists publish, each signed and verified, and that every release tag resolves to the signed digest. 3. Confirm the `sha-<commit>` tag exists on that same digest, and that the signature verifies against it. 4. Confirm `docker inspect` on the arm64 `-graviton` image reports a `-graviton` `org.opencontainers.image.version`, and the default does not. 5. Force the graviton build leg to fail; confirm the default variant still publishes and signs, and that the graviton merge fails on the leg guard rather than being skipped. 6. Confirm the build log for the graviton leg prints `tuning arm64 for neoverse-512tvb`, and that the baseline legs skip that step. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Aleksander Arruda <aleksander.arruda@xa.epicgames.com>

## Summary Making the arm64 image portable by default contradicted three documents that were not updated with it. `docs/how-to/deploy-local-lore-server.md` is the primary Run with Docker how-to, and it still told Apple Silicon and Windows readers to build and run with `--platform linux/amd64` because the arm64 image targets Graviton3 SVE — the restriction the baseline default removed, and the opposite of what DOCKER.md now says. Following it meant a release Rust build under QEMU and an emulated runtime to work around nothing. The path-chooser bullet carried the same caveat. `contrib/aws/README.md` deploys c8gd.8xlarge Graviton instances and used to inherit `-C target-cpu=neoverse-512tvb` from `.cargo/config.toml`. The Dockerfile assembles RUSTFLAGS itself now, so that command silently shipped an untuned baseline binary to the one place tuning was meant for. It passes `--build-arg ARM64_TARGET_CPU=neoverse-512tvb`. ## Also in DOCKER.md The new Published images heading was inserted above an existing paragraph, which left "generates self-signed TLS certificates for QUIC using scripts/server/make-certs.sh" reading as a claim about the published GHCR images. It was wrong twice over: that script has never existed in this repository, and the Dockerfile generates no certificates. The server generates an ephemeral self-signed one at startup, which is now stated under Running, where someone whose client rejects the certificate will look for it. The `docker.toml` bullet claimed to configure QUIC certificates for the same reason and had the same problem; the file it describes sets two store paths. The tag table was missing `X.Y-graviton`, which `meta-graviton` emits from the same `{{major}}.{{minor}}` pattern the default row already documents, so pinning a minor series on Graviton was undiscoverable. It also now describes the `sha-<commit>` tag, which the two-phase signing change publishes alongside each release and which would otherwise look like stray litter in the package list. ## Test Plan 1. docs-lint.sh passes with vale, markdownlint and lychee installed. 2. Build the image on Apple Silicon following the how-to verbatim, with no `--platform` flag, and confirm loreserver starts. 3. Build with the contrib/aws command and confirm the log line reports `-C target-cpu=neoverse-512tvb`. 4. Confirm no document still references `scripts/server/make-certs.sh` as a step in building the Docker image. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Aleksander Arruda <aleksander.arruda@xa.epicgames.com>

## Summary Comment-only. The explanations added while fixing this workflow had grown longer than what they explain, so they are cut back to the reasoning that is not recoverable from the code beside them: why `merge` waits on the legs rather than on their success, why the index is staged before it is signed, why the target CPU comes from the workflow env, and why the smoke test does not prove what it looks like it proves. 101 comment lines down to 81. No expression, step, or shell line is touched: diff <(git show HEAD:<file> | grep -vE '^\s*#') \ <(grep -vE '^\s*#' <file>) is empty, so run 33642540774 still attests to this behaviour and the workflow needs no re-run. ## Test Plan 1. YAML parses and actionlint passes. 2. The non-comment diff against the previous revision is empty. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Aleksander Arruda <aleksander.arruda@xa.epicgames.com>

Rewriting the gate as a loop introduced `for json in "$JSON" ...`, a local differing from the environment variable beside it only in case. That is what SC2153 looks for, so actionlint failed the workflow: SC2153:info:2:14: Possible misspelling: JSON may not be assigned. Did you mean json? Name the loop variable `tags_json`. Behaviour is unchanged. Checked the rest of the file for the same pattern: no other script assigns a local whose uppercase form matches an environment variable it reads, and shellcheck now passes on all thirteen `run` blocks. Signed-off-by: Aleksander Arruda <aleksander.arruda@xa.epicgames.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Aleksander Arruda <aleksander.arruda@xa.epicgames.com>

## Summary This workflow compiled loreserver from source, making it a second place that builds Lore's client. It now fetches the binaries from a published release and only packages them, so the release remains the single build. That costs the portable image. The release ships exactly one arm64 Linux binary and it is tuned for Graviton3+: loreserver-<version>-x86_64-unknown-linux-gnu.tar.gz loreserver-<version>-aarch64-unknown-linux-gnu-neoverse-512tvb.tar.gz so there is no baseline armv8-a binary left to build a portable variant from. Rather than hand that arm64 image to Apple Silicon and Ampere hosts under an unsuffixed tag — the SIGILL fb89c97 fixed — every tag keeps the -graviton suffix, `latest-graviton` included, and no unsuffixed tag is published at all. Claiming the portable name for a non-portable image is worse than leaving it unclaimed. When a baseline binary ships, the portable stream can take it back. ## Trigger `release: published`, not the tag push. Releases here are drafted, have their assets uploaded, and are published by hand hours or days later: v0.9.0 assets Aug 28 14:31 published Aug 31 07:22 v0.8.6 assets Jul 30 11:25 published Jul 31 09:37 v0.8.5 assets Jul 15 16:25 published Jul 16 12:02 v0.8.4 assets Jun 25 15:26 published Jun 26 19:14 The tag therefore exists long before the assets are complete, so a tag-push trigger would race the upload and, when it won, publish an image and move `latest` for a draft nobody had announced. `published` fires with every asset already in place. It also covers prereleases, which publish their own tag and nothing else, as before. `workflow_dispatch` gains a required `release` input: with no release event there is nothing to infer the version from, and it is the only way to backfill a release that predates this workflow. `type=ref,event=branch` is dropped — no branch-shaped asset exists on the release to fetch. ## Shape Copying a prebuilt binary needs no emulation, so the per-arch native runners the Rust compile required are gone, and with them the per-leg digest artifacts and the `imagetools create` stitching: buildx pushes the finished index itself. Five jobs become four, none of which compile. meta validate the version and tag inputs, resolve tags and labels build fetch, package both platforms in one pass, push by digest smoke-arm64 pull by digest on a native arm64 runner, run --version publish sign, verify, rehearse the copy, promote, re-resolve Sign-before-tag is unchanged, staging-tag rehearsal included: that is about signature integrity, not architecture. push-by-digest still leaves the index unnamed until the signature exists. arm64 is smoke tested on ubuntu-24.04-arm rather than under QEMU, because the released binary is tuned past baseline and emulation need not implement what it was tuned for. As before, that runner reports SVE and so does not prove the binary runs on older arm64 — nothing here does, which is what the tag says. The releases publish no checksums or signatures, so each tarball's sha256 is computed after download and recorded in the image labels under com.epicgames.lore.asset.<arch>.sha256, and in the run summary. That is the only thing tying a published image back to exact bytes. lore-server/Dockerfile is left alone: it still builds from source, which is what a checkout without this workflow needs. The packaging stage lives beside it as Dockerfile.release and each names the other. ## Test Plan Not yet run in CI — it cannot be, as `release: published` needs a release and the dispatch path needs this file on a branch first. Locally: actionlint (the repo's pinned v1.7.1 hook), trailing-whitespace, end-of-file-fixer, remove-tabs and codespell all pass. Extracted all twelve `run` blocks and ran shellcheck 0.11.0 and `bash -n` over them: clean, and no block contains a `${{ }}` expression, so everything reaches shell through `env`. Note actionlint skips shell linting silently when shellcheck is absent, which it was, so that green was confirmed only after checking shellcheck flags a planted SC2086. Asset layout verified against the real v0.9.0 tarball: ./loreserver, ./LICENSE.txt and ./THIRD-PARTY-NOTICES.txt at the archive root. Three things the first run has to confirm: that `latest=auto` reads prerelease-ness from `type=semver,value=` rather than the ref; that the GHCR package is public, since a workflow-created package starts private; and that releases are not published by a workflow using GITHUB_TOKEN, which would trigger nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Aleksander Arruda <aleksander.arruda@xa.epicgames.com>

## Summary Packaging the released binaries left `lore-server/DOCKER.md` describing tags the workflow no longer publishes. Its table still offered `X.Y.Z`, `X.Y` and `latest` as a baseline `armv8-a` stream, so the document told readers to pull an image that will not exist. Replace that table with the one stream that ships, and say why there is no unsuffixed tag rather than leaving its absence to be discovered: releases carry exactly one arm64 Linux binary and it is tuned for Graviton3+, so an unsuffixed tag would hand it to Apple Silicon and Ampere hosts that fault on it. Building from source is still the portable route, and still documented above it. Also name the two Dockerfiles, since there are now two: `Dockerfile` builds from source and is what that section describes, `Dockerfile.release` packages a published asset and is used only by the workflow. The `sha-<commit>` tag is `sha-<commit>-graviton` now that every tag carries the suffix. ## Restored in the workflow Rewriting the summary step dropped the `cosign verify` invocation it used to print, which DOCKER.md documents and which is the one thing in the summary a reader acts on. Put it back, unchanged, alongside the release assets and their SHA-256. `docs/how-to/deploy-local-lore-server.md` and `contrib/aws/README.md` need no change: both describe building from source with `lore-server/Dockerfile` and `--build-arg ARM64_TARGET_CPU`, which this branch leaves alone. ## Test Plan actionlint, trailing-whitespace, end-of-file-fixer, remove-tabs and codespell pass. Grepped DOCKER.md for surviving references to unsuffixed tags and to there being two variants: none left. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Aleksander Arruda <aleksander.arruda@xa.epicgames.com>

## Summary Taking the version from an input rather than the ref made a dispatch resolve the release tags. Dispatching from this pull request branch to prove the workflow would therefore have published `0.9.0-graviton`, `0.9-graviton` and `latest-graviton`, pointing the real release tags at an image built from unreviewed code and signed with this workflow's identity. Gate the two semver tags on the run being a release, or a dispatch from the default branch. A dispatch from any other branch now publishes only its `tag` input, and `latest=auto` follows the semver tag so it stays off too. Backfilling a release that predates this workflow still works: dispatch from the default branch, where the gate opens. When the gate is closed and no `tag` is passed, no tags resolve at all. The existing check catches that before anything is built; its message now says which input is missing. ## Test Plan Not yet dispatched — this is what makes that dispatch safe to run, and it is next. actionlint passes. Re-extracted all twelve `run` blocks and re-ran shellcheck 0.11.0 and `bash -n`: clean, still no `${{ }}` inside any block. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Aleksander Arruda <aleksander.arruda@xa.epicgames.com>

## Summary The first dispatch failed on the assertion that the unpacked binary is executable. It is not: both Linux archives store it 0644. $ tar tvzf loreserver-v0.9.0-x86_64-unknown-linux-gnu.tar.gz drwx------ runner runner 0 ./ -rw-r--r-- runner runner 1073 ./LICENSE.txt -rw-r--r-- runner runner 986559 ./THIRD-PARTY-NOTICES.txt -rw-r--r-- runner runner 40046992 ./loreserver The aarch64 archive is identical in this respect, so the assertion was simply wrong to expect the bit rather than set it. `chmod +x` after unpacking. This is load-bearing, not cosmetic: `COPY` carries the context's mode into the image, so without it the entrypoint would be non-executable and every container would fail at startup — an error the build and the push would both report as success. Split the check while here. It asserted `-x`, which is also false for a file that is absent, so a missing binary and an unexecutable one gave the same message. Existence is now its own failure. ## Test Plan Verified against the real v0.9.0 archives, downloaded and unpacked locally. The amd64 archive's sha256 matches what the failed run recorded, so these are the same bytes CI saw: d30324ce10f5498a749e6dea65100d01103474a5373cb5db0f270831c967a1d7 actionlint passes; all twelve `run` blocks still clean under shellcheck 0.11.0 and `bash -n`. Re-dispatching next. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Aleksander Arruda <aleksander.arruda@xa.epicgames.com>

## Summary Pulling a `-graviton` image on a Mac gets you `linux/arm64` and a SIGILL, and there is no unsuffixed tag to reach for instead. Restore one, carrying `linux/amd64` alone: | Tag | Platforms | | --- | --- | | `X.Y.Z`, `X.Y`, `latest` | `linux/amd64` only | | `X.Y.Z-graviton`, ... | `linux/amd64`, and `linux/arm64` tuned for Graviton3+ | An amd64 image is not a portable arm64 one, so this is a fallback rather than a fix, and the release still ships nothing that would make a portable arm64 image possible. What it buys is that the obvious command now works everywhere instead of faulting on the second-most-common developer machine. ## Why a plain manifest, not a single-entry index Because an index listing only `linux/amd64` is refused outright: $ docker run --rm <amd64-only index> no matching manifest for linux/arm64/v8 in the manifest list entries exit=125 $ docker run --rm <amd64-only plain manifest> WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) ... x86_64 exit=0 Both measured on an M3 Max against a local registry. So the variant has to be a plain manifest, which is what `provenance: false` is doing: buildx attaches provenance as a second manifest, and anything with two manifests is an index. The cost is that this variant carries no provenance attestation. It is still cosign-signed. That also breaks promotion, which was `imagetools create`. Given a lone plain manifest it rewraps it in an index, and the digest changes with it: imagetools create sha256:fbeeabe8... -> sha256:d9da8d55... imagetools create --prefer-index=false sha256:fbeeabe8... -> sha256:fbeeabe8... Without the flag the default variant would land on its release tags as an index, off its signature and having lost the fallback it exists for. The existing digest-preserving check would have caught it and failed the run, but only after the staging tag had moved, so the flag is the fix and the check stays as the guard. ## Shape is now asserted A variant quietly becoming the wrong shape passes every other step in this workflow, so `publish` now checks the media type and platform set against what the variant claims before signing anything. That is what would catch the default variant gaining an arm64 manifest, or turning back into an index. ## Test Plan actionlint passes; all thirteen `run` blocks clean under shellcheck 0.11.0 and `bash -n`, none containing a `${{ }}` expression. The manifest-resolution and digest-preservation behaviour above was measured directly, not assumed — an earlier version of this reasoning had it backwards and would have shipped an amd64-only index that no arm64 host could pull. Dispatching next to prove both variants end to end. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Aleksander Arruda <aleksander.arruda@xa.epicgames.com>

## Summary The default variant's build failed: ERROR: failed to solve: index annotations not supported for single platform export It was handed the `index,manifest` annotations the tuned variant needs. buildx rejects index annotations on a single-platform export rather than ignoring them, and this variant exports one platform as a plain manifest by design. Drop the level override for that variant, leaving metadata-action's default of manifest-level annotations. The tuned variant keeps `index,manifest`, because its index is what gets signed and what a puller resolves. The comment claimed the override was "harmless on the default variant, which has no index". That was exactly backwards: having no index is what makes it an error. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Aleksander Arruda <aleksander.arruda@xa.epicgames.com>

## Summary Comment-only. The explanations added while reworking this workflow had grown back past what 65c336f trimmed them to — 140 comment lines in a 513-line file, 46 of them a header block restating the jobs below it. Down to 57, keeping only the places where a reasonable edit silently breaks something: the release-tag gate, `provenance: false`, `--prefer-index=false`, the annotation levels, the `chmod`, push-by-digest, the native arm64 runner, the shape check, and the identity anchor. Most of those were found the hard way rather than reasoned out, and none of them are recoverable from the code. The tag table and the reasoning behind the two variants now live only in lore-server/DOCKER.md, which is where a reader looking for them would go. `lore-server/Dockerfile.release` gets the same treatment, 17 lines to 11. ## Test Plan No behaviour change, and the same check 65c336f used shows it: diff <(git show HEAD:<file> | grep -vE '^\s*#' | grep -vE '^\s*$') \ <(grep -vE '^\s*#' <file> | grep -vE '^\s*$') is empty for both files, but for a trailing `# One variant failing should not withhold the other` on `fail-fast: false`, which that pattern does not strip. So run 34378951799 still attests to this behaviour and needs no re-run. actionlint, trailing-whitespace, end-of-file-fixer, remove-tabs and codespell all pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Aleksander Arruda <aleksander.arruda@xa.epicgames.com>

`flavor: latest=auto` gives `latest` to any non-prerelease semver without consulting what is already published, so backfilling an older release -- which this workflow documents as a supported dispatch -- or hotfixing an older line would move `latest` and `latest-graviton` onto the older image. Running the pinned metadata-action against a v0.9.0 backfill alongside a newer v0.11.0 emits exactly that. Resolve the newest stable release first and gate an explicit raw `latest` tag on it; the flavor suffix reaches a raw tag, so the tuned stream still gets `latest-graviton` and not `latest`. Take `default.toml` from the release tag rather than the checkout. Only a release run checks out the tag; a dispatch gets whatever ref it came from, and config keys move between releases -- main has since added `permit_timeout_ms` and moved `lock_service` under `.general`, either of which can fail an older binary at startup. The Dockerfile still comes from the workflow's own ref, because a release predating this workflow has no Dockerfile.release at its tag. EXPOSE defaults to TCP, so it was not declaring the QUIC listener that shares 41337; `docker run -P` left it unreachable. Signed-off-by: Aleksander Arruda <aleksander.arruda@xa.epicgames.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

The `latest` gate added in the previous commit left three ways round it. `X.Y` is a moving tag too, and was enabled by RELEASE_TAGS alone, so a backfill of v0.8.5 would republish `0.8` over v0.8.6. It now needs to be the newest of its own series, and `latest` the newest overall. `latest` compared VERSION against the newest *stable* release and took the tag if it sorted higher. A release GitHub flags as a prerelease is absent from that list whatever its tag looks like, so v1.0.0 published as a prerelease outranked the stable it could not see and took `latest`. The comparison is now equality, which requires membership. The `tag` input reaches the same tag list with only a charset check, so a dispatch from any branch could publish `latest` from unreviewed code -- precisely what the RELEASE_TAGS gate exists to prevent. Release-tag shapes are now refused unless release tags are open. Also from review: The staging tag was the commit alone, but every backfill dispatch runs from the default branch and shares its SHA. Two would overwrite each other's tag and read back the other's digest, failing after cosign has signed and before any release tag moves. It carries the version now. `ARM64_TARGET_CPU=""` only ever reached RUSTFLAGS, while lore-base pins `-mcpu=neoverse-512tvb` for its cc build of rpmalloc regardless. That flag raises the architecture floor as well as the tuning: GCC 14 emits `stlur` (FEAT_LRCPC2, armv8.4) and `ldapr` (armv8.3), which are undefined on Neoverse N1 parts such as Ampere Altra and Graviton2 -- the hosts DOCKER.md promises a baseline build runs on. Verified by disassembling both builds; with the tuning off the allocator is clean armv8-a. `Dockerfile.release` now chmods the binary itself. The workflow did it, but DOCKER.md documents unpacking the asset and building by hand, and the archive stores it 0644, so that path produced an image whose entrypoint could not execute. `oci-mediatypes=true` is explicit rather than a BuildKit default the publish job then asserts on, and the stale several-GB RAM note now agrees with the 16 GB the fat-LTO link needs. Signed-off-by: Aleksander Arruda <aleksander.arruda@xa.epicgames.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

No behaviour change: stripping comments and blank lines from each file before and after leaves the four identical. Signed-off-by: Aleksander Arruda <aleksander.arruda@xa.epicgames.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

duncangrist · 2026-09-10T17:29:04Z

+            platforms: linux/amd64
+          - variant: graviton
+            media: application/vnd.oci.image.index.v1+json
+            platforms: linux/amd64,linux/arm64


I'm confused as to why graviton also has linux/amd64, the -graviton image should only run on Graviton machines - i.e. only package linux/arm64.

It's leftover from the previous implementation. Claude justified keeping it because in theory it's good practice to have the amd64 version, so that docker pull still works in emulation mode. But that's only the case for portable arm64, not graviton. I've removed it.

duncangrist · 2026-09-10T17:39:32Z

now you have your own Dockerfile.release, changes to this one should be in a different PR

True. Moved to #201, together with some other changes that were no longer needed for this PR.

Nothing pulls a Graviton amd64 image; that half only duplicated the unsuffixed variant over identical layers. `-graviton` is now linux/arm64 alone, and carries only the arm64 asset's SHA-256 label. `lore-server/Dockerfile` goes back to `--release`. It built the real release-lto profile because the published image was compiled from it; the published image now comes from release assets, so that only slowed local builds down and needed a 16 GB note in two guides. The arm64 tuning stays: .cargo/config.toml pins Graviton3+ for aarch64 regardless of who is building, and source is the only route to a portable arm64 image. Signed-off-by: Aleksander Arruda <aleksander.arruda@xa.epicgames.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

It came in when the workflow still compiled the image from source, where publishing a portable arm64 tag required it. The workflow packages release assets now and never builds lore-server/Dockerfile, and main already documents a working path for Apple Silicon. Moved to aarruda/native-arm64-docker-build. This branch is back to the workflow, Dockerfile.release, and the docs for what gets published. Signed-off-by: Aleksander Arruda <aleksander.arruda@xa.epicgames.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

epic-lore-bot · 2026-09-10T19:02:20Z

Imported as Lore CR-589.

## Summary Fills in `publish-loreserver-image.yml`. It packages the `loreserver` binaries from a published GitHub release into signed images at `ghcr.io/epicgames/lore/loreserver`, so the community Helm chart can reference a verifiable tag. Nothing is compiled — the release assets are the artifacts. ## Tags | Tag | Platforms | Shape | | --- | --- | --- | | `X.Y.Z`, `X.Y`, `latest` | `linux/amd64` | plain manifest | | `X.Y.Z-graviton`, `X.Y-graviton`, `latest-graviton` | `linux/arm64` tuned for Graviton3+ | index | **On Graviton3 or newer**, pull `-graviton`. **Anywhere else**, pull the unsuffixed tag. Two decisions worth a reviewer's attention, both covered at more length in `DOCKER.md`: **arm64 is only under the suffixed tag.** Releases ship no baseline `armv8-a` Linux binary — the only `aarch64-unknown-linux-gnu` build is tuned for Graviton3+, and the Apple one is Mach-O. OCI cannot express the difference: `platform.variant` covers `v6`/`v7`/`v8`, not microarchitecture, so Graviton3, Ampere Altra and Apple Silicon are all just `linux/arm64` and nothing in the pull path consults CPU features. A separate tag is the only honest mechanism. **The unsuffixed variant is a plain manifest, not a single-entry index.** An index listing only `linux/amd64` makes an arm64 host *fail* the pull (`no matching manifest for linux/arm64/v8`); a plain manifest emulates with a warning. That is why it passes `provenance: false` — provenance would be a second manifest, and two manifests force an index. Both variants are signed regardless; only this one lacks a provenance attestation. ## Trigger `release: published`, not the tag push: releases here are drafted, have assets uploaded, and are published by hand hours or days later, so a tag-push trigger would race the upload and publish an image for a draft nobody had announced. The draft is created by automation but published by hand, so the event fires from a person rather than from `GITHUB_TOKEN` — which would have triggered nothing at all. `workflow_dispatch` takes a required `release` and an optional `tag`, for backfilling a release that predates this workflow and for proving the workflow from a branch. ## Shape | Job | | | --- | --- | | `meta` | validate inputs, decide which moving tags this release may take, resolve tags/labels/annotations | | `build` | fetch the assets, take `default.toml` from the release tag, build both variants, push each by digest | | `smoke-arm64` | pull by digest on a native arm64 runner, run `--version` | | `publish` | per variant: assert the shape, sign, verify, rehearse the copy, promote, re-resolve | `push-by-digest` pushes each variant untagged, so it is signed and verified before it is ever named. Promotion re-resolves every tag and fails if one is off the signed digest. arm64 is smoke tested natively rather than under emulation, because the binary is tuned past baseline — though that runner has SVE, so it proves nothing about older arm64, which is what the tag says. No secrets: `GITHUB_TOKEN` plus OIDC to Fulcio. ## Safeguards Mostly added in review, and the reason for most of the workflow's complexity: - `latest` goes only to the newest release, and `X.Y` only to the newest of its series, so a backfill or an older hotfix cannot drag either backwards. - A release GitHub flags as a prerelease takes neither, whatever its tag looks like. - The `tag` input cannot take a release-tag shape unless release tags are open, so a dispatch from a branch cannot publish `latest` from unreviewed code. - The staging tag carries the release as well as the commit — every backfill runs from the default branch and shares its SHA. - `default.toml` comes from the release tag, not the branch: config keys move between releases and can fail an older binary at startup. Only the packaging comes from the workflow's ref. Each tarball's SHA-256 is recorded in the image labels and the run summary, alongside the `cosign verify` invocation — the releases carry no checksums of their own. ## Test evidence - [Run 34513691611](https://github.com/EpicGames/lore/actions/runs/34513691611): green on all five jobs against the real `v0.9.0` assets. Both moving-tag gates resolved `true` and the branch gate still withheld every release tag, so the two layers are independent. - [Run 34487676120](https://github.com/EpicGames/lore/actions/runs/34487676120): dispatched with `tag: latest` and correctly refused before anything was built. - From an M3 Max: a plain `docker pull` of the unsuffixed tag fetches `linux/amd64` and runs under emulation; `-graviton` refuses to pull as amd64, and its arm64 binary exits 132 (`SIGILL`) on non-Graviton hardware, as the tag says. The baked config is byte-identical to `git show v0.9.0:lore-server/config/default.toml`. - The `latest` regression was reproduced against the pinned `metadata-action` before being fixed. ``` Imported-PR: #180 Imported-From: 2135bc4 Imported-Base: 3e4fd2a Imported-Merge: 3795af3 Imported-Merge-Strategy: verbatim Imported-Merged-Paths: 0 Imported-Author: Aleksander Arruda (aleksanderllada) Signed-off-by: Aleksander Arruda <aleksander.arruda@xa.epicgames.com> GH-URL: #180 ``` Lore-RevId: 1052 Lore-Signature: 27f6a22077101bba1fb0f6986a194987c8dfa13845e4a3a081598892a23b15e6

epic-lore-bot · 2026-09-11T13:24:13Z

Closed by mirrored commit 0aa8f7e.

github-actions Bot added area:server Server, provider integrations, telemetry area:ci CI workflows and GitHub configuration labels Aug 28, 2026

aleksanderllada requested review from duncangrist and peter-lockhart-pub August 31, 2026 12:56

duncangrist requested changes Sep 1, 2026

View reviewed changes

aleksanderllada force-pushed the aarruda/publish-loreserver-image branch from 13fcea7 to aaee068 Compare September 1, 2026 13:04

aleksanderllada changed the title ~~Build and publish a signed multi-arch loreserver image to GHCR~~ Build and publish signed multi-arch loreserver images to GHCR Sep 1, 2026

github-actions Bot added the documentation Improvements or additions to documentation label Sep 2, 2026

aleksanderllada requested a review from duncangrist September 2, 2026 20:50

duncangrist approved these changes Sep 4, 2026

View reviewed changes

aleksanderllada added the ready-to-import Approved by Epic staff for import into Lore label Sep 4, 2026

epic-lore-bot Bot added imported Imported into Lore for internal review and removed ready-to-import Approved by Epic staff for import into Lore labels Sep 4, 2026

aleksanderllada force-pushed the aarruda/publish-loreserver-image branch from 2935aaf to 226ca0c Compare September 9, 2026 16:04

aleksanderllada changed the title ~~Build and publish signed multi-arch loreserver images to GHCR~~ Publish a signed loreserver image from the released binaries Sep 9, 2026

aleksanderllada requested a review from duncangrist September 9, 2026 16:06

aleksanderllada force-pushed the aarruda/publish-loreserver-image branch from 4df42f5 to 4f8e0ce Compare September 9, 2026 23:02

aleksanderllada and others added 10 commits September 10, 2026 09:50

aleksanderllada and others added 8 commits September 10, 2026 09:50

aleksanderllada force-pushed the aarruda/publish-loreserver-image branch from 4f8e0ce to c01582e Compare September 10, 2026 12:51

github-actions Bot added the area:core Core library and its interfaces (lib, C API); revision, storage, transport, protocol internals label Sep 10, 2026

Cut the comments back once more

2247122

No behaviour change: stripping comments and blank lines from each file before and after leaves the four identical. Signed-off-by: Aleksander Arruda <aleksander.arruda@xa.epicgames.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

duncangrist requested changes Sep 10, 2026

View reviewed changes

aleksanderllada and others added 2 commits September 10, 2026 15:20

duncangrist approved these changes Sep 10, 2026

View reviewed changes

aleksanderllada mentioned this pull request Sep 10, 2026

Build a native arm64 image instead of emulating amd64 #201

Open

aleksanderllada added the ready-to-import Approved by Epic staff for import into Lore label Sep 10, 2026

epic-lore-bot Bot removed the ready-to-import Approved by Epic staff for import into Lore label Sep 10, 2026

epic-lore-bot Bot closed this Sep 11, 2026

epic-lore-bot Bot added the merged Merged into Lore codebase label Sep 11, 2026

Navigation Menu

Uh oh!

Publish a signed loreserver image from the released binaries - #180

Publish a signed loreserver image from the released binaries#180
aleksanderllada wants to merge 22 commits into
mainfrom
aarruda/publish-loreserver-image

aleksanderllada commented Aug 28, 2026 •

edited

Loading

Uh oh!

duncangrist left a comment

Uh oh!

aleksanderllada commented Sep 2, 2026

Uh oh!

duncangrist commented Sep 3, 2026

Uh oh!

duncangrist left a comment

Uh oh!

epic-lore-bot Bot commented Sep 4, 2026

Uh oh!

duncangrist Sep 10, 2026

Uh oh!

aleksanderllada Sep 10, 2026

Uh oh!

duncangrist Sep 10, 2026

Uh oh!

aleksanderllada Sep 10, 2026

Uh oh!

epic-lore-bot Bot commented Sep 10, 2026

Uh oh!

epic-lore-bot Bot commented Sep 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

2 participants

Tag	Platforms	Shape
`X.Y.Z`, `X.Y`, `latest`	`linux/amd64`	plain manifest
`X.Y.Z-graviton`, `X.Y-graviton`, `latest-graviton`	`linux/arm64` tuned for Graviton3+	index

Uh oh!

Conversation

aleksanderllada commented Aug 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Tags

Trigger

Shape

Safeguards

Test evidence

Uh oh!

duncangrist left a comment

Choose a reason for hiding this comment

Uh oh!

aleksanderllada commented Sep 2, 2026

Uh oh!

duncangrist commented Sep 3, 2026

Uh oh!

duncangrist left a comment

Choose a reason for hiding this comment

Uh oh!

epic-lore-bot Bot commented Sep 4, 2026

Uh oh!

duncangrist Sep 10, 2026

Choose a reason for hiding this comment

Uh oh!

aleksanderllada Sep 10, 2026

Choose a reason for hiding this comment

Uh oh!

duncangrist Sep 10, 2026

Choose a reason for hiding this comment

Uh oh!

aleksanderllada Sep 10, 2026

Choose a reason for hiding this comment

Uh oh!

epic-lore-bot Bot commented Sep 10, 2026

Uh oh!

epic-lore-bot Bot commented Sep 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

2 participants

aleksanderllada commented Aug 28, 2026 •

edited

Loading