Skip to content

feat: install the aws cli and gh on the runner image - #8

Merged
frogggias merged 2 commits into
mainfrom
feat/6-aws-and-gh-cli
Oct 14, 2026
Merged

frogggias merged 2 commits into
mainfrom
feat/6-aws-and-gh-cli

Conversation

@frogggias

@frogggias frogggias commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator

What & why

Tower's v0.4.0 release (run 38571675894) failed on a ci-bokvi runner at "Publish desktop update feed" with aws is not on PATH. The published :ci-bokvi image (Ubuntu 24.04.5) has neither aws nor gh. The release job needs both: aws s3api with conditional writes for the update feeds, then gh release create in the very next step. If only aws were added, the release would fail one step later, so this PR installs both.

  • AWS CLI v2: a new aws-cli stage, FROM amazon/aws-cli:2.38.0@sha256:…, pinned like the cached Node.js so Dependabot's docker ecosystem bumps it. base copies /usr/local/aws-cli and recreates the /usr/local/bin/aws and aws_completer symlinks. COPY follows symlinks, and a copied binary can't find its bundled libpython; I hit this in the first build. Ubuntu's awscli package isn't used because it lacks put-object --if-match / --if-none-match, which Tower's publisher depends on and which fail closed when missing.
  • gh: installed from GitHub's apt repository (cli.github.com/packages), with its published keyring as signed-by, in the same apt-get layer as the other packages. The source and keyring are removed after the install, so a job's sudo apt-get update (Tower's Linux jobs run one) never depends on cli.github.com or on a keyring that could fall out of date. Like the rest of the apt packages, gh isn't version-pinned. The keyring isn't pinned to a fingerprint because GitHub rotates the key; the current keyring already lists a replacement next to an expired key.
  • make test now also runs gh --version and aws --version as runner, and checks that the gh apt source is gone. It then sends a put-object with both conditional options to a closed port and requires a connection error. An unrecognised option fails earlier with Unknown options, so the test fails.
  • Docs and Dependabot: the README package table and intro, the make test lines in the README and CONTRIBUTING, and the OCI description label. Dependabot ignores AWS CLI majors, like Node majors, because the Dockerfile links into the v2 layout.

Both :latest and :ci-bokvi get the tools, because they're added to base.

Proof

Before: the published :ci-bokvi image.

gh and aws missing from the published ci-bokvi image

After: make test on both architectures. arm64 runs natively on this Mac, and amd64 runs under emulation via DOCKER_DEFAULT_PLATFORM.

make test on linux/arm64

make test on linux/amd64

The probe catches a missing option:

aws rejects an unknown option

End to end: the amd64 image's aws performs conditional writes against a local S3 server (SeaweedFS 4.45). A create with --if-none-match '*' succeeds once and is then refused. --if-match with a stale ETag is refused, and with the current ETag it succeeds. These are the operations Tower's feed publisher uses on R2.

conditional PutObject against a local S3 server

The ci-bokvi target also builds, and has aws, gh and its .npmrc.

Rollout

After merge, Build and publish pushes :latest, :ci-bokvi and :ci-bokvi-<merge sha>. Coolify won't use them on its own: following prewarm/README.md steps 3 and 5, pull the new ci-bokvi image onto each build host, then select it by digest in the server's GitHub Runners settings. That applies to new stacks. After that, finish Tower's release with gh run rerun --failed 38571675894. The failed attempt published nothing; its first step logs whether aws is found.

Follow-ups (not in this PR)

  • Tower's deploy-server-image job adds the latest released CLI to the server image on a best-effort basis, using gh. On these runners it has been skipping that step; the latest main run logs gh is not installed on this runner; the image will not serve CLI updates. This PR fixes the cause. Confirm it on Tower's next push to main.

Review

An automated high-level review raised these, fixed in the second commit:

  • The gh apt source stayed in the image, so jobs' apt-get update could fail after a GitHub key rotation. It's now removed after the install.
  • gh --version | head -n 1 couldn't fail make test, because the script has no pipefail.
  • Dependabot could have proposed an AWS CLI major.
  • The README overstated how fresh gh is in cached builds.

Not changed:

  • aws … help needs groff/mandoc, which the image doesn't have. CI doesn't use help.
  • The gh keyring download relies on curl already being in the base image. It is, and the README already lists curl as something to keep.
  • Splitting the gh install into its own layer (nit).

Linked issue

Closes #6

frogggias and others added 2 commits October 14, 2026 17:15
Tower's release job publishes its update feeds with aws s3api
conditional writes and creates the GitHub Release with gh. The
ci-bokvi runners had neither, so the v0.4.0 release failed with
"aws is not on PATH".

- AWS CLI v2 is copied from the official amazon/aws-cli image, pinned
  by tag and digest so Dependabot bumps it. Ubuntu's awscli package
  lacks put-object --if-match / --if-none-match. The /usr/local/bin
  symlinks are recreated, because COPY follows them and the binary
  then can't find its bundled libraries.
- gh is installed from GitHub's apt repository.
- make test runs both as runner and checks that put-object accepts
  the conditional-write options.

Closes #6

Co-authored-by: Innie of Bokvi <innie@bokvi.com>
- Remove GitHub's apt source and keyring once gh is installed. Jobs that
  run `sudo apt-get update` then no longer fail if GitHub signs the
  repository with a key the image's cached keyring doesn't have.
- Run `gh --version` without a pipe, so a missing or broken gh fails
  `make test` (the script has no pipefail). Also check that the source
  is gone.
- Dependabot ignores AWS CLI majors, since the Dockerfile links into the
  v2 layout.
- Correct the README: gh isn't pinned, and cached builds keep it.

Refs #6

Co-authored-by: Innie of Bokvi <innie@bokvi.com>
@frogggias
frogggias merged commit 3febda8 into main Oct 14, 2026
@frogggias
frogggias deleted the feat/6-aws-and-gh-cli branch October 14, 2026 15:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: install the aws cli and gh on the runner image

1 participant