Repository navigation
feat: install the aws cli and gh on the runner image - #8
Merged
Merged
Conversation
Tower's release job publishes its update feeds with aws s3api conditional writes and creates the GitHub Release with gh. The ci-bokvi runners had neither, so the v0.4.0 release failed with "aws is not on PATH". - AWS CLI v2 is copied from the official amazon/aws-cli image, pinned by tag and digest so Dependabot bumps it. Ubuntu's awscli package lacks put-object --if-match / --if-none-match. The /usr/local/bin symlinks are recreated, because COPY follows them and the binary then can't find its bundled libraries. - gh is installed from GitHub's apt repository. - make test runs both as runner and checks that put-object accepts the conditional-write options. Closes #6 Co-authored-by: Innie of Bokvi <innie@bokvi.com>
- Remove GitHub's apt source and keyring once gh is installed. Jobs that run `sudo apt-get update` then no longer fail if GitHub signs the repository with a key the image's cached keyring doesn't have. - Run `gh --version` without a pipe, so a missing or broken gh fails `make test` (the script has no pipefail). Also check that the source is gone. - Dependabot ignores AWS CLI majors, since the Dockerfile links into the v2 layout. - Correct the README: gh isn't pinned, and cached builds keep it. Refs #6 Co-authored-by: Innie of Bokvi <innie@bokvi.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Tower's v0.4.0 release (run 38571675894) failed on a
ci-bokvirunner at "Publish desktop update feed" withaws is not on PATH. The published:ci-bokviimage (Ubuntu 24.04.5) has neitherawsnorgh. Thereleasejob needs both:aws s3apiwith conditional writes for the update feeds, thengh release createin the very next step. If onlyawswere added, the release would fail one step later, so this PR installs both.aws-clistage,FROM amazon/aws-cli:2.38.0@sha256:…, pinned like the cached Node.js so Dependabot'sdockerecosystem bumps it.basecopies/usr/local/aws-cliand recreates the/usr/local/bin/awsandaws_completersymlinks. COPY follows symlinks, and a copied binary can't find its bundledlibpython; I hit this in the first build. Ubuntu'sawsclipackage isn't used because it lacksput-object --if-match/--if-none-match, which Tower's publisher depends on and which fail closed when missing.cli.github.com/packages), with its published keyring assigned-by, in the sameapt-getlayer as the other packages. The source and keyring are removed after the install, so a job'ssudo apt-get update(Tower's Linux jobs run one) never depends on cli.github.com or on a keyring that could fall out of date. Like the rest of the apt packages, gh isn't version-pinned. The keyring isn't pinned to a fingerprint because GitHub rotates the key; the current keyring already lists a replacement next to an expired key.make testnow also runsgh --versionandaws --versionasrunner, and checks that the gh apt source is gone. It then sends aput-objectwith both conditional options to a closed port and requires a connection error. An unrecognised option fails earlier withUnknown options, so the test fails.make testlines in the README and CONTRIBUTING, and the OCI description label. Dependabot ignores AWS CLI majors, like Node majors, because the Dockerfile links into thev2layout.Both
:latestand:ci-bokviget the tools, because they're added tobase.Proof
Before: the published
:ci-bokviimage.After:
make teston both architectures. arm64 runs natively on this Mac, and amd64 runs under emulation viaDOCKER_DEFAULT_PLATFORM.The probe catches a missing option:
End to end: the amd64 image's
awsperforms conditional writes against a local S3 server (SeaweedFS 4.45). A create with--if-none-match '*'succeeds once and is then refused.--if-matchwith a stale ETag is refused, and with the current ETag it succeeds. These are the operations Tower's feed publisher uses on R2.The
ci-bokvitarget also builds, and hasaws,ghand its.npmrc.Rollout
After merge, Build and publish pushes
:latest,:ci-bokviand:ci-bokvi-<merge sha>. Coolify won't use them on its own: following prewarm/README.md steps 3 and 5, pull the newci-bokviimage onto each build host, then select it by digest in the server's GitHub Runners settings. That applies to new stacks. After that, finish Tower's release withgh run rerun --failed 38571675894. The failed attempt published nothing; its first step logs whetherawsis found.Follow-ups (not in this PR)
deploy-server-imagejob adds the latest released CLI to the server image on a best-effort basis, usinggh. On these runners it has been skipping that step; the latestmainrun logsgh is not installed on this runner; the image will not serve CLI updates. This PR fixes the cause. Confirm it on Tower's next push tomain.Review
An automated high-level review raised these, fixed in the second commit:
apt-get updatecould fail after a GitHub key rotation. It's now removed after the install.gh --version | head -n 1couldn't failmake test, because the script has nopipefail.Not changed:
aws … helpneedsgroff/mandoc, which the image doesn't have. CI doesn't use help.curlalready being in the base image. It is, and the README already lists curl as something to keep.Linked issue
Closes #6