Skip to content

test(linux): an end-to-end test of Flock on Arch, Ubuntu and Fedora - #22

Closed
Nejcc wants to merge 18 commits into
flock-php:developfrom
Nejcc:feat/linux-e2e
Closed

Nejcc wants to merge 18 commits into
flock-php:developfrom
Nejcc:feat/linux-e2e

Conversation

@Nejcc

@Nejcc Nejcc commented Oct 19, 2026 •

Copy link
Copy Markdown

Refs #12.

Summary

This adds an end-to-end test of Flock on Linux: the one #12 promised, and the source of the results reported there. It installs Flock the way a user would, on real systemd-based Arch, Ubuntu and Fedora systems, checks every part of the setup, breaks it on purpose, and uninstalls it again. It also runs in CI whenever the Linux setup code changes.

Merge order: this is stacked on #16, which is stacked on #12 (at 9d631ee). Until those are merged, the diff includes their commits. The commits to review are:

  • bba647a test(linux): an end-to-end test of Flock on Arch, Ubuntu and Fedora
  • de949b6 test(linux): check flock0's NetworkManager state in the test's own shell

Why

Flock's Linux setup changes things a unit test can't reach, and each of them depends on how the distribution is set up:

  • systemd-resolved
  • a dummy network link (flock0)
  • NetworkManager
  • file capabilities for ports 80 and 443
  • the system and browser trust stores

Two setups in particular have already broken it, and only a real boot shows it:

  • Global DNS servers in resolved: the setup that a TLD in resolved's global scope can't handle.
  • An admin's NetworkManager config: a drop-in that reclaims flock0.

How it works

tools/linux-e2e/run.sh [--from <ref>] [arch] [ubuntu] [fedora] first builds the release archives from the working tree (make linux). Then, for each distribution, it boots a rootless Podman container running systemd, set up like a typical desktop:

  • systemd-resolved with global DNS servers (DNS=9.9.9.9 1.1.1.1)
  • a user session with linger
  • a Chrome NSS database
  • on Ubuntu and Fedora, NetworkManager, with an admin-style uplink.conf drop-in that uses unmanaged-devices=

Flock is installed with the real install.sh, through a curl stand-in that serves the release downloads from the local archives. With --from <ref>, that ref is installed first, so the run also covers the upgrade.

What it checks

Area Checks
Setup
  • Only flock-php is installed; util-linux's flock still comes first in a new shell.
  • flock-dns.service, the flock0 link and zz-flock.conf are in place, with no drop-in in resolved's global scope.
  • setcap lets Flock bind ports 80 and 443.
  • The CA is in the system store and in NSS.
  • *.test resolves through resolvectl and getent, including after a lookup outside the TLD.
  • HTTPS works.
NetworkManager
  • flock0 stays unmanaged.
  • The setup survives networking off/on followed by a resolved restart.
  • managed yes is refused.
  • A zzz-admin.conf read after Flock's is reported by flock-php status.
Drift
  • ip address flush dev flock0 is reported within 70 s, including while *.test still resolves.
  • systemctl restart flock-dns.service clears it.
Restarts
  • The setup survives a systemd-resolved restart and a full reboot.
Uninstall
  • uninstall --purge removes the unit, the link, the drop-in, the CA, the PATH line and the data.
  • Other NetworkManager drop-ins are kept.

CI

.github/workflows/linux-e2e.yml runs one job per distribution on ubuntu-24.04, with fail-fast off and a 30-minute timeout.

  • Triggers: changes to internal/cli, internal/dns, internal/certs, internal/ports, install.sh, the Makefile, the harness or the workflow itself. It can also be started by hand (workflow_dispatch).
  • Kernel module: the runner loads the dummy module itself (sudo modprobe dummy numdummies=0), because a rootless container can't.
  • Fork PRs: the first run on a fork PR needs a maintainer's approval.

Results

On 9d631ee, upgraded from 18d4050:

Distribution Checks Failed
Arch (no NetworkManager) 31 0
Ubuntu 24.04 + NetworkManager 1.46 41 0
Fedora + NetworkManager 41 0

On Ubuntu and Fedora, the drift is reported while *.test still resolves. That's the hidden case this test was written for. On Arch, lookups fail as soon as the address is gone, and the drift is reported as well.

Running it locally

sudo modprobe dummy numdummies=0      # once; rootless containers can't load it
tools/linux-e2e/run.sh --from 18d4050 # all three, upgrading from 18d4050
tools/linux-e2e/run.sh ubuntu         # or a single distribution

You need Podman. A full run takes about 15 minutes, mostly the one-minute status checks and the reboot.

sineld and others added 17 commits October 19, 2026 02:44
…nstalls the newest one

make linux builds flock-<version>-linux-{x64,arm64}.tar.gz with their .sha256; build.yml keeps
them as artifacts and betas and releases require them. install.sh picks the newest release (the
newest beta until there is a stable one), checks the archive and runs flock install.
util-linux owns /usr/bin/flock, so the package installs /usr/lib/flock-php/flock with a
/usr/bin/flock-php link; flock-php install sets Flock up for the user.
The archives hold flock-php, install.sh runs it, and the AUR package installs /usr/bin/flock-php
directly. The comments name the right folder, ~/.config/Flock/bin.
tools/linux-e2e/run.sh installs Flock with install.sh, from archives built from the tree, in a
rootless Podman container per distribution that boots systemd (systemd-resolved with global DNS
servers, a user session, NetworkManager on Ubuntu and Fedora). It checks the setup (flock-php,
flock-dns.service and flock0, the NetworkManager drop-in, setcap, the CA in the system and NSS,
HTTPS), NetworkManager taking networking down and a drop-in read after Flock's, drift reported
while lookups still work, a resolved restart and a reboot, and that uninstall --purge leaves
nothing. --from <ref> installs that ref first and upgrades from it.

linux-e2e.yml runs it per distribution when the Linux setup changes, and by hand.

@sineld sineld left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @Nejcc. This is the test #12 needed. The scenarios match the failures we actually hit, and having the conf.d ordering and the latent drift as regression checks is great. A few things I noticed in bba647a/de949b6, none of them blocking:

  • run.sh:151: install.sh's exit status is lost in the pipe to grep | sed. A failed install only shows up through the checks after it, so set -o pipefail or a check "install" would make that failure direct.
  • archlinux:latest and fedora:latest: CI can start failing without a change here. That's probably the point of an e2e test, just worth knowing when a run goes red.
  • CI: the workflow hasn't run yet, since it needs a maintainer's approval on a fork PR. So rootless Podman with --systemd=always on ubuntu-24.04 is still unproven there.

Comment thread tools/linux-e2e/run.sh Outdated

if [ -n "$from" ]; then
echo "== install $from"
user 'FLOCK_VERSION=0.0.0-from sh /test/install.sh </dev/null >/tmp/from.log 2>&1' || { cat "$work/from.log" 2>/dev/null; }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The log goes to /tmp/from.log inside the container, but the fallback reads $work/from.log on the host. That file never exists, and /test is mounted read-only anyway. So when the --from install fails, nothing is printed and nothing is counted. The run then carries on as a fresh install, while the summary still reads as an upgrade.

Maybe:

check "install $from" user 'FLOCK_VERSION=0.0.0-from sh /test/install.sh </dev/null >/tmp/from.log 2>&1 || { cat /tmp/from.log; exit 1; }'

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, that fallback could never fire. In ebfdb53 both installs are now checks that print the tail of the log from inside the container. So a failed --from install counts as a failure and doesn't quietly turn the run into a fresh install. I re-ran Arch with --from 89d0784, and PASS install 89d0784 now shows in the summary.

Comment thread tools/linux-e2e/run.sh Outdated

build() { # build <version> <source dir>
make -s -C "$2" -f "$repo/Makefile" linux VERSION="$1" >/dev/null &&
mv "$2"/dist/flock-"$1"-linux-x64.tar.gz* "$work/dist/"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make linux builds both linux-x64 and linux-arm64, but only the x64 archive is moved. On an arm64 host, install.sh asks for flock-<v>-linux-arm64.tar.gz, and the curl stand-in's cp fails. That covers GitHub's ubuntu-24.04-arm runners and the Ampere/Graviton boxes I run Flock on. flock-"$1"-linux-*.tar.gz* would cover both.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point about the Ampere/Graviton boxes and the arm runners. Fixed in ebfdb53 with flock-"$1"-linux-*.tar.gz*, so both archives go to the curl stand-in.

on:
workflow_dispatch:
pull_request:
paths:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The drift report this test guards lives in internal/daemon/wildcard.go: checkWildcard calls dns.LinkProblem on Linux. internal/daemon/** isn't in the list, so a change there that broke the report wouldn't run the test. go.mod/go.sum (and maybe cmd/flock/**) may be worth adding too.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added internal/daemon/**, cmd/flock/**, go.mod and go.sum in ebfdb53. I'd missed that checkWildcard → dns.LinkProblem path.

Comment thread tools/linux-e2e/run.sh Outdated
# status_says yes|no <seconds>: waits for `flock-php status` to (not) report flock0.
status_says() {
for _ in $(seq "$2"); do
if user '$F status' 2>/dev/null | grep -q flock0; then [ "$1" = yes ] && return 0; else [ "$1" = no ] && return 0; fi

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When $F status fails (the daemon isn't running), the output is empty, grep -q flock0 fails, and status_says no returns success. So "status clean" and "status clean again" also pass with a dead daemon. Requiring $F status to succeed in that branch would close it, e.g. out=$(user '$F status' 2>/dev/null) || { sleep 1; continue; }.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch, a dead daemon passed as clean. In ebfdb53, status_says only counts a flock-php status that succeeds, and a failing one just waits for the next second, close to your suggestion. So both "status clean" checks now need a running daemon.

- Both installs are checks now: the --from log was read from the host,
  where it never exists, and the tree's exit status was lost in a pipe.
- status_says needs a successful flock-php status, so a dead daemon
  no longer passes as "status clean".
- Move the arm64 archives too, for arm64 hosts and runners.
- Run the workflow on internal/daemon (the drift report), cmd/flock,
  go.mod and go.sum.
@Nejcc

Nejcc commented Oct 19, 2026

Copy link
Copy Markdown
Author

Thanks for the review, @sineld. The inline points are fixed in ebfdb53. On the top-level ones:

  • install.sh's exit status: handled by making the install a check (see the run.sh:148 thread) rather than pipefail, so a failed install is now a FAIL line of its own.
  • :latest images: yes, that's on purpose. A red run after an Arch or Fedora update is exactly what this test should catch. If it gets noisy, pinning by digest is a one-line change.
  • CI: agreed, it's still unproven on ubuntu-24.04 until a maintainer approves the first run. Locally, Arch passes all 33 checks with the upgrade from 89d0784.

@lukas-frey

Copy link
Copy Markdown
Member

Since this is dependent on the closed #16 PR, I'm closing this too. Thanks a lot for the PR and sorry for closing, in the future PRs need an issue first so we can discuss whether it makes sense to work on it, before someone actually puts their time and effort into the PR.

@lukas-frey lukas-frey closed this Oct 26, 2026
@Nejcc

Nejcc commented Oct 26, 2026

Copy link
Copy Markdown
Author

No worries, thanks! 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants