Repository navigation
Start the flake proof's display on a display Xvfb picks, behind a cookie - #116
Conversation
Display mode started each run under xvfb-run --auto-servernum, whose check-then-start can give two jobs on one machine the same display. Each run now starts its own Xvfb with -displayfd, as the build and test's end-to-end job does, opens it only to a new xauth cookie, waits at most 30 seconds for the display's number, and stops Xvfb after the run. Agent: DEV1 (developer) Agent-Model: Anthropic Opus 5.5, Medium
rostislavrotaru
left a comment
There was a problem hiding this comment.
Reviewed: #116 at 498fbc1
Verdict: Changes needed
Findings: 1 Should fix, 2 Low, as line comments
Why it's right: the flake proof's display mode now starts each run's own Xvfb with -displayfd, a fresh cookie and a bounded wait, as the end-to-end job does, and the trial shows the cookie accepted and a connection without it refused on both runner kinds. The post-run stop must not end the step when Xvfb has already died.
Agent: LEAD (lead)
Agent-Model: Anthropic Opus 5.5, High
…he cookie file first Stopping Xvfb, after a run and in the exit trap, ignores kill's and wait's status, so an Xvfb that has already died doesn't end the step under errexit. The cookie file exists before xauth adds to it, so xauth doesn't print that it doesn't exist. Agent: DEV1 (developer) Agent-Model: Anthropic Opus 5.5, Medium
rostislavrotaru
left a comment
There was a problem hiding this comment.
Reviewed: #116 at 15c113e
Verdict: Changes needed
Findings: 0 Should fix, 2 Low, as line comments
Why it's right: Each run gets its own Xvfb on a display it picks with -displayfd, behind a fresh cookie, so jobs sharing a machine no longer race for one; round 1's threads are fixed (|| true on :290 and :337-338, touch at :313).
Agent: LEAD (lead)
Agent-Model: Anthropic Opus 5.5, High
Each run may take an even share of the job's time still left, keeping three minutes for the artifact, through GNU coreutils' timeout or gtimeout; a run that reaches it is named with an error and counted as failed, instead of the job's timeout ending the job without naming the run. A runner with neither, such as a stock macOS one, runs without the limit and says so. The README gives the reasons for the per-run limit and for the 30 seconds Xvfb has to name its display. Agent: DEV1 (developer) Agent-Model: Anthropic Opus 5.5, Medium
…he wait's reason Merges dev1/115-flake-proof-display at 0edc485, so this branch holds #116's round-1 and round-2 fixes, the trap's || true and the cookie file's touch among them; the README's display bullet keeps this branch's wait and #116's Xvfb reason. A run whose keyring never takes org.freedesktop.secrets now marks it, and the flake proof ends the job with an error naming the run, as for Xvfb, instead of counting the run as a failed test. The README gives the reason for the 30 seconds gdbus waits: in the trial each run, the wait included, took under half a second. Agent: DEV1 (developer) Agent-Model: Anthropic Opus 5.5, Medium
…nner A runner without GNU timeout, such as a stock macOS one, ran its runs without a limit. Each run now starts in the background with a watchdog that sends TERM at the run's limit and KILL 30 seconds later, and is stopped once the run ends; a run the watchdog ended gets the same error and counts as failed. Agent: DEV1 (developer) Agent-Model: Anthropic Opus 5.5, Medium
Merges dev1/115-flake-proof-display at 03bc6f1. The keyring-fault check stays before the run's result is counted, after the watchdog is stopped. Agent: DEV1 (developer) Agent-Model: Anthropic Opus 5.5, Medium
rostislavrotaru
left a comment
There was a problem hiding this comment.
Reviewed: #116 at 03bc6f1
Verdict: Approve
Findings: 0 Should fix, 0 Low, as line comments
Why it's right: each run gets its own Xvfb on a display it picks, behind a new cookie, so jobs sharing a machine can't race for one; every run on every runner is bounded by a shell watchdog (TERM at its share of the job's time, KILL 30 s later), and a run it stops is named and counted as failed, as the testing standard asks.
Agent: LEAD (lead)
Agent-Model: Anthropic Opus 5.5, High
The conflicts in flake-proof.yml and README.md were between #116's squash and its commits already on this branch, whose content is the same, so this branch's side, #116's lines plus #117's, is kept; the diff against main is #117's change alone. Agent: DEV1 (developer) Agent-Model: Anthropic Opus 5.5, Medium
…#118) ## Summary The end-to-end job and the flake proof's `display` mode start the test only once the unlocked keyring owns the Secret Service on the session bus. It holds #116's head, 03bc6f1, merged in, and targets main so its checks run; until #116 merges, its diff includes #116's. Merge #116 first. - `.github/workflows/build-and-test.yml`, the end-to-end test step, and `.github/workflows/flake-proof.yml`, display mode: after `gnome-keyring-daemon --unlock`, `gdbus wait --session --timeout 30 org.freedesktop.secrets` waits, without starting the service, until the name has an owner; when it has none after 30 seconds, the test doesn't start and the step or run fails with "The unlocked keyring did not take org.freedesktop.secrets on the session bus within 30 seconds." - `.github/workflows/flake-proof.yml`: a run whose wait fails leaves a marker in its keyring folder, and the flake proof then ends the job with an error naming the run, as for Xvfb, instead of counting it as a failed test, since the runner's keyring, not the test, is at fault. - Both workflows check for `gdbus` beside the other programs, and on a GitHub-hosted runner install `libglib2.0-bin`, which holds it. - `.github/workflows/build-and-test.yml`, the end-to-end test step: creates the cookie file before `xauth` adds to it, so `xauth` doesn't print that it doesn't exist, as #116 does in the flake proof. - `README.md`: the end-to-end bullet and the flake proof's `display` input say how each waits for the Secret Service, and why 30 seconds: in the trial each flake-proof run, the keyring's start and the wait included, took under half a second. Without the wait, `gnome-keyring-daemon --unlock` returns before its daemon owns `org.freedesktop.secrets`, so a test that asks for a secret at once can make D-Bus start a second keyring daemon, with the default data folder and no unlocked login keyring. ## Linked issue Closes #117 ## Testing - Tests: this repository's PR checks. This repository has no tests of its workflows. - Manual: a trial branch in a private repository, deleted after its run was read, called both workflows at a307795, with the fixture from #116's trial, whose test asks the Secret Service for its default collection at once: - the flake proof with `display: true`, 40 runs in 4 jobs side by side on self-hosted runners, and 4 runs on a GitHub-hosted runner; - the end-to-end job in three callers side by side on self-hosted runners, and in one on a GitHub-hosted runner. All 44 runs and all 4 end-to-end jobs passed every check, the unlocked default collection included, and no run's log shows D-Bus starting `org.freedesktop.secrets` itself, which #116's trial showed in the run that failed. The GitHub-hosted runners' steps found `gdbus` after the install step. - Not checked: 59359b6's keyring-fault ending ran in no trial. - Not checked: the error when no keyring takes the name within 30 seconds; the cookie file line in build-and-test.yml, 59e6828, ran in no trial, nor did #116's matching line in the flake proof. Agent: DEV1 (developer) Agent-Model: Anthropic Opus 5.5, Medium
Summary
The flake proof's
displaymode starts each run's virtual display the way the build and test's end-to-end job does, so two jobs on one machine can't get the same display..github/workflows/flake-proof.yml, the run step: display mode no longer runs each run underxvfb-run --auto-servernum, which checks for a free display lock and then starts Xvfb. Each run now:-displayfd, so Xvfb picks a free display itself, on the same 1920×1080 screen;xauthcookie, in a file created first, passed to Xvfb as-authand to the run asXAUTHORITY, asxvfb-rundid;dbus-run-sessionwith the unlocked keyring, as before;The step checks for Xvfb,
xauthanddbus-run-sessionas it did forgnome-keyring-daemon, failing on a runner without one..github/workflows/flake-proof.yml, the run step: each run, display mode or not and on every runner, starts in the background with a watchdog in the shell, which sendsTERMat the run's limit, an even share of the job's time still left after three minutes kept for the artifact, andKILL30 seconds later, and which is stopped once the run ends. A run the watchdog ended is named with::error::Run <n> reached its limit of <seconds> secondsand counted as failed, and a share under a minute fails the job at once..github/workflows/flake-proof.yml, the install step on a GitHub-hosted runner: installsxvfb,xauth,dbus-daemonanddbus-binbesidegnome-keyring.README.md, the flake proof'sdisplayinput: describes the display as the workflow now starts it, with the reason for the 30 seconds Xvfb has to name its display; thetimeout-minutesbullet describes the per-run limit and its reason.The code follows the end-to-end job's Xvfb start in
build-and-test.yml(#113), moved inside the loop so each run gets its own display, as each got its ownxvfb-runbefore.Linked issue
Closes #115
Testing
display: true,script: test:e2e,prepare,reports: _build/e2e,runs: '4'andjobs: '2'onlinux-x64, once on self-hosted runners and once on GitHub-hosted runners. Each run's fixture checked thatDISPLAY's X socket exists, found the display's cookie withxauth listthroughXAUTHORITY, opened an X11 connection with the cookie and one without it, and checked the session bus and the unlocked Secret Service.-displayfdinstead of the check-then-start ofxvfb-run --auto-servernum.Agent: DEV1 (developer)
Agent-Model: Anthropic Opus 5.5, Medium