Skip to content

ci: a release-check workflow, run on demand on the release PR: update path, configs, audit log, mixed versions, stress, changelog, timing report #199

Closed

Description

@groupsky

Why

Before a release, the release PR is checked for compatibility with the last releases. Today an agent does this by hand from written steps, on another machine. It should be a workflow on GitHub runners, run only when the owner decides to release, not on each update of the release PR.

What to build

.github/workflows/release-check.yml, started by workflow_dispatch on the release-please branch. It records the commit it checks (the candidate) and writes one result file (JSON artifact + job summary) with each check's result and evidence.

  1. One build script for release.yml and this workflow (scripts/build-release.sh or a reusable workflow): the same Go version, CGO_ENABLED=0, architectures, -trimpath, ldflags, files, tar and gzip settings, and SHA256SUMS. release.yml uses it too; signing and publishing stay in release.yml.
  2. N-1 and N-2: the two highest published, non-draft, non-prerelease vX.Y.Z releases below the candidate (not the nearest tag: a failed release can leave a tag or a draft).
  3. Update path: the real N-1 and N-2 binaries run nodkey update (and their own install.sh) against a local HTTPS fixture that serves the candidate's files. Also: a missing or wrong checksum, and a missing file, are refused and keep the old binary. The production signer cannot sign a candidate, so the signature step is simulated here; ci: check the draft release's files before publishing (checksums, attestation, version, clean install) #200 checks the real one.
  4. Configs: a corpus of configs that N-1 accepts (test/compat/configs/, from the docs of each release, plus refused cases) loads with the candidate the same way.
  5. Audit log and state: N-1 writes them, stops; the candidate goes on with no tamper alarm; audit verify passes with both; a changed or cut copy is refused.
  6. Mixed versions: the end-to-end test takes NODKEY_E2E_BROKER_BIN and NODKEY_E2E_CLIENT_BIN (default: build from source, as today), and runs candidate/candidate, N-1 broker/candidate client, candidate broker/N-1 client. Release builds send version notices: the tests expect them.
  7. Stress: the full suite with -race -count=5 -cpu 1,2,4.
  8. Changelog: every PR merged since N-1 is in the release PR's notes, or has a hidden type; every ! PR has its docs/updating.md section (ci: a behavior-change gate: every PR says if it changes behavior for callers; yes needs '!' and an update note #197).
  9. Timing report (never fails the workflow): an agent-browser login (plugin → broker → a fake bw that sleeps 6 s per call), cold and after a pre-approval: the time against PluginResolveTime (12 s), and the number of bw calls. A reader judges it; timing on shared runners is not stable enough to fail on.

The workflow never needs a secret beyond the default token, and never publishes.

Done when

  • release-check.yml runs checks 1-9 by workflow_dispatch and writes the result file.
  • release.yml builds with the shared script (the files of a build are the same as before: checked byte for byte on one version).
  • The end-to-end test takes the two binary variables.
  • docs/releasing.md says how to run it and how to read the result.

Activity

  1. groupsky commented on Nov 18, 2026

    @groupsky
    ContributorAuthor

    hq-jobs nodkey-release-ci-2 (run r-20261118-210019-7f96)

    Done in #215 (merged). What each "Done when" box shows:

    The timing report never fails the workflow. In the first run, the login after a pre-approval took a few milliseconds, with 0 runs of bw. The cold pre-approval took about 18 s, with 3 runs of bw at 6 s each.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions