Skip to content

ci(release): pin both halves of the zig toolchain, and cut v0.8.1 - #199

Merged
vaderyang merged 4 commits into
mainfrom
fix/zigbuild-aarch64
Sep 12, 2026
Merged

ci(release): pin both halves of the zig toolchain, and cut v0.8.1#199
vaderyang merged 4 commits into
mainfrom
fix/zigbuild-aarch64

Conversation

@vaderyang

Copy link
Copy Markdown
Collaborator

v0.8.0 was tagged and never released: three of four targets built,
aarch64-unknown-linux-musl failed to link, so release.yml never reached the
create-release step. Nothing in Heron caused it — two build dependencies drifted
under a pin that is two years old.

What was broken

cargo-zigbuild was too old. rustc stable passes
-Wl,--fix-cortex-a53-843419 for aarch64 targets, and zig's linker rejects
unknown arguments outright:

error: unsupported linker arg: --fix-cortex-a53-843419

cargo-zigbuild added a filter for it in 0.23.0
(rust-cross/cargo-zigbuild#452).
The pin here was 0.19.6, released 2024-12-16 — before the flag existed.
dtolnay/rust-toolchain@stable moved underneath it.

ziglang was not pinned at all. It is a hard pip dependency of
cargo-zigbuild declared ziglang>=0.9.0, so pip resolved it to whatever was
newest — it had reached 0.16, where both Linux targets silently produce
dynamically linked binaries. The static-linkage check caught that, which is
why it surfaced as "no tarball" rather than as a broken release asset.

So the matrix has never actually built with the zig version this workflow
declares. Both halves are now pinned to 0.13.0 / 0.23.4, and the install step
prints each version — a mismatch between them otherwise appears three steps
later as an unrelated-looking C compiler error.

Both zig installs are needed, for different consumers: the action provides a
zig executable for the libpcap step (CC="zig cc -target …"), while pip's
copy is what links the Rust side. I removed the action at one point on the
assumption it was redundant; libpcap's configure immediately died with "C
compiler cannot create executables". The comment now records why both exist.

Verification

workflow_dispatch on this branch, all four targets:

target Build Link check
x86_64-unknown-linux-musl ✅ static
aarch64-unknown-linux-musl ✅ static
x86_64-apple-darwin
aarch64-apple-darwin

The upload step fails on this branch and that is expected, not a defect: the
package name is heron-${GITHUB_REF_NAME}-<target>, and a branch name
containing / puts the tarball in a subdirectory where dist/*.tar.gz cannot
match it. On a v* tag the ref name has no slash. Worth knowing before using
workflow_dispatch to validate the matrix from a fix/… branch.

The version bump

v0.8.0 stays where it is — deleting a pushed tag buys nothing when no release,
asset, or deploy ever referenced it — so this cuts 0.8.1 with the same
content plus the build fix. The CHANGELOG section is renamed rather than
duplicated: release.yml extracts the section matching the tag and fails when
there is none, and a stub 0.8.1 section pointing at 0.8.0 would have produced
release notes omitting everything the release contains. The section opens by
saying there is no released 0.8.0, so the gap in the sequence is explained where
someone will look.

https://claude.ai/code/session_01GxGWpo3L4BRMkwtoa9X54W

Vader Yang added 4 commits September 12, 2026 18:39
…more

The aarch64-unknown-linux-musl leg of the release matrix fails at link time:

    error: unsupported linker arg: --fix-cortex-a53-843419

rustc stable passes that flag for aarch64 targets (the Cortex-A53 erratum
workaround), and zig's linker rejects unknown args outright rather than warning.
cargo-zigbuild's job is to translate rustc's linker invocation into something
zig accepts, and it learned to filter this one in 0.23.0
(rust-cross/cargo-zigbuild#452, `filter_linker_arg` in src/zig.rs).

The pin was 0.19.6, released 2024-12-16 — long before the flag existed. Nothing
in this repo changed; `dtolnay/rust-toolchain@stable` moved under it, which is
the hazard of pinning one half of a toolchain pair and floating the other. This
broke the v0.8.0 tag: three targets built, aarch64 did not, so no release was
created.

Zig deliberately stays on 0.13.0 — 0.15+ wants clang 18+ for crates using
bindgen, and duckdb-sys does.

Claude-Session: https://claude.ai/code/session_01GxGWpo3L4BRMkwtoa9X54W
…ld fix

v0.8.0 was tagged and built three of four targets; the aarch64 leg failed to
link, so `release.yml` never reached the create-release step and nothing
shipped. The tag stays where it is — deleting a pushed tag buys nothing when no
release, no asset, and no deploy ever referenced it — so the content moves to
0.8.1 instead.

The CHANGELOG section is renamed rather than duplicated: release.yml extracts
the section matching the tag and fails the build if there is none, and a two-line
0.8.1 section pointing at a 0.8.0 section would have produced release notes that
omitted everything the release actually contains. The section now opens by
saying there is no released 0.8.0 to upgrade from, so the gap in the version
sequence is explained where someone will look for it.

Claude-Session: https://claude.ai/code/session_01GxGWpo3L4BRMkwtoa9X54W
…ting

Upgrading cargo-zigbuild fixed the aarch64 link error and broke both Linux
targets a different way: they built, but produced dynamically linked binaries,
which the static-link check rejected, so no tarball was produced.

The cause is that `ziglang` is a pip dependency of cargo-zigbuild with an open
upper bound (`ziglang>=0.9.0`), and the zig it installs lands on PATH ahead of
whatever `setup-zig` provided. So this workflow has never actually built with
the zig version it declared — pip's resolution decided, and it had drifted to
0.16, where the static build silently stops being static.

Both halves are now pinned to what the matrix has really been building with:
cargo-zigbuild 0.23.4 (has the aarch64 arg filter) and ziglang 0.13.0. The
`setup-zig` step is dropped rather than left to imply a version it never
controlled, and the install step now prints the zig version so the next person
does not have to infer it from a libcxx include path in a build log.

Claude-Session: https://claude.ai/code/session_01GxGWpo3L4BRMkwtoa9X54W
My last commit removed it on the reasoning that pip's ziglang wins on PATH and
the action was therefore decorative. That was wrong, and the next run said so:
libpcap's configure died with "C compiler cannot create executables". The two
installs serve different consumers.

  * the action puts a `zig` *executable* on PATH, which the libpcap step needs
    because it builds C with `CC="zig cc -target …"`. The pip package exposes
    `python -m ziglang` and no `zig` binary, so nothing else provides it.
  * pip's `ziglang` is what cargo-zigbuild uses to link the Rust side, and its
    open-bounded requirement (`ziglang>=0.9.0`) is what had drifted to 0.16 and
    quietly turned both Linux builds dynamic.

So both are required and both are now pinned to 0.13.0. The install step prints
each version separately, because a mismatch between them is exactly the kind of
thing that otherwise surfaces three steps later as an unrelated-looking C
compiler error.

Claude-Session: https://claude.ai/code/session_01GxGWpo3L4BRMkwtoa9X54W
@vaderyang
vaderyang merged commit a31dfec into main Sep 12, 2026
5 of 9 checks passed
@vaderyang
vaderyang deleted the fix/zigbuild-aarch64 branch September 12, 2026 11:38

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have what I need. Writing the review.

Summary

This PR fixes the release pipeline: bumps cargo-zigbuild 0.19.6 → 0.23.4 (the aarch64-musl leg was failing on rustc's --fix-cortex-a53-843419), pins ziglang==0.13.0 (which pip had silently dragged to 0.16, turning the Linux builds dynamic), and renumbers the version to 0.8.1 across VERSION / Cargo.toml / package.json / CHANGELOG.md since v0.8.0 never produced a release. The diagnosis is sound and the comments explain the two-zigs/two-consumers split well; the version and changelog edits are self-consistent and I verified the release-note extraction still yields a full 92-line section for [0.8.1]. The one thing worth flagging before merge is that the version bump is one file short: server/Cargo.lock still records all 15 workspace crates at 0.8.0, so a PR whose whole point is a version renumber leaves the lockfile stale. Recommendation: APPROVE — the lockfile drift is cosmetic and no build path uses --locked (I checked), so it can't fail a release, but it should be a follow-up.

Suggestions

  • server/Cargo.lock:1363–1693 — 15 workspace members (h-api, h-capture, h-common, h-ebpf-common, h-export, h-llm, h-metrics, h-pcap-extract, h-protocol, h-storage, h-storage-aglake, h-storage-clickhouse, h-storage-duckdb, h-turn, heron) are still pinned at version = "0.8.0" while server/Cargo.toml:11 now says 0.8.1. Both the linux and macos release jobs (release.yml:191, release.yml:268) run plain cargo zigbuild / cargo build with no --locked, so the lockfile is auto-rewritten in CI and nothing breaks — but the committed file no longer agrees with the manifest, and a local cargo build --locked will fail for anyone who tries it. Regenerate with a plain cargo check -p heron and commit.
  • scripts/routers/shared/bump.sh:63–76check_drift() only compares VERSION, server/Cargo.toml, and console/package.json; Cargo.lock is invisible to it. That is why this drift slipped through: just bump check reports "✓ in sync" while the lockfile is stale. Adding the workspace members' lockfile entries to the drift check (or running cargo update --workspace --offline at the end of a bump) would close the class rather than this instance. Non-blocking, and arguably out of scope for a release-fix PR.
  • CHANGELOG.md:9 — the [0.8.1] header date is 2026-09-12, identical to the [0.8.0] date it replaces, which is correct here (same-day re-cut) but means the two entries are indistinguishable by date if a [0.8.0] section is ever restored. The prose paragraph directly below already disambiguates, so this is fine as-is — noting it only so a future reader doesn't "fix" the date.
  • .github/workflows/release.yml:143pip install cargo-zigbuild==0.23.4 ziglang==0.13.0 pins both, which is right, but setup-zig@v2 at release.yml:122 is still itself unpinned to a major (the version it installs is, though). If the action's version: 0.13.0 input ever changes its accepted format, the failure mode is the libpcap configure error the commit message describes. Low risk; the two echo lines you added make the mismatch immediately visible, which is the right mitigation.

Questions

  • The commit chain is four commits that walk a fix forward and then partly back it out: 3b46b2ea removes setup-zig on the reasoning that pip's ziglang wins on PATH, and cfaec6ff restores it because libpcap's CC="zig cc -target …" needs a real zig executable. That is a good story, but it means the intermediate commits on this branch are each independently broken. If this branch is ever bisected or the commits are replayed individually, two of the four fail. Was squashing considered — or is the step-by-step history intentional as documentation of the diagnosis?
  • PATCH 3/4's message claims the zig on PATH from pip "lands ahead of whatever setup-zig provided," and PATCH 4/4's message contradicts it by saying the pip package "exposes python -m ziglang and no zig binary." The final comment block (release.yml:112–121) states the correct version. Flagging only because the two messages are in the same diff and a future reader consulting git log will hit the wrong one first — worth knowing the third message was superseded rather than trusting it.

Verified

  • Release-note extraction still works. Traced the awk at release.yml:328–332 against the renamed header: ^## \[0.8.1\] matches CHANGELOG.md:9 (regex is right-unanchored, so the trailing em-dash + date are harmless), flag is set and the header printed, output runs to line 100 and exits at the ## [0.7.3] header on line 101 — ~92 lines covering ### Changed / ### Added / ### Fixed. [ -s release-notes.md ] passes; no other line in the file matches ## [0.8.1].
  • Version SSOT chain intact. VERSION = 0.8.1; server/app/heron/src/main.rs:69 uses clap's version attribute and main.rs:355/701/920 use env!("CARGO_PKG_VERSION"), so the binary reports 0.8.1 from server/Cargo.toml:11; console/vite.config.ts:7,12 reads ../VERSION directly, so the console version does not depend on package.json at all — the package.json bump is bookkeeping consistent with the bump script's declared derived files.
  • No --locked anywhere in the release path. Grepped release.yml and ci.yml: the only --locked is cargo install bpf-linker --locked (ci.yml:251), unrelated to the workspace lockfile. This is what makes the Cargo.lock drift cosmetic rather than a release-breaker.
  • Test compatibility. server/app/heron/tests/cli_smoke.rs:13-25 asserts heron --version contains env!("CARGO_PKG_VERSION") — i.e. it reads the same manifest value being bumped, so it tracks the bump rather than pinning a literal. No test hardcodes 0.8.0.
  • CHANGELOG gap is intentional and explained. Confirmed no [0.8.0] section remains (renamed, not duplicated, per the commit message) and the [0.8.1] preamble at CHANGELOG.md:11–14 explains the missing release to anyone reading the version sequence.
  • Leakage check. Read the full release.yml and the CHANGELOG diff: no private IPs, credentials, private keys, internal hostnames, or machine-specific absolute paths. Version pins, public GitHub URLs (tcpdump.org, cargo-zigbuild issue refs), and CI runner paths only.
  • Scope. Changes are confined to .github/workflows/release.yml, CHANGELOG.md, VERSION, console/package.json, server/Cargo.toml — no Rust logic, no console source, no SQL, no capture pipeline. The schema-drift / prompt-cache-key / body-scan / window-width classes from the review checklist are not reachable from this diff.

🤖 Reviewed by the review botworkflow run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant