From fd9fe06bfb5f266662e9a4b84318f43593ee4726 Mon Sep 17 00:00:00 2001 From: Paul Clark Date: Sat, 5 Sep 2026 08:55:24 -0400 Subject: [PATCH] fix: root-cause the macOS reproducibility false-MISMATCH (TMPDIR symlink aliasing) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Controlled experiment: same clone, same SDK 10.0.400, clean obj — packing with cwd inside the clone matched the published DLL byte-for-byte; the script's invocation mismatched by exactly 476 bytes, all in the PE timestamp / MVID / debug-stamp regions (hash-derived, code identical). Diffing the real csc args exposed the cause: on macOS, mktemp -d returns /var/folders/... whose /var is a symlink to /private/var. Built through the unresolved spelling, SourceLink's computed source root (/var/...) disagreed with the compiler's canonical source paths (/private/var/...), so the /_/ path map never applied, absolute paths leaked into the deterministic input hash, and a byte-perfect package was reported MISMATCH. The earlier external macOS-reproducibility report had the same cause. Fix: canonicalize the work dir (pwd -P). Verified: core AND Hybrid v1.7.1 now verify MATCH on macOS arm64 — so the published packages are proven byte-identical across Linux x64 (release CI), Linux arm64, and macOS arm64, and REPRODUCIBLE-BUILDS.md replaces the retracted cross-OS claim with the proven one plus the verify-from-a-physical-path guidance. Also corrected: AUDIT-SCOPE.md pinned the v1.7.1 annotated-tag OBJECT hash (e0d332a) as the commit; the actual commit is f062c10. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01Sym7RJ7ehNhbMXytE5rMmS --- .github/scripts/verify-reproducibility.sh | 7 ++++++- CHANGELOG.md | 10 ++++++++++ docs/AUDIT-SCOPE.md | 4 ++-- docs/REPRODUCIBLE-BUILDS.md | 24 +++++++++++------------ 4 files changed, 30 insertions(+), 15 deletions(-) diff --git a/.github/scripts/verify-reproducibility.sh b/.github/scripts/verify-reproducibility.sh index 5cbadb5..9110283 100644 --- a/.github/scripts/verify-reproducibility.sh +++ b/.github/scripts/verify-reproducibility.sh @@ -21,7 +21,12 @@ if [[ -z "$TAG" || -z "$PKG" ]]; then fi VERSION="${TAG#v}" -WORK="$(mktemp -d)" +# Canonicalize (pwd -P): on macOS mktemp returns a /var/folders/... path, but /var is a +# symlink to /private/var — and building through the unresolved spelling makes SourceLink's +# computed source root disagree with the compiler's canonical source paths, so the /_/ path +# map never applies, absolute paths leak into the deterministic input hash, and verification +# reports a false MISMATCH for a perfectly reproducible package. +WORK="$(cd "$(mktemp -d)" && pwd -P)" trap 'rm -rf "$WORK"' EXIT echo "==> Verifying reproducibility for $PKG $VERSION (tag $TAG)" diff --git a/CHANGELOG.md b/CHANGELOG.md index f13dac1..0481e4c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -105,6 +105,16 @@ independently re-traced before fixing; none affects any on-disk byte or reader a trace on a locked/unreadable file; the pqfe-web page no longer hangs at "Encrypting…" forever if the worker crashes mid-operation; the docs-consistency link checker handles markdown link titles and URL-encoded targets. +- **The reproducibility verifier's macOS false-MISMATCH, root-caused and fixed.** On macOS, + `mktemp -d` returns a `/var/folders/…` path whose `/var` prefix is a symlink to + `/private/var`; building the cloned source through the unresolved spelling made + SourceLink's source root disagree with the compiler's canonical paths, so the `/_/` path + map never applied and the deterministic input hash shifted — reporting a byte-perfect + package as MISMATCH (the earlier external macOS-reproducibility report had the same + cause). The script now canonicalizes its work directory (`pwd -P`); `v1.7.1` is verified + byte-identical on Linux **and macOS arm64**, and REPRODUCIBLE-BUILDS.md now states the + proven cross-OS claim. (Also corrected: AUDIT-SCOPE.md pinned the v1.7.1 annotated-tag + object hash as the commit; it now names the actual commit `f062c10`.) ### Changed (behavior) diff --git a/docs/AUDIT-SCOPE.md b/docs/AUDIT-SCOPE.md index 526d12f..e8f3f25 100644 --- a/docs/AUDIT-SCOPE.md +++ b/docs/AUDIT-SCOPE.md @@ -11,7 +11,7 @@ restating. | | | | --- | --- | | Repository | `https://github.com/systemslibrarian/postquantum-file-encryption` | -| Audit revision | latest release tag — **`v1.7.1`**, commit **`e0d332a`** | +| Audit revision | latest release tag — **`v1.7.1`**, commit **`f062c10`** | | On-disk formats under review | `.pqfe` **v2** container, `.sig` **v1** sidecar, `PQKF` **v1** key file — all **frozen** for the entire `1.x` line | Pin the engagement to the tag, not to `main`: @@ -19,7 +19,7 @@ Pin the engagement to the tag, not to `main`: ```bash git clone https://github.com/systemslibrarian/postquantum-file-encryption cd postquantum-file-encryption -git checkout v1.7.1 # commit e0d332a +git checkout v1.7.1 # commit f062c10 # Release provenance: verify the tag's published artifacts against their build-provenance # attestations instead — see docs/SUPPLY-CHAIN.md ("gh attestation verify"). ``` diff --git a/docs/REPRODUCIBLE-BUILDS.md b/docs/REPRODUCIBLE-BUILDS.md index 86a969a..629e251 100644 --- a/docs/REPRODUCIBLE-BUILDS.md +++ b/docs/REPRODUCIBLE-BUILDS.md @@ -53,18 +53,18 @@ What does **not** affect reproducibility (with the above in place): - Local NuGet caches. The compiler reads from `obj/`, which is regenerated from the same inputs. -What **should not** affect it, but currently does: - -- **The operating system of the verifier.** In principle `.gitattributes`-normalised sources - plus `Deterministic=true` yield byte-identical assemblies on any OS. In practice, a macOS - arm64 rebuild of `v1.7.1` produced managed DLLs that differ from the Linux-built published - package — observed with this repository's own verifier, and independently reported. The - release-time verification on Linux passed for the same tag, so this is platform/toolchain - drift, not evidence of tampering — but it means **Linux (the CI environment) is the - demonstrated reproducibility envelope today**. Verify on Linux, or when investigating a - mismatch elsewhere, first match the exact SDK (`dotnet --version`) and OS of the release - build before suspecting the artifact. Root-causing (and either fixing or permanently - scoping) the cross-OS difference is tracked work. +- **The operating system of the verifier — proven, not just principled.** For `v1.7.1` the + rebuild is byte-identical on Linux (x64 release CI and arm64) **and macOS arm64** with SDK + `10.0.400`. An earlier macOS "mismatch" — reported externally and initially reproduced + here — turned out to be a bug in this repository's own verification script, not in the + artifacts: on macOS `mktemp -d` returns a `/var/folders/…` path, but `/var` is a symlink + to `/private/var`, and building through the unresolved spelling made SourceLink's computed + source root disagree with the compiler's canonical source paths — the `/_/` path map never + applied, absolute build paths leaked into the deterministic input hash, and the diff + flagged a perfectly reproducible package as MISMATCH. The script now canonicalizes its + work directory (`pwd -P`). If you verify by hand, build from a symlink-free (physical) + path — a mismatch that appears only under an aliased path is the verifier's environment, + not the artifact. What **does** matter: