From 6d48f2948cd1400f99230fa768cb7d31b03a71c3 Mon Sep 17 00:00:00 2001 From: Alexander Date: Mon, 27 Jul 2026 09:02:55 -0400 Subject: [PATCH 1/3] fix(releases): sync install.sh with hardened bootstrap + serve manifest bundles MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two live symptoms of one cause — this repo's serving layer is behind the hal0 release contract. 1. public/install.sh was 95 lines behind hal0:installer/bootstrap.sh. Provenance, since this is the second sync attempt: #44 (e1cef95, parent of this commit) was CORRECT when it landed. Its 267 bytes-for-bytes match hal0's installer/bootstrap.sh at c0d5538b (2026-07-11) exactly — verified with cmp. hal0 then re-hardened the installer on 2026-07-22 across six commits (c3466c98, 8d1afefd, 65611a69, 3f7145e0, c41db8e5, ba644c68, +208/-113), which is the drift the daily parity job now reports. This is staleness, not a deliberately reduced installer. What the 2026-07-22 hardening reinstates, all of it absent from the currently-served copy: - cosign is a REQUIRED bootstrap dependency again. The 2026-07-04 commit (3ef73d59) had made it optional because cosign isn't in apt; HAL0_INSTALL_REQUIRE_COSIGN is now gone entirely and a missing cosign is fatal. - The channel manifest is authenticated before it is parsed: cosign verify-blob against a sibling .bundle with a client-pinned, per-channel release-workflow OIDC identity. - A fail-closed jq policy pass over the authenticated bytes (_schema/channel/release_kind/prerelease_stage/version consistency) and exact signer_identity equality against a client-derived value. - jq becomes a new hard dependency; ambiguous manifest input is rejected; workdir cleanup is trap-guarded. - Default manifest URL moves from GitHub /latest/download/.json to https://releases.hal0.dev/.json — which is precisely why item 2 below is a launch blocker and not a nicety. Replaced wholesale with the canonical bytes. hal0's `Bootstrap parity (daily)` workflow diffs the two with no normalisation, so this is exit 0. 2. functions/_middleware.ts only proxied `{stable,nightly,dev}.json`. It knew nothing about `preview` (a real channel per hal0's src/hal0/release/policy.py manifest_targets) and nothing about the sibling `.json.bundle` that release.yml uploads alongside every manifest. Hardened clients require BOTH halves, so no external pointer was operational: stable.json 200, everything else 404. Route `/{stable,preview,nightly}.json` and `/….json.bundle`, resolving both from ONE release selection driven by the manifest asset — never by the bundle — so a manifest request and a bundle request land on the same release even if a tag publishes between a client's two fetches. If the selected release carries a manifest but not its bundle (broken publish), fail closed with `no-sibling::` instead of walking back to an older release and handing out a mismatched pair. Bundles get no static backstop: a placeholder signature reads as tampering, not as not-yet-published. Also bump the releases-list window 10 → 50. Nightlies publish daily and carry only nightly.json, so at per_page=10 the newest release carrying stable.json scrolled out of the window in about a week and the stable pointer silently degraded to the static placeholder. `dev.json` stays routed as a legacy alias with byte-for-byte identical behaviour (no release publishes it → annotated fallthrough). README: the release-hosting section still described the retired sig_url/cert_url verification path and omitted preview and bundles entirely. Rewritten around the bundle contract, plus the channel matrix, the both-halves-must-be-200 probe, the install.sh mirror rule, and the two reasons a channel can be non-operational upstream. Co-Authored-By: Claude Opus 5 --- README.md | 157 +++++++++++++++---- functions/_middleware.ts | 77 ++++++++-- public/install.sh | 321 +++++++++++++++++++++++++-------------- 3 files changed, 401 insertions(+), 154 deletions(-) diff --git a/README.md b/README.md index 3243e5c..c9bf093 100644 --- a/README.md +++ b/README.md @@ -115,43 +115,75 @@ vercel --prod # ad-hoc prod ship (CI handles the normal path) ## Release manifest hosting (`releases.hal0.dev`) -The hal0 self-updater (`src/hal0/updater/updater.py`) fetches per-channel -release manifests from: +The hal0 self-updater (`src/hal0/updater/updater.py`) and the one-line +installer (`installer/bootstrap.sh`) fetch per-channel release manifests — +**and the sibling Sigstore bundle that authenticates each one** — from: ``` -https://releases.hal0.dev/{stable|nightly}.json +https://releases.hal0.dev/{stable|preview|nightly}.json +https://releases.hal0.dev/{stable|preview|nightly}.json.bundle ``` +Both halves of the pair are mandatory. Hardened clients cosign-verify the +exact manifest bytes against the bundle, using a client-pinned release +workflow OIDC identity, *before* parsing a single artifact URL out of the +manifest. A manifest served without its bundle is not consumable — the +client refuses it rather than trusting unauthenticated URLs. + Schema: `hal0.releases.v1` — see -[`hal0/docs/release-manifest.md`](https://github.com/hal0ai/hal0/blob/main/docs/release-manifest.md) -for the full field reference. The schema's `cert_url` field is -**required** — cosign 3.x keyless `verify-blob` needs `--certificate`, -so the manifest carries the URL to the `.crt` artifact alongside -`url` (tarball) and `sig_url` (signature). +[`hal0/docs/internal/release-manifest.md`](https://github.com/hal0ai/hal0/blob/main/docs/internal/release-manifest.md) +for the full field reference. The trust-carrying fields are `bundle_url` +(Sigstore bundle for the tarball), `digest_sha256`, `signer_identity`, and +`signer_issuer`. `sig_url`/`cert_url` are still emitted but are no longer +the verification path: the bundle embeds the Fulcio certificate, the +signature, and the Rekor inclusion proof + SET, so `cosign verify-blob +--bundle` keeps working after the short-lived cert expires. + +### Channels + +Which manifests a tag publishes is decided by hal0's +`src/hal0/release/policy.py` (`manifest_targets`), not by this repo: + +| tag | manifests published | +|---|---| +| `v1.2.3` (final) | `stable.json`, `preview.json` | +| `v1.2.3-alpha.2` / `-beta.N` / `-rc.N` | `preview.json` | +| `v1.2.3-nightly.YYYYMMDD` | `nightly.json` | + +So `preview` tracks the newest of (latest prerelease, latest final) — a +final tag lands on both channels — and `stable` only ever moves on a final +tag. `release.yml` uploads `.json` **and** `.json.bundle` +as a pair for every target. -### How it works (as of v0.1.0-alpha.1, 2026-05-21) +### How it works -`releases.hal0.dev` is **fully operational** and serves the live -manifest. The subdomain lives on a small Cloudflare Pages project -whose middleware proxies the canonical asset off the latest GitHub -Release on `hal0ai/hal0`: +The subdomain lives on a small Cloudflare Pages project whose middleware +(`functions/_middleware.ts`) proxies the canonical assets off the newest +GitHub Release on `hal0ai/hal0` that carries them: 1. Tag `vX.Y.Z` on `hal0ai/hal0` triggers `.github/workflows/release.yml`. -2. The workflow builds `hal0-X.Y.Z.tar.gz`, computes its sha256, signs - it with cosign keyless against the GH Actions OIDC identity, and - uploads tarball + `.sig` + `.crt` to the GH Release alongside the - generated `stable.json` (manifest schema `hal0.releases.v1`). -3. The CF Pages middleware on `releases.hal0.dev` fetches - `stable.json` from the latest GH Release and serves it with a short - cache (~60s). A `v*` tag propagates end-to-end within about a - minute — **no hal0-web deploy required**. -4. Updater clients verify the tarball with `cosign verify-blob - --certificate --signature …` against the - `signer_identity` regex in the manifest. - -Updater clients should point at `https://releases.hal0.dev/stable.json` -(or `…/nightly.json`). The old GitHub-direct URL form is not used; the -canonical asset name is `stable.json`, not `latest.json`. +2. The workflow builds `hal0-X.Y.Z.tar.gz`, computes its sha256, cosign + keyless-signs it against the GH Actions OIDC identity, generates each + target channel manifest, signs each manifest into a sibling `.bundle`, + self-verifies both, and uploads the lot as Release assets. +3. The middleware on `releases.hal0.dev` resolves + `/.json[.bundle]` from the newest non-draft release carrying + `.json` and serves it with a short cache (~60s). A `v*` tag + propagates end-to-end within about a minute — **no hal0-web deploy + required**. +4. Clients verify the manifest against its bundle, then verify the tarball + digest and publisher signature again as defence in depth. + +Release selection is always driven by the **manifest** asset, never the +bundle, so a `.json` and a `.json.bundle` request resolve to the same +release even if a new tag lands between a client's two fetches. If the +selected release carries the manifest but not its bundle, the bundle +request fails closed (`x-hal0-proxy-failed: no-sibling:…`) rather than +pairing a manifest with an older release's signature. + +Response headers worth probing: `x-hal0-source: github-release/`, +`x-hal0-channel`, `x-hal0-artifact: manifest|bundle`, and on any +fallthrough `x-hal0-proxy-failed: `. ### Static fallback in this repo @@ -166,11 +198,76 @@ land at `https://hal0.dev/releases/{stable,nightly}.json`. They're kept as a backstop and as a schema example; the live manifest is whatever the CF Pages middleware on `releases.hal0.dev` returns. +There is deliberately **no static backstop for `.bundle`** — a placeholder +signature cannot verify, and a client that fetched one would read the +failure as tampering rather than as "not published yet". A bundle that +can't be proxied is an annotated 404. + ### Verify ```sh -curl -s https://releases.hal0.dev/stable.json | jq . -curl -sI https://releases.hal0.dev/stable.json | grep -i cache-control +# manifest + its bundle must BOTH be 200 for a channel to be operational +for c in stable preview nightly; do + for f in "$c.json" "$c.json.bundle"; do + printf '%-24s %s\n' "$f" \ + "$(curl -so /dev/null -w '%{http_code}' "https://releases.hal0.dev/$f")" + done +done + +curl -sI https://releases.hal0.dev/stable.json | grep -iE 'cache-control|x-hal0' + +# end-to-end: authenticate the manifest exactly as a client does +curl -fsSLO https://releases.hal0.dev/preview.json +curl -fsSLO https://releases.hal0.dev/preview.json.bundle +cosign verify-blob --bundle preview.json.bundle \ + --certificate-identity-regexp '^https://github\.com/(Hal0ai|hal0ai)/hal0/\.github/workflows/release\.yml@refs/tags/v\d+\.\d+\.\d+(-(alpha|beta|rc)\.(0|[1-9]\d*))?$' \ + --certificate-oidc-issuer https://token.actions.githubusercontent.com \ + preview.json +``` + +(That identity regex is the `preview` admission pattern from +`installer/bootstrap.sh`; `stable` and `nightly` pin narrower ones. Keep the +copies in lockstep with bootstrap.sh and `updater.py` — they are the trust +root, not documentation.) + +A channel only becomes operational once a tag has published *both* halves, +in the current manifest schema. Two things can hold a channel back, and +neither is fixable in this repo: + +- **No bundle.** Releases cut before manifest-bundle signing carry + `.json` with no `.bundle`. The serving layer cannot synthesise a + missing signature, so `.bundle` stays 404 until the next tag. +- **Pre-hardening manifest fields.** The client's strict policy pass + requires `release_kind` and `prerelease_stage`, and requires + `signer_identity` to equal the exact per-release identity it derives + itself. Manifests generated before those fields existed are rejected even + when perfectly signed. + +As of 2026-07-27 that means `stable` is **not** operational end-to-end: the +newest release carrying `stable.json` is `v0.9.8`, which has no +`stable.json.bundle` and whose manifest predates `release_kind`. `preview` +(from `v1.0.0-alpha.2`) satisfies both conditions. Both clear on the next +final tag cut by the current `release.yml`. + +## One-line installer (`hal0.dev/install.sh`) + +`public/install.sh` is a **byte-identical mirror** of +[`hal0:installer/bootstrap.sh`](https://github.com/hal0ai/hal0/blob/main/installer/bootstrap.sh) +— that file is the audited original and the trust boundary for +`curl https://hal0.dev/install.sh | bash`. Never hand-edit the copy here: +copy the canonical file over it wholesale, in the same PR as the hal0-side +change. + +hal0's `Bootstrap parity (daily)` workflow +(`.github/workflows/bootstrap-parity.yml` → +`scripts/check-bootstrap-parity.sh`) fetches the live URL daily and does a +plain `diff -u` against the in-tree original: exit 0 in sync, 1 drift, +2 operational error. There is no normalisation, so a single byte of drift +fails it. Preview a sync locally from a hal0 checkout with: + +```sh +HAL0_INSTALL_URL="file:///path/to/hal0-web/public/install.sh" \ + bash scripts/check-bootstrap-parity.sh ``` ## Build state diff --git a/functions/_middleware.ts b/functions/_middleware.ts index 1fcd0b1..9d14693 100644 --- a/functions/_middleware.ts +++ b/functions/_middleware.ts @@ -2,13 +2,28 @@ // // Two responsibilities on `releases.hal0.dev`: // -// 1. `releases.hal0.dev/{stable,nightly,dev}.json` is proxied LIVE from -// the most recent GitHub release on `Hal0ai/hal0` that carries an -// asset of that name. Self-syncing: each tagged release on hal0 -// becomes visible at releases.hal0.dev without a hal0-web deploy. +// 1. `releases.hal0.dev/{stable,preview,nightly}.json` — and the sibling +// `.json.bundle` Sigstore bundle for each — are proxied LIVE from the +// most recent GitHub release on `Hal0ai/hal0` that carries the channel +// manifest. Self-syncing: each tagged release on hal0 becomes visible +// at releases.hal0.dev without a hal0-web deploy. // Static `public/releases/*.json` stays as a placeholder backstop // in case the GitHub API is unreachable. // +// Channels mirror hal0's `src/hal0/release/policy.py` manifest targets: +// final tag v1.2.3 → stable.json + preview.json +// preview v1.2.3-alpha.2 → preview.json +// nightly v1.2.3-nightly.20260727 → nightly.json +// release.yml uploads `.json` AND `.json.bundle` as a +// pair for every target, so both are always resolvable from the SAME +// release. Hardened clients (installer/bootstrap.sh `verify_release_manifest`, +// src/hal0/updater/updater.py) require BOTH: they cosign-verify the exact +// manifest bytes against the sibling bundle before parsing any artifact +// URL. Serving the manifest without its bundle leaves the channel +// non-operational, so the two are resolved from one release selection +// (see `proxyChannelArtifact`) — never independently, which could pair a +// manifest from release N with a bundle from release N-1 mid-publish. +// // 2. Anything else on `releases.hal0.dev` (e.g. `/foo`) is rewritten // to `/releases/foo` so the static files under `public/releases/` // are reachable without polluting the marketing-site root. @@ -27,8 +42,17 @@ // limit is permanently exhausted. Authenticated requests get 5000/hr // per token. The token only needs public-repo read scope. -const CHANNEL_RE = /^\/(stable|nightly|dev)\.json$/; -const RELEASES_API = "https://api.github.com/repos/Hal0ai/hal0/releases?per_page=10"; +// Capture 1 = channel, capture 2 = ".bundle" when the sibling Sigstore bundle +// is being requested. `dev` is a legacy alias kept so an old client asking for +// it behaves exactly as before (no release publishes dev.json, so it falls +// through to the static backstop); the live channels are stable/preview/nightly. +const CHANNEL_RE = /^\/(stable|preview|nightly|dev)\.json(\.bundle)?$/; +// per_page must comfortably out-span the nightly cadence: nightlies publish +// daily and carry only `nightly.json`, so at per_page=10 the newest release +// still carrying `stable.json` scrolls out of the window in ~a week and the +// stable pointer silently degrades to the static placeholder. 50 keeps a +// stable/preview tag reachable through a long nightly streak. +const RELEASES_API = "https://api.github.com/repos/Hal0ai/hal0/releases?per_page=50"; function authHeaders(token: string | undefined): Record { const base: Record = { @@ -55,10 +79,23 @@ type ProxyOutcome = | { ok: true; response: Response } | { ok: false; reason: string }; -async function proxyChannelManifest( +// Resolve `.json` or its sibling `.json.bundle` from the most +// recent non-draft release that carries the channel manifest. +// +// Release selection is driven by the MANIFEST asset in both cases, never by the +// bundle. That keeps a bundle request pinned to the same release a manifest +// request would resolve to, so `cosign verify-blob --bundle` sees a matching +// pair even if a new release lands between a client's two fetches. If the +// selected release carries the manifest but not its bundle (a broken publish), +// we fail loudly rather than walking back to an older release and handing out a +// mismatched pair. +async function proxyChannelArtifact( channel: string, + wantBundle: boolean, token: string | undefined, ): Promise { + const manifestName = `${channel}.json`; + const assetName = wantBundle ? `${manifestName}.bundle` : manifestName; let listResp: Response; try { listResp = await fetch(RELEASES_API, { @@ -89,8 +126,15 @@ async function proxyChannelManifest( for (const release of releases) { if (release.draft) continue; - const asset = release.assets?.find((a) => a.name === `${channel}.json`); - if (!asset) continue; + // The manifest decides which release serves this channel. + if (!release.assets?.some((a) => a.name === manifestName)) continue; + + const asset = release.assets?.find((a) => a.name === assetName); + if (!asset) { + const reason = `no-sibling:${assetName}:${release.tag_name}`; + console.warn(`[releases-proxy] ${reason}`); + return { ok: false, reason }; + } // Use the api.github.com asset endpoint with octet-stream Accept. // Returns a 302 to objects.githubusercontent.com with the asset @@ -123,17 +167,21 @@ async function proxyChannelManifest( response: new Response(body, { status: 200, headers: { + // Sigstore bundles are JSON too (`cosign sign-blob --bundle` + // writes a JSON-serialized protobuf bundle), so one + // content-type covers manifest and bundle alike. "content-type": "application/json; charset=utf-8", "access-control-allow-origin": "*", "cache-control": "public, max-age=60, must-revalidate", "x-content-type-options": "nosniff", "x-hal0-source": `github-release/${release.tag_name}`, "x-hal0-channel": channel, + "x-hal0-artifact": wantBundle ? "bundle" : "manifest", }, }), }; } - const reason = `no-asset:${channel}.json:${releases.length}releases`; + const reason = `no-asset:${manifestName}:${releases.length}releases`; console.warn(`[releases-proxy] ${reason}`); return { ok: false, reason }; } @@ -172,9 +220,16 @@ export const onRequest: PagesFunction = async (context) => { if (url.hostname === "releases.hal0.dev") { const channelMatch = url.pathname.match(CHANNEL_RE); if (channelMatch) { - const outcome = await proxyChannelManifest(channelMatch[1], context.env.GITHUB_TOKEN); + const outcome = await proxyChannelArtifact( + channelMatch[1], + channelMatch[2] === ".bundle", + context.env.GITHUB_TOKEN, + ); if (outcome.ok) return outcome.response; // Fall through to the static placeholder, but annotate why. + // Bundles have no static backstop (a placeholder signature would be + // worse than a 404 — it would fail cosign verification and read as + // tampering), so a bundle fallthrough is an annotated 404. const rewritten = new URL(context.request.url); rewritten.pathname = "/releases" + url.pathname; return annotateFallthrough(context, rewritten.toString(), outcome.reason); diff --git a/public/install.sh b/public/install.sh index 7c2fba9..29c299d 100755 --- a/public/install.sh +++ b/public/install.sh @@ -19,22 +19,17 @@ # # Env overrides: # HAL0_RELEASES_URL full URL to a hal0.releases.v1 manifest -# (default: GitHub Releases /latest/download/stable.json) -# HAL0_CHANNEL channel name when using the default URL (default: stable) -# HAL0_INSTALL_REQUIRE_COSIGN=1 -# fail the install if cosign is not present -# (restores the old hard requirement for -# security-conscious / enterprise installs) +# (default: https://releases.hal0.dev/.json) +# HAL0_CHANNEL stable, preview, or nightly (default: stable) # HAL0_BOOTSTRAP_KEEP_TMP=1 don't delete the work directory on exit # (debugging the unpacked tree) # -# This script is the trust boundary for the one-line install. The tarball's -# sha256 is ALWAYS checked against the manifest digest (fatal on mismatch) -# before anything is executed. cosign then verifies the publisher signature -# against the workflow OIDC identity — but cosign is no longer a hard -# install-time dependency: if the binary is absent, the install proceeds on -# the sha256 check alone with a loud warning (opt back into strict mode with -# HAL0_INSTALL_REQUIRE_COSIGN=1). See docs/internal/release-manifest.md. +# This script is the trust boundary for the one-line install. It first verifies +# the exact channel-manifest bytes with their sibling Sigstore bundle and a +# client-pinned release-workflow identity. Only then does it parse artifact +# URLs. The tarball's sha256 and publisher signature are both checked again as +# defense-in-depth before anything is executed. cosign is therefore a required +# bootstrap dependency. See docs/internal/release-manifest.md. # # Schema reference: docs/internal/release-manifest.md (hal0.releases.v1). @@ -42,7 +37,15 @@ set -euo pipefail IFS=$'\n\t' HAL0_CHANNEL="${HAL0_CHANNEL:-stable}" -HAL0_RELEASES_URL="${HAL0_RELEASES_URL:-https://github.com/Hal0ai/hal0/releases/latest/download/${HAL0_CHANNEL}.json}" +HAL0_RELEASES_URL="${HAL0_RELEASES_URL:-}" + +# Keep these trust roots in lockstep with src/hal0/updater/updater.py. The +# requested channel selects admission before any manifest JSON is parsed. +_MANIFEST_IDENTITY_PREFIX='^https://github\.com/(Hal0ai|hal0ai)/hal0/\.github/workflows/release\.yml@' +_STABLE_MANIFEST_ADMISSION_IDENTITY="${_MANIFEST_IDENTITY_PREFIX}refs/tags/v\\d+\\.\\d+\\.\\d+$" +_PREVIEW_MANIFEST_ADMISSION_IDENTITY="${_MANIFEST_IDENTITY_PREFIX}refs/tags/v\\d+\\.\\d+\\.\\d+(-(alpha|beta|rc)\\.(0|[1-9]\\d*))?$" +_NIGHTLY_MANIFEST_IDENTITY="${_MANIFEST_IDENTITY_PREFIX}refs/heads/main$" +_MANIFEST_SIGNER_ISSUER='https://token.actions.githubusercontent.com' # ── tiny output helpers ──────────────────────────────────────────────────── if [[ -t 1 && -z "${NO_COLOR:-}" ]]; then @@ -57,6 +60,13 @@ warn() { printf '%s! %s%s\n' "${_C_YEL}" "$*" "${_C_RST}" >&2; } err() { printf '%s✗ %s%s\n' "${_C_RED}" "$*" "${_C_RST}" >&2; } die() { err "$*"; exit 1; } +_BOOTSTRAP_WORK="" +cleanup_workdir() { + if [[ -n "${_BOOTSTRAP_WORK}" ]]; then + rm -rf -- "${_BOOTSTRAP_WORK}" + fi +} + banner() { printf '\n%shal0%s — open-source home AI inference platform\n' "${_C_BLD}" "${_C_RST}" printf '%s%s%s\n\n' "${_C_DIM}" "https://hal0.dev" "${_C_RST}" @@ -72,43 +82,154 @@ preflight() { need curl need tar need sha256sum + need jq need python3 } -# ── manifest fetch + parse ──────────────────────────────────────────────── +validate_channel() { + case "${HAL0_CHANNEL}" in + stable|preview|nightly) ;; + *) die "HAL0_CHANNEL must be one of: stable, preview, nightly (got ${HAL0_CHANNEL})" ;; + esac +} + +manifest_admission_identity() { + case "$1" in + stable) printf '%s\n' "${_STABLE_MANIFEST_ADMISSION_IDENTITY}" ;; + preview) printf '%s\n' "${_PREVIEW_MANIFEST_ADMISSION_IDENTITY}" ;; + nightly) printf '%s\n' "${_NIGHTLY_MANIFEST_IDENTITY}" ;; + *) return 1 ;; + esac +} + +exact_manifest_identity() { + local release_kind="$1" version="$2" escaped_version + case "${release_kind}" in + stable) + [[ "${version}" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]] || return 1 + ;; + preview) + [[ "${version}" =~ ^[0-9]+\.[0-9]+\.[0-9]+-(alpha|beta|rc)\.(0|[1-9][0-9]*)$ ]] \ + || return 1 + ;; + nightly) + [[ "${version}" =~ ^[0-9]+\.[0-9]+\.[0-9]+-nightly\.([0-9]{8}|[0-9]{14})$ ]] \ + || return 1 + printf '%s\n' "${_NIGHTLY_MANIFEST_IDENTITY}" + return 0 + ;; + *) return 1 ;; + esac + escaped_version="${version//./\\.}" + printf '%srefs/tags/v%s$\n' "${_MANIFEST_IDENTITY_PREFIX}" "${escaped_version}" +} + +resolve_release_manifest_url() { + if [[ -z "${HAL0_RELEASES_URL}" ]]; then + HAL0_RELEASES_URL="https://releases.hal0.dev/${HAL0_CHANNEL}.json" + fi +} + +release_manifest_bundle_url() { + python3 - "$1" <<'PY' +import sys +from urllib.parse import urlsplit, urlunsplit + +scheme, netloc, path, query, fragment = urlsplit(sys.argv[1]) +print(urlunsplit((scheme, netloc, f"{path}.bundle", query, fragment))) +PY +} + +# ── manifest fetch + authenticate + parse ───────────────────────────────── fetch_manifest() { local out="$1" info "fetching release manifest" info " ${_C_DIM}${HAL0_RELEASES_URL}${_C_RST}" - if ! curl -fsSL --retry 3 --retry-delay 2 -o "${out}" "${HAL0_RELEASES_URL}"; then + if ! curl -fsSL --retry 3 --retry-delay 2 -o "${out}" --url "${HAL0_RELEASES_URL}"; then die "could not download release manifest from ${HAL0_RELEASES_URL}" fi } -parse_manifest_field() { - local file="$1" field="$2" - python3 -c " -import json, sys -try: - v = json.load(open('${file}')).get('${field}') - if v is None: - sys.exit('manifest missing required field: ${field}') - print(v) -except json.JSONDecodeError as e: - sys.exit(f'manifest is not valid JSON: {e}') -" +verify_release_manifest() { + local manifest="$1" bundle="$2" identity="$3" + command -v cosign >/dev/null 2>&1 \ + || die "cosign is required to verify the release manifest but is not installed. + install it from https://docs.sigstore.dev/cosign/installation/" + + info "verifying release manifest with pinned workflow identity" + if ! cosign verify-blob \ + --bundle "${bundle}" \ + --certificate-identity-regexp "${identity}" \ + --certificate-oidc-issuer "${_MANIFEST_SIGNER_ISSUER}" \ + "${manifest}" >/dev/null 2>&1; then + die "release manifest signature verification FAILED — refusing to trust artifact URLs" + fi + ok "release manifest signature OK" } -# Like parse_manifest_field but prints nothing (rather than dying) when the -# field is absent — used for the transition-window bundle_url/sig_url/ -# cert_url fields, which are mutually optional (see cosign_verify below). -parse_manifest_field_optional() { +validate_manifest_for_channel() { + local manifest="$1" requested_channel="$2" normalized="$3" + + # This is deliberately one fail-closed jq policy pass over the exact bytes + # authenticated above. It emits a normalized manifest only when every + # bootstrap-required field and channel/kind/stage relationship is valid. + if ! jq -e -s \ + --arg requested "${requested_channel}" \ + --arg trusted_issuer "${_MANIFEST_SIGNER_ISSUER}" ' + def nonempty_string: type == "string" and length > 0; + select(length == 1) + | .[0] + | select( + type == "object" + and ._schema == "hal0.releases.v1" + and (.version | nonempty_string) + and (.url | nonempty_string) + and (.bundle_url | nonempty_string) + and (.signer_identity | nonempty_string) + and .signer_issuer == $trusted_issuer + and (try (.digest_sha256 | test("^(sha256:)?[0-9A-Fa-f]{64}$")) catch false) + and (.channel == "stable" or .channel == "preview" or .channel == "nightly") + and (.release_kind == "stable" or .release_kind == "preview" or .release_kind == "nightly") + and .channel == $requested + and ( + ($requested == "stable" and .release_kind == "stable") + or ($requested == "preview" and (.release_kind == "preview" or .release_kind == "stable")) + or ($requested == "nightly" and .release_kind == "nightly") + ) + and ( + (.release_kind == "preview" and ( + .prerelease_stage == "alpha" + or .prerelease_stage == "beta" + or .prerelease_stage == "rc" + )) + or ((.release_kind == "stable" or .release_kind == "nightly") and .prerelease_stage == null) + ) + and ( + (.release_kind == "stable" + and (try (.version | test("^[0-9]+\\.[0-9]+\\.[0-9]+$")) catch false)) + or (.release_kind == "preview" and ( + (.prerelease_stage == "alpha" and + (try (.version | test("^[0-9]+\\.[0-9]+\\.[0-9]+-alpha\\.(0|[1-9][0-9]*)$")) catch false)) + or (.prerelease_stage == "beta" and + (try (.version | test("^[0-9]+\\.[0-9]+\\.[0-9]+-beta\\.(0|[1-9][0-9]*)$")) catch false)) + or (.prerelease_stage == "rc" and + (try (.version | test("^[0-9]+\\.[0-9]+\\.[0-9]+-rc\\.(0|[1-9][0-9]*)$")) catch false)) + )) + or (.release_kind == "nightly" and + (try (.version | test("^[0-9]+\\.[0-9]+\\.[0-9]+-nightly\\.([0-9]{8}|[0-9]{14})$")) catch false)) + ) + ) + | .digest_sha256 |= (ascii_downcase | sub("^sha256:"; "")) + ' "${manifest}" >"${normalized}"; then + rm -f -- "${normalized}" + die "authenticated release manifest failed strict policy validation" + fi +} + +parse_manifest_field() { local file="$1" field="$2" - python3 -c " -import json -v = json.load(open('${file}')).get('${field}') -print(v if v is not None else '') -" + jq -er --arg field "${field}" '.[$field] | select(type == "string")' "${file}" \ + || die "validated release manifest field extraction failed: ${field}" } # ── tarball fetch + sha256 verify ───────────────────────────────────────── @@ -116,7 +237,7 @@ fetch_and_hash_check() { local url="$1" expected_digest="$2" out="$3" info "downloading tarball" info " ${_C_DIM}${url}${_C_RST}" - curl -fsSL --retry 3 --retry-delay 2 -o "${out}" "${url}" \ + curl -fsSL --retry 3 --retry-delay 2 -o "${out}" --url "${url}" \ || die "could not download tarball" info "verifying sha256" @@ -128,67 +249,32 @@ fetch_and_hash_check() { ok "sha256 OK (${actual:0:12}…)" } -# ── cosign verify (or documented skip) ──────────────────────────────────── +# ── tarball cosign verify (defense-in-depth) ─────────────────────────────── fetch_sidecar() { local label="$1" url="$2" out="$3" info "downloading ${label}" info " ${_C_DIM}${url}${_C_RST}" - curl -fsSL --retry 3 --retry-delay 2 -o "${out}" "${url}" \ + curl -fsSL --retry 3 --retry-delay 2 -o "${out}" --url "${url}" \ || die "could not download ${label}" } cosign_verify() { - # bundle may be empty — in that case sig and cert must both be set (the - # transition-window fallback for manifests without bundle_url). See main(). - local tarball="$1" bundle="$2" sig="$3" cert="$4" identity="$5" issuer="$6" + local tarball="$1" bundle="$2" identity="$3" - if ! command -v cosign >/dev/null 2>&1; then - if [[ "${HAL0_INSTALL_REQUIRE_COSIGN:-0}" == "1" ]]; then - die "cosign is required (HAL0_INSTALL_REQUIRE_COSIGN=1) but not installed. - install it from https://docs.sigstore.dev/cosign/installation/" - fi - # cosign is not a hard dependency: the tarball's sha256 was already - # verified against the manifest digest (fetch_and_hash_check, fatal on - # mismatch), so integrity relative to the manifest holds. What we lose - # by skipping is proof that the tarball was built by hal0's signing - # workflow rather than substituted upstream of the manifest — hence the - # loud warning. Install cosign, or set HAL0_INSTALL_REQUIRE_COSIGN=1, to - # keep that guarantee. - warn "cosign not installed — skipping publisher signature verification" - warn " the tarball sha256 was verified against the manifest, but its" - warn " cosign/OIDC signature was NOT checked." - warn " for full supply-chain verification install cosign:" - warn " ${_C_DIM}https://docs.sigstore.dev/cosign/installation/${_C_RST}" - return 0 - fi + command -v cosign >/dev/null 2>&1 \ + || die "cosign disappeared after release manifest verification — refusing to install" info "verifying signature with cosign keyless OIDC" info " identity-regex: ${_C_DIM}${identity}${_C_RST}" - info " issuer: ${_C_DIM}${issuer}${_C_RST}" - - local -a verify_args - if [[ -n "${bundle}" ]]; then - # Keyless verification uses a Sigstore bundle. The bundle carries the - # Fulcio cert, the signature, AND the Rekor Signed Entry Timestamp - # (SET) — the trusted timestamp that lets verify-blob succeed after - # the short-lived (~10 min) signing cert has expired, which is - # always the case by the time a user runs the installer. - # --certificate-identity-regexp is matched against the cert SAN - # carried in the bundle. (A detached .sig + .crt had no SET and - # failed on every client — #1159.) - verify_args=(--bundle "${bundle}") - else - # Transition-window fallback: manifest had no bundle_url (older - # release, or a manifest generated before #1159 shipped). This path - # inherits the known post-expiry failure the bundle fixes — kept - # only so manifests without bundle_url still attempt verification - # instead of erroring outright. - verify_args=(--signature "${sig}" --certificate "${cert}") - fi + info " issuer: ${_C_DIM}${_MANIFEST_SIGNER_ISSUER}${_C_RST}" + + # The authenticated manifest must provide a Sigstore bundle. Detached + # signature/certificate sidecars are not an accepted bootstrap scheme. + local -a verify_args=(--bundle "${bundle}") if ! cosign verify-blob \ "${verify_args[@]}" \ --certificate-identity-regexp "${identity}" \ - --certificate-oidc-issuer "${issuer}" \ + --certificate-oidc-issuer "${_MANIFEST_SIGNER_ISSUER}" \ "${tarball}" >/dev/null 2>&1; then die "cosign signature verification FAILED — refusing to install" fi @@ -197,56 +283,65 @@ cosign_verify() { # ── main ────────────────────────────────────────────────────────────────── main() { + validate_channel + local admission_identity + admission_identity="$(manifest_admission_identity "${HAL0_CHANNEL}")" \ + || die "could not derive manifest admission identity" banner preflight + resolve_release_manifest_url local work work="$(mktemp -d -t hal0-install-XXXXXX)" if [[ "${HAL0_BOOTSTRAP_KEEP_TMP:-0}" != "1" ]]; then - trap 'rm -rf "${work}"' EXIT + _BOOTSTRAP_WORK="${work}" + trap cleanup_workdir EXIT else warn "HAL0_BOOTSTRAP_KEEP_TMP=1 — leaving work dir ${work}" fi local manifest="${work}/manifest.json" + local manifest_bundle="${work}/manifest.json.bundle" fetch_manifest "${manifest}" + fetch_sidecar \ + "release manifest signature bundle" \ + "$(release_manifest_bundle_url "${HAL0_RELEASES_URL}")" \ + "${manifest_bundle}" + verify_release_manifest "${manifest}" "${manifest_bundle}" "${admission_identity}" - local version url bundle_url sig_url cert_url digest identity issuer - version="$(parse_manifest_field "${manifest}" version)" - url="$(parse_manifest_field "${manifest}" url)" - bundle_url="$(parse_manifest_field_optional "${manifest}" bundle_url)" - sig_url="$(parse_manifest_field_optional "${manifest}" sig_url)" - cert_url="$(parse_manifest_field_optional "${manifest}" cert_url)" - digest="$(parse_manifest_field "${manifest}" digest_sha256)" - identity="$(parse_manifest_field "${manifest}" signer_identity)" - issuer="$(parse_manifest_field "${manifest}" signer_issuer)" - - if [[ -z "${bundle_url}" && ( -z "${sig_url}" || -z "${cert_url}" ) ]]; then - die "manifest has no usable signing scheme (need bundle_url, or both sig_url and cert_url)" + local validated_manifest="${work}/manifest.validated.json" + validate_manifest_for_channel "${manifest}" "${HAL0_CHANNEL}" "${validated_manifest}" + + local version release_kind url bundle_url digest manifest_identity expected_identity + version="$(parse_manifest_field "${validated_manifest}" version)" + release_kind="$(parse_manifest_field "${validated_manifest}" release_kind)" + url="$(parse_manifest_field "${validated_manifest}" url)" + bundle_url="$(parse_manifest_field "${validated_manifest}" bundle_url)" + digest="$(parse_manifest_field "${validated_manifest}" digest_sha256)" + manifest_identity="$(parse_manifest_field "${validated_manifest}" signer_identity)" + expected_identity="$(exact_manifest_identity "${release_kind}" "${version}")" \ + || die "validated release manifest has unsupported release identity policy" + if [[ "${manifest_identity}" != "${expected_identity}" ]]; then + die "authenticated release manifest signer_identity does not match exact release identity" + fi + if [[ "${admission_identity}" != "${expected_identity}" ]]; then + verify_release_manifest "${manifest}" "${manifest_bundle}" "${expected_identity}" fi info "release: ${_C_BLD}hal0 v${version}${_C_RST} (${HAL0_CHANNEL})" - local tarball="${work}/hal0-${version}.tar.gz" + # Manifest strings never become shell syntax or path components. + local tarball="${work}/artifact.tar.gz" fetch_and_hash_check "${url}" "${digest}" "${tarball}" - # Prefer the Sigstore bundle (survives cert expiry, #1159); fall back to - # the transition-window sig_url/cert_url pair when bundle_url is absent. - local bundle="" sig="" cert="" - if [[ -n "${bundle_url}" ]]; then - bundle="${tarball}.bundle" - fetch_sidecar "signature bundle" "${bundle_url}" "${bundle}" - else - sig="${tarball}.sig" - cert="${tarball}.crt" - fetch_sidecar "signature" "${sig_url}" "${sig}" - fetch_sidecar "certificate" "${cert_url}" "${cert}" - fi - cosign_verify "${tarball}" "${bundle}" "${sig}" "${cert}" "${identity}" "${issuer}" + local bundle="${tarball}.bundle" + fetch_sidecar "signature bundle" "${bundle_url}" "${bundle}" + cosign_verify "${tarball}" "${bundle}" "${expected_identity}" info "extracting tarball" - tar -xzf "${tarball}" -C "${work}" - local unpacked="${work}/hal0-${version}" + local unpacked="${work}/unpacked" + mkdir "${unpacked}" + tar -xzf "${tarball}" --strip-components=1 -C "${unpacked}" [[ -x "${unpacked}/installer/install.sh" ]] \ || die "extracted tree is missing installer/install.sh — corrupt tarball?" From ea10d2d16d3ef358366559f8549877ea6095410f Mon Sep 17 00:00:00 2001 From: Alexander Date: Mon, 27 Jul 2026 09:33:59 -0400 Subject: [PATCH 2/3] test(releases): pin the middleware manifest/bundle pairing invariants MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit functions/_middleware.ts decides which GitHub Release a hardened client's manifest AND its signature bundle come from. If those two ever come from different releases, cosign verify-blob fails and the client reads a routing bug as tampering. That makes the same-release pairing rule and the fail-closed-on-missing-bundle rule security invariants, not style choices — and exactly the kind of thing a later well-meaning refactor drops silently. This repo had no test coverage at all, so nothing would have caught it. 15 cases across 5 groups, driving the real exported onRequest against a stubbed GitHub Releases API: pairing (manifest-driven selection, sibling bundle from the same release, fail-closed when the bundle is absent), channel semantics (preview from newest prerelease, preview falling back to a final tag, nightly, stable not served by a nightly, drafts skipped), response contract (content-type, CORS, cache, nosniff), degradation (annotated static fallthrough, rate-limit fallthrough, legacy dev alias) and host-conditional routing (rewrite, no double-prefix, other hosts untouched and no upstream call). Node's built-in runner and type stripping — no dev dependencies, no build step, no framework for 15 cases. Lives in test/ rather than functions/ because Cloudflare Pages deploys everything under functions/ as routes. Confirmed load-bearing by mutation: flipping the release-selection predicate from the manifest asset to the requested asset turns "selects the release by the MANIFEST asset, never by the bundle" red and npm test exits 1. Reverted after. Deliberately NOT wired into CI — hal0-web's only workflow is mirror-docs.yml and adding a job is a separate call. README: document npm test, and record that astro check's 5 errors (src/content.config.ts, src/pages/changelog.astro) are a pre-existing baseline so the next person doesn't think they caused them. Co-Authored-By: Claude Opus 5 --- README.md | 15 +- package.json | 1 + test/middleware.test.mjs | 311 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 326 insertions(+), 1 deletion(-) create mode 100644 test/middleware.test.mjs diff --git a/README.md b/README.md index c9bf093..5c7ac6f 100644 --- a/README.md +++ b/README.md @@ -98,11 +98,24 @@ See [`NOTES.md`](./NOTES.md) for the full rationale. ```sh npm install # install deps npm run dev # dev server at http://localhost:4321 -npm run astro check # type check (must be 0 errors / 0 warnings) +npm test # behavioural tests for functions/_middleware.ts +npm run astro check # type check npm run build # static site → ./dist/ npm run preview # serve ./dist/ for smoke-testing ``` +`npm test` uses Node's built-in runner and type stripping — no dev +dependencies, no build step. It covers `functions/_middleware.ts`, which +decides which GitHub Release a hardened client's manifest *and* its +signature bundle come from; the same-release pairing rule it pins is a +security invariant, not a style preference. See +[Release manifest hosting](#release-manifest-hosting-releaseshal0dev). + +`astro check` currently reports **5 pre-existing errors** (2 in +`src/content.config.ts` from starlight-blog's Zod `SchemaContext`, 3 implicit +`any` in `src/pages/changelog.astro`). They are unrelated to the middleware; +treat 5 as the baseline and anything above it as yours. + ## Deploy The marketing site + docs ship to **Vercel** on every push to `master`. diff --git a/package.json b/package.json index 3b2c07b..3addc5f 100644 --- a/package.json +++ b/package.json @@ -8,6 +8,7 @@ "prebuild": "node scripts/sync-changelog.mjs", "build": "astro build", "preview": "astro preview", + "test": "node --test \"test/**/*.test.mjs\"", "astro": "astro", "sync:changelog": "node scripts/sync-changelog.mjs", "capture:screenshots": "node scripts/capture-screenshots.mjs" diff --git a/test/middleware.test.mjs b/test/middleware.test.mjs new file mode 100644 index 0000000..c4e4782 --- /dev/null +++ b/test/middleware.test.mjs @@ -0,0 +1,311 @@ +// Behavioural tests for functions/_middleware.ts — the releases.hal0.dev +// channel-manifest + Sigstore-bundle proxy. +// +// Why these exist: the middleware decides which GitHub Release a hardened +// client's manifest AND its signature bundle come from. If those two ever +// come from different releases, `cosign verify-blob` fails and the client +// reads a routing bug as tampering. The same-release pairing rule and the +// fail-closed-on-missing-bundle rule are load-bearing security invariants +// that a later refactor could silently break, so they are pinned here. +// +// Runs the real exported `onRequest` against a stubbed GitHub Releases API. +// Node's built-in test runner and type stripping only — no dev dependencies, +// no build step. `npm test`. + +import assert from "node:assert/strict"; +import { after, before, describe, it } from "node:test"; + +const MIDDLEWARE = new URL("../functions/_middleware.ts", import.meta.url); + +// ── fixture ──────────────────────────────────────────────────────────────── +// Shaped like GET /repos/Hal0ai/hal0/releases, newest first as the real API +// returns. Channel targets mirror hal0's src/hal0/release/policy.py. +const asset = (name) => ({ + id: name.length, + name, + url: `https://api.github.com/asset/${name}`, + browser_download_url: `https://github.com/dl/${name}`, +}); + +const NIGHTLY_TAG = "v1.0.0-nightly.20260727"; +const PREVIEW_TAG = "v1.0.0-alpha.2"; +const STABLE_TAG = "v0.9.8"; + +const RELEASES = [ + { + tag_name: NIGHTLY_TAG, + draft: false, + prerelease: true, + assets: [asset("nightly.json"), asset("nightly.json.bundle")], + }, + { + tag_name: PREVIEW_TAG, + draft: false, + prerelease: true, + assets: [asset("preview.json"), asset("preview.json.bundle")], + }, + { + // A final tag publishes to BOTH stable and preview. + tag_name: STABLE_TAG, + draft: false, + prerelease: false, + assets: [ + asset("stable.json"), + asset("stable.json.bundle"), + asset("preview.json"), + asset("preview.json.bundle"), + ], + }, +]; + +let releases = RELEASES; +let assetFetches = []; +let apiCalls = 0; +let realFetch; + +function stubFetch() { + return async (input, init) => { + const url = typeof input === "string" ? input : input.url; + if (url.startsWith("https://api.github.com/repos/")) { + apiCalls++; + return new Response(JSON.stringify(releases), { + status: 200, + headers: { "content-type": "application/json" }, + }); + } + if (url.startsWith("https://api.github.com/asset/")) { + const name = url.slice(url.lastIndexOf("/") + 1); + assetFetches.push(name); + assert.equal( + init.headers.Accept, + "application/octet-stream", + "asset download must use the documented octet-stream path", + ); + return new Response(`BYTES:${name}`, { status: 200 }); + } + throw new Error(`unexpected fetch: ${url}`); + }; +} + +before(() => { + realFetch = globalThis.fetch; + globalThis.fetch = stubFetch(); +}); +after(() => { + globalThis.fetch = realFetch; +}); + +const { onRequest } = await import(MIDDLEWARE.href); + +// `next` stands in for the static-asset pipeline: it records the path it was +// handed (so rewrites are observable) and 404s anything not in public/releases/. +const STATIC_FILES = new Set(["/releases/stable.json", "/releases/nightly.json"]); + +function request(url) { + const seen = []; + const context = { + request: new Request(url), + env: {}, + next: async (req) => { + const path = new URL(req ? req.url : url).pathname; + seen.push(path); + return STATIC_FILES.has(path) + ? new Response(`{"_placeholder":true,"path":"${path}"}`, { status: 200 }) + : new Response("not found", { status: 404 }); + }, + }; + return { context, seen }; +} + +function reset() { + releases = RELEASES; + assetFetches = []; + apiCalls = 0; +} + +// ── the pairing invariant ────────────────────────────────────────────────── +describe("manifest/bundle pairing", () => { + it("serves a channel manifest from the newest release carrying it", async () => { + reset(); + const { context } = request("https://releases.hal0.dev/stable.json"); + const res = await onRequest(context); + + assert.equal(res.status, 200); + assert.equal(await res.text(), "BYTES:stable.json"); + assert.equal(res.headers.get("x-hal0-source"), `github-release/${STABLE_TAG}`); + assert.equal(res.headers.get("x-hal0-channel"), "stable"); + assert.equal(res.headers.get("x-hal0-artifact"), "manifest"); + }); + + it("serves the sibling bundle from the SAME release as the manifest", async () => { + reset(); + const { context } = request("https://releases.hal0.dev/stable.json.bundle"); + const res = await onRequest(context); + + assert.equal(res.status, 200); + assert.equal(await res.text(), "BYTES:stable.json.bundle"); + assert.equal(res.headers.get("x-hal0-source"), `github-release/${STABLE_TAG}`); + assert.equal(res.headers.get("x-hal0-artifact"), "bundle"); + }); + + it("selects the release by the MANIFEST asset, never by the bundle", async () => { + // v1.1.0 is newer and carries stable.json but its bundle upload broke. + // Selecting by bundle would silently skip to v0.9.8 and pair v1.1.0's + // manifest with v0.9.8's signature on a later request. Fail closed. + reset(); + releases = [ + { tag_name: "v1.1.0", draft: false, prerelease: false, assets: [asset("stable.json")] }, + ...RELEASES, + ]; + const { context, seen } = request("https://releases.hal0.dev/stable.json.bundle"); + const res = await onRequest(context); + + assert.equal(res.status, 404, "bundles have no static backstop"); + assert.equal( + res.headers.get("x-hal0-proxy-failed"), + "no-sibling:stable.json.bundle:v1.1.0", + ); + assert.deepEqual(seen, ["/releases/stable.json.bundle"]); + assert.deepEqual(assetFetches, [], "must not download an older release's bundle"); + }); +}); + +// ── channel semantics (hal0 src/hal0/release/policy.py) ──────────────────── +describe("channel semantics", () => { + it("resolves preview from the newest prerelease", async () => { + reset(); + for (const [path, body] of [ + ["/preview.json", "BYTES:preview.json"], + ["/preview.json.bundle", "BYTES:preview.json.bundle"], + ]) { + const { context } = request(`https://releases.hal0.dev${path}`); + const res = await onRequest(context); + assert.equal(res.status, 200, path); + assert.equal(await res.text(), body); + assert.equal(res.headers.get("x-hal0-source"), `github-release/${PREVIEW_TAG}`); + assert.equal(res.headers.get("x-hal0-channel"), "preview"); + } + }); + + it("falls back to a final tag for preview when no prerelease is newer", async () => { + // A final tag publishes stable.json AND preview.json. + reset(); + releases = [RELEASES[2]]; + const { context } = request("https://releases.hal0.dev/preview.json"); + const res = await onRequest(context); + assert.equal(res.status, 200); + assert.equal(res.headers.get("x-hal0-source"), `github-release/${STABLE_TAG}`); + }); + + it("resolves nightly from the newest nightly", async () => { + reset(); + for (const path of ["/nightly.json", "/nightly.json.bundle"]) { + const { context } = request(`https://releases.hal0.dev${path}`); + const res = await onRequest(context); + assert.equal(res.status, 200, path); + assert.equal(res.headers.get("x-hal0-source"), `github-release/${NIGHTLY_TAG}`); + } + }); + + it("does not let a nightly serve the stable pointer", async () => { + reset(); + const { context } = request("https://releases.hal0.dev/stable.json"); + const res = await onRequest(context); + assert.equal(res.headers.get("x-hal0-source"), `github-release/${STABLE_TAG}`); + }); + + it("skips draft releases", async () => { + reset(); + releases = [ + { tag_name: "v2.0.0", draft: true, prerelease: false, assets: [asset("stable.json")] }, + ...RELEASES, + ]; + const { context } = request("https://releases.hal0.dev/stable.json"); + const res = await onRequest(context); + assert.equal(res.headers.get("x-hal0-source"), `github-release/${STABLE_TAG}`); + }); +}); + +// ── response contract ───────────────────────────────────────────────────── +describe("response headers", () => { + it("sets JSON content-type, permissive CORS, short cache and nosniff", async () => { + reset(); + const { context } = request("https://releases.hal0.dev/preview.json"); + const res = await onRequest(context); + + assert.equal(res.headers.get("content-type"), "application/json; charset=utf-8"); + assert.equal(res.headers.get("access-control-allow-origin"), "*"); + assert.equal(res.headers.get("cache-control"), "public, max-age=60, must-revalidate"); + assert.equal(res.headers.get("x-content-type-options"), "nosniff"); + }); +}); + +// ── degradation ─────────────────────────────────────────────────────────── +describe("fallthrough", () => { + it("annotates the static placeholder when no release carries the channel", async () => { + reset(); + releases = []; + const { context, seen } = request("https://releases.hal0.dev/stable.json"); + const res = await onRequest(context); + + assert.equal(res.status, 200, "static placeholder still served"); + assert.equal(res.headers.get("x-hal0-proxy-failed"), "no-asset:stable.json:0releases"); + assert.deepEqual(seen, ["/releases/stable.json"]); + }); + + it("annotates rather than throwing when the GitHub API rate-limits", async () => { + reset(); + const stubbed = globalThis.fetch; + globalThis.fetch = async (input, init) => { + const url = typeof input === "string" ? input : input.url; + if (url.startsWith("https://api.github.com/repos/")) + return new Response("rate limited", { status: 403 }); + return stubbed(input, init); + }; + try { + const { context } = request("https://releases.hal0.dev/nightly.json"); + const res = await onRequest(context); + assert.equal(res.status, 200); + assert.equal(res.headers.get("x-hal0-proxy-failed"), "gh-list-403-anon"); + } finally { + globalThis.fetch = stubbed; + } + }); + + it("keeps the legacy dev.json alias behaving exactly as before", async () => { + reset(); + const { context, seen } = request("https://releases.hal0.dev/dev.json"); + const res = await onRequest(context); + + assert.equal(res.status, 404); + assert.equal(res.headers.get("x-hal0-proxy-failed"), "no-asset:dev.json:3releases"); + assert.deepEqual(seen, ["/releases/dev.json"]); + }); +}); + +// ── routing ─────────────────────────────────────────────────────────────── +describe("host-conditional rewrites", () => { + it("rewrites non-channel paths under /releases/", async () => { + reset(); + const { context, seen } = request("https://releases.hal0.dev/foo"); + await onRequest(context); + assert.deepEqual(seen, ["/releases/foo"]); + }); + + it("does not double-prefix an already-/releases/ path", async () => { + reset(); + const { context, seen } = request("https://releases.hal0.dev/releases/stable.json"); + await onRequest(context); + assert.deepEqual(seen, ["/releases/stable.json"]); + }); + + it("leaves other hosts untouched and makes no GitHub API call", async () => { + reset(); + for (const url of ["https://hal0.dev/install.sh", "https://hal0.dev/stable.json"]) { + const { context, seen } = request(url); + await onRequest(context); + assert.deepEqual(seen, [new URL(url).pathname]); + } + assert.equal(apiCalls, 0, "no upstream call for non-releases hosts"); + }); +}); From 020578e0df90501d652c07eb2277c190375996e6 Mon Sep 17 00:00:00 2001 From: Alexander Date: Mon, 27 Jul 2026 12:07:04 -0400 Subject: [PATCH 3/3] fix(releases): re-sync install.sh with the cosign-fetching bootstrap MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit hal0 #1363 (f6f92691) added the digest-pinned cosign fetch to installer/bootstrap.sh, taking it 362 -> 504 lines. check-bootstrap-parity.sh is a byte-exact diff, so the served copy has to follow. Verified: cmp against hal0 f6f92691:installer/bootstrap.sh is byte-identical, bash -n clean, and check-bootstrap-parity.sh run from a clean main worktree reports 'OK: in-tree bootstrap.sh matches the live install.sh — no drift.' Co-Authored-By: Claude Opus 5 (1M context) --- public/install.sh | 156 +++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 149 insertions(+), 7 deletions(-) diff --git a/public/install.sh b/public/install.sh index 29c299d..8cf1176 100755 --- a/public/install.sh +++ b/public/install.sh @@ -29,7 +29,9 @@ # client-pinned release-workflow identity. Only then does it parse artifact # URLs. The tarball's sha256 and publisher signature are both checked again as # defense-in-depth before anything is executed. cosign is therefore a required -# bootstrap dependency. See docs/internal/release-manifest.md. +# bootstrap dependency — see ensure_cosign() below, which uses a system cosign +# when one exists and otherwise fetches a digest-pinned official build into the +# throwaway work directory. See docs/internal/release-manifest.md. # # Schema reference: docs/internal/release-manifest.md (hal0.releases.v1). @@ -47,6 +49,58 @@ _PREVIEW_MANIFEST_ADMISSION_IDENTITY="${_MANIFEST_IDENTITY_PREFIX}refs/tags/v\\d _NIGHTLY_MANIFEST_IDENTITY="${_MANIFEST_IDENTITY_PREFIX}refs/heads/main$" _MANIFEST_SIGNER_ISSUER='https://token.actions.githubusercontent.com' +# ── pinned cosign (see ensure_cosign) ────────────────────────────────────── +# +# cosign is what turns "some bytes off a CDN" into "bytes signed by the hal0 +# release workflow", so it is a hard requirement. Distros that package it +# (Arch, Fedora, Alpine, openSUSE, nixpkgs) are used as-is. Debian/Ubuntu do +# NOT package cosign — and rather than degrade to an unverified install on +# the single most common hal0 host, this script fetches the official sigstore +# release binary itself and checks it against a digest pinned right here. +# +# Pinning the digest in this file introduces NO new trust root. The user +# already trusted these exact bytes the moment they piped this script to +# bash; a constant inside a script you have already decided to execute +# cannot be less trustworthy than the script executing it. The resulting +# chain is: +# +# trusted script -> digest-pinned cosign -> OIDC-pinned release manifest +# -> digest-pinned + signature-verified release tarball +# +# There is deliberately NO opt-out environment variable. A flag such as +# HAL0_INSTALL_REQUIRE_COSIGN=0 becomes the copy-pasted default in forum +# answers and CI snippets within a week, and silently un-does the signature +# hardening this file exists to provide. Unsupported platform, failed +# download, or digest mismatch all fail closed with manual-install guidance. +# +# ── MAINTENANCE: this pin must be bumped by hand ────────────────────────── +# There is no automated bump path in this repo: there is no +# .github/dependabot.yml and no renovate config, and +# scripts/update-toolbox-digests.sh only refreshes ghcr.io *image* digests in +# the repo-root manifest.json — it never touches this file. Bump manually +# when sigstore cuts a release (aim to stay within a release or two): +# +# V=v3.1.2 +# curl -fsSL "https://github.com/sigstore/cosign/releases/download/${V}/cosign_checksums.txt" \ +# | grep -E 'cosign-linux-(amd64|arm64)$' +# +# then paste both digests below and update _COSIGN_VERSION. That checksums +# file is itself keyless-signed (cosign_checksums.txt.sigstore.json on the +# same release) — verify it with an existing cosign before trusting it. +# tests/installer/test_bootstrap_cosign_fetch.py pins the shape of these +# constants so a malformed or half-finished bump fails CI. +_COSIGN_VERSION='v3.1.2' +_COSIGN_BASE_URL='https://github.com/sigstore/cosign/releases/download' +# Digests below are the published sha256 of the official release assets for +# _COSIGN_VERSION, taken from that release's cosign_checksums.txt. +_COSIGN_SHA256_LINUX_AMD64='f7622ed3cf22e55e1ae6377c080979ff77a22da9981c11df222a2e444991e7cf' +_COSIGN_SHA256_LINUX_ARM64='90e7ae0b5dfd60f20816b52c012addf7fc055ebcc7bea4ce81c428ca8518c302' + +# Resolved exactly once by ensure_cosign(); every cosign invocation goes +# through it so a fetched binary and a system one are indistinguishable to +# the verification code below. +_COSIGN_BIN="" + # ── tiny output helpers ──────────────────────────────────────────────────── if [[ -t 1 && -z "${NO_COLOR:-}" ]]; then _C_DIM=$'\033[2m'; _C_RED=$'\033[31m'; _C_YEL=$'\033[33m' @@ -93,6 +147,90 @@ validate_channel() { esac } +# ── cosign acquisition ───────────────────────────────────────────────────── +# Map `uname -m` onto the sigstore release asset name. Fails closed: an +# architecture we hold no pinned digest for is an unsupported platform, not a +# reason to skip verification. +cosign_asset_for_machine() { + case "$1" in + x86_64|amd64) printf 'cosign-linux-amd64\n' ;; + aarch64|arm64) printf 'cosign-linux-arm64\n' ;; + *) return 1 ;; + esac +} + +cosign_pinned_sha256() { + case "$1" in + cosign-linux-amd64) printf '%s\n' "${_COSIGN_SHA256_LINUX_AMD64}" ;; + cosign-linux-arm64) printf '%s\n' "${_COSIGN_SHA256_LINUX_ARM64}" ;; + *) return 1 ;; + esac +} + +cosign_manual_install_hint() { + printf '%s' "\ + install cosign manually and re-run: + Arch: sudo pacman -S cosign + Fedora: sudo dnf install cosign + Alpine: sudo apk add cosign + other: https://docs.sigstore.dev/cosign/system_config/installation/" +} + +# Resolve _COSIGN_BIN. Prefers a distro-packaged cosign (it rides the +# distro's own update track and needs no pin from us); otherwise downloads +# the pinned official build into the trap-guarded work dir, verifies its +# sha256 against the constant above, and uses it from there. Nothing is +# installed anywhere persistent. +ensure_cosign() { + local work="$1" + + if command -v cosign >/dev/null 2>&1; then + _COSIGN_BIN="cosign" + info "using system cosign ($(command -v cosign))" + return 0 + fi + + local machine asset expected out actual + machine="$(uname -m)" + if ! asset="$(cosign_asset_for_machine "${machine}")"; then + die "cosign is required to verify the release, and hal0 pins no cosign + build for this architecture (uname -m: ${machine}). +$(cosign_manual_install_hint)" + fi + expected="$(cosign_pinned_sha256 "${asset}")" \ + || die "internal error: no pinned cosign sha256 for ${asset}" + + out="${work}/cosign" + info "cosign not found — fetching pinned ${_COSIGN_VERSION} (${asset})" + info " ${_C_DIM}${_COSIGN_BASE_URL}/${_COSIGN_VERSION}/${asset}${_C_RST}" + if ! curl -fsSL --retry 3 --retry-delay 2 -o "${out}" \ + --url "${_COSIGN_BASE_URL}/${_COSIGN_VERSION}/${asset}"; then + rm -f -- "${out}" + die "could not download pinned cosign ${_COSIGN_VERSION} (${asset}). +$(cosign_manual_install_hint)" + fi + + actual="$(sha256sum "${out}" | awk '{print $1}')" + if [[ "${actual}" != "${expected}" ]]; then + rm -f -- "${out}" + die "pinned cosign sha256 mismatch — expected ${expected}, got ${actual} + refusing to run an unverified cosign binary. +$(cosign_manual_install_hint)" + fi + + chmod +x "${out}" + # A hardened host may mount the temp filesystem noexec, which would + # otherwise surface as an inscrutable verification failure later. + if ! "${out}" version >/dev/null 2>&1; then + rm -f -- "${out}" + die "fetched cosign could not be executed from ${work} + (is that filesystem mounted noexec? retry with TMPDIR=/var/tmp) +$(cosign_manual_install_hint)" + fi + _COSIGN_BIN="${out}" + ok "pinned cosign ${_COSIGN_VERSION} sha256 OK (${actual:0:12}…)" +} + manifest_admission_identity() { case "$1" in stable) printf '%s\n' "${_STABLE_MANIFEST_ADMISSION_IDENTITY}" ;; @@ -152,12 +290,12 @@ fetch_manifest() { verify_release_manifest() { local manifest="$1" bundle="$2" identity="$3" - command -v cosign >/dev/null 2>&1 \ - || die "cosign is required to verify the release manifest but is not installed. - install it from https://docs.sigstore.dev/cosign/installation/" + [[ -n "${_COSIGN_BIN}" ]] \ + || die "cosign is required to verify the release manifest but is not available. +$(cosign_manual_install_hint)" info "verifying release manifest with pinned workflow identity" - if ! cosign verify-blob \ + if ! "${_COSIGN_BIN}" verify-blob \ --bundle "${bundle}" \ --certificate-identity-regexp "${identity}" \ --certificate-oidc-issuer "${_MANIFEST_SIGNER_ISSUER}" \ @@ -261,7 +399,7 @@ fetch_sidecar() { cosign_verify() { local tarball="$1" bundle="$2" identity="$3" - command -v cosign >/dev/null 2>&1 \ + [[ -n "${_COSIGN_BIN}" ]] \ || die "cosign disappeared after release manifest verification — refusing to install" info "verifying signature with cosign keyless OIDC" @@ -271,7 +409,7 @@ cosign_verify() { # The authenticated manifest must provide a Sigstore bundle. Detached # signature/certificate sidecars are not an accepted bootstrap scheme. local -a verify_args=(--bundle "${bundle}") - if ! cosign verify-blob \ + if ! "${_COSIGN_BIN}" verify-blob \ "${verify_args[@]}" \ --certificate-identity-regexp "${identity}" \ --certificate-oidc-issuer "${_MANIFEST_SIGNER_ISSUER}" \ @@ -300,6 +438,10 @@ main() { warn "HAL0_BOOTSTRAP_KEEP_TMP=1 — leaving work dir ${work}" fi + # Resolve cosign before any release bytes are fetched: if we cannot get a + # verifier we must not go on to download things we cannot verify. + ensure_cosign "${work}" + local manifest="${work}/manifest.json" local manifest_bundle="${work}/manifest.json.bundle" fetch_manifest "${manifest}"