Skip to content

fix(releases): sync install.sh with hardened bootstrap + serve manifest bundles #45

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
thinmintdev wants to merge 3 commits into
base: master
Choose a base branch
from
Open

fix(releases): sync install.sh with hardened bootstrap + serve manifest bundles #45

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
6d48f29
fix(releases): sync install.sh with hardened bootstrap + serve manife…
thinmintdev Jul 27, 2026
ea10d2d
test(releases): pin the middleware manifest/bundle pairing invariants
thinmintdev Jul 27, 2026
020578e
fix(releases): re-sync install.sh with the cosign-fetching bootstrap
thinmintdev Jul 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
172 changes: 141 additions & 31 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,11 +98,24 @@ See [`NOTES.md`](./NOTES.md) for the full rationale.
```sh
npm install # install deps
npm run dev # dev server at http://localhost:4321
npm run astro check # type check (must be 0 errors / 0 warnings)
npm test # behavioural tests for functions/_middleware.ts
npm run astro check # type check
npm run build # static site → ./dist/
npm run preview # serve ./dist/ for smoke-testing
```

`npm test` uses Node's built-in runner and type stripping — no dev
dependencies, no build step. It covers `functions/_middleware.ts`, which
decides which GitHub Release a hardened client's manifest *and* its
signature bundle come from; the same-release pairing rule it pins is a
security invariant, not a style preference. See
[Release manifest hosting](#release-manifest-hosting-releaseshal0dev).

`astro check` currently reports **5 pre-existing errors** (2 in
`src/content.config.ts` from starlight-blog's Zod `SchemaContext`, 3 implicit
`any` in `src/pages/changelog.astro`). They are unrelated to the middleware;
treat 5 as the baseline and anything above it as yours.

## Deploy

The marketing site + docs ship to **Vercel** on every push to `master`.
Expand All @@ -115,43 +128,75 @@ vercel --prod # ad-hoc prod ship (CI handles the normal path)

## Release manifest hosting (`releases.hal0.dev`)

The hal0 self-updater (`src/hal0/updater/updater.py`) fetches per-channel
release manifests from:
The hal0 self-updater (`src/hal0/updater/updater.py`) and the one-line
installer (`installer/bootstrap.sh`) fetch per-channel release manifests —
**and the sibling Sigstore bundle that authenticates each one** — from:

```
https://releases.hal0.dev/{stable|nightly}.json
https://releases.hal0.dev/{stable|preview|nightly}.json
https://releases.hal0.dev/{stable|preview|nightly}.json.bundle
```

Both halves of the pair are mandatory. Hardened clients cosign-verify the
exact manifest bytes against the bundle, using a client-pinned release
workflow OIDC identity, *before* parsing a single artifact URL out of the
manifest. A manifest served without its bundle is not consumable — the
client refuses it rather than trusting unauthenticated URLs.

Schema: `hal0.releases.v1` — see
[`hal0/docs/release-manifest.md`](https://github.com/hal0ai/hal0/blob/main/docs/release-manifest.md)
for the full field reference. The schema's `cert_url` field is
**required** — cosign 3.x keyless `verify-blob` needs `--certificate`,
so the manifest carries the URL to the `.crt` artifact alongside
`url` (tarball) and `sig_url` (signature).
[`hal0/docs/internal/release-manifest.md`](https://github.com/hal0ai/hal0/blob/main/docs/internal/release-manifest.md)
for the full field reference. The trust-carrying fields are `bundle_url`
(Sigstore bundle for the tarball), `digest_sha256`, `signer_identity`, and
`signer_issuer`. `sig_url`/`cert_url` are still emitted but are no longer
the verification path: the bundle embeds the Fulcio certificate, the
signature, and the Rekor inclusion proof + SET, so `cosign verify-blob
--bundle` keeps working after the short-lived cert expires.

### Channels

Which manifests a tag publishes is decided by hal0's
`src/hal0/release/policy.py` (`manifest_targets`), not by this repo:

| tag | manifests published |
|---|---|
| `v1.2.3` (final) | `stable.json`, `preview.json` |
| `v1.2.3-alpha.2` / `-beta.N` / `-rc.N` | `preview.json` |
| `v1.2.3-nightly.YYYYMMDD` | `nightly.json` |

### How it works (as of v0.1.0-alpha.1, 2026-05-21)
So `preview` tracks the newest of (latest prerelease, latest final) — a
final tag lands on both channels — and `stable` only ever moves on a final
tag. `release.yml` uploads `<channel>.json` **and** `<channel>.json.bundle`
as a pair for every target.

`releases.hal0.dev` is **fully operational** and serves the live
manifest. The subdomain lives on a small Cloudflare Pages project
whose middleware proxies the canonical asset off the latest GitHub
Release on `hal0ai/hal0`:
### How it works

The subdomain lives on a small Cloudflare Pages project whose middleware
(`functions/_middleware.ts`) proxies the canonical assets off the newest
GitHub Release on `hal0ai/hal0` that carries them:

1. Tag `vX.Y.Z` on `hal0ai/hal0` triggers `.github/workflows/release.yml`.
2. The workflow builds `hal0-X.Y.Z.tar.gz`, computes its sha256, signs
it with cosign keyless against the GH Actions OIDC identity, and
uploads tarball + `.sig` + `.crt` to the GH Release alongside the
generated `stable.json` (manifest schema `hal0.releases.v1`).
3. The CF Pages middleware on `releases.hal0.dev` fetches
`stable.json` from the latest GH Release and serves it with a short
cache (~60s). A `v*` tag propagates end-to-end within about a
minute — **no hal0-web deploy required**.
4. Updater clients verify the tarball with `cosign verify-blob
--certificate <cert_url> --signature <sig_url> …` against the
`signer_identity` regex in the manifest.

Updater clients should point at `https://releases.hal0.dev/stable.json`
(or `…/nightly.json`). The old GitHub-direct URL form is not used; the
canonical asset name is `stable.json`, not `latest.json`.
2. The workflow builds `hal0-X.Y.Z.tar.gz`, computes its sha256, cosign
keyless-signs it against the GH Actions OIDC identity, generates each
target channel manifest, signs each manifest into a sibling `.bundle`,
self-verifies both, and uploads the lot as Release assets.
3. The middleware on `releases.hal0.dev` resolves
`/<channel>.json[.bundle]` from the newest non-draft release carrying
`<channel>.json` and serves it with a short cache (~60s). A `v*` tag
propagates end-to-end within about a minute — **no hal0-web deploy
required**.
4. Clients verify the manifest against its bundle, then verify the tarball
digest and publisher signature again as defence in depth.

Release selection is always driven by the **manifest** asset, never the
bundle, so a `.json` and a `.json.bundle` request resolve to the same
release even if a new tag lands between a client's two fetches. If the
selected release carries the manifest but not its bundle, the bundle
request fails closed (`x-hal0-proxy-failed: no-sibling:…`) rather than
pairing a manifest with an older release's signature.

Response headers worth probing: `x-hal0-source: github-release/<tag>`,
`x-hal0-channel`, `x-hal0-artifact: manifest|bundle`, and on any
fallthrough `x-hal0-proxy-failed: <reason>`.

### Static fallback in this repo

Expand All @@ -166,11 +211,76 @@ land at `https://hal0.dev/releases/{stable,nightly}.json`. They're
kept as a backstop and as a schema example; the live manifest is
whatever the CF Pages middleware on `releases.hal0.dev` returns.

There is deliberately **no static backstop for `.bundle`** — a placeholder
signature cannot verify, and a client that fetched one would read the
failure as tampering rather than as "not published yet". A bundle that
can't be proxied is an annotated 404.

### Verify

```sh
curl -s https://releases.hal0.dev/stable.json | jq .
curl -sI https://releases.hal0.dev/stable.json | grep -i cache-control
# manifest + its bundle must BOTH be 200 for a channel to be operational
for c in stable preview nightly; do
for f in "$c.json" "$c.json.bundle"; do
printf '%-24s %s\n' "$f" \
"$(curl -so /dev/null -w '%{http_code}' "https://releases.hal0.dev/$f")"
done
done

curl -sI https://releases.hal0.dev/stable.json | grep -iE 'cache-control|x-hal0'

# end-to-end: authenticate the manifest exactly as a client does
curl -fsSLO https://releases.hal0.dev/preview.json
curl -fsSLO https://releases.hal0.dev/preview.json.bundle
cosign verify-blob --bundle preview.json.bundle \
--certificate-identity-regexp '^https://github\.com/(Hal0ai|hal0ai)/hal0/\.github/workflows/release\.yml@refs/tags/v\d+\.\d+\.\d+(-(alpha|beta|rc)\.(0|[1-9]\d*))?$' \
--certificate-oidc-issuer https://token.actions.githubusercontent.com \
preview.json
```

(That identity regex is the `preview` admission pattern from
`installer/bootstrap.sh`; `stable` and `nightly` pin narrower ones. Keep the
copies in lockstep with bootstrap.sh and `updater.py` — they are the trust
root, not documentation.)

A channel only becomes operational once a tag has published *both* halves,
in the current manifest schema. Two things can hold a channel back, and
neither is fixable in this repo:

- **No bundle.** Releases cut before manifest-bundle signing carry
`<channel>.json` with no `.bundle`. The serving layer cannot synthesise a
missing signature, so `.bundle` stays 404 until the next tag.
- **Pre-hardening manifest fields.** The client's strict policy pass
requires `release_kind` and `prerelease_stage`, and requires
`signer_identity` to equal the exact per-release identity it derives
itself. Manifests generated before those fields existed are rejected even
when perfectly signed.

As of 2026-07-27 that means `stable` is **not** operational end-to-end: the
newest release carrying `stable.json` is `v0.9.8`, which has no
`stable.json.bundle` and whose manifest predates `release_kind`. `preview`
(from `v1.0.0-alpha.2`) satisfies both conditions. Both clear on the next
final tag cut by the current `release.yml`.

## One-line installer (`hal0.dev/install.sh`)

`public/install.sh` is a **byte-identical mirror** of
[`hal0:installer/bootstrap.sh`](https://github.com/hal0ai/hal0/blob/main/installer/bootstrap.sh)
— that file is the audited original and the trust boundary for
`curl https://hal0.dev/install.sh | bash`. Never hand-edit the copy here:
copy the canonical file over it wholesale, in the same PR as the hal0-side
change.

hal0's `Bootstrap parity (daily)` workflow
(`.github/workflows/bootstrap-parity.yml` →
`scripts/check-bootstrap-parity.sh`) fetches the live URL daily and does a
plain `diff -u` against the in-tree original: exit 0 in sync, 1 drift,
2 operational error. There is no normalisation, so a single byte of drift
fails it. Preview a sync locally from a hal0 checkout with:

```sh
HAL0_INSTALL_URL="file:///path/to/hal0-web/public/install.sh" \
bash scripts/check-bootstrap-parity.sh
```

## Build state
Expand Down
77 changes: 66 additions & 11 deletions functions/_middleware.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,28 @@
//
// Two responsibilities on `releases.hal0.dev`:
//
// 1. `releases.hal0.dev/{stable,nightly,dev}.json` is proxied LIVE from
// the most recent GitHub release on `Hal0ai/hal0` that carries an
// asset of that name. Self-syncing: each tagged release on hal0
// becomes visible at releases.hal0.dev without a hal0-web deploy.
// 1. `releases.hal0.dev/{stable,preview,nightly}.json` — and the sibling
// `<channel>.json.bundle` Sigstore bundle for each — are proxied LIVE from the
// most recent GitHub release on `Hal0ai/hal0` that carries the channel
// manifest. Self-syncing: each tagged release on hal0 becomes visible
// at releases.hal0.dev without a hal0-web deploy.
// Static `public/releases/*.json` stays as a placeholder backstop
// in case the GitHub API is unreachable.
//
// Channels mirror hal0's `src/hal0/release/policy.py` manifest targets:
// final tag v1.2.3 → stable.json + preview.json
// preview v1.2.3-alpha.2 → preview.json
// nightly v1.2.3-nightly.20260727 → nightly.json
// release.yml uploads `<channel>.json` AND `<channel>.json.bundle` as a
// pair for every target, so both are always resolvable from the SAME
// release. Hardened clients (installer/bootstrap.sh `verify_release_manifest`,
// src/hal0/updater/updater.py) require BOTH: they cosign-verify the exact
// manifest bytes against the sibling bundle before parsing any artifact
// URL. Serving the manifest without its bundle leaves the channel
// non-operational, so the two are resolved from one release selection
// (see `proxyChannelArtifact`) — never independently, which could pair a
// manifest from release N with a bundle from release N-1 mid-publish.
//
// 2. Anything else on `releases.hal0.dev` (e.g. `/foo`) is rewritten
// to `/releases/foo` so the static files under `public/releases/`
// are reachable without polluting the marketing-site root.
Expand All @@ -27,8 +42,17 @@
// limit is permanently exhausted. Authenticated requests get 5000/hr
// per token. The token only needs public-repo read scope.

const CHANNEL_RE = /^\/(stable|nightly|dev)\.json$/;
const RELEASES_API = "https://api.github.com/repos/Hal0ai/hal0/releases?per_page=10";
// Capture 1 = channel, capture 2 = ".bundle" when the sibling Sigstore bundle
// is being requested. `dev` is a legacy alias kept so an old client asking for
// it behaves exactly as before (no release publishes dev.json, so it falls
// through to the static backstop); the live channels are stable/preview/nightly.
const CHANNEL_RE = /^\/(stable|preview|nightly|dev)\.json(\.bundle)?$/;
// per_page must comfortably out-span the nightly cadence: nightlies publish
// daily and carry only `nightly.json`, so at per_page=10 the newest release
// still carrying `stable.json` scrolls out of the window in ~a week and the
// stable pointer silently degrades to the static placeholder. 50 keeps a
// stable/preview tag reachable through a long nightly streak.
const RELEASES_API = "https://api.github.com/repos/Hal0ai/hal0/releases?per_page=50";

function authHeaders(token: string | undefined): Record<string, string> {
const base: Record<string, string> = {
Expand All @@ -55,10 +79,23 @@ type ProxyOutcome =
| { ok: true; response: Response }
| { ok: false; reason: string };

async function proxyChannelManifest(
// Resolve `<channel>.json` or its sibling `<channel>.json.bundle` from the most
// recent non-draft release that carries the channel manifest.
//
// Release selection is driven by the MANIFEST asset in both cases, never by the
// bundle. That keeps a bundle request pinned to the same release a manifest
// request would resolve to, so `cosign verify-blob --bundle` sees a matching
// pair even if a new release lands between a client's two fetches. If the
// selected release carries the manifest but not its bundle (a broken publish),
// we fail loudly rather than walking back to an older release and handing out a
// mismatched pair.
async function proxyChannelArtifact(
channel: string,
wantBundle: boolean,
token: string | undefined,
): Promise<ProxyOutcome> {
const manifestName = `${channel}.json`;
const assetName = wantBundle ? `${manifestName}.bundle` : manifestName;
let listResp: Response;
try {
listResp = await fetch(RELEASES_API, {
Expand Down Expand Up @@ -89,8 +126,15 @@ async function proxyChannelManifest(

for (const release of releases) {
if (release.draft) continue;
const asset = release.assets?.find((a) => a.name === `${channel}.json`);
if (!asset) continue;
// The manifest decides which release serves this channel.
if (!release.assets?.some((a) => a.name === manifestName)) continue;

const asset = release.assets?.find((a) => a.name === assetName);
if (!asset) {
const reason = `no-sibling:${assetName}:${release.tag_name}`;
console.warn(`[releases-proxy] ${reason}`);
return { ok: false, reason };
}

// Use the api.github.com asset endpoint with octet-stream Accept.
// Returns a 302 to objects.githubusercontent.com with the asset
Expand Down Expand Up @@ -123,17 +167,21 @@ async function proxyChannelManifest(
response: new Response(body, {
status: 200,
headers: {
// Sigstore bundles are JSON too (`cosign sign-blob --bundle`
// writes a JSON-serialized protobuf bundle), so one
// content-type covers manifest and bundle alike.
"content-type": "application/json; charset=utf-8",
"access-control-allow-origin": "*",
"cache-control": "public, max-age=60, must-revalidate",
"x-content-type-options": "nosniff",
"x-hal0-source": `github-release/${release.tag_name}`,
"x-hal0-channel": channel,
"x-hal0-artifact": wantBundle ? "bundle" : "manifest",
},
}),
};
}
const reason = `no-asset:${channel}.json:${releases.length}releases`;
const reason = `no-asset:${manifestName}:${releases.length}releases`;
console.warn(`[releases-proxy] ${reason}`);
return { ok: false, reason };
}
Expand Down Expand Up @@ -172,9 +220,16 @@ export const onRequest: PagesFunction = async (context) => {
if (url.hostname === "releases.hal0.dev") {
const channelMatch = url.pathname.match(CHANNEL_RE);
if (channelMatch) {
const outcome = await proxyChannelManifest(channelMatch[1], context.env.GITHUB_TOKEN);
const outcome = await proxyChannelArtifact(
channelMatch[1],
channelMatch[2] === ".bundle",
context.env.GITHUB_TOKEN,
);
if (outcome.ok) return outcome.response;
// Fall through to the static placeholder, but annotate why.
// Bundles have no static backstop (a placeholder signature would be
// worse than a 404 — it would fail cosign verification and read as
// tampering), so a bundle fallthrough is an annotated 404.
const rewritten = new URL(context.request.url);
rewritten.pathname = "/releases" + url.pathname;
return annotateFallthrough(context, rewritten.toString(), outcome.reason);
Expand Down
1 change: 1 addition & 0 deletions package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
"prebuild": "node scripts/sync-changelog.mjs",
"build": "astro build",
"preview": "astro preview",
"test": "node --test \"test/**/*.test.mjs\"",
"astro": "astro",
"sync:changelog": "node scripts/sync-changelog.mjs",
"capture:screenshots": "node scripts/capture-screenshots.mjs"
Expand Down
Loading