Skip to content

Distribute via platform subpackages so bunx works securely#18

Closed
tombeckenham wants to merge 3 commits into
upstash:masterfrom
tombeckenham:bunx-via-platform-subpackages
Closed

Distribute via platform subpackages so bunx works securely#18
tombeckenham wants to merge 3 commits into
upstash:masterfrom
tombeckenham:bunx-via-platform-subpackages

Conversation

@tombeckenham
Copy link
Copy Markdown

TL;DR

bunx @upstash/qstash-cli does not work today — Bun deliberately skips lifecycle scripts for untrusted dependencies, so the existing postinstall never runs, the Go binary is never downloaded, and bin/qstash (an empty file from touch in prepublishOnly) executes as a no-op.

This PR replaces the postinstall-download model with the standard platform-specific optionalDependencies pattern used by esbuild, swc, turbo, biome, sharp, lightningcss, rolldown, and oxc. bunx (and npx, pnpm dlx, yarn dlx) all work out of the box with no --trust flag, and the supply-chain story gets substantially stronger as a side effect.

Why bunx doesn't work today

Bun ships secure-by-default: lifecycle scripts (postinstall, preinstall, etc.) are skipped unless the package is listed in the user's trustedDependencies. Under bunx, transient installs follow the same rule. So the current flow:

  1. bunx @upstash/qstash-cli dev installs the package
  2. postinstall: node install.js is skipped (Bun's default)
  3. bin/qstash is an empty file (prepublishOnly: touch bin/qstash)
  4. Bun execs the empty file → no error, no output, no help

The fix is not to ask users for a trust flag — that would weaken the very protection that makes ephemeral installs safe. The fix is to remove the postinstall entirely.

How the new model works

Package Contents Notes
`@upstash/qstash-cli` tiny JS shim (`bin/qstash.js`) Wrapper users install. Pure JS, ~50 lines.
`@upstash/qstash-cli-darwin-arm64` the Go binary Constrained via `os`+`cpu` fields
`@upstash/qstash-cli-darwin-x64` the Go binary Constrained via `os`+`cpu` fields
`@upstash/qstash-cli-linux-arm64` the Go binary Constrained via `os`+`cpu` fields
`@upstash/qstash-cli-linux-x64` the Go binary Constrained via `os`+`cpu` fields
`@upstash/qstash-cli-win32-arm64` the Go binary Constrained via `os`+`cpu` fields
`@upstash/qstash-cli-win32-x64` the Go binary Constrained via `os`+`cpu` fields

The wrapper lists all six binary packages as `optionalDependencies`. npm/bun/pnpm install only the matching one — the `os` + `cpu` constraints cause every other platform's package to be silently skipped. A macOS-arm64 user downloads only the wrapper + one binary package; nothing else.

At runtime the shim does a single `require.resolve('@upstash/qstash-cli--/package.json')`, derives the binary path, and `spawnSync`s it with stdio inherited. No shell. No network. No `shell: true`. Exit code propagated. About 50 lines total.

Security benefits

1. No postinstall script anywhere. Removes an entire class of supply-chain risk — the same vector that's hit the JS ecosystem repeatedly (`event-stream`, `ua-parser-js`, etc.).

2. npm provenance now covers the actual Go binary, not just a downloader. Today the wrapper has `--provenance` but the binary is fetched out-of-band over plain HTTPS with no integrity check at all — not even a SHA-256. With this change, each platform subpackage is published with `npm publish --provenance`, producing a sigstore-backed attestation tied to a specific GitHub workflow run + commit SHA. The bits the user executes are covered by the same attestation chain that covers the wrapper.

3. Single trust anchor. Today users implicitly trust both npm and `artifacts.upstash.com` (a separate domain, separate TLS chain, no integrity verification on download). After this PR the entire user-facing chain is npm + GitHub OIDC. `artifacts.upstash.com` is only consulted at release time, inside CI.

4. Defense-in-depth at release time. The release workflow now:

  • Fetches `qstash-server__checksums.txt` (the goreleaser-style manifest that's already published next to each release) and verifies each archive's SHA-256 against it with `sha256sum -c` before packaging.
  • Sanity-checks each extracted binary's file-type magic bytes against the platform/arch slot (Mach-O+arm64/x86_64, ELF+aarch64/x86-64, PE32+Aarch64/x86-64). Defends against the wrong binary landing in the wrong slot.
  • Extracts archives with `tar --no-same-owner --no-same-permissions` into a fresh `mktemp -d`.

5. No runtime network access. Works air-gapped, works in CI containers with no egress, works on restricted laptops, works behind corporate proxies that block `artifacts.upstash.com`.

6. Tamper-evident at the npm layer. npm tarball integrity hashes are recorded in user lockfiles; a swap post-publish is detectable.

7. Compatible with `--ignore-scripts` and strict-mode installers. pnpm strict mode, Bun's default behavior, `npm ci --ignore-scripts`, and Yarn PnP all work without special configuration.

8. Release pipeline is now atomic. Restructured into three jobs: `build` → `publish-platform` → `publish-wrapper`. All six platforms are validated (download + checksum + magic-byte) before any publish runs, so a verification failure no longer leaves orphan publishes on npm. The wrapper publishes last, after polling npm registry with backoff to confirm every `optionalDependency` is resolvable — closing the registry-propagation gap where a user installing in the wrong second would hit the shim's "could not locate subpackage" error.

9. Every GitHub Action pinned to a 40-char commit SHA with a version comment, blocking the tag-rewrite class of supply-chain attacks against the actions we depend on. Actions are on absolute-latest stable majors (`checkout@v6.0.2`, `setup-node@v6.4.0`, `upload-artifact@v7.0.1`, `download-artifact@v8.0.1`).

10. Workflow concurrency guard prevents two simultaneous releases for the same tag from racing each other.

Benefits to users of bun and other package managers

Package runner Before this PR After this PR
`bunx @upstash/qstash-cli dev` Silently does nothing (postinstall skipped) ✅ Works
`bunx --bun @upstash/qstash-cli dev` Silently does nothing ✅ Works (node-less Bun)
`npx @upstash/qstash-cli dev` ✅ Works (postinstall runs) ✅ Works (no postinstall needed)
`pnpm dlx @upstash/qstash-cli dev` Works only if `onlyBuiltDependencies` permits ✅ Works
`yarn dlx @upstash/qstash-cli dev` Works (postinstall runs) ✅ Works
`npm install --ignore-scripts @upstash/qstash-cli` ❌ Binary never downloaded ✅ Works
`bun install` in a strict-mode project ❌ Binary never downloaded ✅ Works
Air-gapped / no-egress install ❌ Cannot reach `artifacts.upstash.com` ✅ Works (binary comes via npm)

For end users this is a strict improvement: every package manager works, no flags required, no security defaults to disable.

Release pipeline

The new workflow is `build` → `publish-platform` (matrix, both with all 6 platforms) → `publish-wrapper`. Each step:

  • `build` (matrix): runs `scripts/build-platform-package.sh `, which curls the archive + the checksums manifest, verifies SHA-256, extracts, magic-byte-checks the binary, assembles the subpackage, and uploads it as a GH Actions artifact.
  • `publish-platform` (matrix, `needs: build`): downloads each artifact and runs `npm publish --access public --provenance` (`--tag=canary` on prerelease).
  • `publish-wrapper` (`needs: publish-platform`): `jq`-pins the wrapper's `version` and every `optionalDependency` value to the release version, polls the registry for each subpackage with linear backoff (5s + 10s + … up to ~105s), then `npm publish --provenance`.

The wrapper publishes last so that the moment users can resolve `@upstash/qstash-cli@`, every `optionalDependency@` it references is already resolvable.

Test plan

Verified locally:

  • `bin/qstash.js` shim resolves and execs the real `qstash` binary under both Node 24 and Bun 1.3.x; `stdio: inherit` propagates output and exit codes correctly.
  • Missing subpackage path prints a clear error message, not a stack trace.
  • `QSTASH_CLI_BIN` env override works for local contributor builds.
  • `scripts/build-platform-package.sh` runs end-to-end against all 6 real platform tarballs at v2.37.18 — including Windows zips → `qstash.exe`.
  • Negative control: archive corrupted before verification → `sha256sum -c` rejects with `FAILED`, non-zero exit.
  • Negative control: ELF binary placed in a Mach-O slot → file-type check rejects with clear "expected … got …" message.
  • `bun install` against the current `package.json` exits 0 (placeholder `0.0.0` subpackages 404 silently, as expected for optional deps).
  • Workflow YAML parses; every `uses:` ends in a 40-char SHA + version comment; all SHAs round-tripped through the GitHub API as matching their claimed tags; `needs:` chain and artifact name pairs are consistent.

Needs CI verification post-merge:

  • Cut a canary release tag; confirm all 7 packages (6 platforms + wrapper) publish with provenance.
  • `bunx @upstash/qstash-cli@canary dev --help` on darwin/arm64, linux/x64, win32/x64.
  • `npm view @upstash/qstash-cli-darwin-arm64@canary --json | jq .dist.attestations` shows the sigstore attestation.

Backwards compatibility

  • The wrapper package name (`@upstash/qstash-cli`), the `bin` entry names (`qstash`, `qstash-cli`, `@upstash/qstash-cli`), and the CLI surface are all unchanged.
  • `npx @upstash/qstash-cli dev` keeps working exactly as before.
  • Existing global installs (`npm install -g`) work the same way once they update.
  • The release tag format is unchanged.
  • Six new npm package names need to be claimed once before the first release using this workflow (`@upstash/qstash-cli--` for each combo). `npm publish --provenance` against a never-published name from this org will create them.

Out-of-PR follow-ups (maintainer choices)

These would tighten things further but are repo-settings or org-level decisions:

  1. GitHub Environment with required reviewers on the publishing jobs (`environment: npm-publish`) — adds manual approval for prod releases.
  2. `SECURITY.md` describing the supply-chain model (npm provenance + checksums verification + SHA-pinned actions).
  3. Long-term: sign `qstash-server__checksums.txt` with cosign or minisign on the upstash side, and verify the signature in `scripts/build-platform-package.sh`. Closes the last remaining "coordinated swap of binary + manifest" attack against `artifacts.upstash.com`.

Disclosure

This PR was developed with Claude Code (Opus 4.7) assistance. The commit messages carry `Co-Authored-By` trailers reflecting this. All changes were reviewed and the verification steps above were executed before push.

tombeckenham and others added 3 commits May 15, 2026 13:28
Replace the postinstall-download model with the standard
optionalDependencies + os/cpu pattern (esbuild / swc / turbo / biome).
The qstash binary ships inside per-platform npm subpackages instead of
being fetched from artifacts.upstash.com at install time.

Why this matters for bunx: Bun skips lifecycle scripts by default for
untrusted dependencies, so the existing postinstall never ran under
`bunx @upstash/qstash-cli` and the CLI silently did nothing. Asking
users for a `--trust` flag would weaken bunx's security model. This
change removes the postinstall entirely, so bunx works without any
opt-in flags.

Security improvements:
- No postinstall script (removes a whole class of supply-chain risk).
- npm provenance attestation now covers the actual Go binary, not just
  the JS downloader; trust chain is npm + GitHub OIDC only.
- No network access at install or run time.
- Compatible with `--ignore-scripts`, pnpm strict mode, bun defaults.

Layout:
- `@upstash/qstash-cli` is now a tiny JS wrapper that resolves the
  matching `@upstash/qstash-cli-<platform>-<arch>` subpackage via
  `require.resolve` and spawns the binary with stdio inherit.
- Each platform subpackage carries one Go binary and is constrained by
  `os` + `cpu` so unrelated platforms are skipped during install.
- The release workflow downloads platform binaries from the existing
  artifact host, assembles each subpackage, and publishes all 7
  packages (6 platforms + wrapper) with `npm publish --provenance`.
- Local contributors can override binary resolution with the
  `QSTASH_CLI_BIN` env var.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…line

Tightens the release pipeline against several realistic failure modes
the original change deferred:

- scripts/build-platform-package.sh now downloads
  qstash-server_<VERSION>_checksums.txt alongside each archive and
  verifies the archive's SHA-256 with sha256sum/shasum. The manifest
  must contain a line matching the archive filename — otherwise the
  script aborts. Closes the asymmetric-swap attack where the binary is
  replaced but the manifest is left intact.
- After extraction, the script sanity-checks the binary's file-type
  against the platform/arch slot it is being packaged into (Mach-O /
  ELF / PE32+ family plus arch). Defends against the wrong binary
  landing in the wrong slot.
- tar extraction now uses --no-same-owner --no-same-permissions.
- .github/workflows/release.yml is restructured into three jobs:
  build -> publish-platform -> publish-wrapper. The build job
  validates (download + checksum + magic-byte) all six platforms
  before any publish runs, so a verification failure no longer leaves
  orphan publishes on npm.
- Every actions/* use is pinned to a 40-char commit SHA with a version
  comment, blocking the tag-rewrite class of supply-chain attacks.
- Workflow-level concurrency group prevents two simultaneous releases
  for the same tag from racing.
- publish-wrapper polls npm with backoff for each optionalDependency
  before publishing the wrapper, closing the registry-propagation gap
  where users installing in the propagation window would land in the
  shim's "could not locate subpackage" error.

Negative-control smoke tests confirmed: corrupted archive is rejected
by sha256sum -c; ELF binary placed in a Mach-O slot is rejected by
the file-type check. All six happy-path builds still succeed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- actions/checkout         v4.3.1 -> v6.0.2
- actions/setup-node       v4.4.0 -> v6.4.0
- actions/upload-artifact  v4.6.2 -> v7.0.1
- actions/download-artifact v4.3.0 -> v8.0.1

Each SHA verified against the GitHub API as the head of the claimed
tag. Pinning to commit SHA still provides the supply-chain protection
regardless of version freshness.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@tombeckenham
Copy link
Copy Markdown
Author

Closing in favor of #17 (@smorimoto) — same architectural approach, opened 9 days earlier; I missed it before opening this. The security-hardening from this branch is posted as suggestions on #17.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant