Add multi-registry OIDC release pipeline and packaging infrastructure#19
Draft
kjanat wants to merge 3 commits into
Draft
Add multi-registry OIDC release pipeline and packaging infrastructure#19kjanat wants to merge 3 commits into
kjanat wants to merge 3 commits into
Conversation
…I + GitHub Releases)
Adds .github/workflows/release.yml driving a single dual-trigger pipeline:
- `vars.MANUAL` unset (default): tag push builds and publishes in one run.
- `vars.MANUAL` set: tag push builds and creates a DRAFT GitHub Release; the
human clicks Publish, the `release: published` event re-enters the same
workflow file in publish-only mode and fans out to all registries using
cross-run `actions/download-artifact@v4`.
Stable workflow filename `release.yml` is required by all three OIDC
trusted-publisher subject claims.
Composite actions under .github/actions/:
- build-target: cargo / cross / cargo-build-std / cargo-cross-toolchain matrix
builder, including the rustup bootstrap windows-11-arm still needs. Produces
the existing binstall-compatible flat tar.gz/zip layout + SHA256 sidecar.
- pack-npm-platform: pure bash + jq replacement for the missing
npm/scripts/build-packages.ts referenced by the old justfile. Stamps
per-target package.json (name/version/os/cpu/libc/bin) and runs npm pack.
- pack-npm-facade: reuses npm/facade/{bin,lib}/*.cjs unchanged; injects
optionalDependencies for tier-1/tier-2 targets only.
- pack-wheel: thin PyO3/maturin-action wrapper using `--bindings bin` so both
[[bin]] targets ship as wheel scripts (`pip install runner-run`).
pyproject.toml: maturin bin bindings, no PyO3 dep.
Provenance: actions/attest-build-provenance@v3 attests every binary archive,
every npm sub-package tarball, every wheel, the sdist, the .crate, and the
facade. PyPI emits PEP 740 attestations automatically; npm provenance is
emitted automatically via OIDC.
Idempotency: every publish step short-circuits on already-published versions
and retries network failures with exponential backoff. Tier-3 BSD targets are
continue-on-error and excluded from required-builds gate, npm publish list,
and wheel matrix.
justfile: drop the missing build-packages.ts reference; rewrite test-release
to mirror what pack-npm-platform does so local sanity checks remain useful.
Add `just bump <version>` to keep Cargo.toml + pyproject.toml + npm facade
version in lockstep with the tag (verify-versions step in the workflow
fails fast otherwise).
…-script@v9
Composite action.yml files become wiring-only — every multi-line bash block
moves to a sibling pack.sh invoked as `bash $GITHUB_ACTION_PATH/pack.sh`.
This gives proper editor highlighting, shellcheck linting, and lets
justfile's `test-release` source the same scripts CI runs (real local
fidelity instead of duplicated inline bash).
In .github/workflows/release.yml, every block that previously called gh
CLI or iterated structured data flips to actions/github-script@v9:
- plan.matrix: read npm/targets.json via require(), emit derived matrices
with core.setOutput. Replaces jq-on-bash.
- plan.verify-versions: read Cargo.toml + pyproject.toml + facade
package.json, fail with core.setFailed on drift. Replaces awk + jq.
- resolve-publish-run: github.rest.actions.listWorkflowRuns to find the
prior tag-push build run. Replaces gh run list shell pipeline.
- release-assets: github.rest.repos.{getReleaseByTag,createRelease,
deleteReleaseAsset,uploadReleaseAsset} with @actions/glob for asset
discovery and core.summary for the upload table. Replaces gh release
create/upload bash.
- required-builds-passed and summary: small github-script blocks for
consistent error reporting and core.summary tables.
Inline `run:` blocks remain only for ≤ 10-line non-API bash: the mode
selector and the per-publish-step retry loops.
Drop the Windows .zip branch in build-target/pack.sh — both consumers
(action.yml line 113 and install.sh) only fetch .tar.gz, matching the
binstall pkg-fmt = "tgz" pin in Cargo.toml. Git Bash on windows-latest
and windows-11-arm provides tar(1) and sha256sum(1) so a single bash
script handles all three OSes.
Applies the repo's dprint configuration to the pipeline files: - YAML (.github/**): pretty_yaml plugin reflow (printWidth 160). - TOML (pyproject.toml): tombi reflow. - Bash (.github/actions/*/pack.sh): shfmt -i 0 -ln bash -bn -ci. No behavioral change.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces a comprehensive multi-registry release pipeline with OIDC trusted publishing support, along with modular packaging infrastructure for distributing binaries across npm, PyPI, and crates.io registries.
Key Changes
Release Workflow (
release.yml)auto: Build and publish in a single run (default on tag push)draft-only: Build and create a draft release (whenvars.MANUALis set)publish-only: Skip build, fetch artifacts from original tag-push run, publish to registries (triggered byrelease.publishedevent)plan: Determines mode, derives build matrix fromnpm/targets.json, verifies version consistency across Cargo.toml, pyproject.toml, and package.jsonbuild-binaries: Matrix-based compilation for all target triples with build provenance attestationbuild-wheels: Per-target Python wheels (excluding BSD targets)build-sdist-and-crate: Python source distribution and Rust crate packagebuild-npm-facade: Facade package with injected platform sub-package dependenciesrelease-assets: Aggregates artifacts, creates/updates GitHub Release, uploads assetspublish-*: Registry-specific publishing jobs with idempotent retry logicsummary: Rollup status reportingComposite Actions
build-target: Compiles binaries for a single target triple, stages them into a flat tar.gz matching binstall layout, computes SHA256 sidecarpack-npm-platform: Creates per-target npm sub-packages with prebuilt binaries, platform/CPU/libc selectorspack-npm-facade: Stamps facade package with version and optionalDependencies for tier-1/tier-2 targetspack-wheel: Wraps maturin to build Python wheels with both binaries as wheel scriptsConfiguration Files
pyproject.toml: New Python project configuration for maturin-based wheel buildingjustfile: Refactoredtest-releaserecipe to exercise the exact packaging code paths used in CI (calls pack.sh scripts directly)Notable Implementation Details
planjob validates that Cargo.toml, pyproject.toml, and npm/facade/package.json all declare the same version before proceedingpublish-onlymode resolves the original build run ID and downloads artifacts from it, enabling deferred publishingactions/attest-build-provenancepack.shscripts for local testability via justfilehttps://claude.ai/code/session_0113xsqWsztLh8uWjR4radJM