diff --git a/.claude/knowledge/deployment-release.md b/.claude/knowledge/deployment-release.md index a612839..0d9f818 100644 --- a/.claude/knowledge/deployment-release.md +++ b/.claude/knowledge/deployment-release.md @@ -11,6 +11,9 @@ How code gets to production. Release processes, environment promotion, rollback ## npm + PyPI packaging (packaging/) - npm/PyPI ship thin wrappers around the prebuilt GoReleaser binaries — no Go toolchain at install time. `packaging/extract-binaries.sh` downloads release assets, `npm/build.sh` and `pypi/build_wheels.py` assemble artifacts. `publish-npm`/`publish-pypi` jobs in `release.yml` run for stable releases only. See `packaging/README.md`. -- **GOTCHA — trusted publishing + reusable workflow:** the publish jobs live in `release.yml`, but npm and PyPI OIDC trusted publishing match the **top-level *calling* workflow**, not the reusable one. So the trusted publisher on npmjs.com / PyPI must be configured with workflow filename **`cd.yml`**, NOT `release.yml`. PyPI additionally forbids naming a reusable workflow as the publisher outright (warehouse#11096). `id-token: write` must be present on both the `cd.yml` caller jobs (it is) and the `release.yml` publish jobs. +- **Publish jobs (`publish-npm`, `publish-pypi`) live in `cd.yml`, NOT in the reusable `release.yml`.** Both trusted publishers (npm + PyPI) are configured with workflow filename **`cd.yml`**. They run after `stable-release` (`needs: [release-please, stable-release]`, gated on `release_created == 'true'`) and download the just-built binaries from the GitHub release via `extract-binaries.sh`. +- **WHY they're in cd.yml, not release.yml (the whole saga):** OIDC trusted publishing matches a workflow filename, and PEP 740 attestations also embed it in the signing cert's Build Config URI (OID 1.3.6.1.4.1.57264.1.18). When the publish step runs inside a *reusable* workflow, the two OIDC claims split: `workflow_ref` = caller (`cd.yml`), `job_workflow_ref` = reusable file (`release.yml`). PyPI auth matches `job_workflow_ref` (release.yml) but the attestation Build Config URI is `workflow_ref` (cd.yml) — no single publisher satisfies both, so PyPI rejects with `400 Invalid attestations ... Build Config URI (...cd.yml...) does not match expected Trusted Publisher (release.yml ...)`. Running the publish jobs directly in `cd.yml` collapses both claims to `cd.yml`, so one publisher works for both registries with attestations on. (This is why szr "just worked": szr's `release.yml` is triggered directly by the tag push, so it *is* the entry workflow — both claims are `release.yml`. codeguard's `release.yml` is reusable-only, so the entry workflow is `cd.yml`.) +- Debug tip: on `invalid-publisher` or `400 Invalid attestations`, the pypi-publish action prints the actual OIDC claims — compare `workflow_ref`/`job_workflow_ref` to the configured publisher. +- Note: manual `workflow_dispatch` of `release.yml` (stable) does NOT publish to npm/PyPI — publishing only happens through the release-please → `cd.yml` path. cd.yml blocks manual stable runs (`guard-manual-stable`) anyway. - First automated publish needs one-time manual bootstrap: `packaging/npm/bootstrap-publish.sh` (npm requires a package to exist before a trusted publisher can be added) and a PyPI *pending publisher* for project `devr-codeguard`. Both are documented step-by-step in `packaging/README.md`. - **PyPI project name is `devr-codeguard`, NOT `codeguard`** (the plain name was already taken on PyPI). The npm scope `@devr-tools/codeguard` is unaffected. The installed command is still `codeguard` regardless — in `build_wheels.py`, `PROJECT`/`DIST` (distribution name, hyphen vs. PEP 427 escaped `devr_codeguard`) are decoupled from `BIN`/the data-script name (`codeguard`), so `pip install devr-codeguard` yields a `codeguard` command. diff --git a/.github/workflows/cd.yml b/.github/workflows/cd.yml index 12ca299..d06bee5 100644 --- a/.github/workflows/cd.yml +++ b/.github/workflows/cd.yml @@ -165,3 +165,132 @@ jobs: create_missing_tag: true secrets: RELEASE_PLEASE_TOKEN: ${{ secrets.RELEASE_PLEASE_TOKEN }} + + # npm/PyPI publishing lives here in cd.yml (the top-level entry workflow), NOT + # in the reusable release.yml. Running here means the OIDC workflow_ref and + # job_workflow_ref claims both resolve to cd.yml, so a single trusted publisher + # (cd.yml) satisfies both auth and PEP 740 attestation verification. Publishing + # from the reusable release.yml splits those claims (workflow_ref=cd.yml, + # job_workflow_ref=release.yml) and PyPI rejects the attestation. Both jobs run + # only after a stable release-please release, and pull the built binaries from + # the GitHub release that stable-release just created. + publish-npm: + needs: [release-please, stable-release] + if: needs.release-please.outputs.release_created == 'true' + runs-on: ubuntu-latest + permissions: + contents: read + id-token: write # npm trusted publishing (OIDC) — no NPM_TOKEN needed. + steps: + - name: Check out code + uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7 + + - name: Set up Node.js + uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4 + with: + node-version: "22" + + - name: Upgrade npm for trusted publishing + # Trusted publishing (OIDC) requires npm >= 11.5.1; the runner ships an + # older npm by default. + run: npm install -g npm@latest + + - name: Resolve package version + id: version + shell: bash + env: + RELEASE_TAG: ${{ needs.release-please.outputs.tag_name }} + run: | + set -euo pipefail + if [[ "$RELEASE_TAG" == *-v* ]]; then + version="${RELEASE_TAG##*-v}" + else + version="${RELEASE_TAG#v}" + fi + echo "version=$version" >>"$GITHUB_OUTPUT" + + - name: Stage release binaries + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + set -euo pipefail + ./packaging/extract-binaries.sh "${{ needs.release-please.outputs.tag_name }}" "${{ steps.version.outputs.version }}" + + - name: Build npm packages + run: ./packaging/npm/build.sh "${{ steps.version.outputs.version }}" + + - name: Publish to npm + # Authentication is via OIDC trusted publishing (id-token: write above); + # no NODE_AUTH_TOKEN/.npmrc. Provenance is attached automatically. + shell: bash + run: | + set -euo pipefail + + publish_dir() { + local dir="$1" + local name ver + name="$(node -p "require('./$dir/package.json').name")" + ver="$(node -p "require('./$dir/package.json').version")" + if npm view "$name@$ver" version >/dev/null 2>&1; then + echo "$name@$ver already published, skipping" + else + npm publish "$dir" --access public + fi + } + + # Platform packages first so the launcher's optional deps resolve. + for dir in packaging/npm/dist/codeguard-*/package; do + publish_dir "$dir" + done + publish_dir packaging/npm/dist/codeguard/package + + publish-pypi: + needs: [release-please, stable-release] + if: needs.release-please.outputs.release_created == 'true' + runs-on: ubuntu-latest + permissions: + contents: read + id-token: write # PyPI trusted publishing (OIDC) — no PYPI_API_TOKEN needed. + steps: + - name: Check out code + uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7 + + - name: Set up Python + uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5 + with: + python-version: "3.12" + + - name: Resolve package version + id: version + shell: bash + env: + RELEASE_TAG: ${{ needs.release-please.outputs.tag_name }} + run: | + set -euo pipefail + if [[ "$RELEASE_TAG" == *-v* ]]; then + version="${RELEASE_TAG##*-v}" + else + version="${RELEASE_TAG#v}" + fi + echo "version=$version" >>"$GITHUB_OUTPUT" + + - name: Stage release binaries + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + set -euo pipefail + ./packaging/extract-binaries.sh "${{ needs.release-please.outputs.tag_name }}" "${{ steps.version.outputs.version }}" + + - name: Build wheels + run: | + set -euo pipefail + python3 packaging/pypi/build_wheels.py "${{ steps.version.outputs.version }}" packaging/.staging packaging/pypi/dist + + - name: Publish to PyPI + # Attestations stay on (the default): running in cd.yml makes both OIDC + # claims resolve to cd.yml, so the PEP 740 attestation matches the cd.yml + # trusted publisher. + uses: pypa/gh-action-pypi-publish@cef221092ed1bacb1cc03d23a2d87d1d172e277b # v1 + with: + packages-dir: packaging/pypi/dist + skip-existing: true diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index d9adf95..e5c1768 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -409,125 +409,3 @@ jobs: echo "- SHA256: `$FORMULA_SHA256`" echo "- Suggested branch: `$FORMULA_BRANCH`" } >>"$GITHUB_STEP_SUMMARY" - - publish-npm: - needs: build-release - if: needs.build-release.outputs.prerelease != 'true' - runs-on: ubuntu-latest - permissions: - contents: read - # Required for npm trusted publishing (OIDC) — no NPM_TOKEN needed. - # Matches the top-level caller (cd.yml) configured as the trusted - # publisher; the id-token: write on cd.yml's stable-release job propagates - # here because release.yml is called from it. - id-token: write - steps: - - name: Check out code - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7 - - - name: Set up Node.js - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4 - with: - node-version: "22" - - - name: Upgrade npm for trusted publishing - # Trusted publishing (OIDC) requires npm >= 11.5.1; the runner ships an - # older npm by default. - run: npm install -g npm@latest - - - name: Resolve package version - id: version - shell: bash - env: - RELEASE_TAG: ${{ needs.build-release.outputs.tag }} - run: | - set -euo pipefail - if [[ "$RELEASE_TAG" == *-v* ]]; then - version="${RELEASE_TAG##*-v}" - else - version="${RELEASE_TAG#v}" - fi - echo "version=$version" >>"$GITHUB_OUTPUT" - - - name: Stage release binaries - env: - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - run: | - set -euo pipefail - ./packaging/extract-binaries.sh "${{ needs.build-release.outputs.tag }}" "${{ steps.version.outputs.version }}" - - - name: Build npm packages - run: ./packaging/npm/build.sh "${{ steps.version.outputs.version }}" - - - name: Publish to npm - # Authentication is via OIDC trusted publishing (id-token: write above); - # no NODE_AUTH_TOKEN/.npmrc. Provenance is attached automatically. - shell: bash - run: | - set -euo pipefail - - publish_dir() { - local dir="$1" - local name ver - name="$(node -p "require('./$dir/package.json').name")" - ver="$(node -p "require('./$dir/package.json').version")" - if npm view "$name@$ver" version >/dev/null 2>&1; then - echo "$name@$ver already published, skipping" - else - npm publish "$dir" --access public - fi - } - - # Platform packages first so the launcher's optional deps resolve. - for dir in packaging/npm/dist/codeguard-*/package; do - publish_dir "$dir" - done - publish_dir packaging/npm/dist/codeguard/package - - publish-pypi: - needs: build-release - if: needs.build-release.outputs.prerelease != 'true' - runs-on: ubuntu-latest - permissions: - contents: read - id-token: write # PyPI trusted publishing (OIDC) — no PYPI_API_TOKEN needed. - steps: - - name: Check out code - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7 - - - name: Set up Python - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5 - with: - python-version: "3.12" - - - name: Resolve package version - id: version - shell: bash - env: - RELEASE_TAG: ${{ needs.build-release.outputs.tag }} - run: | - set -euo pipefail - if [[ "$RELEASE_TAG" == *-v* ]]; then - version="${RELEASE_TAG##*-v}" - else - version="${RELEASE_TAG#v}" - fi - echo "version=$version" >>"$GITHUB_OUTPUT" - - - name: Stage release binaries - env: - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - run: | - set -euo pipefail - ./packaging/extract-binaries.sh "${{ needs.build-release.outputs.tag }}" "${{ steps.version.outputs.version }}" - - - name: Build wheels - run: | - set -euo pipefail - python3 packaging/pypi/build_wheels.py "${{ steps.version.outputs.version }}" packaging/.staging packaging/pypi/dist - - - name: Publish to PyPI - uses: pypa/gh-action-pypi-publish@cef221092ed1bacb1cc03d23a2d87d1d172e277b # v1 - with: - packages-dir: packaging/pypi/dist - skip-existing: true diff --git a/packaging/README.md b/packaging/README.md index 6baa887..7637bc8 100644 --- a/packaging/README.md +++ b/packaging/README.md @@ -64,14 +64,28 @@ Unlike a repo that triggers `release.yml` directly on a tag push, codeguard's push to main -> cd.yml (release-please) -> stable-release job -> release.yml ``` -The `publish-npm` and `publish-pypi` jobs live inside `release.yml` (where the -release binaries are built), but **npm and PyPI trusted publishing match the -top-level *calling* workflow, not the reusable one.** So the trusted publisher -must be configured against **`cd.yml`**, and `id-token: write` is set on both the -`cd.yml` caller jobs and the `release.yml` publish jobs. - -> PyPI does not allow a *reusable* workflow to be named as the trusted publisher -> (warehouse#11096); naming the `cd.yml` caller is the supported path. +The `publish-npm` and `publish-pypi` jobs live in **`cd.yml`** (the top-level +entry workflow), NOT in the reusable `release.yml`. They run after the +`stable-release` job (which builds the binaries and cuts the GitHub release) and +pull those binaries back down to assemble the packages. + +**Why cd.yml and not release.yml:** OIDC trusted publishing matches the workflow +filename, and PEP 740 attestations additionally embed it in the signing +certificate's Build Config URI. When a publish step runs inside a *reusable* +workflow, the two OIDC claims split — `workflow_ref` = the caller (`cd.yml`), +`job_workflow_ref` = the reusable file (`release.yml`) — and no single publisher +can satisfy both: auth matches `job_workflow_ref` (release.yml) but the +attestation's Build Config URI comes from `workflow_ref` (cd.yml), so PyPI +rejects the upload with `400 Invalid attestations`. Publishing from `cd.yml` +directly collapses both claims to `cd.yml`, so **one publisher (`cd.yml`) works +for both npm and PyPI, with attestations on.** + +> This mirrors how szr works: szr's `release.yml` is triggered directly on the +> tag push (it *is* the entry workflow), so both claims are `release.yml` there. +> codeguard's `release.yml` is reusable-only, so the entry workflow is `cd.yml`. + +Configure both trusted publishers with workflow filename **`cd.yml`**. +`id-token: write` is set on the `cd.yml` publish jobs. ## One-time prerequisites (before the first automated release) @@ -112,7 +126,7 @@ registries use OIDC trusted publishing — no long-lived tokens live in CI. project `devr-codeguard` (the plain `codeguard` name is taken; the installed command is still `codeguard`): - Owner / repo: `devr-tools/codeguard` - - Workflow filename: `cd.yml` ← the caller, not release.yml + - Workflow filename: `cd.yml` ← the publish job lives in cd.yml (same as npm) - Environment: *(leave blank — the job sets none)* This lets the `publish-pypi` job authenticate via `id-token: write` with no diff --git a/packaging/npm/bootstrap-publish.sh b/packaging/npm/bootstrap-publish.sh index aa86889..a0b0eda 100755 --- a/packaging/npm/bootstrap-publish.sh +++ b/packaging/npm/bootstrap-publish.sh @@ -39,9 +39,9 @@ Next: on npmjs.com, open each package's Settings -> Trusted Publisher and add: Environment: (leave blank) Allowed actions: npm publish -Use cd.yml (NOT release.yml): the publish-npm job lives in the reusable -release.yml, but npm/PyPI trusted publishing matches the *top-level calling* -workflow, which is cd.yml (release-please -> stable-release -> release.yml). +Use cd.yml: the publish-npm and publish-pypi jobs both live in cd.yml (the +top-level entry workflow), so a single trusted publisher (cd.yml) works for +both npm and PyPI. See packaging/README.md for why. Packages to configure: @devr-tools/codeguard