From 576d549691c13d7f499136d41cf0ac3c16687dd6 Mon Sep 17 00:00:00 2001 From: "Joseph T. French" Date: Wed, 14 Oct 2026 20:52:44 -0500 Subject: [PATCH] chore(ci): publish from the release run instead of a release-branch push The publish job now lives in create-release.yml, after the tag and under the production environment, so the approval prompt lands on the run being watched. A failed publish is retried with "Re-run failed jobs" on that run; the version and tag are already cut. PyPI's trusted publisher must name create-release.yml. --- .claude/commands/create-pr.md | 6 +- .claude/commands/publish.md | 20 ++++--- .claude/commands/release-notes.md | 2 +- .claude/commands/staged-review.md | 2 +- .github/workflows/create-release.yml | 84 ++++++++++++++++++++++++++-- .github/workflows/publish.yml | 64 --------------------- 6 files changed, 94 insertions(+), 84 deletions(-) delete mode 100644 .github/workflows/publish.yml diff --git a/.claude/commands/create-pr.md b/.claude/commands/create-pr.md index 7d21c0ed..e9276d95 100644 --- a/.claude/commands/create-pr.md +++ b/.claude/commands/create-pr.md @@ -11,7 +11,7 @@ The previous flow outsourced PR-description authoring to a GitHub Action that on This is `robosystems-client` — a **published, post-1.0 Python SDK**, largely generated from the RoboSystems API's OpenAPI document and GraphQL schema. Its description is read by integrators deciding whether an upgrade is safe, so precision about the public surface matters more here than in an application repo. -**This repository is public.** The PR title and body are world-readable the moment they're pushed — and, because publishing is triggered by a push to `release/**` rather than by a merge, the text is often public well before the version that carries it. Treat the description as a publication. +**This repository is public.** The PR title and body are world-readable the moment they're pushed — and, because publishing happens only when a release is dispatched rather than on merge, the text is often public well before the version that carries it. Treat the description as a publication. ## Instructions @@ -26,7 +26,7 @@ TARGET=${1:-main} # override target via the first argument ``` - **Never PR from the default branch.** If `CURRENT` is `main` (or `master`/`staging`), stop and tell the user to switch to a feature branch first. New branches are created via `just create-feature `, not by hand. -- **Never target a release branch.** `release/**` is what `publish.yml` watches; a PR into one is a publish trigger, not a code review. Target `main`. +- **Never target a release branch.** `release/` is a cut record that `create-release.yml` tags and publishes from; a PR into one changes nothing on `main` and is not a code review. Target `main`. - **Source ≠ target.** If `CURRENT == TARGET`, stop. - **Uncommitted changes.** Run `git status --porcelain`. If there are uncommitted/staged changes, surface them and ask whether to commit them (respecting the repo's commit rules — never on `main`, stage files by name, no `git add -A`) or proceed without them. The PR description must reflect committed state. - **Existing PR.** Check `gh pr list --head "$CURRENT" --base "$TARGET" --json url,number`. If a PR already exists, do **not** create a duplicate — offer to update its title/body with `gh pr edit` instead. @@ -66,7 +66,7 @@ This is the whole point — ground the description in what actually happened: A regeneration is not automatically additive: an API schema change can narrow a type or drop a field, and that reaches consumers as a break even though no hand-written line changed. Diff the emitted types before classifying. -- **Version and publish are not this PR's job.** `create-release.yml` bumps the version on `main` and cuts `release/`; the push to that branch is what triggers `publish.yml`. Never bump the version in `pyproject.toml` in a feature PR and never imply the PR publishes anything. +- **Version and publish are not this PR's job.** `create-release.yml` bumps the version on `main` and cuts `release/`, tags it, and publishes from the tag in the same run. Never bump the version in `pyproject.toml` in a feature PR and never imply the PR publishes anything. - **Security-fix disclosure.** If the PR fixes a security issue, the prose is often _more_ actionable than the diff — keep it terse and non-actionable. Name the area hardened, never the mechanism. No exploit mechanics, attack scenarios, endpoint enumerations, or payloads. For coordinated disclosure use a private GitHub Security Advisory, never a public issue. diff --git a/.claude/commands/publish.md b/.claude/commands/publish.md index a7b147de..736d2a01 100644 --- a/.claude/commands/publish.md +++ b/.claude/commands/publish.md @@ -7,12 +7,13 @@ Monitor a release and publish run — pinpoint why it failed, and confirm the ve ## How a release actually happens here -Two workflows, and the trigger between them is the part that surprises people: +One workflow, `create-release.yml` (`workflow_dispatch`, or `bin/create-release.sh`), with three jobs in sequence: -1. **`create-release.yml`** (`workflow_dispatch`, or `bin/create-release.sh`) — reads the current version from `pyproject.toml`, computes the next one from the requested bump, commits the bump **to `main`**, cuts `release/` from that commit, and tags it. -2. **`publish.yml`** — triggered by **a push to `release/**`**, not by a merge and not by the tag. It reads the version from `pyproject.toml`, builds the distribution, checks the PyPI JSON API for that version, and if it isn't there, publishes over OIDC trusted publishing. +1. **`create-release`** — reads the current version from `pyproject.toml`, computes the next one from the requested bump, commits the bump **to `main`**, and cuts `release/` from that commit. +2. **`create-tag`** — `tag-release.yml` tags that branch and cuts the GitHub Release. +3. **`publish`** — runs once the tag exists, under the `production` environment, so its required-reviewer prompt lands on this same run. It checks out the tag, verifies the version is the one just cut, builds the distribution, checks the PyPI JSON API for that version, and if it isn't there, publishes over OIDC trusted publishing. PyPI's trusted publisher is registered against `create-release.yml` and the `production` environment. -So: **merging a PR to `main` publishes nothing.** The release branch push is the publishing event. And because `publish.yml` short-circuits when the version already exists on PyPI, a re-run of a successful publish is a no-op rather than an error — useful, but it also means "the run went green" is not by itself proof that _this_ run published anything. Note the build step runs *before* the existence check, so a green build tells you nothing about whether a publish happened. +So: **merging a PR to `main` publishes nothing.** Dispatching the workflow is the publishing event, and nothing uploads until the deployment is approved on the run page. A failed publish is retried with **Re-run failed jobs** on that run — the version and tag are already cut, and a new release is warranted only when the code itself is at fault. The job refuses a version that already exists on PyPI, so a re-run after a successful upload fails at the existence check rather than uploading twice. Note the build step runs *before* that check, so a green build tells you nothing about whether a publish happened. `tag-release.yml` writes the GitHub release body separately; see `/release-notes` for the curated-notes override. @@ -26,7 +27,6 @@ So: **merging a PR to `main` publishes nothing.** The release branch push is the ## 1. Find the run ```bash -gh run list --workflow=publish.yml --limit 5 gh run list --workflow=create-release.yml --limit 5 gh run view gh run watch # live, if it's in flight @@ -42,9 +42,11 @@ Classify by stage: - **`create-release.yml` — branch already exists.** The workflow checks for `release/` before creating it. A failure here usually means a previous run got partway, and the fix is to resolve the leftover branch, not to re-dispatch blindly. - **`create-release.yml` — push to `main` rejected.** The version bump commits directly to a protected branch and needs `ACTIONS_TOKEN`; a permissions failure here looks like an auth error at the push step. -- **`publish.yml` — "version exists on PyPI".** Not a failure. The upload step is skipped by condition. Read it as "nothing to do," and if you expected a publish, the version wasn't bumped. -- **`publish.yml` — build.** `pip install build twine` then `python -m build`. A build failure here is a packaging problem — usually `pyproject.toml` metadata or a missing file — that the test suite does not cover, since `just test-all` never builds a distribution. -- **`publish.yml` — the upload.** OIDC trusted publishing. Failures are usually the PyPI-side trusted-publisher configuration (environment or workflow name mismatch) rather than anything in the code. +- **`publish` — waiting.** The job is paused on the `production` environment's required reviewer. Nothing is wrong; approve it from the run page (_Review deployments_). +- **`publish` — "checked out version X, expected Y".** The tag the job checked out does not carry the version this run cut. Nothing was uploaded; look at what `create-tag` tagged. +- **`publish` — "version exists on PyPI".** The job fails at the existence check and uploads nothing. After a successful publish this is the expected result of a re-run; otherwise the version wasn't bumped. +- **`publish` — build.** `pip install build twine` then `python -m build`. A build failure here is a packaging problem — usually `pyproject.toml` metadata or a missing file — that the test suite does not cover, since `just test-all` never builds a distribution. +- **`publish` — the upload.** OIDC trusted publishing. Failures are usually the PyPI-side trusted-publisher configuration (the publisher must name `create-release.yml` and the `production` environment) rather than anything in the code. ## 3. Verify it actually landed @@ -68,6 +70,6 @@ If the version is a major, downstream consumers need coordinated adoption — sa ## Output -A short status: which workflow, what failed and at which step, the root cause, the re-run link if any, and the verified published version from PyPI. If nothing failed, say so — don't manufacture work. +A short status: which job, what failed and at which step, the root cause, the re-run link if any, and the verified published version from PyPI. If nothing failed, say so — don't manufacture work. $ARGUMENTS diff --git a/.claude/commands/release-notes.md b/.claude/commands/release-notes.md index 47b65740..fb906bf3 100644 --- a/.claude/commands/release-notes.md +++ b/.claude/commands/release-notes.md @@ -60,6 +60,6 @@ Write `.github/release-notes/v.md` — **body only**: ### 6. Hand off — sequencing matters -The file must exist **at the tagged ref**, and there is no window to add it late: `create-release.yml` bumps the version on `main`, cuts `release/` from the result, and tags it in the same run. Pushing that release branch is also what triggers `publish.yml`, so by the time the package is on PyPI the notes are already fixed. They have to be **merged into `main` before the workflow is dispatched**. +The file must exist **at the tagged ref**, and there is no window to add it late: `create-release.yml` bumps the version on `main`, cuts `release/` from the result, and tags it in the same run. The same run then publishes from that tag, so by the time the package is on PyPI the notes are already fixed. They have to be **merged into `main` before the workflow is dispatched**. Write the draft on a feature branch (created via `just create-feature`), never on `main`. Present it for review and leave the merge and the dispatch to the user. diff --git a/.claude/commands/staged-review.md b/.claude/commands/staged-review.md index b1d98238..51cffe63 100644 --- a/.claude/commands/staged-review.md +++ b/.claude/commands/staged-review.md @@ -47,7 +47,7 @@ Post-1.0, the emitted type surface **is** the contract. For anything staged here ## Packaging - Changes to `pyproject.toml` — dependency pins, optional extras, `requires-python`, included packages, `py.typed` — change **what ships and who can install it**. A dropped `py.typed` silently turns a typed SDK into an untyped one for every consumer; a narrowed `requires-python` locks out supported versions (3.11–3.13). -- Never stage a version bump in `pyproject.toml` in a feature branch: `create-release.yml` owns the bump on `main`, and pushing `release/**` is what triggers `publish.yml`. +- Never stage a version bump in `pyproject.toml` in a feature branch: `create-release.yml` owns the bump on `main` and publishes from the tag it cuts. ## Testing diff --git a/.github/workflows/create-release.yml b/.github/workflows/create-release.yml index 5820074c..3c57d52a 100644 --- a/.github/workflows/create-release.yml +++ b/.github/workflows/create-release.yml @@ -136,8 +136,79 @@ jobs: project_kind: 'Python SDK' secrets: inherit - create-summary: + # PyPI's trusted publisher is pinned to this workflow file and the production + # environment, whose protection rules (required reviewer, deployment + # branches) gate every publish. The approval prompt lands on this run. + publish: needs: [create-release, create-tag] + runs-on: ubuntu-24.04 + environment: production + + permissions: + contents: read + id-token: write # For PyPI trusted publishing + + steps: + - name: Checkout release tag + uses: actions/checkout@v7 + with: + ref: ${{ needs.create-tag.outputs.tag_name }} + + - name: Setup Python + uses: actions/setup-python@v7 + with: + python-version: "3.13" + + - name: Install dependencies + run: | + python -m pip install --upgrade pip + pip install build twine + + - name: Get current version + id: current-version + run: | + CURRENT_VERSION=$(python -c "import tomllib; print(tomllib.load(open('pyproject.toml', 'rb'))['project']['version'])") + echo "current_version=$CURRENT_VERSION" >> $GITHUB_OUTPUT + echo "Current version: $CURRENT_VERSION" + + - name: Verify the checkout is the released version + run: | + EXPECTED="${{ needs.create-release.outputs.new_version }}" + ACTUAL="${{ steps.current-version.outputs.current_version }}" + + if [ "$ACTUAL" != "$EXPECTED" ]; then + echo "❌ Checked out version $ACTUAL, expected $EXPECTED" + exit 1 + fi + echo "✅ Checked out the released version $ACTUAL" + + - name: Build package + run: python -m build + + - name: Check if version exists on PyPI + id: check-pypi + run: | + VERSION="${{ steps.current-version.outputs.current_version }}" + PACKAGE_NAME="robosystems-client" + + # Check if version exists on PyPI + if curl -s "https://pypi.org/pypi/${PACKAGE_NAME}/${VERSION}/json" | grep -q "\"version\""; then + echo "❌ Version ${VERSION} already exists on PyPI" + echo "version_exists=true" >> $GITHUB_OUTPUT + exit 1 + else + echo "✅ Version ${VERSION} does not exist on PyPI, safe to publish" + echo "version_exists=false" >> $GITHUB_OUTPUT + fi + + - name: Publish to PyPI + if: steps.check-pypi.outputs.version_exists == 'false' + uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2 + with: + verbose: true + + create-summary: + needs: [create-release, create-tag, publish] if: always() runs-on: ubuntu-24.04 permissions: @@ -151,10 +222,11 @@ jobs: echo "**Tag:** \`${{ needs.create-tag.outputs.tag_name }}\`" >> $GITHUB_STEP_SUMMARY echo "**Branch:** \`${{ needs.create-release.outputs.branch_name }}\`" >> $GITHUB_STEP_SUMMARY echo "**Type:** ${{ inputs.version_type }}" >> $GITHUB_STEP_SUMMARY + echo "**Publish:** ${{ needs.publish.result }}" >> $GITHUB_STEP_SUMMARY echo "" >> $GITHUB_STEP_SUMMARY echo "**Release URL:** ${{ needs.create-tag.outputs.release_url }}" >> $GITHUB_STEP_SUMMARY - echo "" >> $GITHUB_STEP_SUMMARY - echo "### Next Steps" >> $GITHUB_STEP_SUMMARY - echo "The PyPI package will be automatically published when the release tag is created." >> $GITHUB_STEP_SUMMARY - echo "" >> $GITHUB_STEP_SUMMARY - echo "Monitor the publish workflow: [View Workflows](https://github.com/RoboFinSystems/robosystems-python-client/actions/workflows/publish.yml)" >> $GITHUB_STEP_SUMMARY + + if [ "${{ needs.publish.result }}" != "success" ]; then + echo "" >> $GITHUB_STEP_SUMMARY + echo "The version and tag are cut but the package is not on PyPI. Fix the cause, then **Re-run failed jobs** on this run; a new release is only needed when the code itself is at fault." >> $GITHUB_STEP_SUMMARY + fi diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml deleted file mode 100644 index 0205f37c..00000000 --- a/.github/workflows/publish.yml +++ /dev/null @@ -1,64 +0,0 @@ -name: Publish to PyPI - -on: - push: - branches: - - 'release/**' - workflow_dispatch: - -jobs: - publish: - runs-on: ubuntu-24.04 - # The production environment's protection rules (required reviewer, - # deployment branches) gate every publish here. - environment: production - - permissions: - contents: read - id-token: write # For PyPI trusted publishing - - steps: - - name: Checkout repository - uses: actions/checkout@v7 - - - name: Setup Python - uses: actions/setup-python@v7 - with: - python-version: "3.13" - - - name: Install dependencies - run: | - python -m pip install --upgrade pip - pip install build twine - - - name: Get current version - id: current-version - run: | - CURRENT_VERSION=$(python -c "import tomllib; print(tomllib.load(open('pyproject.toml', 'rb'))['project']['version'])") - echo "current_version=$CURRENT_VERSION" >> $GITHUB_OUTPUT - echo "Current version: $CURRENT_VERSION" - - - name: Build package - run: python -m build - - - name: Check if version exists on PyPI - id: check-pypi - run: | - VERSION="${{ steps.current-version.outputs.current_version }}" - PACKAGE_NAME="robosystems-client" - - # Check if version exists on PyPI - if curl -s "https://pypi.org/pypi/${PACKAGE_NAME}/${VERSION}/json" | grep -q "\"version\""; then - echo "❌ Version ${VERSION} already exists on PyPI" - echo "version_exists=true" >> $GITHUB_OUTPUT - exit 1 - else - echo "✅ Version ${VERSION} does not exist on PyPI, safe to publish" - echo "version_exists=false" >> $GITHUB_OUTPUT - fi - - - name: Publish to PyPI - if: steps.check-pypi.outputs.version_exists == 'false' - uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33 # v1.14.2 - with: - verbose: true