Skip to content

ci: test and smoke-pack before releasing, push only after publish - #59

Closed
dinwwwh wants to merge 1 commit into
mainfrom
claude/intelligent-cray-6jmbq9
Closed

dinwwwh wants to merge 1 commit into
mainfrom
claude/intelligent-cray-6jmbq9

Conversation

@dinwwwh

@dinwwwh dinwwwh commented Oct 20, 2026

Copy link
Copy Markdown
Member

Why

release.yaml could publish code that was never checked or tested:

  • It never ran pnpm run check or pnpm test.
  • With skip_git_check: true it published from any branch, and there was no branch guard otherwise either.
  • bumpp pushed the commit and tag before pnpm publish. A publish that failed left a pushed vX.Y.Z tag with no npm release, and re-running then failed because the tag already existed.
  • The tests run against src (the dev exports), so nothing ever checked the dist tarball that actually ships.

This matters because @orpc/openapi 2.x depends on @openapi-spec/downgrader and @openapi-spec/types with ~0.3.0, so every 0.3.x patch reaches every oRPC install automatically.

New step order

The job now runs only when github.ref == 'refs/heads/main' || inputs.skip_git_check. skip_git_check was already the explicit "publish from elsewhere" switch (--no-git-checks), so it doubles as the opt-out.

# Step What changed
1 Validate version / tag inputs unchanged (still env-based, no inline expressions)
2 checkout, pnpm setup, pnpm i unchanged
3 pnpm run check new
4 pnpm test new
5 git identity unchanged
6 bumpp -r "$VERSION" --yes --no-push now --no-push: commit and tag exist only on the runner
7 pnpm run test:pack new: smoke-tests the real tarballs at the bumped version
8 git push --dry-run --atomic --follow-tags new: fails before publishing if the branch moved since checkout
9 pnpm publish -r --provenance … unchanged
10 git push --atomic --follow-tags new: the push moved here from bumpp
11 changelogithub --draft moved from before publish to last

Notes on the ordering:

  • Nothing reaches GitHub or npm until check, tests and the pack smoke test pass. If any of them fail, the run ends with no tag, no commit and no npm release.
  • A failed publish leaves no tag, so recovering is just re-running the same dispatch. pnpm publish -r skips versions already on npm, so a partly failed publish (say types went out but downgrader didn't) finishes on the re-run.
  • Push is --atomic, so the commit and tag land together or not at all. bumpp's tag is annotated, so --follow-tags carries it.
  • Why a dry-run push (step 8): pnpm's own "remote history differs" check only compares against the local tracking ref, which is never refreshed after checkout. The dry run asks the real remote, so a PR merged to main during check/test fails the run before publishing, instead of leaving an npm release whose tag can't be pushed. It doesn't run server-side hooks, so a branch-protection rejection would still only show up at step 10.
  • Why changelog moved to the end: changelogithub refuses to create a release unless the tag already exists on GitHub (hasTagOnGitHub), so it has to run after the push. It's also the least critical step: before, a GitHub API hiccup there aborted the run after the tag was pushed but before publishing.
  • skip_bump now also skips both push steps, since there's nothing to push.

pnpm run test:pack (scripts/smoke-pack.ts)

No CI pack job exists yet, so this is a standalone script that a future CI job can call too:

  1. pnpm --filter=./packages/* pack into a temp dir. This runs prepack (unbuild), applies publishConfig.exports and resolves workspace:^, the same as publish does.
  2. npm install all the tarballs into a scratch project outside the workspace. They're installed together so downgrader's ^X.Y.Z dependency on types resolves to the local tarball, with no registry lookup for a version that isn't published yet.
  3. Imports every exports entry point at runtime.
  4. Runs tsc (nodenext, strict, skipLibCheck: false) on a consumer that import types every entry point, so the .d.mts files are checked as a consumer like oRPC sees them.

Validation

  • pnpm run check and pnpm test pass locally (197 tests). The pre-commit hook passed on the commit.
  • pnpm run test:pack passes: 6 entry points. Checked that it fails on:
    • a types subpath removed from publishConfig.exports. downgrader's .d.mts breaks with TS2307 while the unit tests still pass, which is exactly the gap this closes.
    • a module that throws at import time.
    • a missing exports target (unbuild already fails that one at prepack).
  • In a scratch clone with a bare remote:
    • bumpp --no-push commits and tags locally and pushes nothing.
    • The dry-run push succeeds when the branch hasn't moved and is rejected (exit 1) when it has.
    • The atomic push lands commit and tag together.
    • pnpm publish -r --dry-run accepts a branch one commit ahead of upstream (the new post-bump state) and still rejects one that's behind.
  • actionlint 1.7.7 + shellcheck 0.10.0: the only finding is the existing SC2086 info on the unchanged publish line. The $NO_GIT_CHECKS word splitting is deliberate (an empty value must disappear), and main reports the same finding.

Possible follow-up (not in this PR)

Check, tests and the smoke test run in the same job that holds contents: write and id-token: write (npm trusted publishing). Splitting them into a read-only verify job that release needs: would keep test-time dependencies away from publish credentials.


Generated by Claude Code

The release workflow published without running checks or tests, from any
branch when skip_git_check was set, and pushed the release tag before
publishing, so a failed publish left a tag with no npm release.

- Run the job only on main unless skip_git_check is set.
- Run `pnpm run check` and `pnpm test` before bumping.
- Bump with `--no-push`, then pack every package, install the tarballs
  outside the workspace, import every exports entry and typecheck a
  consumer of each (`pnpm run test:pack`).
- Dry-run the push, publish, then push the commit and tag atomically.
- Draft the changelog last, since changelogithub needs the tag on GitHub.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CU1qN3CiSyADEfwPinsqCx
@codecov

codecov Bot commented Oct 20, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@codspeed

codspeed Bot commented Oct 20, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 20 untouched benchmarks


Comparing claude/intelligent-cray-6jmbq9 (b0d098b) with main (308cb29)1

Open in CodSpeed

Footnotes

  1. No successful run was found on main (cda4e92) during the generation of this report, so 308cb29 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes

This is the initial pullfrog review of the release-hardening PR (base cda4e92, head b0d098b).

  • release.yaml job gate and step reorder — publishes only from main (or the explicit skip_git_check opt-out), runs check / test / test:pack before publishing, bumps and tags locally with bumpp --no-push, dry-run pushes before publish and pushes atomically after, and moves changelogithub to the end so it runs once the tag is on GitHub.
  • scripts/smoke-pack.ts — packs the workspace tarballs (prepack: unbuild), installs them into a scratch consumer outside the workspace, imports every exports entry point at runtime, and type-checks the built .d.mts as a consumer would see it.
  • package.json — adds the test:pack script.

I verified the load-bearing claims locally: pnpm run test:pack passes against all 6 entry points and pnpm run check passes, both leaving a clean tree; bumpp emits an annotated tag so --follow-tags carries it; and pnpm's recursive publish skips versions already on the registry, so the partial-publish recovery story holds. The credential-sharing and PR-time smoke-test gaps are already scoped as follow-ups in the PR description.

Pullfrog  | View workflow run | Using deepseek-v4.1-flash (free via Pullfrog for OSS) | 𝕏

@dinwwwh dinwwwh closed this Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants