Skip to content

ci: shared tests and release on merge to stable, version from Cargo.toml - #110

Merged
pjhartlieb merged 28 commits into
devfrom
ci/publish-on-tag
Nov 3, 2026
Merged

pjhartlieb merged 28 commits into
devfrom
ci/publish-on-tag

Conversation

@singlerider

Copy link
Copy Markdown
Contributor

What changed

  • Publishing removed from rust-tests.yml and python-tests.yml, which used to publish on every stable push with long-lived tokens. Neither triggers on push: stable any more, so they no longer duplicate the calls from publish.yml.
  • python-tests.yml and rust-tests.yml call CLA's shared workflows. The Python matrix comes from requires-python. ruff format --check now runs, and the extension is built with maturin develop. Rust runs --all-features --locked on the rust-toolchain.toml channel. CODECOV_TOKEN is passed by name. Until the default branch ruleset requires python / passed, a required job reports test (3.11) to (3.15) from the real result.
  • publish.yml runs on a push to stable: CLA's release-check.yml, then rust and python tests, then CLA's maturin-wheels.yml (every interpreter requires-python allows, on Windows and macOS too), then PyPI with OIDC in release, then crates.io through CLA's crates.yml with trusted publishing, then CLA's publish.yml (tag and GitHub Release with SBOM and wheels, created last). No caller passes App credentials.
  • stable-guard.yml: release ready on pull requests into stable refuses a non-MAJOR.MINOR.PATCH version and git or URL dependencies.
  • cla.yml passes APP_ID/APP_PRIVATE_KEY by name. No workflow uses secrets: inherit. benchmark.yml declares contents: read and pull-requests: write and pins its runner.
  • Cargo.toml is the single version source, now 4.3.1: 4.3.0 is on PyPI and crates.io with no v4.3.0 tag.
  • Dead poetry.lock and .flake8 removed. Merges current dev (dependabot grouping, shared auto-merge).
  • Every CLA reference is pinned to CLA#11 at 07e14bcbba5e737e1b29bd9e3f9128fbf2483be9. Re-pin to its merge commit before this merges.

Before promoting to stable

  • release environment with reviewers and a stable-only deployment policy, holding BLS_RELEASE_APP_ID and BLS_RELEASE_PRIVATE_KEY.
  • Trusted publishers on PyPI and crates.io for publish.yml, environment release. Then delete the PyPI and crates.io token secrets.
  • Ruleset protecc (11169658), and the matching .github-config#1/Prune / Defragment branches to save space #2 Terraform: require python / passed, rust / passed and release ready, then drop the required job.

Validation

$ cargo test --all-features --locked
86 passed
$ cargo metadata --locked --no-deps
(lock consistent at 4.3.1)
$ uvx --from actionlint-py actionlint
(no findings)

Related issues

  • Part of blacklanternsecurity/bbot-enterprise#139
  • Part of blacklanternsecurity/bbot-enterprise#140
  • Part of blacklanternsecurity/bbot-enterprise#149
  • Part of blacklanternsecurity/bbot-enterprise#155
  • Part of blacklanternsecurity/bbot-enterprise#156

Supersedes #96, moved from a fork to an upstream branch.

TheTechromancer and others added 25 commits December 16, 2025 10:23
ubuntu-latest is a label GitHub moves. The image changes under us and
a green pipeline turns red on a morning nobody touched the code.
Pinning to ubuntu-24.04 means the runner moves when we move it.

dtolnay/rust-toolchain was called at @stable, a branch, not a release.
It is pinned to a commit, dated in the trailing comment because that
repo publishes no release tag for it. Version-tagged actions are
untouched.
The lint job ran an unpinned `pipx install ruff`, which picked up 0.16.9
and failed on 8 pre-existing errors (import order, __slots__ sort,
SIM102, SIM201). The Rust job tracked `stable`, whose clippy now denies
for_kv_map in src/node/base.rs. Both blocked unrelated PRs such as #92.

Pin ruff==0.16.9 in the workflow and dev group, pin Rust 1.97.1 with
clippy and rustfmt in rust-toolchain.toml and rust-tests.yml, and fix
the errors in source.
Fork PRs get a read-only GITHUB_TOKEN, so the comment step failed the
job after the benchmark itself passed.
windows-latest and macos-latest were still floating in the wheel
matrix.
rust-tests.yml and python-tests.yml published to crates.io and PyPI on
every push to stable, each with a long-lived token. Publishing moves to
publish.yml, triggered only by v* tags. Its first job refuses tags
outside vMAJOR.MINOR.PATCH or vMAJOR.MINOR.PATCH-rc.N and any tag that
does not equal the Cargo.toml version. Both test workflows run before
any wheel is built, every build must succeed before a registry sees
the release, PyPI uses trusted publishing in the pypi environment,
crates.io runs in the release environment, and the last job creates
the GitHub Release with an SPDX JSON SBOM attached.

actions-rs/toolchain is archived and is replaced by
actions-rust-lang/setup-rust-toolchain reading rust-toolchain.toml.
Every workflow declares least-privilege permissions, every action is
pinned to a commit SHA, and CLA calls the shared reusable workflow.
ruff 0.16 formats Python blocks in Markdown, so ruff format --check
failed on README.md and the lint job only ran ruff check. Formatting
it lets lint run both checks.
pyproject.toml and Cargo.toml both declared 4.2.0, two sources that
publish.yml could only compare against one of. maturin fills a dynamic
[project] version from Cargo.toml, so Cargo.toml is now the single
source for crates.io, PyPI, and the tag check.

poetry.lock and .flake8 are left over from before the move to uv and
ruff. Nothing reads them.
#139 replaces long-lived registry tokens with OIDC. crates.io supports
trusted publishing: rust-lang/crates-io-auth-action exchanges the job's
OIDC token for a short-lived publish token, so CARGO_REGISTRY_TOKEN no
longer needs to exist as a secret.
rust-tests.yml, python-tests.yml, the CLA caller, and the release
steps of publish.yml now use the org workflows in
blacklanternsecurity/CLA at one pinned SHA instead of carrying copies.

- python: python-tests.yml derives the matrix from requires-python,
  replacing the literal 3.9 to 3.14 list, runs ruff check and ruff
  format --check (format was never checked here), and builds the
  extension with maturin develop when maturin is the backend.
- rust: rust-tests.yml with --all-features --locked on the
  rust-toolchain.toml channel.
- publish: release-check.yml validates the tag against Cargo.toml (now
  the single version source). The wheel matrix and the crates.io and
  PyPI uploads stay here because they are radixtarget-specific and PyPI
  trusted publishing does not work in a reusable workflow. publish.yml
  creates the release with the SPDX SBOM and attaches the wheels.
- archive/** is no longer excluded: archive tags are not created.
Picks up secret-env skipping absent secrets on fork PRs and rust-tests
running setup-script before clippy. No caller input changes.
e532142 sits on feat/shared-workflows, which was squash-merged and
can stop resolving once the branch is deleted. 40c6e18 is CLA main and
a superset: it adds optional registry login for image SBOMs to
publish.yml, with defaults that leave existing callers unchanged.

Refs blacklanternsecurity/bbot-enterprise#139
Follows .github-config#17 through CLA#11: publish.yml runs on push to
the trunk instead of a v* tag, publishes in the release environment a
bbot-leads member approves, re-runs safely (PyPI skip-existing,
crates.io only when the version is new), and leaves the tag and the
GitHub release to the bls-release App at the end of the run. The
pypi environment and the long-lived registry token are gone.
# Conflicts:
#	.github/workflows/benchmark.yml
#	.github/workflows/python-tests.yml
#	pyproject.toml
#	radixtarget/test/test_rust_radixtarget.py
#	uv.lock
- publish: build wheels through CLA's maturin-wheels, which installs
  every interpreter requires-python allows, so Windows and macOS ship
  per-version wheels again; publish crates.io through CLA's crates.yml
  after PyPI; pass no App credentials
- python-tests, rust-tests, cla: CODECOV_TOKEN and the CLA App
  credentials by name instead of secrets: inherit; drop the stable push
  triggers that duplicated the publish calls
- python-tests: keep reporting the required test (3.11) to (3.15)
  contexts from the real result until the ruleset moves
- stable-guard: refuse a pull request into stable whose version or
  dependencies cannot release
- benchmark: explicit permissions and a pinned runner
- re-pin every CLA reference to CLA#11
# Conflicts:
#	.github/workflows/benchmark.yml
#	.github/workflows/python-tests.yml
#	.github/workflows/rust-tests.yml
4.3.0 is on PyPI and crates.io with no v4.3.0 tag, so release-check
would refuse it as partly published.
@singlerider singlerider added the github_actions Pull requests that update GitHub Actions code label Oct 26, 2026
@singlerider singlerider self-assigned this Oct 26, 2026
@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

RadixTarget Benchmark Results

100000 networks per table, 100000 lookups, median of 5 runs (seed 1337).

Benchmark dev (ms) ci/publish-on-tag (ms) Change
IPv4 insert 21.44 21.94 +2.3%
IPv4 lookup 18.43 19.79 +7.4%
IPv6 insert 49.07 51.82 +5.6%
IPv6 lookup 42.23 46.89 🔴 +11.0%

Change is in time taken: negative is faster.

…t releasing

publish.yml drops to permissions: {} and grants contents read per job.
check also gets actions and pull-requests read for CLA#11's provenance
check and environment guard.

dependabot-auto-merge.yml is removed. Once dev drops to zero approvals
it would merge Dependabot PRs on CI alone (pjhartlieb).

The README gains a Releasing section: the four release approvals per
run, and that PyPy and free-threaded interpreters get the sdist since
the shared wheel build covers default CPython only.
@codecov-commenter

codecov-commenter commented Oct 26, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100%. Comparing base (9fc51ef) to head (83aec27).
⚠️ Report is 37 commits behind head on dev.

Additional details and impacted files
@@         Coverage Diff         @@
##            dev   #110   +/-   ##
===================================
  Coverage   100%   100%           
===================================
  Files         3      3           
  Lines       604    605    +1     
===================================
+ Hits        604    605    +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@en0f en0f left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved, FYI:

.github/workflows/cla.yml: the CLA check will fail to start once this reaches stable.

The file’s top-level permissions are only pull-requests: write and statuses: write. When you list any permissions, everything you leave out is set to none, so contents becomes none.
At the pinned commit 07e14bc, the shared cla-reusable.yml job asks for contents: read.
A called workflow can’t ask for more than its caller gives it. GitHub rejects the run before it starts with “nested job ‘CLAAssistant’ is requesting ‘contents: read’, but is only allowed ‘contents: none’”. This happens on every pull_request_target and issue_comment event, so no PR gets a CLA status.
The PR checks don’t show this. pull_request_target runs the default branch’s workflow, and the “CLA Assistant” check on this PR came from the old cla.yml already on stable.
CLAAssistant isn’t a required check in the protecc ruleset, so merges won’t be blocked. The CLA gate would just stop running without any error on the PR.
Fix: add contents: read to the permissions in cla.yml.

@singlerider
singlerider requested a review from en0f October 26, 2026 17:00
en0f
en0f previously approved these changes Oct 26, 2026

@en0f en0f left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved, see comment:

Still broke: head is still 3510c97,

.github/workflows/cla.yml: the CLA check won’t start once this reaches stable.

The caller grants only pull-requests: write and statuses: write, so contents falls to none.
At the pinned commit 07e14bc, the CLAAssistant job in cla-reusable.yml asks for contents: read. GitHub refuses to start the run, so no PR gets a CLA status.
The green “CLA Assistant” check on this PR comes from the old cla.yml on the default branch. That’s how pull_request_target works, so the PR checks can’t catch this.
The fix is to add contents: read to cla.yml.
It won’t block merges, because protecc only requires test (3.11) through test (3.15).

@pjhartlieb

Copy link
Copy Markdown
Contributor

release-workflows v1.0.0 is released. To move this PR onto it:

  1. Replace every blacklanternsecurity/CLA/.github/workflows/<file>@07e14bcb… (8 calls in cla.yml, publish.yml, python-tests.yml, rust-tests.yml, stable-guard.yml) with:
    uses: blacklanternsecurity/release-workflows/.github/workflows/<file>@c53c202eedeb26a86088936363e6cb06ce99ac15 # v1.0.0
  2. Map the release key on check (release-check.yml) and release (publish.yml). A reusable workflow gets the release environment's secret only when the caller passes it (README):
    secrets:
      BLS_RELEASE_PRIVATE_KEY: ${{ secrets.BLS_RELEASE_PRIVATE_KEY }}
  3. Rebase onto dev: the branch has conflicts.

Inputs, secrets and permissions otherwise match v1.0.0. Please hold the merge until this repo's release environment, trusted publishers and tag rulesets are in place and the sandbox pilot has released.

Move every CLA reusable workflow call to
blacklanternsecurity/release-workflows at v1.0.0 and pass
BLS_RELEASE_PRIVATE_KEY to the release-check and publish calls,
which receive release-environment secrets only when mapped.

@en0f en0f left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pjhartlieb -
I found one blocker. It won’t stop this PR merging into dev, but it will make the first release fail when dev is promoted to stable.

Blocker: the first release from publish.yml will fail the provenance check.

Where it fails: release-check.yml at v1.0.0 runs provenance.py. That script walks the first-parent history of stable back to the last v* tag and requires every commit since then to belong to a PR merged into stable.

Why: The only v* tag reachable from stable is v3.0.0 (4.3.0 shipped without a tag). That leaves 34 first-parent commits to check, and I looked each one up with the GitHub API. 30 of them don’t pass:
5 direct pushes with no PR: the CLA commits cbd63aa, ff0ac17, e2787a1, ab32e6a and 0ecc48e.
25 merges of PRs whose base is master, not stable: #20–#27, #43–#51, #60, #62, #63 and #65–#69, all from before the branch rename. merged_into() checks the base name exactly.

What happens: The check job exits with “30 commit(s) since v3.0.0 bypassed review”. The tests, wheels, PyPI, crates.io and release jobs never run.

Fix: Before promoting, push a v4.3.0 tag on 3317d3e, the merge of #105 that shipped 4.3.0. The check would then start from there and only see #108 and #111 (both fine) plus the promotion merge. That also fills the tag gap the PR description mentions.

@pjhartlieb
pjhartlieb merged commit 914c8e7 into dev Nov 3, 2026
19 checks passed
@pjhartlieb
pjhartlieb deleted the ci/publish-on-tag branch November 3, 2026 20:59
@pjhartlieb pjhartlieb mentioned this pull request Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

github_actions Pull requests that update GitHub Actions code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants