Repository navigation
ci: shared tests and release on merge to stable, version from Cargo.toml - #96
singlerider wants to merge 25 commits into
Conversation
Branch renaming
Optimize memory usage
Dev -> Stable 4.2.0
ubuntu-latest is a label GitHub moves. The image changes under us and a green pipeline turns red on a morning nobody touched the code. Pinning to ubuntu-24.04 means the runner moves when we move it. dtolnay/rust-toolchain was called at @stable, a branch, not a release. It is pinned to a commit, dated in the trailing comment because that repo publishes no release tag for it. Version-tagged actions are untouched.
The lint job ran an unpinned `pipx install ruff`, which picked up 0.16.9 and failed on 8 pre-existing errors (import order, __slots__ sort, SIM102, SIM201). The Rust job tracked `stable`, whose clippy now denies for_kv_map in src/node/base.rs. Both blocked unrelated PRs such as #92. Pin ruff==0.16.9 in the workflow and dev group, pin Rust 1.97.1 with clippy and rustfmt in rust-toolchain.toml and rust-tests.yml, and fix the errors in source.
…ore/pin-actions-sha
Fork PRs get a read-only GITHUB_TOKEN, so the comment step failed the job after the benchmark itself passed.
windows-latest and macos-latest were still floating in the wheel matrix.
rust-tests.yml and python-tests.yml published to crates.io and PyPI on every push to stable, each with a long-lived token. Publishing moves to publish.yml, triggered only by v* tags. Its first job refuses tags outside vMAJOR.MINOR.PATCH or vMAJOR.MINOR.PATCH-rc.N and any tag that does not equal the Cargo.toml version. Both test workflows run before any wheel is built, every build must succeed before a registry sees the release, PyPI uses trusted publishing in the pypi environment, crates.io runs in the release environment, and the last job creates the GitHub Release with an SPDX JSON SBOM attached. actions-rs/toolchain is archived and is replaced by actions-rust-lang/setup-rust-toolchain reading rust-toolchain.toml. Every workflow declares least-privilege permissions, every action is pinned to a commit SHA, and CLA calls the shared reusable workflow.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## dev #96 +/- ##
===================================
Coverage 100% 100%
===================================
Files 3 3
Lines 604 605 +1
===================================
+ Hits 604 605 +1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
ruff 0.16 formats Python blocks in Markdown, so ruff format --check failed on README.md and the lint job only ran ruff check. Formatting it lets lint run both checks.
pyproject.toml and Cargo.toml both declared 4.2.0, two sources that publish.yml could only compare against one of. maturin fills a dynamic [project] version from Cargo.toml, so Cargo.toml is now the single source for crates.io, PyPI, and the tag check. poetry.lock and .flake8 are left over from before the move to uv and ruff. Nothing reads them.
#139 replaces long-lived registry tokens with OIDC. crates.io supports trusted publishing: rust-lang/crates-io-auth-action exchanges the job's OIDC token for a short-lived publish token, so CARGO_REGISTRY_TOKEN no longer needs to exist as a secret.
rust-tests.yml, python-tests.yml, the CLA caller, and the release steps of publish.yml now use the org workflows in blacklanternsecurity/CLA at one pinned SHA instead of carrying copies. - python: python-tests.yml derives the matrix from requires-python, replacing the literal 3.9 to 3.14 list, runs ruff check and ruff format --check (format was never checked here), and builds the extension with maturin develop when maturin is the backend. - rust: rust-tests.yml with --all-features --locked on the rust-toolchain.toml channel. - publish: release-check.yml validates the tag against Cargo.toml (now the single version source). The wheel matrix and the crates.io and PyPI uploads stay here because they are radixtarget-specific and PyPI trusted publishing does not work in a reusable workflow. publish.yml creates the release with the SPDX SBOM and attaches the wheels. - archive/** is no longer excluded: archive tags are not created.
Picks up secret-env skipping absent secrets on fork PRs and rust-tests running setup-script before clippy. No caller input changes.
e532142 sits on feat/shared-workflows, which was squash-merged and can stop resolving once the branch is deleted. 40c6e18 is CLA main and a superset: it adds optional registry login for image SBOMs to publish.yml, with defaults that leave existing callers unchanged. Refs blacklanternsecurity/bbot-enterprise#139
liquidsec
left a comment
There was a problem hiding this comment.
This will mean anyone with write access can publish. This needs to be discussed.
Follows .github-config#17 through CLA#11: publish.yml runs on push to the trunk instead of a v* tag, publishes in the release environment a bbot-leads member approves, re-runs safely (PyPI skip-existing, crates.io only when the version is new), and leaves the tag and the GitHub release to the bls-release App at the end of the run. The pypi environment and the long-lived registry token are gone.
# Conflicts: # .github/workflows/benchmark.yml # .github/workflows/python-tests.yml # pyproject.toml # radixtarget/test/test_rust_radixtarget.py # uv.lock
|
@liquidsec reworked to the approved publishing policy (.github-config#17), through CLA#11. This answers "anyone with write access can publish" and the review and pipeline gates:
|
pjhartlieb
left a comment
There was a problem hiding this comment.
@singlerider, I reviewed this at 2e983b3b (based on current dev, 950cd0dd) against the .github-config#16 spec part B, the publishing policy, and CLA#11 at 4894c4ad. The inputs match CLA#11's interface.
The structure is right: push: stable, release-check, then rust and python tests, then the wheel matrix, then crates.io (trusted publishing, existence check) and PyPI (release, OIDC, --skip-existing), then the release last. The old publishers are gone: python-tests.yml (maturin publish --password ${{ secrets.PYPI_TOKEN }} at line 61, and the MATURIN_PYPI_TOKEN upload at line 240) and rust-tests.yml (cargo publish --token at line 58), on both dev and stable.
High
H1. The release App key can't come from the release environment (C1).
publish.yml:219 and :225 pass vars.RELEASE_APP_ID / secrets.RELEASE_APP_KEY from a uses: job, which can't hold environment:. Neither exists, and the spec's names are BLS_RELEASE_APP_ID / BLS_RELEASE_PRIVATE_KEY, held in release. As written, the release fails after crates.io and PyPI have published, or the key moves to repo or org level.
Requested: once CLA#11 reads them inside its environment job, drop release-app-id and the secrets: block.
H2. The required checks on stable will never report.
Ruleset protecc (11169658) requires test (3.11)–test (3.15). This head reports python / test (3.11)…(3.15), python / passed, rust / test and rust / passed.
Requested: change the ruleset (and .github-config#1/#2, which the spec says now lists test (3.11)–(3.15)) to the new contexts before the promotion, or keep the live names until then.
Medium
M1. Windows and macOS wheels drop to one Python version.
publish.yml:111–114 and :139–141 set python-version: 3.x, so --find-interpreter sees one interpreter. dev lists 3.11–3.15 for exactly this reason (its comment reads "maturin --find-interpreter only builds for interpreters on the runner"). The crate isn't abi3 (Cargo.toml:19; pyproject.toml features ["py"]), and 4.3.0 on PyPI has cp311, cp312 and cp313 wheels for win32, win_amd64 and both macOS targets. After this change, users on the other versions would build from the sdist.
Requested: call CLA's maturin-wheels.yml (interpreters come from requires-python), as blastdns#139 does, or restore the version list.
M2. crates.io publishes in parallel with PyPI, not after it.
crates (publish.yml:168–170) and pypi (:195–197) both depend only on the builds. The policy and spec order is PyPI with skip-existing, then crates.io if absent, then the tag.
Requested: crates: needs: [check, pypi]. You could also replace the inline job with CLA's crates.yml (same existence check and OIDC) so there's one copy.
M3. 4.3.0 is already on PyPI and crates.io with no v4.3.0 release (C3).
The first stable merge would tag a different commit as 4.3.0 and attach wheels built from it.
Requested: bump in the promotion PR, or tag 4.3.0's real commit before cut-over.
M4. Stale description and pin (C2, C4, C5).
The description says v* tags, and its admin step registers PyPI against environment pypi.
Requested: change the environment to release, re-pin to the CLA#11 merge commit (the test files and cla.yml are on 40c6e18), and hold the promotion until release is protected.
Low
- L1.
secrets: inherit(publish.yml:27,python-tests.yml:22,cla.yml:16) passesCARGO_REGISTRY_TOKEN,PYPI_TOKENand the CLA App key to code that needsCODECOV_TOKEN(orAPP_ID/APP_PRIVATE_KEY). Pass these by name (C6). - L2. Duplicate test runs.
python-tests.ymlandrust-tests.ymlstill run onpush: stablealongside the calls frompublish.yml, in the same groups (C7). - L3.
benchmark.ymlhas nopermissions:block, so it runs with the repo default. Setcontents: readandpull-requests: write.
Verdict
Request changes: H1 (depends on CLA#11), H2 before the promotion, and M1–M4.
Next: we're making these changes ourselves in a follow-up PR, which will be linked here when it opens. This review is the record of what that PR addresses.
en0f
left a comment
There was a problem hiding this comment.
I found one real functional break. The PR is also blocked from merging right now: two reviewers have requested changes. All 14 checks pass, but none of them run publish.yml, so the release path below hasn’t been tested.
Windows and macOS wheels will only be built for one Python version. In .github/workflows/publish.yml, the windows and macos jobs set python-version: 3.x. The old workflow on dev deliberately installed 3.11 through 3.15, with a comment saying --find-interpreter only builds for interpreters that are on the runner. Cargo.toml doesn’t use abi3 builds, so each Python version needs its own wheel. As written, a release would publish Windows and macOS wheels for only the newest Python. Users on 3.11–3.14 would fall back to the sdist, which needs a Rust toolchain to install. Linux and musllinux builds are unaffected because they run in the manylinux containers, which include every interpreter. The fix is to put the old list back in both jobs:
yaml
python-version: |
3.11
3.12
3.13
3.14
3.15
Needs confirming before the first release: PyPI and crates.io must already trust this workflow. The pypi and crates jobs now publish without stored tokens, so both registries need a trusted publisher set up for publish.yml with the release environment. If that isn’t configured, the first push to stable with a new version fails at upload.
- publish: build wheels through CLA's maturin-wheels, which installs every interpreter requires-python allows, so Windows and macOS ship per-version wheels again; publish crates.io through CLA's crates.yml after PyPI; pass no App credentials - python-tests, rust-tests, cla: CODECOV_TOKEN and the CLA App credentials by name instead of secrets: inherit; drop the stable push triggers that duplicated the publish calls - python-tests: keep reporting the required test (3.11) to (3.15) contexts from the real result until the ruleset moves - stable-guard: refuse a pull request into stable whose version or dependencies cannot release - benchmark: explicit permissions and a pinned runner - re-pin every CLA reference to CLA#11
# Conflicts: # .github/workflows/benchmark.yml # .github/workflows/python-tests.yml # .github/workflows/rust-tests.yml
4.3.0 is on PyPI and crates.io with no v4.3.0 tag, so release-check would refuse it as partly published.
en0f
left a comment
There was a problem hiding this comment.
The shared workflows point at an unmerged branch. Every shared workflow call in the PR, including the dependabot auto-merge one that used to point at CLA main, uses blacklanternsecurity/CLA@07e14bc. That commit is only on CLA#11’s open branch (feat/release-on-stable), not on main. The PR description says to switch these to CLA#11’s merge commit before merging, and that hasn’t happened. If this merges first, dev runs its tests, CLA check and dependabot auto-merge off unreviewed CLA code.
pjhartlieb
left a comment
There was a problem hiding this comment.
@singlerider, re-reviewed at 3bdc1486.
From the 2026-10-21 review
Status at 3bdc1486 |
|
|---|---|
| H1 App credentials | Fixed. No App lines in callers; publish.yml:66-76 |
H2 required checks on stable |
Fixed. python-tests.yml:26-38 reports test (3.11)–(3.15), all green |
| M1 Windows/macOS wheels | Fixed for CPython: publish.yml:30-34 calls CLA maturin-wheels.yml; all 14 platforms 4.3.0 shipped are built. See 2 below |
| M2 crates after PyPI | Fixed. publish.yml:57 |
| M3 4.3.0 already published | Fixed. Cargo.toml:3 4.3.1: no tag, not on PyPI or crates.io. A bump into dev only releases on promotion, and CLA's release-check refuses a partly published version |
| M4 description, pin | Description fixed. Pin: see 1 below |
L1 secrets: inherit |
Fixed. publish.yml:27-28, python-tests.yml:23-24, cla.yml:16-18 |
| L2 duplicate runs | Fixed. No test workflow triggers on push: stable |
L3 benchmark.yml permissions |
Fixed. benchmark.yml:6-8 |
Still open
- Pin. Every CLA ref is CLA#11's unmerged head
07e14bcb. Re-pin to the CLA merge commit before this merges, as the description says. dependabot-auto-merge.ymlis still ondev(re-pinned, not removed). Oncedevdrops to 0 approvals it merges Dependabot PRs on CI alone. Delete it in this PR.publish.yml:7-8iscontents: read. Make itpermissions: {}and grantcontents: readoncheck,rust,pythonandwheels.- Before cut-over: 4.3.0 also shipped
pp311(10 wheels),cp314tandcp315t. CLA builds released CPython, default variant only (scripts/manifest.pyat07e14bcb), so those users get the sdist and need Rust. Either accept that and say so in the description, or add an interpreter input tomaturin-wheels.yml. - Before cut-over: a release prompts for
releaseapproval four times: release-check'scredentialsjob,pypi,cratesand the release. Put that in the release docs. The README has no Releasing section (blastdns and blasthttp have one).
Reproduce
TMPDIR=<scratch dir> bash verify-radixtarget-96.sh 3bdc148608c331763dc0f1ed4494c27eeef11f04
# expect: SUMMARY repo=radixtarget pr=96 sha=3bdc1486... pass=19 fail=3 (P13, P14, W02 = items 3, 2, 4)
bash verify-radixtarget-96.sh 2e983b3b6fb1e31baba011fe883e2c154f3a888f
# negative control, the head reviewed 2026-10-21: pass=9 fail=13 (P02 P04 P05 P06 P08 P11 P12 P13 P15.stable P16 W00 W01 W02)
It needs git, an authenticated gh (GET calls only), uv and curl. It downloads actionlint 1.7.12 and checks it against the release checksums.
verify-radixtarget-96.sh
#!/usr/bin/env bash
# verify-radixtarget-96.sh <sha>
#
# Read-only re-check of blacklanternsecurity/radixtarget#96 (publish on merge to stable)
# against the 2026-10-21 review findings and go-live plan item A4 (+ R6, R11).
# Clean blobless clone of the repo at <sha>; GitHub reads are `gh api --method GET`
# only; PyPI and crates.io reads are anonymous GETs. Nothing is written anywhere
# except a temp dir under $TMPDIR (removed on exit).
#
# Prints PASS/FAIL/INFO per check and a final SUMMARY line; exits 1 on any FAIL.
# Needs: git, gh (authenticated), uv, curl, shasum or sha256sum.
# Env: CLA_SHA (expected CLA pin, default the CLA#11 head reviewed 2026-10-23).
set -uo pipefail
OWNER=blacklanternsecurity
REPO=radixtarget
PR=96
SHA=${1:?usage: $0 <commit-sha>}
CLA_SHA=${CLA_SHA:-07e14bcbba5e737e1b29bd9e3f9128fbf2483be9}
ACTIONLINT_VERSION=1.7.12
WORK=$(mktemp -d "${TMPDIR:-$PWD}/verify-$REPO-$PR.XXXXXX")
trap 'rm -rf "$WORK"' EXIT
echo "verify $OWNER/$REPO#$PR at $SHA (CLA_SHA=$CLA_SHA) $(date -u +%Y-%m-%dT%H:%M:%SZ)"
git clone --quiet --filter=blob:none --no-checkout "https://github.com/$OWNER/$REPO.git" "$WORK/repo" || { echo "FAIL clone"; echo "SUMMARY repo=$REPO sha=$SHA pass=0 fail=1"; exit 1; }
git -C "$WORK/repo" fetch --quiet origin "$SHA" 2>/dev/null \
|| git -C "$WORK/repo" fetch --quiet origin "+refs/pull/$PR/head:refs/remotes/origin/pr"
git -C "$WORK/repo" -c advice.detachedHead=false checkout --quiet "$SHA" || { echo "FAIL checkout $SHA"; echo "SUMMARY repo=$REPO sha=$SHA pass=0 fail=1"; exit 1; }
# actionlint, downloaded from the release and verified against the release's checksums file
case "$(uname -s)_$(uname -m)" in
Darwin_arm64) PLAT=darwin_arm64 ;; Darwin_x86_64) PLAT=darwin_amd64 ;;
Linux_x86_64) PLAT=linux_amd64 ;; Linux_aarch64|Linux_arm64) PLAT=linux_arm64 ;;
*) PLAT="" ;;
esac
ACTIONLINT=""
if [ -n "$PLAT" ]; then
base="https://github.com/rhysd/actionlint/releases/download/v$ACTIONLINT_VERSION"
tgz="actionlint_${ACTIONLINT_VERSION}_${PLAT}.tar.gz"
sums="actionlint_${ACTIONLINT_VERSION}_checksums.txt"
mkdir -p "$WORK/al"
if curl -sSfL -o "$WORK/al/$tgz" "$base/$tgz" && curl -sSfL -o "$WORK/al/$sums" "$base/$sums"; then
want=$(grep " $tgz\$" "$WORK/al/$sums" | cut -d' ' -f1)
got=$( (shasum -a 256 "$WORK/al/$tgz" 2>/dev/null || sha256sum "$WORK/al/$tgz") | cut -d' ' -f1)
if [ -n "$want" ] && [ "$want" = "$got" ]; then
tar -xzf "$WORK/al/$tgz" -C "$WORK/al" actionlint && ACTIONLINT="$WORK/al/actionlint"
echo "INFO actionlint $ACTIONLINT_VERSION $PLAT sha256 $got matches the release checksums file"
else
echo "INFO actionlint sha256 mismatch (want=$want got=$got); not used"
fi
fi
fi
cat > "$WORK/check.py" <<'PY'
import base64, json, os, re, subprocess, sys, tomllib, urllib.request, urllib.error
import yaml
owner, repo, pr, sha, cla_sha, root, actionlint = sys.argv[1:8]
wfdir = os.path.join(root, ".github", "workflows")
results = []
def out(status, cid, desc, detail=""):
results.append(status)
print(f"{status} {cid} {desc}" + (f" -- {detail}" if detail else ""))
def check(cid, desc, ok, detail=""):
out("PASS" if ok else "FAIL", cid, desc, detail)
def info(cid, desc, detail=""):
print(f"INFO {cid} {desc}" + (f" -- {detail}" if detail else ""))
def gh(path):
cmd = ["gh", "api", "--method", "GET", path]
p = subprocess.run(cmd, capture_output=True, text=True)
return p.returncode, p.stdout
def http_status(url):
req = urllib.request.Request(url, headers={"User-Agent": f"{owner}-{repo}-pr-review-readonly"})
try:
with urllib.request.urlopen(req, timeout=30) as r:
return r.status, r.read()
except urllib.error.HTTPError as e:
return e.code, b""
def load(path):
with open(path) as f:
return yaml.safe_load(f)
def triggers(wf):
on = wf.get("on", wf.get(True))
if isinstance(on, str):
return {on: None}
if isinstance(on, list):
return {k: None for k in on}
return on or {}
files = sorted(f for f in os.listdir(wfdir) if f.endswith((".yml", ".yaml"))) if os.path.isdir(wfdir) else []
wfs = {f: load(os.path.join(wfdir, f)) for f in files}
texts = {f: open(os.path.join(wfdir, f)).read() for f in files}
pub = wfs.get("publish.yml")
jobs = (pub or {}).get("jobs", {}) or {}
def needs(job):
n = (jobs.get(job) or {}).get("needs", [])
return [n] if isinstance(n, str) else list(n)
def job_using(fragment):
return [j for j, d in jobs.items() if fragment in str((d or {}).get("uses", ""))]
# P01 trigger
t = triggers(pub or {})
push = t.get("push") or {}
check("P01", "publish.yml triggers only on push to stable",
pub is not None and set(t) == {"push"} and push.get("branches") == ["stable"] and not push.get("tags"),
f"on={json.dumps(t, default=str)}")
# P02 job order: tests -> wheels -> pypi -> crates -> release
wheels = job_using("maturin-wheels.yml") or [j for j in jobs if j == "wheels"]
crates = job_using("CLA/.github/workflows/crates.yml") or [j for j in jobs if j == "crates"]
rel = job_using("CLA/.github/workflows/publish.yml")
pypi = [j for j, d in jobs.items() if "pypa/gh-action-pypi-publish" in json.dumps(d) or "\"command\": \"upload\"" in json.dumps(d)]
local_tests = [j for j, d in jobs.items() if str((d or {}).get("uses", "")).startswith("./")]
order = []
if wheels and pypi and crates and rel:
w, p, c, r = wheels[0], pypi[0], crates[0], rel[0]
order = [set(local_tests) <= set(needs(w)) and bool(local_tests), w in needs(p), p in needs(c), {c, p} <= set(needs(r))]
check("P02", "job order tests -> wheels -> pypi -> crates -> release",
bool(order) and all(order),
f"tests={local_tests} wheels.needs={needs(wheels[0]) if wheels else None} pypi.needs={needs(pypi[0]) if pypi else None} "
f"crates.needs={needs(crates[0]) if crates else None} release.needs={needs(rel[0]) if rel else None}")
# P03 pypi job
pj = jobs.get(pypi[0]) if pypi else {}
steps = json.dumps((pj or {}).get("steps", []))
check("P03", "pypi job: environment release, id-token: write, skip-existing",
bool(pj) and pj.get("environment") == "release" and (pj.get("permissions") or {}).get("id-token") == "write"
and re.search(r'"skip-existing": (true|"true")|--skip-existing', steps) is not None,
f"environment={pj.get('environment') if pj else None} permissions={pj.get('permissions') if pj else None}")
# P04 crates via CLA crates.yml with id-token
cj = jobs.get(crates[0]) if crates else {}
check("P04", "crates job calls CLA crates.yml and grants id-token: write",
bool(cj) and "CLA/.github/workflows/crates.yml" in str(cj.get("uses", "")) and (cj.get("permissions") or {}).get("id-token") == "write",
f"permissions={cj.get('permissions') if cj else None}")
alltext = "\n".join(f"{f}:{i+1}:{l}" for f, s in texts.items() for i, l in enumerate(s.splitlines()))
def hits(rx):
return [l for l in alltext.splitlines() if re.search(rx, l.split(":", 2)[2])]
# P05 no App credential lines in callers
h = hits(r"release-app-|RELEASE_APP_|BLS_RELEASE_")
check("P05", "no release App credential lines in callers (CLA reads them in `release`)", not h, "; ".join(h[:4]))
# P06 no secrets: inherit
h = hits(r"secrets:\s*inherit")
check("P06", "no `secrets: inherit` in any workflow", not h, "; ".join(h[:4]))
# P07 SHA pins
bad = []
for l in alltext.splitlines():
m = re.search(r"^\s*(?:-\s+)?uses:\s*([^\s#]+)", l.split(":", 2)[2])
if m and not m.group(1).startswith("./") and not re.search(r"@[0-9a-f]{40}$", m.group(1)):
bad.append(l.strip())
check("P07", "every `uses:` is local or pinned to a full SHA", not bad, "; ".join(bad[:4]))
# P08 / P09 CLA pins
cla_refs = sorted({m.group(1, 2) for m in re.finditer(r"blacklanternsecurity/CLA/(\.github/workflows/[\w.-]+)@([0-9a-f]{7,40})", alltext)})
pins = sorted({s for _, s in cla_refs})
check("P08", f"every CLA reference pinned to CLA_SHA {cla_sha[:8]}", pins == [cla_sha], f"pins={[p[:8] for p in pins]}")
missing = []
for path, s in cla_refs:
rc, _ = gh(f"repos/blacklanternsecurity/CLA/contents/{path}?ref={s}")
if rc != 0:
missing.append(f"{path}@{s[:8]}")
check("P09", "every referenced CLA workflow exists at its pin", bool(cla_refs) and not missing,
f"{len(cla_refs)} refs" + (f"; missing {missing}" if missing else ""))
# P10 old publish paths
h = hits(r"PYPI_TOKEN|PYPI_API_TOKEN|MATURIN_PYPI_TOKEN|secrets\.CARGO_REGISTRY_TOKEN|cargo publish .*--token|maturin publish|twine upload|gh release create|git push .*--tags|git tag ")
extra = [f for f in ("release.toml", "release.sh") if os.path.exists(os.path.join(root, f))]
check("P10", "no old publish path in any workflow (stored tokens, cargo publish --token, maturin publish, tag pushes)", not h and not extra, "; ".join(h[:4] + extra))
# P11 no other workflow runs on push to stable (duplicate test runs, C7)
dup = []
for f, wf in wfs.items():
if f == "publish.yml":
continue
tt = triggers(wf)
if "push" in tt:
b = (tt.get("push") or {}).get("branches")
if b is None or "stable" in b or any("*" in x for x in b):
dup.append(f)
check("P11", "no workflow besides publish.yml triggers on push to stable", not dup, f"{dup}")
# P12 explicit top-level permissions (R6)
noperm = [f for f, wf in wfs.items() if "permissions" not in wf]
check("P12", "every workflow declares top-level permissions (R6)", not noperm, f"{noperm}")
# P13 permissions: {} on publish.yml (A4)
check("P13", "publish.yml top-level `permissions: {}` (A4)", pub is not None and pub.get("permissions") == {},
f"permissions={pub.get('permissions') if pub else None}")
# P14 dependabot-auto-merge removed from dev (R11)
dam = [f for f in files if "dependabot-auto-merge" in f]
check("P14", "dependabot-auto-merge workflow removed (R11)", not dam, f"{dam}")
# P15 live required checks reported at this SHA
p = subprocess.run(["gh", "api", "--method", "GET", "--paginate", f"repos/{owner}/{repo}/commits/{sha}/check-runs?per_page=100",
"--jq", ".check_runs[] | {name, status, conclusion, app: {id: .app.id}}"], capture_output=True, text=True)
runs = [json.loads(l) for l in p.stdout.splitlines() if l.strip()]
reported = {(r["name"], (r.get("app") or {}).get("id")) for r in runs}
for branch in ("dev", "stable"):
rc, body = gh(f"repos/{owner}/{repo}/rules/branches/{branch}")
req = []
for rule in json.loads(body or "[]"):
if rule.get("type") == "required_status_checks":
p = rule["parameters"]
req += [(c["context"], c.get("integration_id"), rule.get("ruleset_id"), p.get("strict_required_status_checks_policy")) for c in p["required_status_checks"]]
miss = [c for c, i, _, _ in req if not any(n == c and (i is None or a == i) for n, a in reported)]
strict = sorted({s for *_, s in req})
check(f"P15.{branch}", f"every live `{branch}` required check is reported at this SHA",
not miss, f"required={len(req)} strict={strict} missing={miss}")
# P16 version not already published (C3)
cargo = tomllib.load(open(os.path.join(root, "Cargo.toml"), "rb"))
pyproj = tomllib.load(open(os.path.join(root, "pyproject.toml"), "rb"))
ver = cargo["package"]["version"]
crate = cargo["package"]["name"]
pyname = pyproj.get("project", {}).get("name")
pyver = pyproj.get("project", {}).get("version")
rc_tag, _ = gh(f"repos/{owner}/{repo}/git/ref/tags/v{ver}")
s_pypi, _ = http_status(f"https://pypi.org/pypi/{pyname}/{ver}/json")
s_crates, _ = http_status(f"https://crates.io/api/v1/crates/{crate}/{ver}")
check("P16", f"manifest version {ver} is unreleased (no v{ver} tag, not on PyPI or crates.io)",
rc_tag != 0 and s_pypi == 404 and s_crates == 404 and (pyver in (None, ver)),
f"tag_exists={rc_tag == 0} pypi={s_pypi} crates={s_crates} pyproject.version={pyver}")
# P17 actionlint
if actionlint:
p = subprocess.run([actionlint, "-shellcheck=", "-pyflakes=", "-no-color"], cwd=root, capture_output=True, text=True)
check("P17", "actionlint 1.7.12 clean (shellcheck/pyflakes integration off)", p.returncode == 0, (p.stdout + p.stderr).strip().replace("\n", " | ")[:600])
else:
out("FAIL", "P17", "actionlint 1.7.12 clean", "actionlint unavailable or checksum mismatch")
# P18 CI at this SHA
bad = sorted({r["name"] for r in runs if r.get("status") == "completed" and r.get("conclusion") not in ("success", "skipped", "neutral")})
pend = sorted({r["name"] for r in runs if r.get("status") != "completed"})
check("P18", "every check run at this SHA concluded success/skipped/neutral", bool(runs) and not bad and not pend,
f"runs={len(runs)} failing={bad} pending={pend}")
# W: wheel matrix through CLA maturin-wheels.yml, compared with the latest PyPI release
wj = jobs.get(wheels[0]) if wheels else None
via_cla = bool(wj) and "CLA/.github/workflows/maturin-wheels.yml" in str(wj.get("uses", ""))
check("W00", "wheels built by CLA maturin-wheels.yml", via_cla, f"wheels job={wheels[0] if wheels else None}")
s, raw = http_status(f"https://pypi.org/pypi/{pyname}/json")
latest = json.loads(raw) if s == 200 else {}
wheels_pypi = [u["filename"] for u in latest.get("urls", []) if u["filename"].endswith(".whl")]
arch = {"x86_64": "x86_64", "i686": "x86", "aarch64": "aarch64", "armv7l": "armv7", "s390x": "s390x", "ppc64le": "ppc64le",
"amd64": "x64", "win32": "x86", "arm64": "aarch64"}
def fam(plat):
a = next((v for k, v in arch.items() if plat.endswith(k)), plat)
for prefix, family in (("musllinux", "musllinux"), ("manylinux", "linux"), ("linux", "linux"), ("macosx", "macos"), ("win", "windows")):
if plat.startswith(prefix):
return family, a
return "other", plat
shipped = set()
interp = set()
for fn in wheels_pypi:
parts = fn[:-4].split("-")
for plat in parts[4].split("."):
f, a = fam(plat)
shipped.add((f, a))
interp.add(f"{parts[2]}-{parts[3]}")
if via_cla:
m = re.search(r"maturin-wheels\.yml@([0-9a-f]{40})", str(wj["uses"]))
rc, src = gh(f"repos/blacklanternsecurity/CLA/contents/.github/workflows/maturin-wheels.yml?ref={m.group(1)}") if m else (1, "")
clawf = yaml.safe_load(base64.b64decode(json.loads(src)["content"]).decode()) if rc == 0 else {}
default = clawf.get("on", clawf.get(True))["workflow_call"]["inputs"]["targets"]["default"]
targets = json.loads((wj.get("with") or {}).get("targets") or default)
built = {(f, a) for f, al in targets.items() for a in al}
gap = sorted(shipped - built)
check("W01", f"platforms built cover every platform of {pyname} {latest.get('info', {}).get('version')} on PyPI",
not gap, f"shipped={len(shipped)} built={len(built)} not_built={gap}")
rc2, src2 = gh(f"repos/blacklanternsecurity/CLA/contents/scripts/manifest.py?ref={m.group(1)}")
mp = base64.b64decode(json.loads(src2)["content"]).decode() if rc2 == 0 else ""
cpython_default_only = 'build["implementation"] != "cpython"' in mp and 'build["variant"] != "default"' in mp
dropped = sorted(i for i in interp if not re.fullmatch(r"cp3(\d+)-cp3\1", i)) if cpython_default_only else []
check("W02", "every interpreter ABI on the latest PyPI release is still built (CLA builds released CPython, default variant, only)",
not dropped, f"pypi_abis={sorted(interp)} not_built={dropped}")
else:
out("FAIL", "W01", "platform coverage", "depends on W00")
out("FAIL", "W02", "interpreter coverage", "depends on W00")
# repo-specific checks
extra = os.environ.get("EXTRA_CHECKS", "")
if "blasthttp" in extra:
rt = wfs.get("rust-tests.yml") or {}
rtt = texts.get("rust-tests.yml", "")
unp = [u for u in re.findall(r"(?m)^\s*(?:-\s+)?uses:\s*([^\s#]+)", rtt) if not re.search(r"@[0-9a-f]{40}$", u)]
tt = triggers(rt)
check("H01", "rust-tests.yml: SHA-pinned, contents: read, no push trigger, persist-credentials: false",
bool(rt) and not unp and rt.get("permissions") == {"contents": "read"} and "push" not in tt and "persist-credentials: false" in rtt,
f"unpinned={unp} permissions={rt.get('permissions')} triggers={list(tt)}")
ok = "rust-tests" in jobs and "rust-tests" in needs(wheels[0] if wheels else "")
check("H02", "publish.yml runs rust-tests.yml before the wheels", ok, f"wheels.needs={needs(wheels[0]) if wheels else None}")
env_tests = "PYO3_USE_ABI3_FORWARD_COMPATIBILITY" in texts.get("tests.yml", "")
env_wheels = "PYO3_USE_ABI3_FORWARD_COMPATIBILITY" in json.dumps(wj or {})
check("H03", "PYO3_USE_ABI3_FORWARD_COMPATIBILITY passed to the shared tests and wheels (en0f)", env_tests and env_wheels,
f"tests={env_tests} wheels={env_wheels}")
readme = open(os.path.join(root, "README.md")).read()
links = sorted(set(re.findall(r"https://github\.com/blacklanternsecurity/\.github[\w.-]*/blob/[^\s)]+", readme)))
st = {l: http_status(l)[0] for l in links}
check("H04", "README release-policy link resolves for an anonymous reader", bool(links) and all(v == 200 for v in st.values()), f"{st}")
n_pass, n_fail = results.count("PASS"), results.count("FAIL")
print(f"SUMMARY repo={repo} pr={pr} sha={sha} pass={n_pass} fail={n_fail}")
sys.exit(1 if n_fail else 0)
PY
echo "INFO env release environment exists: $(gh api --method GET "repos/$OWNER/$REPO/environments" --jq '[.environments[].name] | index("release") != null' 2>/dev/null || echo unknown)"
EXTRA_CHECKS="" uv run --quiet --no-project --python 3.12 --with pyyaml==6.0.2 python -I "$WORK/check.py" \
"$OWNER" "$REPO" "$PR" "$SHA" "$CLA_SHA" "$WORK/repo" "$ACTIONLINT"|
Superseded by #110, moved from a fork to an upstream branch. |
What changed
rust-tests.ymlandpython-tests.yml, which used to publish on everystablepush with long-lived tokens. Neither triggers onpush: stableany more, so they no longer duplicate the calls frompublish.yml.python-tests.ymlandrust-tests.ymlcall CLA's shared workflows. The Python matrix comes fromrequires-python.ruff format --checknow runs, and the extension is built withmaturin develop. Rust runs--all-features --lockedon therust-toolchain.tomlchannel.CODECOV_TOKENis passed by name. Until the default branch ruleset requirespython / passed, arequiredjob reportstest (3.11)to(3.15)from the real result.publish.ymlruns on a push tostable: CLA'srelease-check.yml, then rust and python tests, then CLA'smaturin-wheels.yml(every interpreterrequires-pythonallows, on Windows and macOS too), then PyPI with OIDC inrelease, then crates.io through CLA'scrates.ymlwith trusted publishing, then CLA'spublish.yml(tag and GitHub Release with SBOM and wheels, created last). No caller passes App credentials.stable-guard.yml:release readyon pull requests intostablerefuses a non-MAJOR.MINOR.PATCHversion and git or URL dependencies.cla.ymlpassesAPP_ID/APP_PRIVATE_KEYby name. No workflow usessecrets: inherit.benchmark.ymldeclarescontents: readandpull-requests: writeand pins its runner.Cargo.tomlis the single version source, now 4.3.1: 4.3.0 is on PyPI and crates.io with nov4.3.0tag.poetry.lockand.flake8removed. Merges currentdev(dependabot grouping, shared auto-merge).Before promoting to
stablereleaseenvironment with reviewers and astable-only deployment policy, holdingBLS_RELEASE_APP_IDandBLS_RELEASE_PRIVATE_KEY.publish.yml, environmentrelease. Then delete the PyPI and crates.io token secrets.protecc(11169658), and the matching .github-config#1/Prune / Defragment branches to save space #2 Terraform: requirepython / passed,rust / passedandrelease ready, then drop therequiredjob.Validation
Related issues