Skip to content

Signed release assets with SLSA provenance and SBOMs, Atheris fuzzing, and Scorecard fixes - #51

Merged
laithalsaadoon merged 6 commits into
mainfrom
ci/signed-releases-and-scorecard
Oct 10, 2026
Merged

laithalsaadoon merged 6 commits into
mainfrom
ci/signed-releases-and-scorecard

Conversation

@laithalsaadoon

@laithalsaadoon laithalsaadoon commented Oct 10, 2026 •

Copy link
Copy Markdown
Owner

Raises the OpenSSF Scorecard checks this repository can move by itself: Signed-Releases, Packaging (after the first publish), Fuzzing and Vulnerabilities. Branch protection is already applied in the repository settings, and RELEASING.md describes it (see the end).

What changes

Signed release assets with provenance and SBOMs (publish.yml, mise run release:assets)

  • The build job stages release/ on the tag: the wheel and sdist that go to PyPI, CycloneDX and SPDX SBOMs of the locked runtime closure (uv export --no-dev, converted to SPDX by syft so both describe one set), and SHA256SUMS.
  • A new attest job, the only one holding attestations: write, signs SLSA v1 build provenance over every asset and an SBOM attestation over the two distributions with actions/attest (Sigstore, this workflow's OIDC identity), and writes both bundles into atif_sql-<version>.intoto.jsonl.
  • A new release-assets job runs gh attestation verify --signer-workflow .../publish.yml on every asset, then scripts/verify_release_assets.py --attestations read at the tag, then gh release upload without --clobber. A stray file, a name that disagrees with the tag, a wrong digest, an SBOM for another version or an asset outside the provenance fails the upload. The PyPI job now also waits on attest.
  • RELEASING.md documents the six assets and the verify commands; SECURITY.md points at them.

Vulnerabilities: OpenVEX suppressions reach the docs-site lockfile

  • osv-scanner applies the osv-scanner.toml in a lockfile's own directory only. Scorecard's recursive scan therefore reported GHSA-vfj7-8cjw-p6xm and GHSA-ch52-4w7c-c8xp against site/pnpm-lock.yaml even though the ledger argues both. The renderer writes the same render to site/osv-scanner.toml, and security:vex:check drift-gates both copies. osv-scanner scan source -r . (2.6.0): 2 findings before, "No issues found" after.

Fuzzing with Atheris (fuzz/, fuzz.yml, mise run fuzz)

  • Harnesses drive the Claude Code and Codex conversion use cases end to end; only DomainError is an accepted rejection. Seeds come from the existing fixtures and replay in mise run check without atheris (an optional fuzz group, Linux x86_64 only). fuzz.yml runs weekly, on dispatch and on pull requests touching the converter or fuzz/.

Three crashes the fuzzer found, fixed (two fix(converter) commits)

  • build_edges raised TypeError sorting a non-string uuid; Codex _tool_use_ids and extract_codex_blobs raised TypeError on an unhashable payload type (the third came from this PR's first fuzz.yml run). Every string-set membership test on a transcript field outside the harbor conversion now uses a key that is None for a non-string. The three inputs are kept under fuzz/regressions/ and each fails on the unfixed code. Every input that converted before keeps byte-identical output, so CONVERTER_SCHEMA_VERSION stays 9 and only the source digest is re-pinned.

release.yml pushes through a deploy key

  • main requires pull requests and checks, and GITHUB_TOKEN cannot be a ruleset bypass actor; a deploy key can. The job runs in a release environment and refuses a non-dry run without RELEASE_DEPLOY_KEY. RELEASING.md has the one-time setup and, without the key, a fallback that cuts the bump commit through a pull request (cz bump --version-files-only --changelog) and pushes only the tag.

RELEASING.md describes the three main rulesets

  • main history (deletion and non-fast-forward refused, no bypass), main checks (the six CI jobs on an up-to-date branch, deploy key bypass only) and main review (two approvals, Code Owner review, last-push approval, resolved threads, stale reviews dismissed; deploy key, plus the admin role for pull requests only). It says why they are split (a bypass skips every rule in its ruleset), how the maintainer merges (gh pr merge --admin once the checks are green), why fuzz and the external link crawl are not required, and gives a read-only command that lists what main enforces.

Verification

  • mise run --force check: see the latest CI run; locally 2,604 tests passed, pyright 0 errors, Vale 0 errors at baseline.
  • mise run fuzz at 240 s each on the fixed code: claude_code 319,406 runs, codex 422,633 runs, both exit 0.
  • New tests: 13 for the release gate (each planted defect red, a well-formed release green, and the workflow order pinned), 3 for the site osv-scanner copy, 3 crash regressions.
  • actionlint and zizmor (auditor persona) report no findings on the changed workflows.
  • Not run: the publish workflow end to end. It runs for the first time on the next release; the attest and release-assets jobs are checked only statically here.

Repository settings (applied)

These are in place and match RELEASING.md. gh api repos/laithalsaadoon/atif-sql/rules/branches/main --jq '[.[].type]' lists required_status_checks, deletion, non_fast_forward and pull_request.

  1. The write deploy key release.yml push, the release environment (deployments from main) and its RELEASE_DEPLOY_KEY secret.
  2. Rulesets main history, main checks and main review, as above. This PR therefore reads review required; it merges with gh pr merge --admin once the required checks are green.
  3. Immutable releases stay off: assets are attached after the release is published.

🤖 Generated with Claude Code

laithalsaadoon and others added 6 commits October 10, 2026 16:55
…ockfile too

osv-scanner applies the osv-scanner.toml in a lockfile's own directory and no
other one. A recursive `osv-scanner scan source -r .`, which is what the
OpenSSF Scorecard Vulnerabilities check runs, loaded the root copy for uv.lock
and reported both npm advisories the ledger argues (GHSA-vfj7-8cjw-p6xm,
GHSA-ch52-4w7c-c8xp) against site/pnpm-lock.yaml, so Scorecard scored the
check 8 of 10.

The renderer takes one output per lockfile directory and writes the same
render to each; `security:vex:check` drift-gates both copies, and three tests
cover the site copy green, drifted and missing. With site/osv-scanner.toml in
place the same recursive scan reports no issues.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A release published to PyPI left nothing on the GitHub release a consumer
could verify, so OpenSSF Scorecard scored Signed-Releases as no data and a
user had no SBOM to answer "what does this version ship?".

publish.yml now stages the release on the tag with `mise run release:assets`:
the wheel and sdist uploaded to PyPI, CycloneDX and SPDX SBOMs of the locked
runtime closure the gate tested, and SHA256SUMS. A separate job holding the
only `attestations: write` signs SLSA v1 build provenance over every asset
and an SBOM attestation over the distributions through Sigstore, and writes
both bundles into `atif_sql-<version>.intoto.jsonl`. The upload job checks
each asset with `gh attestation verify --signer-workflow` against
publish.yml, then scripts/verify_release_assets.py --attestations read at the
tag, before `gh release upload`; the PyPI job also waits on the attestations.

release.yml pushes the bump commit and tag through a write deploy key held
in a `release` environment limited to main, so a ruleset on main that
requires pull requests can name deploy keys as its automated bypass actor.

RELEASING.md documents the assets, the verify commands, the ruleset and the
deploy key setup; SECURITY.md points at them, and README shows the Scorecard
score badge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Claude Code and Codex converters parse transcripts nobody here wrote,
and until now only hand-written fixtures exercised that path. Two Atheris
harnesses under fuzz/ feed arbitrary bytes, laid out as a session or a
rollout on disk, through convert_and_audit and convert_codex_and_audit,
the same use cases materialize runs. A DomainError is the converter's
documented rejection; any other exception, or a successful conversion
whose artifacts the corpus writer could not store, is a finding.

The target logic lives in fuzz/transcript_targets.py with no atheris
import, so mise run check replays the committed seed corpus through it on
every machine. mise run fuzz runs each harness for ATIF_SQL_FUZZ_SECONDS
(default 60) and fails on a crash, a timeout, an out-of-memory input, a
missing harness, seeds or dictionary, or a run that executed no input.
atheris ships only manylinux x86_64 wheels, so it sits in an optional fuzz
dependency group behind a platform marker: uv sync on macOS and the
universal lock are unchanged.

fuzz.yml runs the task weekly, on demand, and on pull requests that change
the converter or fuzz/, with the action pins and least-privilege settings
the other workflows use, and uploads any crash input on failure.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…crashing

The Atheris fuzzers found two inputs that raised TypeError past the
converter's DomainError contract, so materialize would crash on a session
rather than record it as failed and move on:

- Claude Code: build_edges sorts on (ts or "", uuid or ""); two records with
  no timestamp, one with a uuid that is a JSON object, raised
  "'<' not supported between instances of 'str' and 'dict'".
- Codex: _tool_use_ids tested `payload.get("type") in _CALL_ID_ITEM_TYPES`;
  a type that is a list or object raised "unhashable type".

The edges sort retries with non-string values treated as missing only when
the original order raised, and the Codex check requires a string type. Every
input that converted before keeps byte-identical artifacts, so
CONVERTER_SCHEMA_VERSION stays 9 and only the source digest is re-pinned:
the inputs whose output changes are the ones that produced no artifact.

Both crash inputs are kept under fuzz/regressions/ and replayed in check;
each test fails on the unfixed code with the original TypeError.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fields

The weekly fuzz job's first pull-request run found a third crash of the
same class: extract_codex_blobs tested a response_item payload `type`
against a set, and a `type` that is a JSON object raised "unhashable type"
before the conversion's error translation, escaping as TypeError.

Every membership test outside the harbor conversion that takes a transcript
value (the Claude Code block type in blob extraction, the Codex payload and
item types in blobs, result signals and enrichment, the tool name and step
source) now goes through a `_str` key that is None for a non-string. A
non-string never matched these string sets, so every input that converted
before keeps byte-identical artifacts: CONVERTER_SCHEMA_VERSION stays 9 and
the source digest is re-pinned. The crash input is kept under
fuzz/regressions/codex/ and fails on the unfixed code; a further 240 s per
harness (742,039 runs) found nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
RELEASING.md described one ruleset named main and a maintainer bypass that
let a local update through. The repository now has three rulesets on the
default branch: main history (no bypass), main checks (deploy key only) and
main review (deploy key, plus the admin role for pull requests only).

Describe them, why they are split, how the maintainer merges, and the
accurate fallback when the release deploy key is unavailable: cut the bump
commit through a pull request and publish only the tag.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@laithalsaadoon
laithalsaadoon merged commit 87693b4 into main Oct 10, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant