Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -51,3 +51,9 @@ indent_size = 2
[uv.lock]
insert_final_newline = unset
trim_trailing_whitespace = unset

# Fuzzer seeds and crash regressions are bytes the harness reads verbatim, NUL separators included.
[fuzz/{corpus,regressions}/**]
charset = unset
insert_final_newline = unset
trim_trailing_whitespace = unset
74 changes: 74 additions & 0 deletions .github/workflows/fuzz.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# Coverage-guided fuzzing of the two transcript converters with Atheris (libFuzzer for
# Python).
#
# The converters read files nobody here wrote: every Claude Code session and Codex rollout on
# the machine. `mise run check` replays the committed seeds through the same targets on every
# push; this job mutates them for a bounded time and fails on any exception that is not the
# converter's own rejection (`DomainError`), on a successful conversion the corpus could not
# write, on a timeout or an out-of-memory input, and on a run that executed no input.
# fuzz/run_fuzzers.py says which is which, and CONTRIBUTING.md how to reproduce a finding.
#
# Weekly as well as on pull requests that touch the converter or the harnesses, because a
# fuzzer finds different inputs on every run: the scheduled run is the one that keeps looking
# when the code does not move. Thursday, offset from the Tuesday and Wednesday scheduled jobs
# so they do not contend for the same runner minute.
#
# No secret and no cloud credential: the harnesses convert bytes in a temporary directory and
# call nothing outside the process.
name: fuzz
on:
pull_request:
paths:
- "packages/atif-converter/**"
- "fuzz/**"
- ".github/workflows/fuzz.yml"
schedule:
- cron: "41 6 * * 4"
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
fuzz:
name: fuzz the transcript converters
runs-on: ubuntu-latest
# Two targets at 60 s each plus a cold toolchain install; a hang is caught per input by
# libFuzzer's own `-timeout`, so this bound only stops a runner that wedged outright.
timeout-minutes: 30
permissions:
contents: read # check out the tree the harnesses fuzz
steps:
# `persist-credentials: false`: nothing in this job pushes, and a token left in
# .git/config is readable by every subsequent step, the fuzzed code included.
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
# `cache: false`, for the reason security.yml gives: a cached tool directory is a tool
# nobody verified, and a cached pipx shebang can point at an interpreter the cache lacks.
- uses: jdx/mise-action@9149ea85001c7435d5a66bb127d6a1b6227cb0a5 # v5.0.0
with:
# `minimum_release_age` because a mise release exists on GitHub before its assets finish
# uploading: the 2026-09-08 scheduled security run asked for the just-cut v2026.9.3, its
# tarball 404'd five times, and every scanner was skipped. A day-old release has its
# assets. `version` stays unpinned so the runner keeps following mise, gated by that day.
minimum_release_age: 24h
cache: false
- run: mise run lock:check
- run: mise run install:ci
# `UV_LOCKED` makes the `uv run --group fuzz` inside the task install atheris from uv.lock
# as locked, and fail rather than re-resolve when the lock is stale.
- run: mise run fuzz
env:
UV_LOCKED: "1"
# Only on failure: the inputs that crashed, timed out or ran out of memory, and each
# target's whole libFuzzer log, so a finding reproduces from the run that found it.
- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: failure()
with:
name: fuzz-findings
path: |
fuzz/out/*/artifacts/
fuzz/out/*/fuzz.log
retention-days: 30
144 changes: 143 additions & 1 deletion .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,8 @@ jobs:
runs-on: ubuntu-latest
permissions:
contents: read
outputs:
version: ${{ steps.version.outputs.version }}
steps:
# The TAG, not the branch. `main` has moved on by the time a maintainer publishes the
# draft, and the bytes that reach PyPI have to be the bytes that were released.
Expand Down Expand Up @@ -85,6 +87,12 @@ jobs:
# nothing installs and gives the preflight below seven extra files to reject.
- run: uv build --out-dir dist

- id: version
name: Read the version from the tag
env:
RELEASE_TAG: ${{ inputs.tag || github.event.release.tag_name }}
run: echo "version=${RELEASE_TAG#v}" >> "$GITHUB_OUTPUT"

# The preflight, on the property a single-wheel distribution can get wrong.
#
# Seven module trees reach PyPI inside ONE wheel, and no sibling is on the index, so the
Expand Down Expand Up @@ -229,6 +237,138 @@ jobs:
retention-days: 7
if-no-files-found: error

# What the GitHub release carries beside PyPI: these same two files, a CycloneDX and an
# SPDX SBOM of the locked runtime closure this job just tested, and SHA256SUMS. Staged
# here, in the job that has no `id-token`, so the attest job below signs bytes it did not
# make; `release:assets` ends in scripts/verify_release_assets.py, so a stray file, a
# name that disagrees with the tag or an SBOM for another version fails before signing.
- name: Stage the release assets
env:
RELEASE_TAG: ${{ inputs.tag || github.event.release.tag_name }}
run: RELEASE_VERSION="${RELEASE_TAG#v}" mise run release:assets

- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: release-assets
path: release/
retention-days: 7
if-no-files-found: error

# SLSA build provenance over every release asset, and an SBOM attestation over the two
# distributions, both signed through Sigstore with this workflow's OIDC identity and stored in
# GitHub's attestation API, where `gh attestation verify` finds them by digest. The bundles
# are also written into the release as `atif_sql-<version>.intoto.jsonl`, one per line, so a
# consumer can verify offline and OpenSSF Scorecard's Signed-Releases check sees provenance.
#
# Its own job because it is the only one that holds `attestations: write`: the build job
# cannot sign, and the upload job below cannot mint an identity.
attest:
name: attest the release assets
needs: build
runs-on: ubuntu-latest
permissions:
id-token: write # request the Sigstore signing certificate for this workflow's identity
attestations: write # store the attestations in the repository's attestation API
steps:
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: release-assets
path: release

- id: provenance
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-path: release/*

- id: sbom
name: Attest the SBOM
uses: actions/attest@1e69f48acb82d1966a394da916b4c1698aa569d6 # v4.2.2
with:
subject-path: |
release/*.whl
release/*.tar.gz
sbom-path: release/atif_sql-${{ needs.build.outputs.version }}.cdx.json

# One compact Sigstore bundle per line: the shape `gh attestation verify --bundle` reads.
- name: Write the attestation bundles
env:
VERSION: ${{ needs.build.outputs.version }}
PROVENANCE: ${{ steps.provenance.outputs.bundle-path }}
SBOM: ${{ steps.sbom.outputs.bundle-path }}
run: |
set -euo pipefail
mkdir attestations
jq -c . "$PROVENANCE" "$SBOM" > "attestations/atif_sql-$VERSION.intoto.jsonl"
wc -l "attestations/atif_sql-$VERSION.intoto.jsonl"

- uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: attestations
path: attestations/
retention-days: 7
if-no-files-found: error

# Uploads the attested assets to the GitHub release, after proving each one verifies against
# THIS workflow's identity and that the attestations cover every asset. An asset nobody can
# verify fails the release here rather than shipping beside the ones that can be verified.
#
# No checkout and no `id-token`: the job holds `contents: write` for one `gh release upload`,
# and its inputs are the two artifacts the jobs above produced. The verifier script is read at
# the TAG through the API, so a change on main cannot loosen the gate an old tag passes.
#
# Assets are added to the release after it is published, so this needs immutable releases
# left off in the repository settings. `gh release upload` without `--clobber` refuses to
# replace an asset: a re-dispatch after a partial upload fails loudly rather than swapping
# bytes a consumer may already have verified.
release-assets:
name: upload the signed release assets
needs: [build, attest]
runs-on: ubuntu-latest
permissions:
contents: write # upload assets to the existing release
steps:
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: release-assets
path: release
- uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: attestations
path: release

- name: Verify every asset against the signing workflow
env:
GH_TOKEN: ${{ github.token }}
VERSION: ${{ needs.build.outputs.version }}
REPO: ${{ github.repository }}
run: |
set -euo pipefail
bundle="release/atif_sql-$VERSION.intoto.jsonl"
for asset in release/*; do
[ "$asset" = "$bundle" ] && continue
gh attestation verify "$asset" --bundle "$bundle" --repo "$REPO" \
--signer-workflow "$REPO/.github/workflows/publish.yml"
done

- name: Check the attestations cover the whole release
env:
GH_TOKEN: ${{ github.token }}
VERSION: ${{ needs.build.outputs.version }}
TAG: ${{ inputs.tag || github.event.release.tag_name }}
REPO: ${{ github.repository }}
run: |
set -euo pipefail
gh api -H "Accept: application/vnd.github.raw+json" \
"repos/$REPO/contents/scripts/verify_release_assets.py?ref=$TAG" > verify_release_assets.py
python3 verify_release_assets.py release "$VERSION" --attestations

- name: Upload the assets to the release
env:
GH_TOKEN: ${{ github.token }}
TAG: ${{ inputs.tag || github.event.release.tag_name }}
REPO: ${{ github.repository }}
run: gh release upload "$TAG" release/* --repo "$REPO"

# WHAT A MAINTAINER CONFIGURES ON PYPI, BEFORE THE FIRST RUN
# ---------------------------------------------------------
# At <https://pypi.org/manage/account/publishing/> (the ACCOUNT-level "Publishing" page, not a
Expand Down Expand Up @@ -275,7 +415,9 @@ jobs:
# single-distribution shape rather than of this file.
publish:
name: publish atif-sql
needs: build
# `attest` as well as `build`: no version reaches PyPI without its provenance in GitHub's
# attestation API.
needs: [build, attest]
runs-on: ubuntu-latest
# The human decision, and the reason it sits HERE rather than earlier: publishing the draft
# release says "these are the release notes", which is reversible. This says "these bytes
Expand Down
40 changes: 29 additions & 11 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,10 @@ jobs:
release:
name: cut the release
runs-on: ubuntu-latest
# The `release` environment holds RELEASE_DEPLOY_KEY and lets only `main` deploy to it, so
# no other branch's workflow and no pull request can read the one credential that may push
# to `main` past its ruleset.
environment: release
permissions:
# Pushes the bump commit and the tag, and creates the draft release. `id-token: write`
# is deliberately absent: nothing here talks to PyPI, so this job cannot mint a
Expand All @@ -66,17 +70,33 @@ jobs:
# the last tag, and a shallow clone would find no tag and treat every commit in the
# repository as unreleased.
#
# `persist-credentials: true` — the exception to this repository's default, because this
# is the one job that pushes. The token it leaves in .git/config is the same
# `GITHUB_TOKEN` scoped to `contents: write` above, this job uploads no artifact for it to
# leak into, and every step after the checkout is either a first-party mise task or `git`
# itself. Writing the token into a remote URL instead would put the same secret in the
# same file with none of that reasoning attached, which is why the audit is silenced here
# rather than worked around.
# `persist-credentials: true`, the exception to this repository's default, because this
# is the one job that pushes. With `ssh-key` set, checkout writes the deploy key to a
# runner temp file and points `core.sshCommand` at it; the key can push to this one
# repository and nothing else, this job uploads no artifact for it to leak into, and every
# step after the checkout is either a first-party mise task or `git` itself. That is why
# the audit is silenced here rather than worked around.
#
# The push below goes through a deploy key, the one actor the `main` ruleset lets past its
# pull-request rule: `GITHUB_TOKEN` pushes as `github-actions[bot]`, which no bypass list
# can name. Checked before the checkout, because an empty `ssh-key` makes the checkout fall
# back to the token silently and the run would fail ten minutes later at the push, with
# `GH006`. A dry run pushes nothing and needs no key.
- name: Require the release deploy key
if: ${{ !inputs.dry_run }}
env:
KEY: ${{ secrets.RELEASE_DEPLOY_KEY }}
run: |
if [ -z "$KEY" ]; then
echo "::error::The release environment has no RELEASE_DEPLOY_KEY secret. RELEASING.md, 'The release push and branch protection', has the setup."
exit 1
fi

- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with: # zizmor: ignore[artipacked] the one job that pushes needs the credential it checks out with
fetch-depth: 0
persist-credentials: true
ssh-key: ${{ secrets.RELEASE_DEPLOY_KEY }}

- uses: jdx/mise-action@9149ea85001c7435d5a66bb127d6a1b6227cb0a5 # v5.0.0
with:
Expand Down Expand Up @@ -215,10 +235,8 @@ jobs:
# `--follow-tags` pushes the annotated tag with the commit, in one ref update, so a tag
# can never exist without the commit it names.
#
# This push is refused with `GH006: Protected branch update failed` when a ruleset on
# `main` requires a pull request, because `GITHUB_TOKEN` pushes as `github-actions[bot]`
# and a ruleset bypass list cannot name it (bypass actors are roles, teams, GitHub Apps,
# and Dependabot). RELEASING.md carries the setting and the local fallback.
# The checkout's deploy key makes this push, and the `main` ruleset names deploy keys as
# its one automated bypass actor. RELEASING.md carries the setting and the local fallback.
- name: Push the bump commit and the tag
if: ${{ !inputs.dry_run }}
run: git push --follow-tags
Expand Down
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ coverage.xml
htmlcov/
.mise/
dist/
release/
build/
*.egg-info/
.DS_Store
Expand Down Expand Up @@ -48,3 +49,6 @@ docs/.packets/

# lychee request cache; links.yml carries it between runs via actions/cache
.lycheecache

# Atheris fuzz runs: the inputs libFuzzer kept, crash artifacts and logs (mise run fuzz).
fuzz/out/
1 change: 1 addition & 0 deletions .vale/styles/config/vocabularies/atif-sql/accept.txt
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ Bedrock
Cohere(?:'s)?
commitizen(?:'s)?
lefthook
[Aa]theris
anyio
fastapi
starlette
Expand Down
6 changes: 6 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -426,6 +426,12 @@ relative `.md` path as GitHub renders it; the build turns each link into the pag
the GitHub blob for a Markdown file the site does not publish.
First-time setup: `mise trust && mise install && mise run install`.

Fuzzing sits beside the gate, not in it: `mise run fuzz` runs the Atheris harnesses in `fuzz/` over
both transcript converters for `ATIF_SQL_FUZZ_SECONDS` each (default 60), on Linux x86_64 only,
and fails on any exception other than the converter's `DomainError`. `check` replays the seeds
in `fuzz/corpus/` through the same targets. CONTRIBUTING.md has the contract and the steps
for a finding.

## Experiments

`experiments/` holds numbered experiment protocols (README per experiment).
Expand Down
Loading
Loading