Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
9eb7909
feat(golden): protect released golden hash files in CI (ITL-946)
Nov 15, 2026
6035e4f
Merge remote-tracking branch 'origin/main' into eywalker/itl-946-chec…
Nov 15, 2026
eb28445
docs(golden): make every golden description match the per-context fol…
Nov 15, 2026
3590a45
fix(golden): trial trees start open; --add plans every group before w…
Nov 15, 2026
2189c24
fix(golden): refuse one new entry name planned by two groups in one -…
Nov 15, 2026
bd11c84
feat(golden): --remove takes an entry out of an unreleased file (ITL-…
Nov 15, 2026
d52f2f3
test(golden): take the newest file and next counter from the copied i…
Nov 15, 2026
c05293b
style(golden): ruff format (ITL-946)
Nov 15, 2026
0e15bf0
test(golden): name the unlisted trial file after the inventory (ITL-946)
Nov 15, 2026
2dcb6d6
fix(golden): the soundness check covers duplicate names and unknown g…
Nov 15, 2026
c128918
test(golden): pin the hash_samples cases no golden case covered (ITL-…
Nov 15, 2026
aae1dfc
test(golden): pin the bytes sample values at today's encoding (ITL-946)
Nov 15, 2026
8f9605c
feat(golden): the Release workflow refuses a release the golden histo…
Nov 16, 2026
f9e2bfa
fix(golden): an edited unreleased file's problem says to restore it (…
Nov 16, 2026
2998644
fix(golden): the release gate requires the release revision to be the…
Nov 16, 2026
5b77b68
fix(golden): harden the store, the tag check and the no-skip rule (IT…
Nov 16, 2026
3223999
feat(golden): the Release workflow also checks earlier release tags (…
Nov 16, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Golden hash files are hashed byte for byte: never convert their line endings (ITL-946).
tests/test_hashing/golden/data/** -text
11 changes: 11 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,17 @@ jobs:
fetch-depth: 0 # required: hatch-vcs reads tag to set version
ref: ${{ inputs.branch }}

- name: Check the golden release step
# Nothing is tagged or published unless the release pull request ran
# generate_golden_hashes.py --release for this version (ITL-946).
run: python3 scripts/check_golden_tags.py --release-version "${{ steps.normalize.outputs.version }}"
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved

- name: Check golden files against release tags
# Every golden file, revision and pin an earlier release tag of the current data
# context shipped is unchanged here (ITL-946). Tags of no such release (older
# data contexts) are not checked; it fails only without git, history or tags.
run: python3 scripts/check_golden_tags.py

- name: Configure git identity
run: |
git config user.name "github-actions[bot]"
Expand Down
5 changes: 5 additions & 0 deletions .github/workflows/run-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,11 @@ jobs:
- name: Install dependencies
run: uv sync --locked --all-extras --dev --python ${{ matrix.python-version }}

- name: Check golden files against release tags
# Every golden file and pin a release tag of the current data context shipped must
# be unchanged in HEAD (ITL-946). Fails when git, the history or the tags are missing.
run: uv run --python ${{ matrix.python-version }} python scripts/check_golden_tags.py

- name: Run tests
run: uv run --python ${{ matrix.python-version }} pytest -v -rsxX --durations=20 -m "not postgres" --cov=src --cov-report=term-missing --cov-report=xml

Expand Down
46 changes: 35 additions & 11 deletions .zed/rules
Original file line number Diff line number Diff line change
Expand Up @@ -287,7 +287,9 @@ A logical type's identifier is its Arrow extension name plus its Arrow extension
differently. It is not a package version. Built-in types use `version_metadata(N)`
(`{"version":N}`); a new built-in type must write one, never empty metadata.
- A changed identifier is a schema change: it moves stored paths and golden values of that
type. Treat it like any other hash change (developer approval, migration assessment).
type. Treat it like any other hash change (developer approval, migration assessment). There
is no exemption for a logical type's own change (developer, November 15, 2026, ITL-946): if
a golden value it moves has shipped in a release, it needs a new data context version.
- The native containers `list[T]` / `set[T]` of a logical type (`ListLogicalType`) have no
version of their own, by design: never add a `"version"` key to their metadata. Their
identifier is the container kind plus the element's identifier; the type hash treats them
Expand Down Expand Up @@ -386,14 +388,34 @@ A class a user defines is one of two kinds (developer, November 12, 2026, ITL-97
## Golden hash values

Hash values must stay identical across releases (developer, November 10, 2026). Every hash
Orcapod stores or embeds in a name is pinned in `tests/test_hashing/golden/data/*.json` and
checked by `tests/test_hashing/golden/test_golden_hashes.py`.
Orcapod stores or embeds in a name is pinned in the golden folder of the current data context,
`tests/test_hashing/golden/data/<folder>/<group>-NN.json`, and checked by
`tests/test_hashing/golden/test_golden_hashes.py`. The mechanism is
`docs/reference/golden-hashes.md` (developer, November 15, 2026, ITL-946).

- A failing golden entry means a hash moved. Treat it as a bug in the change, not a stale
file. Never regenerate the golden files to make a test pass.
- Regenerate (`uv run python tests/test_hashing/golden/generate_golden_hashes.py`) only to add
entries (check the diff adds lines and changes none), or for a hash change the developer has
approved together with a migration assessment.
file. Never edit or regenerate a golden file to make a test pass.
- The unit of immutability is the golden file. The manifest's `golden.history`
(`src/orcapod/contexts/data/<version>.json`) lists every file with the sha256 of its bytes;
a release appends a revision with `added: {}`, pinned in
`src/orcapod/contexts/released_manifests.json`. A file listed by a released revision never
changes and is never removed or renamed.
- Adding golden values is allowed in any release:
`uv run python tests/test_hashing/golden/generate_golden_hashes.py` writes only entries no
file pins yet (into a new `<group>-NN.json` once the group's newest file is released) and
records them in the open revision. Check the diff adds and changes no value. `--check`
reports drift without writing.
- Any change to a released golden value needs a new data context version, with no exemption
(also not a logical type's own identifier change). A new data context version ships only in
a minor release (before 1.0) or a major one, never in a patch. When it starts, the old
context's golden files and code move to `orcapod-sanctuary`; this repository carries only
the current context's golden folder.
- Before a release, a value no release lists changes only with the developer's approval and
a migration assessment: `generate_golden_hashes.py --remove <entry>` (refuses an entry a
release lists), then the generator writes the new value.
- The release pull request runs `generate_golden_hashes.py --release X.Y.Z`. CI checks HEAD
against every release tag (`scripts/check_golden_tags.py`). Never mark a golden entry
`skip` or `xfail`; the only skip is the optional-package rule (`torch`).
- Never move or rename the fixtures in `tests/test_hashing/golden/cases.py`: function pods,
dataclasses and pydantic models hash with their module path.

Expand Down Expand Up @@ -685,10 +707,12 @@ generation of hashing; rows of the current data context are the current entries.
proceed (developer, November 11, 2026). Operator caches and cached sources have no stable
name; PipelineJob.run() reports one total for them. Every database class implements
list_table_paths.
- A released data context manifest (contexts/data/<version>.json) is frozen: its sha256 is
pinned in contexts/released_manifests.json. A change that moves an existing golden hash
value for a reason other than a logical type's own change needs a new data context version
with its own manifest file, never an edit of a released one.
- A released data context manifest (contexts/data/<version>.json) is frozen: the digest of
its canonical JSON without golden is pinned in contexts/released_manifests.json
(manifest), with one releases line per library release (ITL-946). Only its golden
history grows, by appending. A change that moves a released golden hash value, for any
reason (a logical type's own change included), needs a new data context version with its
own manifest file, never an edit of a released one.
- In memory the data context column is still _context_key (ColumnConfig context).
- A writer refuses an input of another data context than its own
(contexts/stored.require_same_data_context, InputValidationError naming both keys) before
Expand Down
46 changes: 35 additions & 11 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -298,7 +298,9 @@ A logical type's identifier is its Arrow extension name plus its Arrow extension
differently. It is not a package version. Built-in types use `version_metadata(N)`
(`{"version":N}`); a new built-in type must write one, never empty metadata.
- A changed identifier is a schema change: it moves stored paths and golden values of that
type. Treat it like any other hash change (developer approval, migration assessment).
type. Treat it like any other hash change (developer approval, migration assessment). There
is no exemption for a logical type's own change (developer, November 15, 2026, ITL-946): if
a golden value it moves has shipped in a release, it needs a new data context version.
- The native containers `list[T]` / `set[T]` of a logical type (`ListLogicalType`) have no
version of their own, by design: never add a `"version"` key to their metadata. Their
identifier is the container kind plus the element's identifier; the type hash treats them
Expand Down Expand Up @@ -397,14 +399,34 @@ A class a user defines is one of two kinds (developer, November 12, 2026, ITL-97
## Golden hash values

Hash values must stay identical across releases (developer, November 10, 2026). Every hash
Orcapod stores or embeds in a name is pinned in `tests/test_hashing/golden/data/*.json` and
checked by `tests/test_hashing/golden/test_golden_hashes.py`.
Orcapod stores or embeds in a name is pinned in the golden folder of the current data context,
`tests/test_hashing/golden/data/<folder>/<group>-NN.json`, and checked by
`tests/test_hashing/golden/test_golden_hashes.py`. The mechanism is
`docs/reference/golden-hashes.md` (developer, November 15, 2026, ITL-946).

- A failing golden entry means a hash moved. Treat it as a bug in the change, not a stale
file. Never regenerate the golden files to make a test pass.
- Regenerate (`uv run python tests/test_hashing/golden/generate_golden_hashes.py`) only to add
entries (check the diff adds lines and changes none), or for a hash change the developer has
approved together with a migration assessment.
file. Never edit or regenerate a golden file to make a test pass.
- The unit of immutability is the golden file. The manifest's `golden.history`
(`src/orcapod/contexts/data/<version>.json`) lists every file with the sha256 of its bytes;
a release appends a revision with `added: {}`, pinned in
`src/orcapod/contexts/released_manifests.json`. A file listed by a released revision never
changes and is never removed or renamed.
- Adding golden values is allowed in any release:
`uv run python tests/test_hashing/golden/generate_golden_hashes.py` writes only entries no
file pins yet (into a new `<group>-NN.json` once the group's newest file is released) and
records them in the open revision. Check the diff adds and changes no value. `--check`
reports drift without writing.
- Any change to a released golden value needs a new data context version, with no exemption
(also not a logical type's own identifier change). A new data context version ships only in
a minor release (before 1.0) or a major one, never in a patch. When it starts, the old
context's golden files and code move to `orcapod-sanctuary`; this repository carries only
the current context's golden folder.
- Before a release, a value no release lists changes only with the developer's approval and
a migration assessment: `generate_golden_hashes.py --remove <entry>` (refuses an entry a
release lists), then the generator writes the new value.
- The release pull request runs `generate_golden_hashes.py --release X.Y.Z`. CI checks HEAD
against every release tag (`scripts/check_golden_tags.py`). Never mark a golden entry
`skip` or `xfail`; the only skip is the optional-package rule (`torch`).
- Never move or rename the fixtures in `tests/test_hashing/golden/cases.py`: function pods,
dataclasses and pydantic models hash with their module path.

Expand Down Expand Up @@ -713,10 +735,12 @@ current data context are the current entries.
proceed (developer, November 11, 2026). Operator caches and cached sources have no stable
name; `PipelineJob.run()` reports one total for them. Every database class implements
`list_table_paths`.
- A released data context manifest (`contexts/data/<version>.json`) is frozen: its sha256 is
pinned in `contexts/released_manifests.json`. A change that moves an existing golden hash
value for a reason other than a logical type's own change needs a new data context version
with its own manifest file, never an edit of a released one.
- A released data context manifest (`contexts/data/<version>.json`) is frozen: the digest of
its canonical JSON without `golden` is pinned in `contexts/released_manifests.json`
(`manifest`), with one `releases` line per library release (ITL-946). Only its `golden`
history grows, by appending. A change that moves a released golden hash value, for any
reason (a logical type's own change included), needs a new data context version with its
own manifest file, never an edit of a released one.
- In memory the data context column is still `_context_key` (`ColumnConfig(context=True)`).
- A writer refuses an input of another data context than its own
(`contexts/stored.require_same_data_context`, `InputValidationError` naming both keys)
Expand Down
54 changes: 49 additions & 5 deletions RELEASING.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,10 +54,29 @@ Do these on `metamorphic-labs` `main` before anything is pushed:

The GitHub Release body is generated from PR titles; `CHANGELOG.md` is the curated
record.
2. **Check that `main` is green** — the `Run Tests` workflow on the commit you are
2. **Record the release in the golden history**, in the same release pull request, as its
last change (see *Golden hash values* below):

```
uv run python tests/test_hashing/golden/generate_golden_hashes.py --release X.Y.Z
```

It appends the release revision to the `golden` history of the current data context's
manifest, writes its digest under `releases` in
`src/orcapod/contexts/released_manifests.json`, and at the data context's first release
sets `released_in` and the `manifest` pin. It computes no hash and refuses a version
already released. Commit both files with the release pull request; the release is tagged
on a commit that contains them. The Release workflow's build job checks this before it
creates the tag (`scripts/check_golden_tags.py --release-version X.Y.Z`) and stops if the
release is not recorded, or if a golden file was added after the release step (the
release revision must be the last revision when the commit is tagged). It also runs the
tag check (`scripts/check_golden_tags.py`) there, so a branch that changed what an earlier
release of the data context shipped is not released. Tags of earlier data contexts are
not checked, so the first release of a data context passes with nothing to check.
3. **Check that `main` is green** — the `Run Tests` workflow on the commit you are
releasing. The Release workflow re-runs the unit tests on Python 3.11, 3.12 and 3.13
and the license check, but not the PostgreSQL or SpiralDB integration jobs.
3. **Check the version is free** — the tag must not exist on origin, and the version must
4. **Check the version is free** — the tag must not exist on origin, and the version must
not exist on PyPI or TestPyPI; neither index allows a version to be uploaded twice, even
after it is deleted.

Expand All @@ -71,12 +90,33 @@ All of these hold before the developer is asked for the go:
1. **The full suite is green**: `uv run pytest tests/` passes on the commit to be pushed.
2. **The release pull request is merged** on `metamorphic-labs` (`CHANGELOG.md` folded, section
dated, `changelog.d/` fragments deleted).
3. **The data context freeze is done**: every released manifest is pinned in
`contexts/released_manifests.json` and the golden hash tests pass unchanged.
3. **The golden release step is done**: `contexts/released_manifests.json` has the current
data context's `releases` line for this version and a pinned `manifest`, and the golden
hash tests and the data context tests pass unchanged.
4. **`main` is green** on `metamorphic-labs`, and `git status` is clean.
5. **The version is free**: the tag does not exist on `nauticalab`, and the version is on
neither PyPI nor TestPyPI.

## Golden hash values

Developer decisions, November 15, 2026 (ITL-946); the mechanism and what each check protects
against are in `docs/reference/golden-hashes.md`.

- A golden file that a release listed never changes, and is never removed or renamed.
- New golden values can be added in any release, a patch included.
- Any change to a released golden value, a logical type's own identifier change included,
needs a new data context version. A new data context version ships only in a minor release
(before 1.0) or a major one, never in a patch.
- When a new data context starts, the old context's golden files and code move to
`orcapod-sanctuary`; this repository carries only the current context's golden folder.
- Release tags are protected on both `metamorphic-labs/orcapod-python` and
`nauticalab/orcapod-python`: the repository ruleset `protect-release-tags` (target
`refs/tags/v*`; rules deletion, update and non-fast-forward; no bypass actors) is in place
since 2026-11-15. A `v*` tag can be created, as the Release workflow does, but never moved
or deleted. This is a precondition of the CI tag check (`scripts/check_golden_tags.py`),
which compares HEAD with every release tag of the current data context. Agents never change
repository settings.

## Cutting a Release

Releases are triggered from the GitHub Actions UI — no manual `git tag` step required.
Expand Down Expand Up @@ -142,7 +182,11 @@ environments:
that name `metamorphic-labs`, if any.
3. Protect `main` on `nauticalab` (no force push, no deletion). It is written only by the
fast-forward push above.
4. Optional: set `CODECOV_TOKEN` there. The Codecov, SpiralDB and Linear jobs run only on
4. Protect release tags on both `metamorphic-labs/orcapod-python` and
`nauticalab/orcapod-python` with the ruleset `protect-release-tags` (target
`refs/tags/v*`; deletion, update, non-fast-forward; no bypass actors), so the golden tag
check has a fixed anchor (ITL-946). Done on 2026-11-15.
5. Optional: set `CODECOV_TOKEN` there. The Codecov, SpiralDB and Linear jobs run only on
`metamorphic-labs` and are skipped on `nauticalab`.

## Pre-releases
Expand Down
16 changes: 8 additions & 8 deletions changelog.d/ITL-944.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,18 +8,18 @@ read its tables as results, and computes every input again into its own tables.

- **One data context key for the whole of Orcapod's hashing.** The default data context is
now `std:v1:default` (manifest `orcapod/contexts/data/v1.json`). The version increments
when, and only when, an existing hash value changes for a reason other than a logical
type's own change. Everything stored before v0.3 (rows that carry `std:v0.1:default`, and
tables without the new column) is the earlier generation. A manifest is frozen at the
when, and only when, a released hash value changes, for any reason (ITL-946). Everything
stored before v0.3 (rows that carry `std:v0.1:default`, and tables without the new column)
is the earlier generation. A manifest is frozen at the
release of the library version that contains it, and may be edited in place only while
that release is prepared: `orcapod/contexts/released_manifests.json` pins the sha256 of
each released one, and a test fails when a pinned file is edited. `v1.json` is pinned
when v0.3.0 is tagged.
that release is prepared: `orcapod/contexts/released_manifests.json` pins the digest of
each released one without its `golden` history, and a test fails when a pinned manifest is
edited (ITL-946). `v1.json` is pinned when v0.3.0 is tagged.
- **Manifest contents.** Besides the wiring of the hashers, a manifest now has
`released_in` (the library version that froze it; `null` while open), `previous` (the data
context it follows), `definitions` (a name for each definition that lives in code),
`libraries` (the libraries the core hashes depend on, with versions), `golden` (a digest
of the golden hash files, set when the manifest is frozen) and `changes` (what differs
`libraries` (the libraries the core hashes depend on, with versions), `golden` (the folder
and the append-only history of the golden hash files, ITL-946) and `changes` (what differs
from `previous`).
- **Logical types moved out of the manifest.** The registered logical types and the value
handlers that hash their Python values are now in
Expand Down
Loading
Loading