Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 15 additions & 3 deletions .zed/rules
Original file line number Diff line number Diff line change
Expand Up @@ -487,6 +487,15 @@ Orcapod stores or embeds in a name is pinned in the golden folder of the current

- A failing golden entry means a hash moved. Treat it as a bug in the change, not a stale
file. Never edit or regenerate a golden file to make a test pass.
- Data context v1 is open until the 0.4.0 release (developer, November 17, 2026, ITL-1255).
0.3.0 was withdrawn on 2026-11-17, before anyone used it: its release revision is out of the
golden history, and the pins it shipped are kept under `withdrawn` in
`released_manifests.json`, with the date and the reason, so no v1 file is released today.
Until 0.4.0, a change to any v1 golden value needs the developer's approval of the exact
list of entries that move, each explained; it is made with `--remove <entry>`, then the
generator writes the new value. `--release 0.4.0` freezes v1. Any version published
before it, a pre-release included, would freeze v1 there: the Release workflow requires
the release step for every version. A withdrawn version is never released again.
- The unit of immutability is the golden file. The manifest's `golden.history`
(`src/orcapod/contexts/data/<version>.json`) lists every file with the sha256 of its bytes;
a release appends a revision with `added: {}`, pinned in
Expand All @@ -506,8 +515,10 @@ Orcapod stores or embeds in a name is pinned in the golden folder of the current
a migration assessment: `generate_golden_hashes.py --remove <entry>` (refuses an entry a
release lists), then the generator writes the new value.
- The release pull request runs `generate_golden_hashes.py --release X.Y.Z`. CI checks HEAD
against every release tag (`scripts/check_golden_tags.py`). Never mark a golden entry
`skip` or `xfail`; the only skip is the optional-package rule (`torch`).
against every release tag (`scripts/check_golden_tags.py`); the tag of a withdrawn release
is checked against its `withdrawn` record instead, so the check passes whether or not that
tag still exists. Never mark a golden entry `skip` or `xfail`; the only skip is the
optional-package rule (`torch`).
- Never move or rename the fixtures in `tests/test_hashing/golden/cases.py`: function pods,
dataclasses and pydantic models hash with their module path.

Expand Down Expand Up @@ -856,7 +867,8 @@ generation of hashing; rows of the current data context are the current entries.
(manifest), with one releases line per library release (ITL-946). Only its golden
history grows, by appending. A change that moves a released golden hash value, for any
reason (a logical type's own change included), needs a new data context version with its
own manifest file, never an edit of a released one.
own manifest file, never an edit of a released one. A withdrawn release freezes nothing:
its pins move to a withdrawn record (0.3.0, ITL-1255: v1 is open until 0.4.0).
- In memory the data context column is still _context_key (ColumnConfig context).
- A writer refuses an input of another data context than its own
(contexts/stored.require_same_data_context, InputValidationError naming both keys) before
Expand Down
5 changes: 4 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
# Changelog

## [0.3.0] — 2026-11-17
## [0.3.0] — 2026-11-17 (withdrawn)
Comment thread
eywalker marked this conversation as resolved.
Resolved

Withdrawn on 2026-11-17, before anyone used it: removed from PyPI, its tag and GitHub Release
deleted. Its changes ship in 0.4.0; data context v1 freezes there.

### Breaking changes and migration

Expand Down
18 changes: 15 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -498,6 +498,15 @@ Orcapod stores or embeds in a name is pinned in the golden folder of the current

- A failing golden entry means a hash moved. Treat it as a bug in the change, not a stale
file. Never edit or regenerate a golden file to make a test pass.
- Data context v1 is open until the 0.4.0 release (developer, November 17, 2026, ITL-1255).
0.3.0 was withdrawn on 2026-11-17, before anyone used it: its release revision is out of the
golden history, and the pins it shipped are kept under `withdrawn` in
`released_manifests.json`, with the date and the reason, so no v1 file is released today.
Until 0.4.0, a change to any v1 golden value needs the developer's approval of the exact
list of entries that move, each explained; it is made with `--remove <entry>`, then the
generator writes the new value. `--release 0.4.0` freezes v1. Any version published
before it, a pre-release included, would freeze v1 there: the Release workflow requires
the release step for every version. A withdrawn version is never released again.
- The unit of immutability is the golden file. The manifest's `golden.history`
(`src/orcapod/contexts/data/<version>.json`) lists every file with the sha256 of its bytes;
a release appends a revision with `added: {}`, pinned in
Expand All @@ -517,8 +526,10 @@ Orcapod stores or embeds in a name is pinned in the golden folder of the current
a migration assessment: `generate_golden_hashes.py --remove <entry>` (refuses an entry a
release lists), then the generator writes the new value.
- The release pull request runs `generate_golden_hashes.py --release X.Y.Z`. CI checks HEAD
against every release tag (`scripts/check_golden_tags.py`). Never mark a golden entry
`skip` or `xfail`; the only skip is the optional-package rule (`torch`).
against every release tag (`scripts/check_golden_tags.py`); the tag of a withdrawn release
is checked against its `withdrawn` record instead, so the check passes whether or not that
tag still exists. Never mark a golden entry `skip` or `xfail`; the only skip is the
optional-package rule (`torch`).
- Never move or rename the fixtures in `tests/test_hashing/golden/cases.py`: function pods,
dataclasses and pydantic models hash with their module path.

Expand Down Expand Up @@ -884,7 +895,8 @@ current data context are the current entries.
(`manifest`), with one `releases` line per library release (ITL-946). Only its `golden`
history grows, by appending. A change that moves a released golden hash value, for any
reason (a logical type's own change included), needs a new data context version with its
own manifest file, never an edit of a released one.
own manifest file, never an edit of a released one. A withdrawn release freezes nothing:
its pins move to a `withdrawn` record (0.3.0, ITL-1255: v1 is open until 0.4.0).
- In memory the data context column is still `_context_key` (`ColumnConfig(context=True)`).
- A writer refuses an input of another data context than its own
(`contexts/stored.require_same_data_context`, `InputValidationError` naming both keys)
Expand Down
31 changes: 18 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,22 +11,25 @@ provenance token.

## Installation

Orcapod supports Python 3.11, 3.12 and 3.13 and is published on PyPI as `orcapod`:
Orcapod supports Python 3.11, 3.12 and 3.13 and is published on PyPI as `orcapod`.
No release after 0.1.4 is available there yet: 0.2.0 was never published, and 0.3.0 was
withdrawn on 2026-11-17. Until 0.4.0 is released, install the development version from the
repository.

The recommended way to install it is [uv](https://docs.astral.sh/uv/): `uv add` adds
Orcapod to a uv project (start one with `uv init` in a new directory), and `uv run` runs
your code in that project's environment:

```bash
uv add orcapod
uv add git+https://github.com/metamorphic-labs/orcapod-python
```

In an environment not managed by uv, use `pip install orcapod` instead. A plain
`uv add orcapod` installs the latest stable release; to try a release candidate, name it,
for example `uv add "orcapod==0.4.0rc1"`.
In an environment not managed by uv, use
`pip install git+https://github.com/metamorphic-labs/orcapod-python` instead. Once 0.4.0 is
out, `uv add orcapod` (or `pip install orcapod`) installs the latest stable release; to try a
release candidate, name it, for example `uv add "orcapod==0.4.0rc1"`.

0.3.0 is the first release on PyPI since 0.1.4; 0.2.0 was never published to PyPI. Tables
written by 0.2.0 or earlier are not read by 0.3.0: see
Tables written by 0.2.0 or earlier are not read by this version: see
[Upgrading from 0.2.0](https://github.com/metamorphic-labs/orcapod-python/blob/main/CHANGELOG.md#upgrading-from-020)
(it also covers 0.1.4).

Expand Down Expand Up @@ -112,20 +115,22 @@ session.

## Status

Orcapod is before version 1.0. In 0.3.0:
Orcapod is before version 1.0. In the development version (0.4.0 is next):

- **Hash values are frozen for data context v1.** Hashes name stored tables and find stored
results again, so a hash value released in 0.3.0 does not change in a later release of the
same data context; CI checks this. See
- **Hash values of data context v1 freeze at 0.4.0.** Hashes name stored tables and find
stored results again, so a hash value released in 0.4.0 will not change in a later release
of the same data context; CI checks this. Until 0.4.0 a v1 hash may still change, only by
an approved and explained change (0.3.0, which first shipped v1, was withdrawn). See
[Golden hashes](https://github.com/metamorphic-labs/orcapod-python/blob/main/docs/reference/golden-hashes.md).
- **Library value types are provisional.** The hashes of `numpy.ndarray`, `torch.Tensor`, and
pandas and polars `DataFrame` and `Series` values are provisional until the library value
type audit (after 0.4.0rc1) and may change then; results computed from them may then be
computed again. Using one issues `ProvisionalHashWarning` once per
type per process.
- **Stores from earlier versions are not read as current.** Tables written by 0.1.x or 0.2.0
are neither read as results nor modified, and results stored by a development build of 0.3
are not found again: 0.3.0 computes them again. Read
are neither read as results nor modified, and results stored by a development build before
v1 was released may not be found again by a later build, which then computes them again.
Read
[Upgrading from 0.2.0](https://github.com/metamorphic-labs/orcapod-python/blob/main/CHANGELOG.md#upgrading-from-020)
before running a job over an old store, in particular one with side-effect nodes.
- **Connector databases cannot hold function nodes yet.** A job over a SQLite, PostgreSQL or
Expand Down
40 changes: 33 additions & 7 deletions RELEASING.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,8 @@ setting, secret or environment.
`nauticalab/orcapod-python`: `main` was pushed there by fast-forward at a release point, the
Release workflow ran there, and the tag was fetched back into `metamorphic-labs`
(ITL-1007). Every release tag up to `v0.3.0` is also here, on the same commit; `v0.2.0` (a
tag only, never published) exists only here and stays as it is.
tag only, never published) exists only here and stays as it is. `0.3.0` was withdrawn on
2026-11-17 (see *Withdrawn releases*); the next release is `0.4.0`.

## Branching Model

Expand Down Expand Up @@ -120,13 +121,38 @@ against are in `docs/reference/golden-hashes.md`.
(before 1.0) or a major one, never in a patch.
- When a new data context starts, the old context's golden files and code move to
`orcapod-sanctuary`; this repository carries only the current context's golden folder.
- Data context v1 is open until `0.4.0` (developer, November 17, 2026, ITL-1255): `0.3.0` is
withdrawn, so no v1 golden file is released. Until `0.4.0`, a change to a v1 value needs the
developer's approval of the exact list of entries that move, each explained. The release
step of `0.4.0` freezes v1. Every published version needs the release step, a pre-release
included (the gate below refuses to tag one without it), so **publishing `0.4.0rc1` would
freeze v1 at the release candidate**: plan the release around it.
- Release tags are protected on `metamorphic-labs/orcapod-python` (and on
`nauticalab/orcapod-python`, the release repository up to `0.3.0`): the repository ruleset
`protect-release-tags` (target `refs/tags/v*`; rules deletion, update and non-fast-forward;
no bypass actors) is in place since 2026-11-15. A `v*` tag can be created, as the Release workflow does, but never moved
or deleted. This is a precondition of the CI tag check (`scripts/check_golden_tags.py`),
which compares HEAD with every release tag of the current data context. Agents never change
repository settings.
repository settings. The one exception is the tag `v0.3.0` of the withdrawn release, which
the developer deletes (see *Withdrawn releases*).

### Withdrawn releases

`0.3.0` (2026-11-17) was withdrawn by the developer on 2026-11-17, before anyone used it, so
that data context v1 is frozen at `0.4.0` instead, after every open question that can move a
golden value is settled (ITL-1255). The developer's acts, never an agent's: withdrawing the
version from PyPI (yanked or deleted), lifting `protect-release-tags` to delete the tag
`v0.3.0` (here and on `nauticalab`), and deleting its GitHub Release. In the repository the
release revision left the v1 golden history and its pins moved to the `withdrawn` record in
`src/orcapod/contexts/released_manifests.json`, with the date and the reason
(`docs/reference/golden-hashes.md`, *Withdrawing a release*). The tag check compares the tag,
while it exists, with that record, so CI passes before and after the tag is deleted.

- The version `0.3.0` is never used again: PyPI never takes a version twice, and
`generate_golden_hashes.py --release` and the release gate refuse a withdrawn version.
- The version of a build comes from the git tags (`hatch-vcs`). After `v0.3.0` is deleted, a
build from `main` reports a `0.2.x` development version until the next tag exists, so a
project that requires `orcapod>=0.3.1.dev0` no longer resolves to a build of `main`.

## Cutting a Release

Expand Down Expand Up @@ -219,10 +245,9 @@ secrets or environments:
5. The `LINEAR_ACCESS_KEY` repository secret, for the Linear jobs (see *Linear Release
System*). Codecov needs no secret (see *Coverage uploads*).
6. Before the first release cut here: the latest GitHub Release in this repository is
`v0.1.2` (the GitHub Release of `v0.3.0` is on `nauticalab`), and the generated
release notes start at the latest earlier Release. Create a GitHub Release for `v0.3.0`
here (its notes copied from `nauticalab`) so that the next notes start at `v0.3.0`, or edit
the next Release's notes after the run.
`v0.1.2` (`0.3.0` is withdrawn and has none), and the generated release notes start at the
latest earlier Release. Edit the notes of `0.4.0` after the run, or create a GitHub Release
for `v0.2.0` here first so that the next notes start there.

### Coverage uploads

Expand Down Expand Up @@ -333,7 +358,8 @@ failure in a later job leaves the tag behind.
| `linear-sync` | Unknown: it needs only `build` and runs in parallel with publishing, so it can fail before anything is published | Check `publish-testpypi` and `publish-pypi` first and recover them by their rows above. Then re-run `linear-sync`; it does not affect publishing |
| `linear-complete` | Release is published (it runs only after `publish-pypi` and `linear-sync` succeed) | Re-run the failed job; it does not affect publishing |

Never delete a tag whose version has been published to any package index.
Never delete a tag whose version has been published to any package index. The one
exception is `v0.3.0`, whose release the developer withdrew (see *Withdrawn releases*).

## Release Lineage

Expand Down
14 changes: 14 additions & 0 deletions changelog.d/ITL-1255.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
### Changed

- **0.3.0 is withdrawn, and data context v1 is open until 0.4.0.** 0.3.0 was withdrawn on
2026-11-17, before anyone used it, so that v1 of every hash freezes at 0.4.0 instead. The
repository records the withdrawal: the 0.3.0 release revision left the v1 golden history,
and the pins it shipped are kept under `withdrawn` in
`src/orcapod/contexts/released_manifests.json`, with the date and the reason. A withdrawn
version is never released again. No hash value changed. Until 0.4.0, a change to a v1 hash
value is made only with the developer's approval of the exact list of entries that move,
each explained; see *Withdrawing a release* in `docs/reference/golden-hashes.md`.
(ITL-1255)
- **Install from the repository until 0.4.0.** No release after 0.1.4 is on PyPI, so a plain
`uv add orcapod` installs 0.1.4; install the development version with
`uv add git+https://github.com/metamorphic-labs/orcapod-python`. (ITL-1255)
8 changes: 6 additions & 2 deletions docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,15 @@ with `uv init` (in a new directory), add Orcapod to it, and run your scripts wit

```bash
uv init
uv add orcapod
uv add git+https://github.com/metamorphic-labs/orcapod-python
uv run python my_pipeline.py
```

In an environment not managed by uv, `pip install orcapod` installs it instead.
In an environment not managed by uv, `pip install git+https://github.com/metamorphic-labs/orcapod-python` installs it instead.

Until 0.4.0 is released, these commands install the development version from the repository:
PyPI has no current version (0.3.0 was withdrawn, and a plain `uv add orcapod` would install
0.1.4). Once 0.4.0 is out, `uv add orcapod` (or `pip install orcapod`) installs it.

## Creating a source

Expand Down
17 changes: 10 additions & 7 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,22 +8,25 @@ content-addressable, and reproducible by design.
## Installation

Orcapod supports Python 3.11, 3.12 and 3.13 and is published on PyPI as `orcapod`, under the
MIT License:
MIT License.
No release after `0.1.4` is available there yet: `0.2.0` was never published, and `0.3.0` was
withdrawn on 2026-11-17. Until `0.4.0` is released, install the development version from the
repository.

The recommended way to install it is [uv](https://docs.astral.sh/uv/): `uv add` adds
Orcapod to a uv project (start one with `uv init` in a new directory), and `uv run` runs
your code in that project's environment:

```bash
uv add orcapod
uv add git+https://github.com/metamorphic-labs/orcapod-python
```

In an environment not managed by uv, use `pip install orcapod` instead. A plain
`uv add orcapod` installs the latest stable release; to try a release candidate, name it,
for example `uv add "orcapod==0.4.0rc1"`.
In an environment not managed by uv, use
`pip install git+https://github.com/metamorphic-labs/orcapod-python` instead. Once `0.4.0` is
out, `uv add orcapod` (or `pip install orcapod`) installs the latest stable release; to try a
release candidate, name it, for example `uv add "orcapod==0.4.0rc1"`.

`0.3.0` is the first release on PyPI since `0.1.4`; `0.2.0` was never published to PyPI.
Tables written by `0.2.0` or earlier are not read by `0.3.0`: see
Tables written by `0.2.0` or earlier are not read by this version: see
`CHANGELOG.md` in the repository, *Upgrading from 0.2.0*
(it also covers `0.1.4`).

Expand Down
Loading
Loading