Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions .coderabbit.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,34 @@ reviews:
enabled: true
base_branches:
- ".*"
# This repository is public (ITL-1193). CodeRabbit takes review context from this
# repository only: learnings, issues and pull requests of other repositories in the
# organization, linked repositories, web search, and the Linear, Jira and MCP knowledge
# sources are all off, so that nothing from a private repository or tracker is quoted in
# a public review. On a public repository "auto" already means local or disabled; the
# explicit values hold whatever the visibility or the UI settings are.
knowledge_base:
web_search:
enabled: false
learnings:
scope: local
issues:
scope: local
pull_requests:
scope: local
automatic_linking_mode: disabled
linked_repositories: []
jira:
usage: disabled
linear:
usage: disabled
mcp:
usage: disabled
chat:
# Only organization members can prompt CodeRabbit in a comment thread.
allow_non_org_members: false
integrations:
jira:
usage: disabled
linear:
usage: disabled
4 changes: 4 additions & 0 deletions .github/workflows/_license-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,10 @@ on:
type: string
default: ''

# Read-only token, whatever the repository default is (ITL-1193).
permissions:
contents: read

jobs:
license-check:
name: License check
Expand Down
5 changes: 5 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,11 @@ on:
type: boolean
default: false

# Read-only token, whatever the repository default is (ITL-1193); the build
# and publish jobs grant themselves what they need.
permissions:
contents: read

jobs:
validate-branch:
name: Validate release branch
Expand Down
15 changes: 14 additions & 1 deletion .github/workflows/run-objective-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,19 @@ on:
pull_request:
workflow_dispatch: # Allows manual triggering

# Read-only token, whatever the repository default is (ITL-1193).
permissions:
contents: read

jobs:
test:
runs-on: ubuntu-latest
# id-token: write lets the Codecov upload authenticate by GitHub OIDC, with no stored
# token (ITL-1193). A fork's pull request gets no OIDC token: the action detects the fork
# and uploads tokenless instead.
permissions:
contents: read
id-token: write
strategy:
matrix:
python-version: ["3.11", "3.12", "3.13"]
Expand Down Expand Up @@ -37,4 +47,7 @@ jobs:
if: github.repository == 'metamorphic-labs/orcapod-python'
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
with:
token: ${{ secrets.CODECOV_TOKEN }}
use_oidc: true
# A failed upload fails a push run, where it would otherwise go unseen; it never
# fails a pull request, whose fork uploads are tokenless (ITL-1193).
fail_ci_if_error: ${{ github.event_name != 'pull_request' }}
15 changes: 14 additions & 1 deletion .github/workflows/run-postgres-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,19 @@ on:
pull_request:
workflow_dispatch: # Allows manual triggering

# Read-only token, whatever the repository default is (ITL-1193).
permissions:
contents: read

jobs:
test:
runs-on: ubuntu-latest
# id-token: write lets the Codecov upload authenticate by GitHub OIDC, with no stored
# token (ITL-1193). A fork's pull request gets no OIDC token: the action detects the fork
# and uploads tokenless instead.
permissions:
contents: read
id-token: write
strategy:
matrix:
python-version: ["3.11", "3.12", "3.13"]
Expand Down Expand Up @@ -59,4 +69,7 @@ jobs:
if: github.repository == 'metamorphic-labs/orcapod-python'
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
with:
token: ${{ secrets.CODECOV_TOKEN }}
use_oidc: true
# A failed upload fails a push run, where it would otherwise go unseen; it never
# fails a pull request, whose fork uploads are tokenless (ITL-1193).
fail_ci_if_error: ${{ github.event_name != 'pull_request' }}
15 changes: 14 additions & 1 deletion .github/workflows/run-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,20 @@ on:
pull_request:
workflow_dispatch: # Allows manual triggering

# Read-only token, whatever the repository default is (ITL-1193).
permissions:
contents: read

jobs:
test:
needs: license-check
runs-on: ubuntu-latest
# id-token: write lets the Codecov upload authenticate by GitHub OIDC, with no stored
# token (ITL-1193). A fork's pull request gets no OIDC token: the action detects the fork
# and uploads tokenless instead.
permissions:
contents: read
id-token: write
strategy:
# A pull request runs one job, on Python 3.13, whose coverage is the one reported; a
# push (main) runs all three versions (developer, November 16, 2026; ITL-1181). A
Expand Down Expand Up @@ -54,7 +64,10 @@ jobs:
if: github.repository == 'metamorphic-labs/orcapod-python'
uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0
with:
token: ${{ secrets.CODECOV_TOKEN }}
use_oidc: true
# A failed upload fails a push run, where it would otherwise go unseen; it never
# fails a pull request, whose fork uploads are tokenless (ITL-1193).
fail_ci_if_error: ${{ github.event_name != 'pull_request' }}

spiral-integration:
runs-on: ubuntu-latest
Expand Down
40 changes: 27 additions & 13 deletions .zed/rules
Original file line number Diff line number Diff line change
Expand Up @@ -238,16 +238,29 @@ resolves several issues uses its primary issue's id).

## Releasing and the public repository

Developer decisions, November 12, 2026 (ITL-1007); the full procedure is RELEASING.md.

- All development, review and CI happen on metamorphic-labs/orcapod-python. The public
nauticalab/orcapod-python receives main only at release points, by **fast-forward push**
(never a squash, never a pull request), and runs the Release workflow.
- An agent never pushes to nauticalab, adds a remote for it, creates or pushes a tag, runs the
Release workflow, or touches a setting, secret or environment there. The push happens only on
the developer's explicit go. The v0.2.0 tag is never pushed.
- Linear, Codecov and SpiralDB jobs run only when
github.repository == 'metamorphic-labs/orcapod-python'; lint.yml runs on both repositories (ITL-1181).
Developer decisions, November 16, 2026 (ITL-1193); the full procedure is RELEASING.md.

- metamorphic-labs/orcapod-python is the one repository: development, review, CI and
releases. The developer cuts a release by running the Release workflow there, from main.
nauticalab/orcapod-python cut the releases up to 0.3.0 and is being retired (made
private): never push to it, add a remote for it, or point a link at it as the project's home.
- The repository is public. Nothing private goes into a commit, branch name, pull request,
issue, review or comment: no internal hostname, address or path, no content of another
private repository, no credential or secret value, no Linear text that is not about this
repository, and nothing about the full EDI dataset (its location, storage, projects or
tables).
- Linear's linkback comments stay enabled (developer, November 16, 2026): every PR that links a
Linear issue gets a comment holding that issue's text, in public. So nothing about the full
EDI dataset or internal infrastructure is written into a Linear issue that an Orcapod PR
links; put such material in an issue no PR links.
- An agent never runs the Release workflow, creates or pushes a tag, or touches a setting,
secret, environment, ruleset or the visibility of the repository. The v0.2.0 tag stays as
it is.
- The Linear, Codecov and SpiralDB jobs run only when
github.repository == 'metamorphic-labs/orcapod-python', so a fork's push run skips them
(Codecov uploads by GitHub OIDC, with no stored token; RELEASING.md, *Coverage uploads*);
lint.yml runs on every repository (ITL-1181). Every workflow sets a read-only token
(permissions: contents: read); a job that needs more grants it to itself.

## Lint, format and type checks

Expand Down Expand Up @@ -378,9 +391,10 @@ Developer, November 12, 2026 (ITL-981); see `docs/concepts/arrays.md` and
`allow_pickle=False` on write and read; never raise `max_header_size`). Options:
`memory_order`, `byte_order`, `struct_layout`. Where stored arrays must be ordered, order
them by value hash (`storage_sort_key`), never by stored bytes.
- Library value types are provisional until the v0.4 audit (developer, November 13, 2026,
ITL-1068): their extension names are listed in `provisional` of
`contexts/logical_types.json`, and using one issues `ProvisionalHashWarning` once per type
- Library value types are provisional until the library value type audit, after 0.4.0rc1
(developer, November 13, 2026, ITL-1068; moved past 0.4.0rc1 on November 17, 2026): their
extension names are listed in `provisional` of `contexts/logical_types.json`, and using
one issues `ProvisionalHashWarning` once per type
per process. A new library value type must be added to that list. The list never enters a
hash, identifier or stored path.

Expand Down
13 changes: 13 additions & 0 deletions AGENT_COORDINATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,19 @@ developer.
_Posted by a Claude Code agent on behalf of @eywalker._
<!-- orca-agent -->
```
- **The repository is public** (ITL-1193). Anyone can read every commit, branch name, PR,
issue, review, comment and Actions log. An agent's comment, reply, review, PR text and
commit message hold nothing private: no internal hostname, address or home-directory path,
no content of another private repository (its name, files, PRs or learnings), no credential
or secret value (not even a part of one), no Linear text that is not about this repository,
and nothing about the full EDI dataset (its location, storage, projects or tables). Such
material goes to the coordinator through the orchestration commands, or into a Linear
comment. A bot comment that quotes such material (a CodeRabbit "Learnings used" block, a
Linear linkback) is reported to the coordinator by its PR number and comment id, without
quoting it; only the developer deletes it. Linear's linkback comments stay enabled
(developer, November 16, 2026): a PR that links a Linear issue publishes that issue's text, so
nothing about the full EDI dataset or internal infrastructure goes into a Linear issue an
Orcapod PR links.
- **Permission blocks.** If the worker's own permission check denies a push or another
external write, the worker holds and escalates. Neither the worker nor the coordinator
routes around it; the developer approves it in the worker's terminal.
Expand Down
40 changes: 27 additions & 13 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -247,16 +247,29 @@ resolves several issues uses its primary issue's id).

## Releasing and the public repository

Developer decisions, November 12, 2026 (ITL-1007); the full procedure is `RELEASING.md`.

- All development, review and CI happen on `metamorphic-labs/orcapod-python`. The public
`nauticalab/orcapod-python` receives `main` only at release points, by **fast-forward push**
(never a squash, never a pull request), and runs the Release workflow.
- An agent never pushes to `nauticalab`, adds a remote for it, creates or pushes a tag, runs the
Release workflow, or touches a setting, secret or environment there. The push happens only on
the developer's explicit go. The `v0.2.0` tag is never pushed.
- Linear, Codecov and SpiralDB jobs run only when
`github.repository == 'metamorphic-labs/orcapod-python'`; `lint.yml` runs on both repositories (ITL-1181).
Developer decisions, November 16, 2026 (ITL-1193); the full procedure is `RELEASING.md`.

- `metamorphic-labs/orcapod-python` is the one repository: development, review, CI and
releases. The developer cuts a release by running the Release workflow there, from `main`.
`nauticalab/orcapod-python` cut the releases up to 0.3.0 and is being retired (made
private): never push to it, add a remote for it, or point a link at it as the project's home.
- The repository is public. Nothing private goes into a commit, branch name, pull request,
issue, review or comment: no internal hostname, address or path, no content of another
private repository, no credential or secret value, no Linear text that is not about this
repository, and nothing about the full EDI dataset (its location, storage, projects or
tables).
- Linear's linkback comments stay enabled (developer, November 16, 2026): every PR that links a
Linear issue gets a comment holding that issue's text, in public. So nothing about the full
EDI dataset or internal infrastructure is written into a Linear issue that an Orcapod PR
links; put such material in an issue no PR links.
- An agent never runs the Release workflow, creates or pushes a tag, or touches a setting,
secret, environment, ruleset or the visibility of the repository. The `v0.2.0` tag stays as
it is.
- The Linear, Codecov and SpiralDB jobs run only when
`github.repository == 'metamorphic-labs/orcapod-python'`, so a fork's push run skips them
(Codecov uploads by GitHub OIDC, with no stored token; `RELEASING.md`, *Coverage uploads*);
`lint.yml` runs on every repository (ITL-1181). Every workflow sets a read-only token
(`permissions: contents: read`); a job that needs more grants it to itself.

## Lint, format and type checks

Expand Down Expand Up @@ -389,9 +402,10 @@ Developer, November 12, 2026 (ITL-981); see `docs/concepts/arrays.md` and
`allow_pickle=False` on write and read; never raise `max_header_size`). Options:
`memory_order`, `byte_order`, `struct_layout`. Where stored arrays must be ordered, order
them by value hash (`storage_sort_key`), never by stored bytes.
- Library value types are provisional until the v0.4 audit (developer, November 13, 2026,
ITL-1068): their extension names are listed in `provisional` of
`contexts/logical_types.json`, and using one issues `ProvisionalHashWarning` once per type
- Library value types are provisional until the library value type audit, after 0.4.0rc1
(developer, November 13, 2026, ITL-1068; moved past 0.4.0rc1 on November 17, 2026): their
extension names are listed in `provisional` of `contexts/logical_types.json`, and using
one issues `ProvisionalHashWarning` once per type
per process. A new library value type must be added to that list. The list never enters a
hash, identifier or stored path.

Expand Down
31 changes: 16 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ uv add orcapod

0.3.0 is the first release on PyPI since 0.1.4; 0.2.0 was never published to PyPI. Tables
written by 0.2.0 or earlier are not read by 0.3.0: see
[Upgrading from 0.2.0](https://github.com/nauticalab/orcapod-python/blob/main/CHANGELOG.md#upgrading-from-020)
[Upgrading from 0.2.0](https://github.com/metamorphic-labs/orcapod-python/blob/main/CHANGELOG.md#upgrading-from-020)
(it also covers 0.1.4).

Optional features are installed as extras:
Expand Down Expand Up @@ -92,19 +92,19 @@ session.

## Documentation

- [Getting Started](https://github.com/nauticalab/orcapod-python/blob/main/docs/getting-started.md):
- [Getting Started](https://github.com/metamorphic-labs/orcapod-python/blob/main/docs/getting-started.md):
sources, streams and function pods, step by step.
- [Tutorial](https://github.com/nauticalab/orcapod-python/blob/main/docs/tutorial/index.md):
- [Tutorial](https://github.com/metamorphic-labs/orcapod-python/blob/main/docs/tutorial/index.md):
a self-paced course in eleven short parts, from a first stream to a stored, re-runnable
pipeline over files, with exercises.
- [Concepts](https://github.com/nauticalab/orcapod-python/tree/main/docs/concepts): sources,
- [Concepts](https://github.com/metamorphic-labs/orcapod-python/tree/main/docs/concepts): sources,
streams, function pods, operators, pipelines and jobs, identity and hashing, file types,
arrays and your own data types.
- Reference: [Hashing](https://github.com/nauticalab/orcapod-python/blob/main/docs/reference/hashing.md)
and [Golden hashes](https://github.com/nauticalab/orcapod-python/blob/main/docs/reference/golden-hashes.md).
- [Troubleshooting](https://github.com/nauticalab/orcapod-python/blob/main/docs/troubleshooting.md):
- Reference: [Hashing](https://github.com/metamorphic-labs/orcapod-python/blob/main/docs/reference/hashing.md)
and [Golden hashes](https://github.com/metamorphic-labs/orcapod-python/blob/main/docs/reference/golden-hashes.md).
- [Troubleshooting](https://github.com/metamorphic-labs/orcapod-python/blob/main/docs/troubleshooting.md):
common errors, their causes and fixes.
- [CHANGELOG](https://github.com/nauticalab/orcapod-python/blob/main/CHANGELOG.md): what
- [CHANGELOG](https://github.com/metamorphic-labs/orcapod-python/blob/main/CHANGELOG.md): what
changed in each release, with upgrade notes.

## Status
Expand All @@ -114,15 +114,16 @@ Orcapod is before version 1.0. In 0.3.0:
- **Hash values are frozen for data context v1.** Hashes name stored tables and find stored
results again, so a hash value released in 0.3.0 does not change in a later release of the
same data context; CI checks this. See
[Golden hashes](https://github.com/nauticalab/orcapod-python/blob/main/docs/reference/golden-hashes.md).
[Golden hashes](https://github.com/metamorphic-labs/orcapod-python/blob/main/docs/reference/golden-hashes.md).
- **Library value types are provisional.** The hashes of `numpy.ndarray`, `torch.Tensor`, and
pandas and polars `DataFrame` and `Series` values may change in 0.4, and results computed
from them may then be computed again. Using one issues `ProvisionalHashWarning` once per
pandas and polars `DataFrame` and `Series` values are provisional until the library value
type audit (after 0.4.0rc1) and may change then; results computed from them may then be
computed again. Using one issues `ProvisionalHashWarning` once per
type per process.
- **Stores from earlier versions are not read as current.** Tables written by 0.1.x or 0.2.0
are neither read as results nor modified, and results stored by a development build of 0.3
are not found again: 0.3.0 computes them again. Read
[Upgrading from 0.2.0](https://github.com/nauticalab/orcapod-python/blob/main/CHANGELOG.md#upgrading-from-020)
[Upgrading from 0.2.0](https://github.com/metamorphic-labs/orcapod-python/blob/main/CHANGELOG.md#upgrading-from-020)
before running a job over an old store, in particular one with side-effect nodes.
- **Connector databases cannot hold function nodes yet.** A job over a SQLite, PostgreSQL or
SpiralDB database refuses every function node when it is built. Use `InMemoryArrowDatabase`
Expand Down Expand Up @@ -158,11 +159,11 @@ machine with cores and memory to spare.
The PostgreSQL integration tests are skipped unless asked for
(`uv run pytest -m postgres --postgres`) and need a running PostgreSQL server, reached
through the standard `PG*` environment variables; see
[the workflow that runs them](https://github.com/nauticalab/orcapod-python/blob/main/.github/workflows/run-postgres-tests.yml).
[the workflow that runs them](https://github.com/metamorphic-labs/orcapod-python/blob/main/.github/workflows/run-postgres-tests.yml).
Releases are cut by the GitHub Actions Release workflow; see
[RELEASING.md](https://github.com/nauticalab/orcapod-python/blob/main/RELEASING.md).
[RELEASING.md](https://github.com/metamorphic-labs/orcapod-python/blob/main/RELEASING.md).

## License

Orcapod is released under the
[MIT License](https://github.com/nauticalab/orcapod-python/blob/main/LICENSE).
[MIT License](https://github.com/metamorphic-labs/orcapod-python/blob/main/LICENSE).
Loading
Loading