Skip to content

docs(readme): rewrite the top-level README for the v0.3.0 release (ITL-1180) - #412

Merged
eywalker merged 3 commits into
mainfrom
eywalker/itl-1180-rewrite-the-top-level-readme-for-the-v030
Nov 17, 2026
Merged

eywalker merged 3 commits into
mainfrom
eywalker/itl-1180-rewrite-the-top-level-readme-for-the-v030

Conversation

@eywalker

@eywalker eywalker commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Fixes ITL-1180

Rewrites README.md (the first page of the public repository and the PyPI description) for the v0.3.0 release: 278 lines down to 160.

New README: sections

  1. Orcapod: what it is (sources, streams of tag + data rows, function pods, operators, jobs that store results and compute only what is not stored, provenance per row and per value).
  2. Installation: pip install orcapod / uv add orcapod, Python 3.11 to 3.13, a table of every extra in pyproject.toml (ray, redis, postgresql, spiraldb, torch (new in the table), all), the optional Graphviz dot binary. The "0.2.0 is not on PyPI, latest is 0.1.4" sentence is replaced by the release PR's note: 0.3.0 is the first PyPI release since 0.1.4, 0.2.0 was never on PyPI, with a link to "Upgrading from 0.2.0".
  3. A quick example: a DictSource, one function pod, a PipelineJob on an InMemoryArrowDatabase; the first run computes 2 rows, then a new job over the same database with one added row computes 1 and reads 2 from the store. Both blocks run in tests/test_docs/test_doc_samples.py, every printed line shown.
  4. Documentation: Getting Started, Tutorial, Concepts, Reference (hashing, golden hashes), Troubleshooting, CHANGELOG. All absolute https://github.com/nauticalab/orcapod-python/... links, so they work on PyPI; no link to a hosted docs site.
  5. Status: hash values frozen for data context v1; library value types provisional (ProvisionalHashWarning); stores from 0.1.x / 0.2.0 / 0.3 development builds not read as current (link to "Upgrading from 0.2.0" in the CHANGELOG); connector databases refuse function nodes.
  6. Related packages: orcapod-extension-spikeinterface (unchanged).
  7. Development: uv sync, the four required CI commands, PostgreSQL tests in one sentence with a link to their workflow, releasing with a link to RELEASING.md.
  8. License: MIT.

Every factual claim and what backs it

Claim Backed by
Sources, streams of tag + data rows, typed and immutable docs/index.md, docs/getting-started.md ("Creating a source"), docs/concepts/streams.md
Function pods compute values with plain Python functions; operators join, filter, group, reshape without computing values docs/getting-started.md, docs/concepts/operators.md, CLAUDE.md operator/function pod boundary
A job stores every function result; a later run computes only what is not stored docs/concepts/pipeline-job.md (first paragraph)
Every row records its source and source row; every value carries a provenance token docs/concepts/sources.md "How sources add provenance columns" (system tags, _source_ columns)
Python 3.11, 3.12, 3.13; PyPI name orcapod pyproject.toml (requires-python, classifiers, name)
Extras: ray → RayExecutor; redis → RedisCacher; postgresql → connector, table source, file hash cache; spiraldb → connector and table source; torch → PyTorch, any installed torch works; all pyproject.toml [project.optional-dependencies] and its torch comment; src/orcapod/hashing/string_cachers.py; docs/concepts/sources.md; docs/concepts/file-hash-caching.md
0.3.0 is the first PyPI release since 0.1.4; 0.2.0 never published to PyPI; tables of 0.2.0 or earlier not read by 0.3.0 the release PR #329's README note (carried in here); CHANGELOG.md "Upgrading from 0.2.0"
Graphviz dot optional, only for rendering graphs previous README, docs/index.md, CLAUDE.md
Example output (2 computed; then 1 computed, 2 from store) executed by tests/test_docs/test_doc_samples.py
Stored results found again by input values docs/concepts/pipeline-job.md "What a re-run computes"
InMemoryArrowDatabase lives for the process; DeltaTableDatabase writes to disk, found again in a new session docs/concepts/pipeline-job.md "The database"
Tutorial is eleven parts docs/tutorial/00–10, docs/index.md
Released hash values do not change within data context v1; CI checks it docs/reference/golden-hashes.md, CHANGELOG.md 0.3.0 (ITL-946)
Library value types (numpy, torch, pandas/polars DataFrame and Series) provisional, may change in 0.4, ProvisionalHashWarning once per type per process docs/concepts/arrays.md ("will be reviewed for v0.4 and may change then"), CHANGELOG.md 0.3.0 (ITL-1068)
0.1.x / 0.2.0 tables neither read as results nor modified; recomputed CHANGELOG.md 0.3.0 (ITL-944, ITL-954, ITL-1154)
Results from a 0.3 development build not found again; side effects delivered again (hence "in particular one with side-effect nodes") CHANGELOG.md 0.3.0 (ITL-1154)
A job over SQLite/PostgreSQL/SpiralDB refuses every function node when built; use in-memory or Delta CHANGELOG.md 0.3.0 Known limitations (ITL-1162), docs/concepts/pipeline-job.md "Connector databases"
SpikeInterface package separate, not on PyPI previous README (kept unchanged)
The four CI commands CLAUDE.md "Lint, format and type checks"
PostgreSQL tests skipped unless --postgres; PG* variables tests/conftest.py, .github/workflows/run-postgres-tests.yml (the old README's command lacked --postgres and so skipped every test)
Releases via the Release workflow RELEASING.md
MIT license pyproject.toml, LICENSE

Note on the #upgrading-from-020 link

The README links CHANGELOG.md#upgrading-from-020 twice. That heading (#### Upgrading from 0.2.0, under 0.3.0's "Breaking changes and migration") came with the release PR (#329), now on main and merged into this branch; the anchor resolves.

Other changes

  • docs/dev/notes/pipeline-workflow-notes.md: the README section "Lower-level details on pipeline workflow (tentative)" moved here, text unchanged (a short preface added). docs/dev/orcapod-design.md and the concept pages do not cover it (observer contextualization, DataLogger, CapturedLogs, the Ray capture path).
  • .gitignore: !docs/dev/notes, since docs/dev/* is ignored except its listed subfolders.
  • No changelog fragment: a README change has nothing to tell users (changelog.d/README.md).

Checks

  • uv run pytest tests/test_docs: 69 passed (README samples, mkdocs build --strict).
  • uv build: the sdist contains README.md; uvx twine check: PASSED for sdist and wheel.
  • ruff check, ruff format --check: clean. Whole-tree basedpyright: exit 0.
  • scripts/design_issues.py --check, scripts/assemble_changelog.py --check: OK.
  • No test or script asserts README text (git grep README tests/ scripts/).

🤖 Generated with Claude Code

…L-1180)

Say what Orcapod is, give installation for 0.3.0 with every extra, a quick
example run by the doc sample test (a job, then a re-run that computes only
the new row), links that work on PyPI, a short status section and a short
development section. Move the internal "Lower-level details on pipeline
workflow (tentative)" notes unchanged to docs/dev/notes/, and drop the
stale "0.2.0 is not on PyPI" sentence from docs/index.md too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@linear-code

linear-code Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
ITL-1180 Rewrite the top-level README for the v0.3.0 public release

Overview

README.md is the first page a visitor sees on the public repository and the project description on PyPI. Today it does not say what Orcapod is, has no example, states that "0.2.0 is not on PyPI, where the latest release is 0.1.4", and ends with about 170 lines headed "Lower-level details on pipeline workflow (tentative)" that are internal design notes. The developer asked for it to be fully updated before the v0.3.0 release (November 16, 2026).

Goals & Success Criteria

  • A reader learns in the first screen what Orcapod is, who it is for and what it gives them, in plain words.
  • Installation instructions that are true for v0.3.0 (PyPI name, supported Python versions, optional extras).
  • One short, complete example that runs: a source, a function pod, a job with a database, a re-run that computes nothing. It is executed by tests/test_docs/test_doc_samples.py like every other README sample, with its printed output shown.
  • Pointers to the documentation by section (getting started, tutorial, concepts, reference, troubleshooting) and to the changelog, with links that work both on GitHub and on PyPI.
  • A short, honest status section: what v0.3.0 freezes (hash values of data context v1), what is provisional (library value types), what a user upgrading from 0.1.x or 0.2.0 must know, with a link to the upgrade notes.
  • Development information (tests, CI, PostgreSQL tests, releasing) is short and points to the files that hold the detail; the "tentative" internal notes leave the README (moved under docs/dev/ if still useful, otherwise removed).
  • No statement about anything removed or moved by ITL-1169 (notebooks, design_issues/ at top level, superpowers/).
  • Nothing internal: no internal hostnames, project ids or private repository names.

Scope & Boundaries

In scope:

  • README.md, and the tests or docs lines that depend on its text.

Out of scope:

  • Rewriting the documentation pages (ITL-1170).
  • Changing library behaviour. A defect found while writing the example gets its own issue.

Dependencies & Risks

Review in Linear

@coderabbitai

coderabbitai Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Explain Change →

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Edgar Y. Walker and others added 2 commits November 16, 2026 18:52
…and link the upgrade notes (ITL-1180)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rite-the-top-level-readme-for-the-v030

# Conflicts:
#	README.md
#	docs/index.md
@eywalker
eywalker merged commit ac7751d into main Nov 17, 2026
16 checks passed
@eywalker
eywalker deleted the eywalker/itl-1180-rewrite-the-top-level-readme-for-the-v030 branch November 17, 2026 02:56
eywalker pushed a commit that referenced this pull request Nov 17, 2026
…al tags slice (ITL-1023)

Clean merge: README rewrite (#412), changelog fold (#329), v0.3.0 date and
golden release record (#413). Data context v1 is frozen as release 0.3.0;
the golden folder and src/orcapod/contexts are byte-identical to main. The
slice's changelog fragments (ITL-1017, ITL-1019, ITL-1022, ITL-1023) stay
fragments for the next release.

Refs ITL-1023

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant