Skip to content

docs: iterate a stream directly; uv add orcapod is the recommended install (ITL-1224, ITL-1228) - #438

Merged
eywalker merged 114 commits into
mainfrom
eywalker/itl-1224-docs-iterate-a-stream-directly-for-tag
Nov 18, 2026
Merged

eywalker merged 114 commits into
mainfrom
eywalker/itl-1224-docs-iterate-a-stream-directly-for-tag

Conversation

@eywalker

Copy link
Copy Markdown
Contributor

Iterate a stream directly (ITL-1224, first commit)

A stream is iterable: StreamBase.__iter__ returns iter_data(). The documentation now writes its loops as for tag, data in stream:, starting with the first page (docs/index.md). Documentation only; no library change.

The rule

A for clause (a loop or a comprehension) iterates the stream itself. iter_data() is written only where the method is the subject of the sentence (the streams page's iter_data() section, lists of read methods) or where the iterator is passed to a function as a value: list(stream.iter_data()), sorted(stream.iter_data(), key=...), islice(stream.iter_data(), 3), next(iter(stream.iter_data())).

Applied to every user-facing page: docs/index.md, docs/getting-started.md, docs/concepts/ (arrays, data-tags, file-types, function-pods, operators, sources, streams), docs/tutorial/ (01, 04, 07). README.md, docs/troubleshooting.md and docs/reference/ had no loop to change (docs/reference/hashing.md keeps next(iter(source.iter_data())), a value). No docstring example in src/ shows such a loop.

docs/getting-started.md, the streams concept page and the first tutorial page each say once that iterating a stream is the same as calling iter_data().

Every stream kind shown is iterable

Sources, operator results, function pod streams (FunctionPodStream has its own __iter__), function and operator nodes and job nodes, and DerivedSource all iterate through StreamBase.__iter__. Two classes have iter_data() but no __iter__: SourceNode and SourceJobNode (a job's source slots, job.nodes["<slot>"]). No documentation sample loops over one, so nothing here depends on it; ITL-1225 covers it. StreamView (what run() returns) has neither and is not looped over in the docs.

Recommended install: uv add orcapod (ITL-1228, second commit)

Every place that tells a user how to install Orcapod shows uv first, as the recommended way: README.md (also the PyPI project description), docs/index.md, docs/getting-started.md (a new short Installing section), docs/tutorial/00-setup.md, docs/troubleshooting.md, and the extras in docs/concepts/sources.md (orcapod[spiraldb], with uv run spiral login), docs/concepts/arrays.md (orcapod[torch]) and docs/concepts/file-hash-caching.md (orcapod[postgresql]). uv init starts a project where a newcomer needs it, and pages that set up a project run code with uv run python .... pip install stays at most once per page, for an environment not managed by uv.

Release candidates: a plain uv add orcapod takes the latest stable release, so the docs say to name the candidate, uv add "orcapod==0.4.0rc1" (index, README, tutorial setup, and a troubleshooting entry). Checked with uv 0.13.7 against a local index holding a stable and an rc version of a package: a plain uv add picked the stable one; uv add "pkg==1.1.0rc1" and uv add "pkg>=1.1.0rc1" picked the rc; --prerelease allow also picked it and recorded >=1.1.0rc1.

No site/ file is changed (the home page hero is PR #431).

Checks

tests/test_docs (every documentation sample runs, and the site content builds): 127 passed after each commit. No changelog fragment: a documentation-only change with nothing to tell library users (changelog.d/README.md).

Fixes ITL-1224. Fixes ITL-1228. Related: ITL-1225.

🤖 Generated with Claude Code

Posted by a Claude Code agent on behalf of @eywalker.

Edgar Y. Walker and others added 30 commits November 12, 2026 21:07
Specification section 15.7: two time series sharing five keys, a third
table keyed on a subset, GroupBy, automatic join and select, a function
reading the timestamp lists and a key by name; function node on the
in-memory and Delta stores, sync and async; second run computes nothing.
Also adds the slice's implementation plan.

Refs ITL-1023

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n, GroupBy, pod checks (ITL-1023)

A referential tag is one column named in both the tag and the data role
(specification section 4.3). Sources make declared tags referential by
default; tag_only_columns keeps chosen tags plain (v0.3 behaviour).
Golden fixtures use tag_only_columns=True and no golden value moves.
Operators that do not handle references yet raise.

Refs ITL-1023

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…in parity; tests (ITL-1023)

Filters, DropNulls, SemiJoin, Pick and Index keep references; MapTags and
MapData work when they leave references alone; Batch, MergeJoin, operator
node storage, DerivedSource and Pipeline.save raise 'not supported with
referential tags yet'. The async Join merges a referenced data column as
the sync path does (section 9). Referential cases join the async/sync
equivalence matrix; its existing sources keep plain tags.

Refs ITL-1023

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he real timestamp shape (ITL-1023)

MapTags renames a referential tag in both roles (the reference survives);
tag_only=True renames the tag only and leaves a plain tag. MapData is always
a data-only move: the old referential tag stays as a plain tag. The EDI test
now has a key column 'timestamp' in both series, renamed with map_tags.

Refs ITL-1023

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rt v0.3 shapes and hashes (ITL-1023)

Existing tests built sources with tag columns and asserted v0.3 schemas,
hashes or behaviour of operators that do not take references yet; they
now pass tag_only_columns=True, which is exactly the v0.3 source. Doc
pages that explain the v0.3 tag/data split do the same; a new concepts
page documents referential tags. The materialization warning is reported
once per operator (a job validates operators again). SQLite and
PostgreSQL sources forward tag_only_columns. Adds the changelog fragment
and design issue RT1.

Refs ITL-1023

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…with plain tags (ITL-1023)

0.3.0 is the first format in which a source may be referential; 0.1.0 and
0.2.0 files hold v0.3 sources, whose configs carry no tag_only_columns and
read back with every declared tag plain, so their saved hashes match. Adds
a job saved by main at f51ce67 as a fixture and a test that it loads with
unchanged hashes.

Refs ITL-1023

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d torch tests that use Batch, MergeJoin or saved jobs (ITL-1023)

Refs ITL-1023

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Developer decision, November 12, 2026: pod(stream) on a stream with
referential tags uses ReferenceFallThroughPod, a variant that calls the
function exactly as the plain pod does (same inputs, result identity and
result store) and emits, per row, the function's outputs plus the row's
own referenced columns, so the tags stay referential. It abbreviates the
composition 'temporary per-row tag, plain pod, join back, drop the tag';
a test checks the two give the same rows with duplicate user tags.
pod.process(stream) stays the plain pod. A function node stores only the
function's result and gives the stored tag column its data role back on
read. Without references the call is exactly the plain pod.

Refs ITL-1023

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…jects; sinks; sorted merged provenance (ITL-1023)

The fall-through columns are added from the input row as Arrow
(Data.with_columns_of) and laid out from the tags, so their Arrow types
are exact and a function's returned objects are not rebuilt. Datagram
select and drop keep the Python values held beside the table. A wrapper
pod called directly keeps its plain behaviour. A side-effect function
gets a referenced column only when it names it. The merged source info of
a joined referenced column is sorted (audit D5), so it is commutative.
ColumnConfig without source drops a referenced column's source info.

Refs ITL-1023

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…kshop-slice-referential-tags-working

# Conflicts:
#	src/orcapod/core/streams/base.py
…be saved (ITL-1023)

ReferenceFallThroughPod takes a FunctionPod and implements to_config and
from_config by raising 'not supported with referential tags yet
(ITL-1019)'; only a plain FunctionPod's call form uses it. Test helpers
typed for basedpyright.

Refs ITL-1023

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…(ITL-1023)

.gitignore excludes *.csv and Parquet files; the fixture needs them.

Refs ITL-1023

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… sources (ITL-1023)

Refs ITL-1023

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…kshop-slice-referential-tags-working

# Conflicts:
#	src/orcapod/pipeline/serialization.py
#	tests/test_pipeline/test_pipeline_job.py
#	tests/test_pipeline/test_serialization.py
#	tests/test_pipeline/test_serialization_helpers.py
…kshop-slice-referential-tags-working

# Conflicts:
#	src/orcapod/core/function_pod.py
#	src/orcapod/core/operators/join.py
Refs ITL-1023

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d from equal elements (ITL-1017)

A referential GroupBy key and a Join key referential on several inputs carry one
token, merged:source_fold_v1:<sha256 hex> of the sorted member tokens
(arrow_utils.fold_source_info_tokens), instead of a list of every member's token.
Provenance keeps the shape of its data column; a list-valued merged key folds per
element; an N-ary join folds once over all inputs, sync and async.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nt RT2 (ITL-1017)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…kshop-slice-referential-tags-working

Conflicts: arrow_table_stream.py (kept reference_tags and main's _identity_table),
semijoin.py imports, function_node.py (data_ids beside the fall-through comment),
test_async_sync_equivalence.py (main's provenance cases, ITL-838 fixed). New tests
from main that assume v0.3 sources pin tag_only_columns=True, as the slice does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in ITL-1070 (GroupBy keeps only its by tags), ITL-1068
(ProvisionalHashWarning) and ITL-1067 (handle_config honours all_info).

GroupBy follows main's rule: a referential tag that is not in `by` is
dropped in both roles, with its source-info column, in the rows, in the
schema prediction and in the output's reference tags. A referential key
keeps its one merged source-info token, unchanged.

Removed the slice tests that only served the old "non-key referential tag
becomes a list" behaviour, rewrote the EDI end-to-end test with timestamp
declared as data, and adapted main's new code to referential-by-default
sources: the side-effect stream's table shows a referential column once,
and the provisional hash warning tests keep their source tags plain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resolves the SpiralDB source docstring: main's https server URL beside the
slice's tag_only_columns argument.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ist (ITL-1023)

Developer decision, November 13, 2026: GroupBy drops a referential tag that
is not in `by` as a tag only, keeping its data. Its data column is aggregated
like every other data column (list, one source-info token per member) and the
members stay ordered by the non-key tags, so the list is in value order. A
plain non-key tag is still dropped entirely (ITL-1070); a referential key is
unchanged, with its pinned merged token.

The EDI end-to-end test is back to the natural pattern: timestamp referential
at the source, renamed with map_tags before grouping, no map_data, no sort in
the function.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflicts resolved:
- sources (CSV, Delta, DB table): main's reload (_build, check_for_new_data,
  ITL-1102) keeps the tag-only choice; _build passes the same reference_tags.
- static_output_pod: main's zero-row and upstream rebuild paths (ITL-912,
  ITL-1131, ITL-1133) carry the referential tags; sync orchestrator uses
  _materialize_upstream.
- join: ITL-829 key nullability reconciliation beside the referential merge.
- operator/function nodes: main's run/zero-row behaviour (ITL-1080, ITL-1095,
  ITL-1111) with the referential result schema and references.
- database renames (ITL-1087) in slice code and tests.
- tests: main's versions, with tag_only_columns=True where the slice pins v0.3
  shapes; unknown job format version parameter stays 0.4.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- SemiJoin: a shared tag is a match key in both roles; a referential key no
  longer triggers SemiJoinIgnoredColumnWarning (ITL-1088).
- function_zero_row_table (ITL-912): a fall-through referential tag is one
  column, laid out from the tags, as rows lay it out.
- Tests from main that build tagged sources and pin v0.3 shapes, derived
  sources or plain slots pass tag_only_columns=True; golden fixtures added on
  main (join key nullability, column selection) likewise, values unchanged.
- New tests: reload keeps the tag-only choice; zero-row fall-through layout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ors after the merge (ITL-1023)

- GroupBy lays its keys out first, the row order every node table keeps
  (ITL-1131); a referential non-key declared before the key no longer leads.
- without_input_meta (ITL-1080) keeps the referential tags of the stream it
  rebuilds; the sync path no longer drops a referenced data column when the
  input carries meta columns (the referential meta cases now pass).
- Join reports a shared plain data column of two types as a type conflict, as
  main does, before the overlap error.
- Main's large-type cases, column selection tests and join helpers pin plain
  tags: they assert v0.3 shapes and run Batch, MergeJoin, SelectColumns and
  DropColumns, which do not handle referential tags yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… pipeline tests (ITL-1023)

- A stream's zero-row table (head before it computes, ITL-912) and a pod
  stream's zero-row upstreams keep their referential tags, so the head has
  the referenced columns' source info as a computed table does.
- test_zero_row_column_order also runs its plain-stream, job-node and
  prediction checks over referential sources for every step that handles
  them (ITL-1131, ITL-1135 rules for referential streams).
- Main's new pipeline tests that save jobs, bind into plain slots (ITL-1112),
  derive sources, use operator storage, Batch or MergeJoin, or hand-build a
  DAG without the pod call's select pass tag_only_columns=True.
- The f51ce67 saved-job fixture loads with an explicit database: it names
  its store under bindings.store (renamed by ITL-1082).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Edgar Y. Walker and others added 22 commits November 17, 2026 01:36
…nto eywalker/itl-1195-operators-that-turn-a-referential-tag-into
…-referential-tags-working' into eywalker/itl-1194-mergejoin-on-referential-tags
…join-on-referential-tags

feat(merge_join): handle referential tags as Join does (ITL-1194)
…ITL-1023.md (ITL-1023)

One fragment per PR; the MergeJoin text is kept as written.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-referential-tags-working' into eywalker/itl-1175-operator-and-stream-method-to-mark

# Conflicts:
#	docs/dev/design_issues/ITL-1023-RT1.md
…nto eywalker/itl-1195-operators-that-turn-a-referential-tag-into
…and data context (ITL-1205)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tors-that-turn-a-referential-tag-into

docs(referential-tags): select and drop act on a referential tag by role; remove ReferenceMaterializationWarning (ITL-1195)
Developer decision, November 17, 2026: the three column roles are tag, data
and data-tag. Prose always writes the hyphenated "data-tag", identifiers
data_tag / data_tags, classes DataTag; never the two words. One mechanical
commit over everything on the slice (#401, #402, #403, #407, #420, #421, #423).

- Files: core/referential_tags.py -> core/data_tags.py; tests/test_core/
  referential_tags/ -> tests/test_core/data_tags/; docs/concepts/
  referential-tags.md -> data-tags.md (mkdocs nav and links); three saved-job
  test files.
- Identifiers: reference_tags -> data_tags (ArrowTableStream, SourceStreamBuilder),
  stream_data_tags, declared_data_tags, stored_data_tags, source_data_tags,
  data_tag_names, mark_data_tags, without_tag_kind_marks, check_data_tag_support,
  DATA_TAGS_ISSUE, TAG_KIND_METADATA_KEY, DATA_TAG_KIND (stored value
  b"data_tag"), ReferenceFallThroughPod -> FallThroughPod (.fall_through_tags),
  fall_through_references -> fall_through_tags (identity element, saved-job key
  FALL_THROUGH_TAGS_KEY), ReferenceMaterializationWarning ->
  DataTagMaterializationWarning, reference_materialization ->
  data_tag_materialization; error text "... with data-tags yet".
- Identities that move (provisional, unpinned): a raw ArrowTableStream with
  data-tags (element "data_tags"), every fall-through pod and node (element
  "fall_through_tags": content and pipeline hash, the node's pipeline-table path
  and every path downstream). Results do not move. Golden files and contexts
  are unchanged.
- Stores and saved jobs written by a development build of the branch (marker
  "referential", key "fall_through_references") are refused, not migrated.
- tests/test_data_tag_spelling.py fails on the two-word spelling outside
  CHANGELOG.md and the dated plans and specs.
- The ITL-1175 and ITL-1195 fragments are folded into changelog.d/ITL-1023.md.
- data_tags.py carries a file-level reportImportCycles=false: the existing
  cycle through types, core_protocols, side_effects and function_node, which
  pyright attributes to the renamed file; breaking it is ITL-1206.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lice (ITL-1023)

Conflicts: tag_by (slice) and add_tags (main) side by side in StreamBase, the
stream protocol, the saved-job operator registry and the async/sync equivalence
cases. AddTags declares no DATA_TAGS_ISSUE, so it refuses a stream with
data-tags ("not supported with data-tags yet"), as every operator that has not
opted in does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Developer decision, November 17, 2026 (fix B): a stream lists its data-tags
first among its data keys and data schema, in tag order, then its other data
columns in table order. A table built from rows lays a data-tag out among the
tags (_in_row_order), so this is every row's order: the async barrier path,
the sync path and job nodes (sync and async, cache OFF and LOG) now agree with
keys() and output_schema(). Before, a source with data a, x, y (x a data-tag)
listed a, x, y while a node's rows and an operator's async result gave x, a, y.

- ArrowTableStream orders _data_columns and its data schema that way; its table
  keeps its column order.
- Every operator prediction applies data_tags.data_tags_first (UnaryOperator,
  BinaryOperator, Join).
- No table layout, stored table or hash changes: Data, schema and table hashes
  do not depend on column order; golden files and contexts are unchanged.
- Tests first: test_data_tag_order.py and ref_mid_* cases (a data-tag that is not
  the table's first column) in test_async_sync_equivalence.py.
- TagBy tests and doc samples that printed the old order are updated; the rule
  is stated in docs/concepts/data-tags.md, CLAUDE.md and .zed/rules; AB1 part 1
  is resolved (part 2, the data context, stays open); changelog in the slice
  fragment.
- Two leftovers of the rename ("Referential in one input", RT6 wording).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…TL-1023)

The workshop combines add_tags with sources whose tags are data-tags, so
AddTags must not refuse them. The input's data-tags stay data-tags, one column
in both roles with its values and source info; the new constant tags are plain.
A new tag named like any column of the input, a data-tag included, is refused
as before.

- DATA_TAGS_ISSUE = None; the sync result passes the input's data-tags to
  ArrowTableStream. The async path already passes each row's data through.
- Tests: tests/test_core/data_tags/test_add_tags.py (keys, values, source info,
  prediction, refusals, job nodes sync and async, OFF and LOG, a function
  reading a data-tag downstream), ref_add_tags / ref_mid_add_tags in the
  async/sync matrix, and AddTags in the ITL-1205 order tests.
- Docs (data-tags.md operator table, operators.md) and the slice changelog.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…dation and zero-row source info (ITL-1023)

CodeRabbit review of PR #342 (heads c4cee17 and 20309a0):

- Streaming async operators passed each row's Data on in its incoming order,
  so after fix B (ITL-1205) DropTagColumns / SelectTagColumns (a data-tag
  losing its tag role), MapTags(drop_unmapped=True) and MapData (renaming a
  data-tag) gave a data order other than their sync result. Every streaming
  operator sends its rows through WithoutMetaWriter, which now also puts the
  row's data-tags first, in tag order (data_tags.with_data_tags_first). A row
  without data-tags passes unchanged. Datagram.select's dict branch now keeps
  the order given, as documented and as its Arrow branch does; no hash, golden
  value or stored column order moves (golden, hashing and stored-order tests
  pass). Matrix cases: the four operators on a two-tag source, and Join /
  GroupBy downstream of a demoted data-tag; they fail without the fix.
- A FallThroughPod called directly (one rebuilt from a saved job) on an input
  with an extra data column was refused: the call's prevalidation left out its
  fall-through tags. It now checks them, as its validation requires.
- The zero-row result of a fall-through pod typed a list-valued data-tag's
  source info (after Batch) as large_string; populated rows have
  large_list<large_string>. It now takes the input's source-info type.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…023)

- function-pods.md: older saved files load within the documented schema
  limits; a data-tag falls through a pod called as pod(stream).
- sources.md: "No data columns found" only when every tag is kept plain and no
  other column is left (a data-tag is a data column).
- operators.md: a function pod never receives a plain tag's content; a
  data-tag's data role it reads.
- hashing.md: source info enters no content or pipeline hash (an operator
  node's record hash includes it); the table-hash default source id is the
  in-memory sources', file and database sources name their location.
- scripts/hash_audit_example.py: tag_only_columns=True on its three sources,
  as the worked example; it ran into a refusal without it, and its 18 printed
  hashes now all appear in hashing.md.
- NoOpArrowDatabase docstring: refused for a node that would store a data-tag.
- CachedSource's unmarked cache table: deferred to ITL-1215, design issue
  ITL-1023-RT8 and a Known limitations line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ags slice (ITL-1023)

Conflict: mkdocs.yml, deleted on main and edited on the slice; main's
deletion is taken. The data-tags concept page joins the site sidebar
(site/astro.config.mjs, after Function Pods, where the mkdocs nav had it).
CLAUDE.md and .zed/rules keep both sides (main's Documentation site section,
the slice's data-tag rules). uv lock --check passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eyword (ITL-1023)

Copilot on PR #342 (head 3817334): the filter that decides which data-tags a
side-effect function receives kept a data-tag whose name matched any
parameter. A sink declared `def deliver(*args, value, ctx)` received a
data-tag named `args` as a keyword and raised TypeError; a positional-only
parameter of the data-tag's name raised the same way. Only a
POSITIONAL_OR_KEYWORD or KEYWORD_ONLY parameter now receives a data-tag by
name (**kwargs still takes every column).

Function pods do not share this filter: PythonDataFunction refuses *args and
**kwargs when it is built, and a fall-through pod passes a data-tag only when
it is in the declared input schema.

Tests first: *args, positional-only (its default is used) and keyword-only
sinks over a source whose data-tag has that name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t and drop keep the data context (ITL-1023)

CodeRabbit review of PR #342 (head 66d6ed7):

- A fall-through function node answered an EmptyData input (an upstream
  ephemeral result that is gone) from its result store, which holds the
  function's plain output: the row lacked the data columns of the data-tags the
  node declares (reproduced: ['half'] for a node of ('t', 'half')). Their
  values are in the tag, but their source info went with the input row, so the
  node now refuses the hit with EphemeralResultMissingError, as a miss, on the
  sync and the async path (one helper, _empty_data_cache_hit). Plain nodes are
  unchanged.
- SelectTagColumns, DropTagColumns, SelectDataColumns, DropDataColumns (as on
  main) and SelectColumns / DropColumns rebuilt their sync result without the
  rows' context column, and the first four without the input's data context, so
  a non-default context became the default. They now request the context column
  and pass the input's data context, as without_input_meta does. The async
  paths pass each row's own datagram on. Test with a second context registered
  under another key; it fails without the fix.
- CLAUDE.md, .zed/rules and two code comments: the column-order rule moves no
  stream, schema, row or table hash; a library value's own hash (a DataFrame in
  a cell) does include that value's column order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…val) into the data-tags slice (ITL-1023)

No conflict: function_node.py, derived_source.py and pipeline/graph.py merge
cleanly. uv lock --check passes; golden files and contexts identical to main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ts the missing-data error (ITL-1023)

Copilot on PR #342 (head ce427c4): Pick and Index called any tag a data-tag
when replacing it in place, so a plain tag (tag_only_columns=True) got the
advice to give out=, which still fails because the column is not data. A
data-tag is a name in both schemas: the check now requires both, and a plain
tag reaches the existing "not found in data schema" error. Tests first: a
plain tag in place and with out= (missing-data error), a data-tag in place
(still asks for out=), for Pick and Index.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ps (ITL-1224)

A stream is iterable (StreamBase.__iter__ returns iter_data()), so the
documentation's loops read `for tag, data in stream:`. One rule: a `for` clause
(a loop or a comprehension) iterates the stream itself; `iter_data()` is
written where the method is the subject of the sentence or where the iterator
is passed to a function as a value (`list(...)`, `sorted(...)`, `islice(...)`,
`next(iter(...))`). Getting started, the streams concept page and the first
tutorial page say once that iterating a stream is the same as `iter_data()`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
README (also the PyPI description), the first page, getting started, the
tutorial setup, troubleshooting and the extras (spiraldb, torch, postgresql)
show uv first: `uv add orcapod` in a project started with `uv init`, code run
with `uv run`. `pip install orcapod` stays once per page, for an environment
not managed by uv. A release candidate is named explicitly,
`uv add "orcapod==0.4.0rc1"`: a plain `uv add orcapod` takes the latest stable
release (checked with uv 0.13.7 against a local package with a stable and an
rc version: a plain add picked the stable one, an `==rc` or `>=rc` specifier
the rc).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1224/ITL-1228

The slice was squashed on main, so the branch's base commits are replaced by
main's tree; the branch's own two commits (aee3791, bf13922) are re-applied
on top unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

  • Run on-demand review

This review includes 16 billable files and costs up to $4.00.

  • Ask an admin to make reviews automatic

Open in CodeRabbit

Reviews can continue after your included limit without a manual trigger. An admin must approve usage-based billing.

Or wait 32 minutes for your next included review.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available. Your 113 included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

Learn how review limits work.

Review configuration:

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 4a754ea and 09ee49c.

Files selected for processing: 16

  • README.md
  • docs/concepts/arrays.md
  • docs/concepts/data-tags.md
  • docs/concepts/file-hash-caching.md
  • docs/concepts/file-types.md
  • docs/concepts/function-pods.md
  • docs/concepts/operators.md
  • docs/concepts/sources.md
  • docs/concepts/streams.md
  • docs/getting-started.md
  • docs/index.md
  • docs/troubleshooting.md
  • docs/tutorial/00-setup.md
  • docs/tutorial/01-first-stream.md
  • docs/tutorial/04-many-rows-into-one.md
  • docs/tutorial/07-files.md
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues remain for the documentation-only updates to stream iteration examples and installation guidance.

No open issues 🎉

What changed in this PR

This documentation-only PR standardizes stream iteration examples and recommends uv for installing Orcapod. It uses existing library behavior; no library or documentation-site implementation changes are included.

Changes:

  • Replace iter_data() calls in loops and comprehensions with direct stream iteration.
  • Present uv add orcapod first, retain pip alternatives, and explain project setup and execution.
  • Update optional-dependency commands and show how to install a specific release candidate.
File Description
README.md Recommend uv and document extras and release-candidate installation.
docs/​tutorial/​07-files.md Use direct iteration in file-processing examples.
docs/​tutorial/​04-many-rows-into-one.md Use direct iteration in batching and aggregation examples.
docs/​tutorial/​01-first-stream.md Introduce direct iteration and explain its equivalence to iter_data().
docs/​tutorial/​00-setup.md Prioritize uv setup and execution; add release-candidate guidance.
docs/​troubleshooting.md Explain how to select a release candidate explicitly.
docs/​index.md Recommend uv and simplify the introductory stream loop.
docs/​getting-started.md Add installation instructions and introduce direct iteration.
docs/​concepts/​streams.md Explain direct iteration while retaining iterator-value examples.
docs/​concepts/​sources.md Update SpiralDB setup commands and source iteration.
docs/​concepts/​operators.md Use direct iteration throughout operator examples.
docs/​concepts/​function-pods.md Use direct iteration in function-pod and source examples.
docs/​concepts/​file-types.md Simplify iteration over file-processing results.
docs/​concepts/​file-hash-caching.md Show uv first for installing the PostgreSQL extra.
docs/​concepts/​data-tags.md Use direct iteration in the batched data-tag example.
docs/​concepts/​arrays.md Simplify array and tensor loops; prioritize uv for the PyTorch extra.

Review effort: Balanced

@codecov

codecov Bot commented Nov 18, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
see 3 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@eywalker
eywalker merged commit ca0ae71 into main Nov 18, 2026
16 checks passed
@eywalker
eywalker deleted the eywalker/itl-1224-docs-iterate-a-stream-directly-for-tag branch November 18, 2026 02:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants