Repository navigation
docs: iterate a stream directly; uv add orcapod is the recommended install (ITL-1224, ITL-1228) - #438
Conversation
Specification section 15.7: two time series sharing five keys, a third table keyed on a subset, GroupBy, automatic join and select, a function reading the timestamp lists and a key by name; function node on the in-memory and Delta stores, sync and async; second run computes nothing. Also adds the slice's implementation plan. Refs ITL-1023 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n, GroupBy, pod checks (ITL-1023) A referential tag is one column named in both the tag and the data role (specification section 4.3). Sources make declared tags referential by default; tag_only_columns keeps chosen tags plain (v0.3 behaviour). Golden fixtures use tag_only_columns=True and no golden value moves. Operators that do not handle references yet raise. Refs ITL-1023 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…in parity; tests (ITL-1023) Filters, DropNulls, SemiJoin, Pick and Index keep references; MapTags and MapData work when they leave references alone; Batch, MergeJoin, operator node storage, DerivedSource and Pipeline.save raise 'not supported with referential tags yet'. The async Join merges a referenced data column as the sync path does (section 9). Referential cases join the async/sync equivalence matrix; its existing sources keep plain tags. Refs ITL-1023 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…he real timestamp shape (ITL-1023) MapTags renames a referential tag in both roles (the reference survives); tag_only=True renames the tag only and leaves a plain tag. MapData is always a data-only move: the old referential tag stays as a plain tag. The EDI test now has a key column 'timestamp' in both series, renamed with map_tags. Refs ITL-1023 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…kshop-slice-referential-tags-working
…rt v0.3 shapes and hashes (ITL-1023) Existing tests built sources with tag columns and asserted v0.3 schemas, hashes or behaviour of operators that do not take references yet; they now pass tag_only_columns=True, which is exactly the v0.3 source. Doc pages that explain the v0.3 tag/data split do the same; a new concepts page documents referential tags. The materialization warning is reported once per operator (a job validates operators again). SQLite and PostgreSQL sources forward tag_only_columns. Adds the changelog fragment and design issue RT1. Refs ITL-1023 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…with plain tags (ITL-1023) 0.3.0 is the first format in which a source may be referential; 0.1.0 and 0.2.0 files hold v0.3 sources, whose configs carry no tag_only_columns and read back with every declared tag plain, so their saved hashes match. Adds a job saved by main at f51ce67 as a fixture and a test that it loads with unchanged hashes. Refs ITL-1023 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…kshop-slice-referential-tags-working
…d torch tests that use Batch, MergeJoin or saved jobs (ITL-1023) Refs ITL-1023 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Developer decision, November 12, 2026: pod(stream) on a stream with referential tags uses ReferenceFallThroughPod, a variant that calls the function exactly as the plain pod does (same inputs, result identity and result store) and emits, per row, the function's outputs plus the row's own referenced columns, so the tags stay referential. It abbreviates the composition 'temporary per-row tag, plain pod, join back, drop the tag'; a test checks the two give the same rows with duplicate user tags. pod.process(stream) stays the plain pod. A function node stores only the function's result and gives the stored tag column its data role back on read. Without references the call is exactly the plain pod. Refs ITL-1023 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…jects; sinks; sorted merged provenance (ITL-1023) The fall-through columns are added from the input row as Arrow (Data.with_columns_of) and laid out from the tags, so their Arrow types are exact and a function's returned objects are not rebuilt. Datagram select and drop keep the Python values held beside the table. A wrapper pod called directly keeps its plain behaviour. A side-effect function gets a referenced column only when it names it. The merged source info of a joined referenced column is sorted (audit D5), so it is commutative. ColumnConfig without source drops a referenced column's source info. Refs ITL-1023 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…kshop-slice-referential-tags-working # Conflicts: # src/orcapod/core/streams/base.py
…be saved (ITL-1023) ReferenceFallThroughPod takes a FunctionPod and implements to_config and from_config by raising 'not supported with referential tags yet (ITL-1019)'; only a plain FunctionPod's call form uses it. Test helpers typed for basedpyright. Refs ITL-1023 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…(ITL-1023) .gitignore excludes *.csv and Parquet files; the fixture needs them. Refs ITL-1023 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… sources (ITL-1023) Refs ITL-1023 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…kshop-slice-referential-tags-working # Conflicts: # src/orcapod/pipeline/serialization.py # tests/test_pipeline/test_pipeline_job.py # tests/test_pipeline/test_serialization.py # tests/test_pipeline/test_serialization_helpers.py
…kshop-slice-referential-tags-working # Conflicts: # src/orcapod/core/function_pod.py # src/orcapod/core/operators/join.py
Refs ITL-1023 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d from equal elements (ITL-1017) A referential GroupBy key and a Join key referential on several inputs carry one token, merged:source_fold_v1:<sha256 hex> of the sorted member tokens (arrow_utils.fold_source_info_tokens), instead of a list of every member's token. Provenance keeps the shape of its data column; a list-valued merged key folds per element; an N-ary join folds once over all inputs, sync and async. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nt RT2 (ITL-1017) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…kshop-slice-referential-tags-working Conflicts: arrow_table_stream.py (kept reference_tags and main's _identity_table), semijoin.py imports, function_node.py (data_ids beside the fall-through comment), test_async_sync_equivalence.py (main's provenance cases, ITL-838 fixed). New tests from main that assume v0.3 sources pin tag_only_columns=True, as the slice does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Brings in ITL-1070 (GroupBy keeps only its by tags), ITL-1068 (ProvisionalHashWarning) and ITL-1067 (handle_config honours all_info). GroupBy follows main's rule: a referential tag that is not in `by` is dropped in both roles, with its source-info column, in the rows, in the schema prediction and in the output's reference tags. A referential key keeps its one merged source-info token, unchanged. Removed the slice tests that only served the old "non-key referential tag becomes a list" behaviour, rewrote the EDI end-to-end test with timestamp declared as data, and adapted main's new code to referential-by-default sources: the side-effect stream's table shows a referential column once, and the provisional hash warning tests keep their source tags plain. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resolves the SpiralDB source docstring: main's https server URL beside the slice's tag_only_columns argument. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ist (ITL-1023) Developer decision, November 13, 2026: GroupBy drops a referential tag that is not in `by` as a tag only, keeping its data. Its data column is aggregated like every other data column (list, one source-info token per member) and the members stay ordered by the non-key tags, so the list is in value order. A plain non-key tag is still dropped entirely (ITL-1070); a referential key is unchanged, with its pinned merged token. The EDI end-to-end test is back to the natural pattern: timestamp referential at the source, renamed with map_tags before grouping, no map_data, no sort in the function. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflicts resolved: - sources (CSV, Delta, DB table): main's reload (_build, check_for_new_data, ITL-1102) keeps the tag-only choice; _build passes the same reference_tags. - static_output_pod: main's zero-row and upstream rebuild paths (ITL-912, ITL-1131, ITL-1133) carry the referential tags; sync orchestrator uses _materialize_upstream. - join: ITL-829 key nullability reconciliation beside the referential merge. - operator/function nodes: main's run/zero-row behaviour (ITL-1080, ITL-1095, ITL-1111) with the referential result schema and references. - database renames (ITL-1087) in slice code and tests. - tests: main's versions, with tag_only_columns=True where the slice pins v0.3 shapes; unknown job format version parameter stays 0.4.0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- SemiJoin: a shared tag is a match key in both roles; a referential key no longer triggers SemiJoinIgnoredColumnWarning (ITL-1088). - function_zero_row_table (ITL-912): a fall-through referential tag is one column, laid out from the tags, as rows lay it out. - Tests from main that build tagged sources and pin v0.3 shapes, derived sources or plain slots pass tag_only_columns=True; golden fixtures added on main (join key nullability, column selection) likewise, values unchanged. - New tests: reload keeps the tag-only choice; zero-row fall-through layout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ors after the merge (ITL-1023) - GroupBy lays its keys out first, the row order every node table keeps (ITL-1131); a referential non-key declared before the key no longer leads. - without_input_meta (ITL-1080) keeps the referential tags of the stream it rebuilds; the sync path no longer drops a referenced data column when the input carries meta columns (the referential meta cases now pass). - Join reports a shared plain data column of two types as a type conflict, as main does, before the overlap error. - Main's large-type cases, column selection tests and join helpers pin plain tags: they assert v0.3 shapes and run Batch, MergeJoin, SelectColumns and DropColumns, which do not handle referential tags yet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… pipeline tests (ITL-1023) - A stream's zero-row table (head before it computes, ITL-912) and a pod stream's zero-row upstreams keep their referential tags, so the head has the referenced columns' source info as a computed table does. - test_zero_row_column_order also runs its plain-stream, job-node and prediction checks over referential sources for every step that handles them (ITL-1131, ITL-1135 rules for referential streams). - Main's new pipeline tests that save jobs, bind into plain slots (ITL-1112), derive sources, use operator storage, Batch or MergeJoin, or hand-build a DAG without the pod call's select pass tag_only_columns=True. - The f51ce67 saved-job fixture loads with an explicit database: it names its store under bindings.store (renamed by ITL-1082). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nto eywalker/itl-1195-operators-that-turn-a-referential-tag-into
…-referential-tags-working' into eywalker/itl-1194-mergejoin-on-referential-tags
…join-on-referential-tags feat(merge_join): handle referential tags as Join does (ITL-1194)
…ITL-1023.md (ITL-1023) One fragment per PR; the MergeJoin text is kept as written. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-referential-tags-working' into eywalker/itl-1175-operator-and-stream-method-to-mark # Conflicts: # docs/dev/design_issues/ITL-1023-RT1.md
…nto eywalker/itl-1195-operators-that-turn-a-referential-tag-into
…and data context (ITL-1205) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tors-that-turn-a-referential-tag-into docs(referential-tags): select and drop act on a referential tag by role; remove ReferenceMaterializationWarning (ITL-1195)
Developer decision, November 17, 2026: the three column roles are tag, data and data-tag. Prose always writes the hyphenated "data-tag", identifiers data_tag / data_tags, classes DataTag; never the two words. One mechanical commit over everything on the slice (#401, #402, #403, #407, #420, #421, #423). - Files: core/referential_tags.py -> core/data_tags.py; tests/test_core/ referential_tags/ -> tests/test_core/data_tags/; docs/concepts/ referential-tags.md -> data-tags.md (mkdocs nav and links); three saved-job test files. - Identifiers: reference_tags -> data_tags (ArrowTableStream, SourceStreamBuilder), stream_data_tags, declared_data_tags, stored_data_tags, source_data_tags, data_tag_names, mark_data_tags, without_tag_kind_marks, check_data_tag_support, DATA_TAGS_ISSUE, TAG_KIND_METADATA_KEY, DATA_TAG_KIND (stored value b"data_tag"), ReferenceFallThroughPod -> FallThroughPod (.fall_through_tags), fall_through_references -> fall_through_tags (identity element, saved-job key FALL_THROUGH_TAGS_KEY), ReferenceMaterializationWarning -> DataTagMaterializationWarning, reference_materialization -> data_tag_materialization; error text "... with data-tags yet". - Identities that move (provisional, unpinned): a raw ArrowTableStream with data-tags (element "data_tags"), every fall-through pod and node (element "fall_through_tags": content and pipeline hash, the node's pipeline-table path and every path downstream). Results do not move. Golden files and contexts are unchanged. - Stores and saved jobs written by a development build of the branch (marker "referential", key "fall_through_references") are refused, not migrated. - tests/test_data_tag_spelling.py fails on the two-word spelling outside CHANGELOG.md and the dated plans and specs. - The ITL-1175 and ITL-1195 fragments are folded into changelog.d/ITL-1023.md. - data_tags.py carries a file-level reportImportCycles=false: the existing cycle through types, core_protocols, side_effects and function_node, which pyright attributes to the renamed file; breaking it is ITL-1206. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lice (ITL-1023)
Conflicts: tag_by (slice) and add_tags (main) side by side in StreamBase, the
stream protocol, the saved-job operator registry and the async/sync equivalence
cases. AddTags declares no DATA_TAGS_ISSUE, so it refuses a stream with
data-tags ("not supported with data-tags yet"), as every operator that has not
opted in does.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Developer decision, November 17, 2026 (fix B): a stream lists its data-tags
first among its data keys and data schema, in tag order, then its other data
columns in table order. A table built from rows lays a data-tag out among the
tags (_in_row_order), so this is every row's order: the async barrier path,
the sync path and job nodes (sync and async, cache OFF and LOG) now agree with
keys() and output_schema(). Before, a source with data a, x, y (x a data-tag)
listed a, x, y while a node's rows and an operator's async result gave x, a, y.
- ArrowTableStream orders _data_columns and its data schema that way; its table
keeps its column order.
- Every operator prediction applies data_tags.data_tags_first (UnaryOperator,
BinaryOperator, Join).
- No table layout, stored table or hash changes: Data, schema and table hashes
do not depend on column order; golden files and contexts are unchanged.
- Tests first: test_data_tag_order.py and ref_mid_* cases (a data-tag that is not
the table's first column) in test_async_sync_equivalence.py.
- TagBy tests and doc samples that printed the old order are updated; the rule
is stated in docs/concepts/data-tags.md, CLAUDE.md and .zed/rules; AB1 part 1
is resolved (part 2, the data context, stays open); changelog in the slice
fragment.
- Two leftovers of the rename ("Referential in one input", RT6 wording).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…TL-1023) The workshop combines add_tags with sources whose tags are data-tags, so AddTags must not refuse them. The input's data-tags stay data-tags, one column in both roles with its values and source info; the new constant tags are plain. A new tag named like any column of the input, a data-tag included, is refused as before. - DATA_TAGS_ISSUE = None; the sync result passes the input's data-tags to ArrowTableStream. The async path already passes each row's data through. - Tests: tests/test_core/data_tags/test_add_tags.py (keys, values, source info, prediction, refusals, job nodes sync and async, OFF and LOG, a function reading a data-tag downstream), ref_add_tags / ref_mid_add_tags in the async/sync matrix, and AddTags in the ITL-1205 order tests. - Docs (data-tags.md operator table, operators.md) and the slice changelog. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…dation and zero-row source info (ITL-1023) CodeRabbit review of PR #342 (heads c4cee17 and 20309a0): - Streaming async operators passed each row's Data on in its incoming order, so after fix B (ITL-1205) DropTagColumns / SelectTagColumns (a data-tag losing its tag role), MapTags(drop_unmapped=True) and MapData (renaming a data-tag) gave a data order other than their sync result. Every streaming operator sends its rows through WithoutMetaWriter, which now also puts the row's data-tags first, in tag order (data_tags.with_data_tags_first). A row without data-tags passes unchanged. Datagram.select's dict branch now keeps the order given, as documented and as its Arrow branch does; no hash, golden value or stored column order moves (golden, hashing and stored-order tests pass). Matrix cases: the four operators on a two-tag source, and Join / GroupBy downstream of a demoted data-tag; they fail without the fix. - A FallThroughPod called directly (one rebuilt from a saved job) on an input with an extra data column was refused: the call's prevalidation left out its fall-through tags. It now checks them, as its validation requires. - The zero-row result of a fall-through pod typed a list-valued data-tag's source info (after Batch) as large_string; populated rows have large_list<large_string>. It now takes the input's source-info type. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…023) - function-pods.md: older saved files load within the documented schema limits; a data-tag falls through a pod called as pod(stream). - sources.md: "No data columns found" only when every tag is kept plain and no other column is left (a data-tag is a data column). - operators.md: a function pod never receives a plain tag's content; a data-tag's data role it reads. - hashing.md: source info enters no content or pipeline hash (an operator node's record hash includes it); the table-hash default source id is the in-memory sources', file and database sources name their location. - scripts/hash_audit_example.py: tag_only_columns=True on its three sources, as the worked example; it ran into a refusal without it, and its 18 printed hashes now all appear in hashing.md. - NoOpArrowDatabase docstring: refused for a node that would store a data-tag. - CachedSource's unmarked cache table: deferred to ITL-1215, design issue ITL-1023-RT8 and a Known limitations line. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ags slice (ITL-1023) Conflict: mkdocs.yml, deleted on main and edited on the slice; main's deletion is taken. The data-tags concept page joins the site sidebar (site/astro.config.mjs, after Function Pods, where the mkdocs nav had it). CLAUDE.md and .zed/rules keep both sides (main's Documentation site section, the slice's data-tag rules). uv lock --check passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eyword (ITL-1023) Copilot on PR #342 (head 3817334): the filter that decides which data-tags a side-effect function receives kept a data-tag whose name matched any parameter. A sink declared `def deliver(*args, value, ctx)` received a data-tag named `args` as a keyword and raised TypeError; a positional-only parameter of the data-tag's name raised the same way. Only a POSITIONAL_OR_KEYWORD or KEYWORD_ONLY parameter now receives a data-tag by name (**kwargs still takes every column). Function pods do not share this filter: PythonDataFunction refuses *args and **kwargs when it is built, and a fall-through pod passes a data-tag only when it is in the declared input schema. Tests first: *args, positional-only (its default is used) and keyword-only sinks over a source whose data-tag has that name. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t and drop keep the data context (ITL-1023) CodeRabbit review of PR #342 (head 66d6ed7): - A fall-through function node answered an EmptyData input (an upstream ephemeral result that is gone) from its result store, which holds the function's plain output: the row lacked the data columns of the data-tags the node declares (reproduced: ['half'] for a node of ('t', 'half')). Their values are in the tag, but their source info went with the input row, so the node now refuses the hit with EphemeralResultMissingError, as a miss, on the sync and the async path (one helper, _empty_data_cache_hit). Plain nodes are unchanged. - SelectTagColumns, DropTagColumns, SelectDataColumns, DropDataColumns (as on main) and SelectColumns / DropColumns rebuilt their sync result without the rows' context column, and the first four without the input's data context, so a non-default context became the default. They now request the context column and pass the input's data context, as without_input_meta does. The async paths pass each row's own datagram on. Test with a second context registered under another key; it fails without the fix. - CLAUDE.md, .zed/rules and two code comments: the column-order rule moves no stream, schema, row or table hash; a library value's own hash (a DataFrame in a cell) does include that value's column order. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…val) into the data-tags slice (ITL-1023) No conflict: function_node.py, derived_source.py and pipeline/graph.py merge cleanly. uv lock --check passes; golden files and contexts identical to main. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ts the missing-data error (ITL-1023) Copilot on PR #342 (head ce427c4): Pick and Index called any tag a data-tag when replacing it in place, so a plain tag (tag_only_columns=True) got the advice to give out=, which still fails because the column is not data. A data-tag is a name in both schemas: the check now requires both, and a plain tag reaches the existing "not found in data schema" error. Tests first: a plain tag in place and with out= (missing-data error), a data-tag in place (still asks for out=), for Pick and Index. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ps (ITL-1224) A stream is iterable (StreamBase.__iter__ returns iter_data()), so the documentation's loops read `for tag, data in stream:`. One rule: a `for` clause (a loop or a comprehension) iterates the stream itself; `iter_data()` is written where the method is the subject of the sentence or where the iterator is passed to a function as a value (`list(...)`, `sorted(...)`, `islice(...)`, `next(iter(...))`). Getting started, the streams concept page and the first tutorial page say once that iterating a stream is the same as `iter_data()`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
README (also the PyPI description), the first page, getting started, the tutorial setup, troubleshooting and the extras (spiraldb, torch, postgresql) show uv first: `uv add orcapod` in a project started with `uv init`, code run with `uv run`. `pip install orcapod` stays once per page, for an environment not managed by uv. A release candidate is named explicitly, `uv add "orcapod==0.4.0rc1"`: a plain `uv add orcapod` takes the latest stable release (checked with uv 0.13.7 against a local package with a stable and an rc version: a plain add picked the stable one, an `==rc` or `>=rc` specifier the rc). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Warning Review limit reached
This review includes 16 billable files and costs up to $4.00.
Reviews can continue after your included limit without a manual trigger. An admin must approve usage-based billing. Or wait 32 minutes for your next included review. View limit details
|
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues remain for the documentation-only updates to stream iteration examples and installation guidance.
No open issues 🎉
What changed in this PR
This documentation-only PR standardizes stream iteration examples and recommends uv for installing Orcapod. It uses existing library behavior; no library or documentation-site implementation changes are included.
Changes:
- Replace
iter_data()calls in loops and comprehensions with direct stream iteration. - Present
uv add orcapodfirst, retain pip alternatives, and explain project setup and execution. - Update optional-dependency commands and show how to install a specific release candidate.
| File | Description |
|---|---|
README.md |
Recommend uv and document extras and release-candidate installation. |
docs/tutorial/07-files.md |
Use direct iteration in file-processing examples. |
docs/tutorial/04-many-rows-into-one.md |
Use direct iteration in batching and aggregation examples. |
docs/tutorial/01-first-stream.md |
Introduce direct iteration and explain its equivalence to iter_data(). |
docs/tutorial/00-setup.md |
Prioritize uv setup and execution; add release-candidate guidance. |
docs/troubleshooting.md |
Explain how to select a release candidate explicitly. |
docs/index.md |
Recommend uv and simplify the introductory stream loop. |
docs/getting-started.md |
Add installation instructions and introduce direct iteration. |
docs/concepts/streams.md |
Explain direct iteration while retaining iterator-value examples. |
docs/concepts/sources.md |
Update SpiralDB setup commands and source iteration. |
docs/concepts/operators.md |
Use direct iteration throughout operator examples. |
docs/concepts/function-pods.md |
Use direct iteration in function-pod and source examples. |
docs/concepts/file-types.md |
Simplify iteration over file-processing results. |
docs/concepts/file-hash-caching.md |
Show uv first for installing the PostgreSQL extra. |
docs/concepts/data-tags.md |
Use direct iteration in the batched data-tag example. |
docs/concepts/arrays.md |
Simplify array and tensor loops; prioritize uv for the PyTorch extra. |
Review effort: Balanced
Codecov Report✅ All modified and coverable lines are covered by tests. 🚀 New features to boost your workflow:
|
Iterate a stream directly (ITL-1224, first commit)
A stream is iterable:
StreamBase.__iter__returnsiter_data(). The documentation now writes its loops asfor tag, data in stream:, starting with the first page (docs/index.md). Documentation only; no library change.The rule
A
forclause (a loop or a comprehension) iterates the stream itself.iter_data()is written only where the method is the subject of the sentence (the streams page'siter_data()section, lists of read methods) or where the iterator is passed to a function as a value:list(stream.iter_data()),sorted(stream.iter_data(), key=...),islice(stream.iter_data(), 3),next(iter(stream.iter_data())).Applied to every user-facing page:
docs/index.md,docs/getting-started.md,docs/concepts/(arrays, data-tags, file-types, function-pods, operators, sources, streams),docs/tutorial/(01, 04, 07).README.md,docs/troubleshooting.mdanddocs/reference/had no loop to change (docs/reference/hashing.mdkeepsnext(iter(source.iter_data())), a value). No docstring example insrc/shows such a loop.docs/getting-started.md, the streams concept page and the first tutorial page each say once that iterating a stream is the same as callingiter_data().Every stream kind shown is iterable
Sources, operator results, function pod streams (
FunctionPodStreamhas its own__iter__), function and operator nodes and job nodes, andDerivedSourceall iterate throughStreamBase.__iter__. Two classes haveiter_data()but no__iter__:SourceNodeandSourceJobNode(a job's source slots,job.nodes["<slot>"]). No documentation sample loops over one, so nothing here depends on it; ITL-1225 covers it.StreamView(whatrun()returns) has neither and is not looped over in the docs.Recommended install:
uv add orcapod(ITL-1228, second commit)Every place that tells a user how to install Orcapod shows uv first, as the recommended way:
README.md(also the PyPI project description),docs/index.md,docs/getting-started.md(a new short Installing section),docs/tutorial/00-setup.md,docs/troubleshooting.md, and the extras indocs/concepts/sources.md(orcapod[spiraldb], withuv run spiral login),docs/concepts/arrays.md(orcapod[torch]) anddocs/concepts/file-hash-caching.md(orcapod[postgresql]).uv initstarts a project where a newcomer needs it, and pages that set up a project run code withuv run python ....pip installstays at most once per page, for an environment not managed by uv.Release candidates: a plain
uv add orcapodtakes the latest stable release, so the docs say to name the candidate,uv add "orcapod==0.4.0rc1"(index, README, tutorial setup, and a troubleshooting entry). Checked with uv 0.13.7 against a local index holding a stable and an rc version of a package: a plainuv addpicked the stable one;uv add "pkg==1.1.0rc1"anduv add "pkg>=1.1.0rc1"picked the rc;--prerelease allowalso picked it and recorded>=1.1.0rc1.No
site/file is changed (the home page hero is PR #431).Checks
tests/test_docs(every documentation sample runs, and the site content builds): 127 passed after each commit. No changelog fragment: a documentation-only change with nothing to tell library users (changelog.d/README.md).Fixes ITL-1224. Fixes ITL-1228. Related: ITL-1225.
🤖 Generated with Claude Code
Posted by a Claude Code agent on behalf of @eywalker.