Repository navigation
fix(function_pod): interleaved reads of a pod stream compute each row once (ITL-1065) #360
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
eywalker
merged 5 commits into
main
from
eywalker/itl-1065-regression-from-355-two-interleaved-reads
Nov 13, 2026
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
b524713
test(function_pod): interleaved and resumed reads compute each row on…
2c270f1
fix(function_pod): a read in progress is judged by its input's noted …
b87f2d2
fix(function_pod): a read in progress also sees a reset further up th…
d1229b6
test(function_pod): a reset two links up between interleaved reads re…
f1d5bef
test(function_pod): the head after an abandoned read shows its rows (…
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| ### Fixed | ||
|
|
||
| #### Two reads of a pod stream open at once compute each row once (ITL-1065) | ||
|
|
||
| With two `iter_data()` iterators open on one function pod stream fed by an operator stream | ||
| (for example the column selection a pod call adds when its input has a column the function | ||
| does not take), the second iterator threw away the rows the first had computed and computed | ||
| them again: one call too many, and a row the first iterator had already handed out was no | ||
| longer the stream's row. A read left partway and read again did the same. Such reads now share | ||
| the stream's results, whatever feeds it: each row is computed once and every read yields the | ||
| same rows. An upstream changed with `clear_cache()` between the reads still makes the next | ||
| read start again. (ITL-1065) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,58 @@ | ||
| ## `src/orcapod/core/function_pod.py` | ||
|
|
||
| ### F19 — Interleaved reads of a pod stream fed by an operator stream recomputed a yielded row | ||
| **Status:** resolved | ||
| **Severity:** high | ||
| **Issue:** ITL-1065 | ||
|
|
||
| A regression from F18 (PR #355). With two iterators open on one function pod stream whose | ||
| input is an operator stream (for example the automatic `SelectDataColumns` that `pod(src)` | ||
| inserts when `src` has a data column the function does not take), the second iterator dropped | ||
| the stream's results and computed them again: 7 calls of the function for 6 rows, and the row | ||
| the first iterator had already yielded was no longer the stream's row. A read left after one | ||
| row and then read again did the same. | ||
|
|
||
| **Cause** (modified times printed at each step): the first read found the stream stale (its | ||
| input, the operator stream, had no modified time before it ran), so `clear_cache()` stamped | ||
| the stream at t1; then opening the input made the operator stream compute and take t2 > t1. | ||
| The stream stamps again only when its input is read to the end (F18), so until then it was | ||
| older than its input: `last_modified` was `None` and `is_stale` True in the middle of the read, | ||
| and the next `iter_data()` cleared it. Before F18 the stream stamped right after opening (and | ||
| so computing) the operator input, which hid this. A source is older than the stream, and a | ||
| pod-stream input stamps only at its end, so neither showed it directly; a pod stream over an | ||
| operator-fed pod stream did. | ||
|
|
||
| **Rule:** a pod stream's results are judged in one of two ways. | ||
|
|
||
| - Completed results (the input read to the end, or nothing read since a reset) are current | ||
| while neither a direct upstream nor the pod is newer or without a modified time (F18). | ||
| - Results in progress (a read has noted its input's state and not reached the input's end) are | ||
| current while that state is unchanged, the input is not stale itself and the pod did not | ||
| change. The state is the reset generation of a pod-stream input, and the modified time of | ||
| any other input, noted when its first row (or its end) arrives. An upstream completing, or | ||
| computing, for the first time during the read is not a change of the stream's input. A | ||
| pod-stream input is also asked whether it is stale, because its generation does not move | ||
| when a stream further up is reset; any other input's modified time already reflects its | ||
| upstreams. | ||
| - A read in progress ends when any read reaches the input's end (the stream stamps, or clears | ||
| its modified time if the input changed) or when the stream is reset (`clear_cache()`, also | ||
| the one a stale stream's next read makes). An abandoned iterator does not end it: the | ||
| stream keeps its partial results and its open input, and the next read resumes it | ||
| (ITL-1039) unless the input changed, in which case that read finds the stream stale and | ||
| starts again. | ||
|
|
||
| An operator stream needs no reset generation of its own: it computes when it is opened, so by | ||
| the time the pod stream notes its modified time it has computed, and that time moves only when | ||
| it is invalidated or recomputed, which are real changes. | ||
|
|
||
| **Fix:** `FunctionPodStream.is_stale` applies the in-progress rule while a noted input state | ||
| exists; `_input_read_to_end` ends the read in progress after stamping (or clearing) the | ||
| modified time. A reset upstream during a read still changes the noted state: the next read of | ||
| the stream starts again, and the stream ends a read that saw the reset stale (F18 unchanged). | ||
| In-memory bookkeeping only: no hash, saved file or store sees it. Regressions: | ||
| `tests/test_core/function_pod/test_pod_stream_interleaved_reads.py` (source, operator, pod | ||
| stream and pod-over-operator inputs; sequential and concurrent paths; plain and cached pods; | ||
| interleaved reads, an abandoned partial read, a reset one or two links up between | ||
| interleaved reads or after an abandoned read, `head()` after an abandoned read (partial, not | ||
| stale), and `as_table()` or `head()` while an iterator is | ||
| open). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
127 changes: 127 additions & 0 deletions
127
superpowers/plans/2026-11-13-itl-1065-interleaved-reads.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,127 @@ | ||
| # ITL-1065 Interleaved reads of a pod stream recompute a yielded row — Implementation Plan | ||
|
|
||
| > **For agentic workers:** REQUIRED SUB-SKILL: Use sensei:executing-plans to implement this plan | ||
| > task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. | ||
|
|
||
| **Goal:** Interleaved and resumed reads of a function pod stream call the function once per | ||
| input row and yield the same datagrams as a later full read, whatever its upstream kind, while | ||
| every staleness guarantee of PR #355 (ITL-1042, ITL-1052) still holds. | ||
|
|
||
| **Architecture:** `FunctionPodStream.is_stale` gets two rules: completed results are judged by | ||
| modified-time order (as now); results in progress (a read has noted its input's state and has | ||
| not reached the end) are judged by whether the input's noted state changed. In-memory | ||
| bookkeeping only. | ||
|
|
||
| **Tech Stack:** Python 3.12, pytest, `uv run`. | ||
|
|
||
| --- | ||
|
|
||
| ## Cause (confirmed on 89160c47, probe in the session scratchpad) | ||
|
|
||
| `s = f.pod(DictSource(rows with an extra column))` reads an automatic `SelectDataColumns` | ||
| stream (`DynamicPodStream`, last_modified `None` until it runs). | ||
|
|
||
| ``` | ||
| start s.own=.655051 s.lm=None gen=0 stale=True up.lm=None calls=0 | ||
| next(first) s.own=.655087 s.lm=None gen=1 stale=True up.lm=.655445 calls=1 | ||
| 3 x next(second) s.own=.656093 s.lm=.656093 gen=2 stale=False up.lm=.655445 calls=4 | ||
| drain first/second/third: calls=7, first read's row 0 is not the third read's | ||
| ``` | ||
|
|
||
| 1. `first`'s `iter_data` sees the stream stale (input time unknown), `clear_cache()` stamps | ||
| `s` at t1 (generation 1). | ||
| 2. `_ensure_iterator` opens the operator stream, which computes now and takes t2 > t1. | ||
| 3. Until the read completes, `s` is older than its input, so `last_modified` is `None` and | ||
| `is_stale` is True mid-read. `second`'s `iter_data` clears the cache (generation 2) and | ||
| recomputes row 0, which `first` had already yielded. | ||
|
|
||
| Before #355, `_ensure_iterator` stamped `s` after opening (and so computing) the operator | ||
| input, which hid this. Without the extra column the input is a source, older than `s`. | ||
|
|
||
| ## Rule after the fix | ||
|
|
||
| - Completed results (input read to the end and stamped, or nothing read since a reset) are | ||
| current while no direct upstream and not the pod is newer: modified-time order, as #355. | ||
| - Results in progress (a read has noted its input's state, the end not reached) are current | ||
| while the input's state is the noted one and the pod did not change: a pod-stream input's | ||
| reset generation; any other input's modified time, noted once it has delivered a row (an | ||
| operator stream computes when opened, so its time then moves only when it is recomputed or | ||
| invalidated). An upstream completing for the first time during the read is not a change. | ||
| - So the operator stream needs no reset generation: its modified time, noted after it has | ||
| computed, is one. | ||
| - A reset during a read still changes the noted state: the next read of the stream clears it, | ||
| and the stream ends its read stale (unchanged from #355). | ||
|
|
||
| ## Files | ||
|
|
||
| - Modify: `src/orcapod/core/function_pod.py` (`FunctionPodStream`: `is_stale` override, | ||
| `_input_read_to_end` ends the read in progress). | ||
| - Create: `tests/test_core/function_pod/test_pod_stream_interleaved_reads.py`. | ||
| - Create: `design_issues/ITL-1065-F19.md`; update `design_issues/ITL-1052-F18.md` (rule). | ||
| - Create: `changelog.d/ITL-1065.md`. | ||
|
|
||
| ### Task 1: Failing tests | ||
|
|
||
| Public interface only: `iter_data`, `is_stale`, `clear_cache`, call counts, `is` identity of | ||
| yielded datagrams. | ||
|
|
||
| Upstream kinds: `source` (`pod.process(src)`), `operator` (`pod(src_with_extra_column)`, the | ||
| automatic select), `pod_stream` (`g.pod(f.pod(src))`), `pod_over_operator` | ||
| (`g.pod(f.pod(src_with_extra))`). Executors: sequential, concurrent. Pods: plain, cached | ||
| (`CachedFunctionPod`, fresh in-memory store). | ||
|
|
||
| - [ ] `test_interleaved_reads_compute_each_row_once`: `first`, `second` iterators; one row from | ||
| `first`, three from `second`; drain both; a third full read. Every function called once per | ||
| row; the three reads yield the same datagram objects; nothing stale. | ||
| - [ ] `test_a_read_after_an_abandoned_partial_read_computes_each_row_once`: one row, abandon, | ||
| full read: once per row, same row 0 object. | ||
| - [ ] `test_a_reset_between_interleaved_reads_gives_one_state`: stamped (epoch) upstream pod, | ||
| directly or through a join + automatic select; reset it between the two iterators; drain; | ||
| third read is all epoch 1, derived from the upstream's rows, computes nothing, not stale. | ||
| - [ ] Run, expect `operator` / `pod_over_operator` sequential cases to fail (7 calls). | ||
| - [ ] WIP commit. | ||
|
|
||
| ### Task 2: Fix | ||
|
|
||
| ```python | ||
| @property | ||
| def is_stale(self) -> bool: | ||
| if self._input_state is None: | ||
| return super().is_stale | ||
| current = self._current_input_state() | ||
| if current != self._input_state or None in current: | ||
| return True | ||
| own = self._modified_time | ||
| pod_time = self._function_pod.last_modified | ||
| return own is None or pod_time is None or pod_time > own | ||
| ``` | ||
|
|
||
| `_input_read_to_end` sets `self._input_state = None` after stamping (the read is no longer in | ||
| progress). `clear_cache` already clears it. | ||
|
|
||
| - [ ] Implement; run the new tests, `test_pod_stream_staleness.py`, | ||
| `tests/test_pipeline/test_node_staleness_after_run.py`, `tests/test_core/function_pod`, | ||
| `tests/test_core/streams` (one pytest process, Python 3.12). | ||
| - [ ] Re-run the probe: 6 calls. | ||
| - [ ] Commit. | ||
|
|
||
| ### Task 3: Docs and records | ||
|
|
||
| - [ ] `design_issues/ITL-1065-F19.md` (cause, rule, fix); F18 rule paragraph points to F19. | ||
| - [ ] `changelog.d/ITL-1065.md` under `### Fixed`. | ||
| - [ ] `uv run python scripts/design_issues.py --check`, `scripts/assemble_changelog.py --check`, | ||
| ruff, basedpyright. | ||
| - [ ] Commit, merge origin/main, push, PR (Fixes ITL-1065), Copilot review. | ||
|
|
||
| ## Notes for a successor | ||
|
|
||
| - Status (2026-11-13): tests (b5247136) failed 8 sequential cases before the fix; fix in | ||
| `FunctionPodStream.is_stale` / `_input_read_to_end`; targeted suites pass (1472); repro and | ||
| the ITL-1034 suite's interleaved cases (`--runxfail`, 117) pass. Docs, F19, F18 pointer and | ||
| changelog fragment written. Next: merge origin/main, push, PR, Copilot. | ||
| - Coordinator follow-up (msg_7bf21fb00ea3): added `as_table`/`head` during an open read and | ||
| an abandoned read followed by a reset. The latter found a gap two links up (`s <- s1 <- p`, | ||
| only `s1`'s generation noted): the in-progress rule now also asks a pod-stream input whether | ||
| it is stale. Merge origin/main after #358 (as_table clears a stale cache) and rerun. | ||
| - The ITL-1034 branch's strict `ITL_1065` xfail marks will XPASS once this merges: that branch | ||
| must drop them. |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.