Repository navigation
fix(pipeline): job node view defects found by a full review of #380 (ITL-1121) #388
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
eywalker
merged 79 commits into
main
from
eywalker/itl-1121-job-node-view-itl-1095-defects-found-by-a
Nov 14, 2026
Merged
Changes from all commits
Commits
Show all changes
79 commits
Select commit
Hold shift + click to select a range
3c8a69c
test(function_node): failing tests for the rows a job node returns af…
2e9972e
fix(function_node): after a run a job node returns exactly the rows o…
51cb1b5
docs(plans): ITL-1095 diagnosis and proposal (WIP)
1d99017
test(nodes): failing tests: a job node reads empty and stale before a…
b1a8d21
feat(nodes)!: a job node reads empty and stale before a run (ITL-1095)
7111a20
Merge remote-tracking branch 'origin/main' into eywalker/itl-1093-aft…
39aa109
test: read job nodes after a run, not before (ITL-1095)
f1c36f2
test: read stored results of loaded jobs through as_source() (ITL-1095)
bea357f
fix(function_node): count a run input only when it gave a row; docstr…
55e01eb
test: update operator, delta and stored-context reads to the current-…
bfefb42
Merge remote-tracking branch 'origin/main' into eywalker/itl-1095-des…
3301092
fix(operator_node): the stored view of a loaded operator raises when …
c8f66c0
test(loaded_job): read loaded nodes' stored results through as_source…
da66b83
docs: a job node shows its current run, as_source() what is stored (I…
7411a89
test(pipeline): failing tests: job.run(load_only=True) (ITL-1095)
6541425
feat(pipeline): job.run(load_only=True) fills the nodes from what is …
c90eb96
docs: loading without computing, job.run(load_only=True) (ITL-1095)
aacd8ce
Merge remote-tracking branch 'origin/eywalker/itl-1093-after-an-async…
d389828
test(pipeline): failing tests: node.run() cascade, state rule and loa…
db1b80a
feat(nodes): node.run() runs the job nodes it depends on, by their st…
48a2015
fix(pipeline): a run whose end-of-run commit fails leaves no node hol…
9108423
docs: running one node, node states, and the run() argument rule (ITL…
a9d07f0
fix(function_node): any raise of a run clears the node's rows, also a…
2930456
test(function_node): a CACHE_ONLY node's run() raises and points to a…
03fa3aa
Merge remote-tracking branch 'origin/eywalker/itl-1093-after-an-async…
b693b50
test: #378's new raised-run tests read the stored view (ITL-1095); lo…
3610fd2
feat(pipeline): a loaded job is filled from its store by run(load_onl…
ee3e485
test(objective): a second node runs before it is read (ITL-1095)
40c54f0
test(arrays): read the v0.2-store jobs after a load-only run (ITL-1095)
74a2ddf
fix(pipeline): type-narrowing in the node.run() cascade (ITL-1095)
32bb766
fix(function_node): an async run whose output close fails defines no …
0b4c231
Merge remote-tracking branch 'origin/eywalker/itl-1093-after-an-async…
4d10024
Merge remote-tracking branch 'origin/main' into eywalker/itl-1095-des…
2846dde
feat(side_effects): a side-effect job node follows the job node state…
c1a97de
Merge remote-tracking branch 'origin/main' into eywalker/itl-1095-des…
4281755
test(pipeline): failing tests: job.run() follows the state rule (ITL-…
b1b9944
feat(pipeline)!: job.run() follows the job node state rule (ITL-1095)
3cd6bef
fix(nodes): review fixes for the current-run rule and load-only runs …
075b8e9
fix(pipeline): second review round for the current-run rule (ITL-1095)
9f145ce
Merge remote-tracking branch 'origin/main' into eywalker/itl-1095-des…
fd812c1
test(pipeline): #379's async error-policy tests read a failed run's n…
f02d72a
fix(side_effects): a side-effect node filled by a load-only run shows…
8314c93
fix(pipeline): third review round for the current-run rule (ITL-1095)
b22763f
fix(pipeline): an interrupted end-of-run commit of job.run clears the…
7de3512
fix(pipeline): a run that leaves a node out still reports pre-v0.3 ta…
86902ec
chore(repro): reproduction scripts for the ITL-1121 review comments
ef2bb44
docs(plan): ITL-1121 implementation plan
b62feae
fix(function_node): stamp a node before on_node_end, so an observer r…
a4a9504
fix(operator_node): a run whose lazy output fails to compute holds no…
3404b7e
fix(run_summary): say 'nothing to do' only when the run ran no node (…
207f4ec
fix(side_effects): a load-only execute whose input raises clears the …
e3d4984
chore(repro): ruff-clean the ITL-1121 reproduction scripts
16a6210
fix(node_run): a load-only run refuses an input that delivers when re…
2c0f1f8
docs(function_pods): a held side-effect node delivers nothing on a se…
902e4e9
docs(changelog): ITL-1121 fragment and design issue entry
de87d16
test(itl-1121): type-clean the regression tests and reproduction scripts
979396f
style: ruff format
1d66a27
chore(repro): reproduce the PR #388 review's load-only delivery paths…
75324ab
fix(pipeline): PR #388 review round 1: more load-only delivery paths,…
6fe3c6e
chore(repro): a held node.run() keeps the previous last_run (observat…
019618d
Merge remote-tracking branch 'origin/main' into eywalker/itl-1121-job…
c76dee0
Merge remote-tracking branch 'origin/main' into eywalker/itl-1121-job…
d06ffcf
feat(node_run): a load-only run refuses an input that computes when r…
ac63354
docs(plan): ITL-1121 progress notes
5dd8d6a
docs(function_pods): name ITL-1130, which re-evaluates the load-only …
c906e56
Merge origin/main (ITL-912, #375) into ITL-1095
5a960d1
test(pipeline): a job node's current-run view has one Arrow schema in…
410420e
Merge remote-tracking branch 'origin/main' into eywalker/itl-1095-des…
8c44199
test(pipeline): docstrings for the current-run view schema tests (ITL…
1aa7ca2
Merge #380's final head (8c44199b, with #375 and #387) into ITL-1121
6b5db05
Merge origin/main (e4b4319c, #380 squash-merged) into ITL-1121
8455c2f
docs(claude): a failed end-of-run commit of node.run() clears only th…
d35d889
docs(plan): ITL-1121 merge of main after #380 merged
03a8717
docs(node_run): a job node behind an outside stream is read, not run …
82a6851
Merge origin/main (653c138e, #390) into ITL-1121
43bf5fd
test(scripts): type the c11 reproduction's pods for basedpyright (ITL…
fec5ab8
Merge origin/main (810d1b03, #389) into ITL-1121
9fc663d
docs(plan): ITL-1121 merges of #390 and #389
83350de
fix(node_run): the load-only refusal reads a held table and refuses a…
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| ### Changed | ||
|
|
||
| - A load-only run (`job.run(load_only=True)`, `run_async(load_only=True)`, `node.run(load_only=True)`, a source slot's `run(load_only=True)`) reads only job nodes, sources and streams that neither compute nor deliver (developer, November 14, 2026; ITL-1121). A pod's stream built outside the job is refused with a `ValueError` before anything is read: a function pod's stream (a cached pod's included), which computes when it is read, and a side-effect pod's stream or a `SideEffectNode` outside a job, which deliver their side effects (before, the run read them and delivered every row). An `ArrowTableStream` holds its rows and is read whatever its `upstreams` name. This applies whether a job node reads the stream, a source slot is bound to it or an operator's stream outside the job reads it. Build the pod inside the job, or run without `load_only`; a run without `load_only` is unchanged. ITL-1130 re-evaluates the choice. | ||
|
|
||
| ### Fixed | ||
|
|
||
| - An observer that read a function job node in `on_node_end` emptied it: the node was marked current only after that hook, so the read found it stale and cleared the run's rows (ITL-1121). The node is now current before the hook, on sync, async and load-only runs. | ||
| - An operator job node without a database, or in `CacheMode.OFF`, whose output failed to compute in `run()` kept claiming a current run and its reads raised; it now reads as not run, like every node whose run raised. | ||
| - The run summary said "nothing to do; every node holds a current run" when the run ran a side-effect node (and delivered) or an operator node below a held function node. It now says so only when the run ran no node; `RunSummary` takes `ran_nothing`. | ||
| - A side-effect job node whose load-only `execute()` failed on its input stayed current, and so did the nodes below it; it is now cleared, as on the async path. | ||
| - An operator job node without a database, run with `load_only=True`, showed its rows as a full run's (`rows` instead of `loaded_only`) and a later full `run()` took them as its own. | ||
| - A side-effect job node over a stream outside a job, with a store attached, kept its rows as a current run when its `run()` was interrupted (`KeyboardInterrupt`) while committing the invocation log, so the next `run()` neither ran nor committed again; it is now cleared. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| ## `src/orcapod/core/nodes/function_node.py` | ||
|
|
||
| ### NV1 — Job node view: a run's rows lost to an observer, failed runs left current, a misleading summary | ||
| **Status:** resolved | ||
| **Severity:** medium | ||
| **Issue:** ITL-1121 | ||
|
|
||
| A full review of the job node current-run view (ITL-1095) found five defects, each reproduced | ||
| through the public interface (`superpowers/scripts/itl_1121/`): | ||
|
|
||
| - A function job node was stamped as modified after `on_node_end`, so an observer reading the | ||
| node there found it stale and cleared the run's rows; the node read empty after the run (sync, | ||
| async and both load-only paths). | ||
| - An operator job node without a database, or in `CacheMode.OFF`, computed its lazy output while | ||
| building the `run()` view, outside the block that clears a failed run: after a failure it | ||
| claimed a current run and its reads raised. | ||
| - `RunSummary` said "nothing to do; every node holds a current run" when the run ran only a | ||
| side-effect node (which delivered) or an operator node below a held function node: the | ||
| headline judged only the function nodes' counts. | ||
| - A side-effect job node's sync `execute(load_only=True)` whose input raised kept its modified | ||
| time, so it and the nodes below it were not stale. | ||
| - A load-only run read a side-effect pod's stream outside a job (directly, through another | ||
| stream outside a job, or bound to a source slot), which delivered every row, although a | ||
| load-only run delivers nothing; it also read a function pod's stream outside a job, which | ||
| computed. | ||
| - An operator job node without a database did not record a load-only run as one. | ||
| - A standalone side-effect job node interrupted while committing its invocation log kept a | ||
| current run, so the next run neither ran nor committed. | ||
|
|
||
| **Fix:** stamp before `on_node_end`; build the computed view inside the clearing block; | ||
| `RunSummary(ran_nothing=...)` from the run plan; clear the node when the load-only input | ||
| raises; the run plan and a source slot's `run()` refuse (`ValueError`, before reading) a | ||
| load-only run that would read a `SideEffectPodStream` or a `FunctionPodStream` | ||
| (`node_run.refuse_unloadable_inputs`, walking streams outside a job and bound source slots; | ||
| developer, November 14, 2026: "Refuse is consistent"); the no-database operator branch | ||
| records `load_only`; the side-effect node's log commit clears the node on any interruption. Two review comments did not | ||
| reproduce or are tracked elsewhere: a node run outside `job.run()` commits every row, so no | ||
| pre-run flush is needed; a function node with no database holding no rows after an async run is | ||
| ITL-1097. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.