Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
3b1b1df
docs(plans): ITL-1034 row-alignment suite plan
Nov 13, 2026
ed33341
test(alignment): row-alignment checker, forced completion orders and …
Nov 13, 2026
0168287
Merge remote-tracking branch 'origin/main' into eywalker/itl-1034-row…
Nov 13, 2026
621cddf
docs(specs): repro scripts from the ITL-1034 code sweep (ITL-1043, IT…
Nov 13, 2026
2c22011
test(alignment): one view per test; strict xfails for ITL-1040, ITL-1…
Nov 13, 2026
48b3793
test(alignment): operators, streams, job run summary, read patterns; …
Nov 13, 2026
3f968e0
Merge remote-tracking branch 'origin/main' into eywalker/itl-1034-row…
Nov 13, 2026
007bed6
docs(hashing): the source record-id namespace is the one the code use…
Nov 13, 2026
45b532f
test(alignment): type annotations; changelog fragment (ITL-1034)
Nov 13, 2026
eaa894d
Merge remote-tracking branch 'origin/main' into eywalker/itl-1034-row…
Nov 13, 2026
9f49979
test(alignment): drop the ITL-1039 and ITL-1040 marks, both fixed on …
Nov 13, 2026
bd85147
Merge remote-tracking branch 'origin/main' into eywalker/itl-1034-row…
Nov 13, 2026
e009af7
test(alignment): drop the ITL-1046 marks, fixed on main by #353 (ITL-…
Nov 13, 2026
17ef8ed
WIP test(alignment): CodeRabbit round 1 strictness (not pushed)
Nov 13, 2026
7ee50d6
test(alignment): the checker and tests are as strict as they claim; I…
Nov 13, 2026
27045b0
Merge remote-tracking branch 'origin/main' into eywalker/itl-1034-row…
Nov 13, 2026
126ef03
test(alignment): drop the ITL-1043 marks, fixed on main by #352 (ITL-…
Nov 13, 2026
c450c2d
Merge remote-tracking branch 'origin/main' into eywalker/itl-1034-row…
Nov 13, 2026
554cfb0
test(alignment): drop the ITL-1042 marks, fixed on main by #355 (ITL-…
Nov 13, 2026
b8fe604
test(alignment): strict xfails for ITL-1065 (interleaved reads recomp…
Nov 13, 2026
7a4665a
test(alignment): requested provenance, sorted order and exact row mem…
Nov 13, 2026
6d29494
WIP test(alignment): Copilot M1-M3 (not pushed)
Nov 13, 2026
d3f09e7
test(alignment): batch coverage per configuration, SpiralDB multi-bat…
Nov 13, 2026
f038536
test(alignment): an empty Batch fails coverage; previews check functi…
Nov 13, 2026
6094c1f
Merge remote-tracking branch 'origin/main' into eywalker/itl-1034-row…
Nov 13, 2026
ed5a017
test(alignment): drop the marks of fixed defects; head takes every co…
Nov 13, 2026
7f06e5b
Merge remote-tracking branch 'origin/main' into eywalker/itl-1034-row…
Nov 13, 2026
d855515
Merge remote-tracking branch 'origin/main' into eywalker/itl-1034-row…
Nov 13, 2026
a8bc36a
test(alignment): drop the ITL-1066 marks; the suite has no xfails lef…
Nov 13, 2026
215c7d7
Merge remote-tracking branch 'origin/main' into eywalker/itl-1034-row…
Nov 14, 2026
eae2391
test(alignment): GroupBy drops its non-key tags; assert the tag is ab…
Nov 14, 2026
532b8a2
test(alignment): bind system-tag pairs to their column; check stored …
Nov 14, 2026
e2d0a2a
test(alignment): sort contract, pair coverage, null lookup and a side…
Nov 14, 2026
8fff6fa
Merge commit '875aeaaa' into eywalker/itl-1034-row-alignment-test-sui…
Nov 14, 2026
116eda5
test(alignment): PipelineJob takes database=, not store= (ITL-1082) (…
Nov 14, 2026
2926e55
Merge commit '6a379238' into eywalker/itl-1034-row-alignment-test-sui…
Nov 14, 2026
647c4d4
test(alignment): resolve ambiguous rows by their tags; SpiralDB previ…
Nov 14, 2026
4641557
test(alignment): recover origins source by source; check a recovered …
Nov 14, 2026
d7afc0e
Merge commit 'b4fde330' into eywalker/itl-1034-row-alignment-test-sui…
Nov 14, 2026
74623ea
Merge commit '5537291f' into eywalker/itl-1034-row-alignment-test-sui…
Nov 14, 2026
2adc4eb
Merge commit '3ac7f898' into eywalker/itl-1034-row-alignment-test-sui…
Nov 14, 2026
4070eb4
Merge commit 'b746c630' into eywalker/itl-1034-row-alignment-test-sui…
Nov 14, 2026
1228a18
test(alignment): side-effect node reads after a run pass now that ITL…
Nov 14, 2026
ba9bafa
Merge commit 'a9b50350' into eywalker/itl-1034-row-alignment-test-sui…
Nov 14, 2026
d673c3b
Merge commit 'edc17c57' into eywalker/itl-1034-row-alignment-test-sui…
Nov 14, 2026
6a45eaa
Merge commit '98c1da40' into eywalker/itl-1034-row-alignment-test-sui…
Nov 14, 2026
d019df2
Merge commit '3a7f3850' into eywalker/itl-1034-row-alignment-test-sui…
Nov 14, 2026
43ae5e1
Merge commit '52d67858' into eywalker/itl-1034-row-alignment-test-sui…
Nov 14, 2026
15c8503
Merge commit '0208dca4' into eywalker/itl-1034-row-alignment-test-sui…
Nov 14, 2026
70fae22
Merge commit 'e4b4319c' into eywalker/itl-1034-row-alignment-test-sui…
Nov 14, 2026
3cdbe7a
test(alignment): a loaded job is read after a load-only run (ITL-1034)
Nov 14, 2026
605108c
Merge commit '653c138e' into eywalker/itl-1034-row-alignment-test-sui…
Nov 14, 2026
015549e
Merge commit '810d1b03' into eywalker/itl-1034-row-alignment-test-sui…
Nov 14, 2026
7708f48
Merge commit '834ee639' into eywalker/itl-1034-row-alignment-test-sui…
Nov 14, 2026
96515aa
Merge commit '548d3df4' into eywalker/itl-1034-row-alignment-test-sui…
Nov 14, 2026
143961d
Merge commit 'fe433eab' into eywalker/itl-1034-row-alignment-test-sui…
Nov 15, 2026
3cd28c2
Merge commit 'aed0663a' into eywalker/itl-1034-row-alignment-test-sui…
Nov 15, 2026
62959d9
Merge commit 'a899d1e7' into eywalker/itl-1034-row-alignment-test-sui…
Nov 15, 2026
ff37397
test(alignment): check a derived token against an origin known from a…
Nov 15, 2026
8e33fb2
Merge commit '118b0379' into eywalker/itl-1034-row-alignment-test-sui…
Nov 15, 2026
a6b35e9
Merge commit '13865271' into eywalker/itl-1034-row-alignment-test-sui…
Nov 15, 2026
97cdc94
fix(alignment): resolve folded members before checking folded system …
Nov 15, 2026
c777cff
fix(alignment): count ambiguous-candidate comparisons and compare Arr…
Nov 15, 2026
8b78172
fix(alignment): follow the whole rename chain in the oracle (ITL-1034)
Nov 15, 2026
7c67858
fix(alignment): check tokens of ambiguous origins and pair completene…
Nov 15, 2026
633bd70
fix(alignment): resolve renames before derivations; check empty previ…
Nov 15, 2026
6342406
test(alignment): check requested columns of empty operator results to…
Nov 15, 2026
aafcf8a
Merge origin/main into eywalker/itl-1034-row-alignment-test-suite-eve…
Nov 15, 2026
cfdc199
fix(alignment): require one element per input in a merged column (ITL…
Nov 15, 2026
f50d804
fix(alignment): find a derivation anywhere in a rename chain (ITL-1034)
Nov 15, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions changelog.d/ITL-1034.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
### Added

- A row-alignment test suite, `tests/test_core/alignment/` (ITL-1034). For every row of every table Orcapod produces, it recomputes each per-row derived column from that row alone: `_content_hash`, source-info tokens, system tags and record ids, `_context_key`, function outputs, and the stored records' hashes and ids. It covers sources, function pod streams, function and operator job nodes (computed, rerun from the store and loaded), `head()`, `as_source()`, every operator and the job's run summary, across every column configuration. Async completion orders are forced deterministically with events, not sleeps. Input rows come shuffled, tag-sorted or reverse-sorted, with duplicate and null tags. Sync and async tables are compared row by row by record id.

### Fixed

- `docs/reference/hashing.md` gave the wrong namespace for a source row's record id (site 5). It is `uuid5(NAMESPACE_URL, "https://orcapod.org/namespaces/source-record-id")`, as in the code and the golden values. A runnable sample on the page now recomputes a pinned record id from the documented recipe.
26 changes: 25 additions & 1 deletion docs/reference/hashing.md
Original file line number Diff line number Diff line change
Expand Up @@ -720,11 +720,35 @@ topology suffix, encoding the full join lineage into each column name.
| Field | Value |
|---|---|
| **Inputs** | `source_id` string + provenance token: `"{col}={value}"` when `record_id_column` is specified, otherwise `"row_{index}"` |
| **Algorithm** | `uuid.uuid5(_SOURCE_RECORD_ID_NAMESPACE, f"{source_id}::{provenance_token}")` where `_SOURCE_RECORD_ID_NAMESPACE = uuid.uuid5(NAMESPACE_URL, "orcapod::record_id")` is a fixed constant |
| **Algorithm** | `uuid.uuid5(_SOURCE_RECORD_ID_NAMESPACE, f"{source_id}::{provenance_token}")` where `_SOURCE_RECORD_ID_NAMESPACE = uuid.uuid5(NAMESPACE_URL, "https://orcapod.org/namespaces/source-record-id")` is a fixed constant (`core/sources/stream_builder.py`) |
| **Output format** | `bytes` (16 bytes, UUID v5 bit pattern), stored in a `large_binary` Arrow column; the column keeps that type through tags, operators and stored tables (ITL-942: before v0.3 a source gave it `fixed_size_binary[16]`, so the operator cache key of one row differed between a direct run and the async orchestrator) |
| **Uniqueness guarantee** | Deterministic per `(source_id, row_identity)`; stable across identical re-runs of the same source |
| **Known exclusions** | When `record_id_column` is not specified, the row index is used as the provenance token; `record_id` changes if rows are **reordered** within the source table |

The recipe recomputes a record id from the source id and the token alone. The value below is
the one pinned for this row in the golden hash file (`pipeline.json`):

```python
import uuid

record_id_namespace = uuid.uuid5(
uuid.NAMESPACE_URL, "https://orcapod.org/namespaces/source-record-id"
)
subjects = DictSource(
data=[{"subject": "s1", "age": 3, "weight": 20.5}],
tag_columns=["subject"],
source_id="subjects",
)
first_row = subjects.as_table(columns={"system_tags": True}).to_pylist()[0]
stored_record_id = next(
value for name, value in first_row.items() if name.startswith("_tag_record_id")
)
print(stored_record_id == uuid.uuid5(record_id_namespace, "subjects::row_0").bytes)
# True
print(stored_record_id.hex())
# e8e02084627f5e1aad83839e421538e8
```

---

### Site 6 — Join system tag suffix
Expand Down
201 changes: 201 additions & 0 deletions superpowers/plans/2026-11-12-itl-1034-row-alignment-suite.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,201 @@
# ITL-1034 Row-alignment test suite Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use sensei:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** A test suite that proves, by recomputation, that every per-row derived column of every
table Orcapod produces belongs to its own row, on sync and async paths, under forced completion
orders; plus a code sweep for the "built in one order, attached by position in another" pattern.

**Spec:** Linear ITL-1034 plus the coordinator's brief (task_a7a8603f2f52). The brief is the design;
no separate spec. ITL-1029 (the `_shape_table` site) is fixed on another branch: this suite must
FAIL on main because of it and pass once that fix is merged. No xfail, no fix of that site here.

**Architecture:** `tests/test_core/alignment/` holds three helper modules and the test modules.
`gate.py` forces completion order with asyncio events (no sleeps). `oracle.py` derives, from the
Python rows the test itself wrote, every origin row and its record id by the documented recipe
(`docs/reference/hashing.md` sites 5 and 6b). `checker.py` walks every row of a produced table and
recomputes each derived column from that row alone (content hash, source-info tokens, system tags,
record ids, context key, function outputs), falling back to an identity-keyed comparison with the
stored records only where a column embeds a per-run uuid.

**Tech stack:** pytest, pyarrow, asyncio; public Orcapod API (`ArrowTableSource`, `function_pod`,
operators, `PipelineJob`, `node.as_table / head / get_all_records / as_source`).

---

## Measured facts (prototype in the scratchpad, Nov 12 2026)

* ITL-1029 reproduces deterministically: async job, completion order reversed, 5 rows,
`node.as_table(all_info=True)` shows each row with the hash of the mirrored row.
* The sync path is affected too whenever `sort_by_tags` permutes the rows (input not in tag
order): `_shape_table` sorts, then appends hashes in `iter_data` order. Same site, same fix.
* Node `head()` goes through `_shape_table` too, but `head_column_config` forces
`sort_by_tags=False`, so it is aligned (sweep).
* `_content_hash` recomputes as `data_context.arrow_hasher.hash_table(row.select(data_keys)).to_string()`
on a one-row slice of the produced table (verified equal on source and node tables).
* Source tokens: `"<source_id>::<token>::<column>"`, token `"<col>=<value>"` with
`record_id_column`, else `"row_<i>"`. Record id: `uuid5(_SOURCE_RECORD_ID_NAMESPACE,
f"{source_id}::{token}")`, namespace `uuid5(NAMESPACE_URL, "https://orcapod.org/namespaces/source-record-id")`.
(docs/reference/hashing.md site 5 states a different namespace string: a docs defect, report it.)
* Function output token: `"<pod uri>::<output datagram uuid hex>::<column>"`; the stored record's
`__data_id` is that uuid. `__output_data_hash` / `__input_data_hash` are prefixed digests.
* Async concurrency is unlimited by default (`resolve_concurrency` -> None), so gating every row
on an event cannot deadlock for small tables; each wait still has a 30 s safety timeout that
raises a named error (a deadlock guard, never the mechanism of an assertion).

## File structure

* `tests/test_core/alignment/__init__.py` — empty.
* `tests/test_core/alignment/gate.py` — `CompletionGate`, `ORDERS` (natural, reversed,
interleaved, one_slow, random seeds), the gated function pods (`async def` functions keyed on a
unique data value).
* `tests/test_core/alignment/oracle.py` — `Origin`, `Oracle` (built from the test's own rows and
source ids; record id -> origin, token -> origin), recipe functions `source_record_id`,
`fold_record_ids`, `fold_values`.
* `tests/test_core/alignment/checker.py` — `assert_rows_aligned(table, *, data_keys, tag_keys,
oracle, data_context_key, ...)` and the per-column rules; `assert_same_rows_by_identity(a, b)`
for sync == async keyed on record ids.
* `tests/test_core/alignment/fixtures.py` — sources for the three tie scenarios (unique tags in
shuffled order, duplicate tags, null tags), `COLUMN_CONFIGS`.
* `tests/test_core/alignment/test_checker.py` — the checker is not vacuous: each rule rejects a
table with that one column permuted.
* `tests/test_core/alignment/test_gate.py` — the gate realises each order (observed via the
node's `iter_data` order).
* `tests/test_core/alignment/test_streams.py` — `ArrowTableStream`, sources, `FunctionPodStream`
(sync and the concurrent async-function path), `head()`.
* `tests/test_core/alignment/test_operators.py` — every operator x column config x tie scenario
(pairwise reduction over operators x configs, full over scenarios).
* `tests/test_core/alignment/test_nodes.py` — function and operator job nodes: sync/async x
orders x stores (in-memory, Delta) x fresh/readback (rerun, `PipelineJob.load`) x column
configs x scenarios (full product), `get_all_records`, `head()`, `as_source()` (DerivedSource).
* `tests/test_core/alignment/test_job.py` — `run`, `run_async`, run inside a running loop; run
summary under forced orders with failing rows; sync == async by identity.

## Checker rules (per produced row)

1. `_content_hash` (or a custom name): recompute from the row's data columns.
2. `_context_key`: equals the stream's `data_context_key`.
3. `_source_<c>`: scalar token -> its origin (oracle by `(source_id, token)`) -> row[c] must equal
the origin's value of the column `c` came from (rename map), or the declared derivation
(function output: `y == f(origin)`). A list token (Batch, GroupBy, MergeJoin) is checked element
by element against the list value. A token whose prefix is a function uri is checked by
identity against the stored record (`__data_id` hex) when records are given.
4. System tags, grouped by column-name suffix: `_tag_source_id::S` / `_tag_record_id::S`. A scalar
record id must be an oracle record id whose source id equals the row's source-id value, and
every source token of a column of that source must give the same record id. Folded (aggregated)
ones must equal the documented fold of the member record ids recomputed from the row's
list-valued tokens.
5. User tags and data without provenance columns: (tag, data) must match the row's origin (via
record id when system tags are present, else by the set of origins with equal tags; with
duplicate tags the value must be one of the candidates' values).
6. Stored records (`get_all_records(all_info=True)`): `__output_data_hash` recomputed from the
data, `__input_data_hash` = content hash of the origin's input data (recomputed from the
origin's own source row through `Data.content_hash` of the source stream), `__data_context`
= context key, `_source_y` embeds `__data_id`.

## Coverage matrix (as designed)

| Path | sync/async | orders | ColumnConfig | scenarios | stores |
|---|---|---|---|---|---|
| ArrowTableStream / source as_table, head | sync | n/a | all 8 | 3 | n/a |
| FunctionPodStream (`pod(src)`), head | sync + async gather | 5 | all 8 | 3 | n/a |
| FunctionJobNode as_table/head/get_all_records/as_source | sync + async (run, run_async, in-loop) | 5 (+3 seeds) | all 8 | 3 | mem, delta; fresh + rerun + load |
| OperatorJobNode (Join, GroupBy downstream of the gated fn node) | sync + async | 5 | all 8 | 3 | mem, delta; fresh + rerun |
| Each operator on a source stream | sync (+ async via job for 3 ops) | n/a | pairwise | 3 | n/a |
| Run summary (failing rows) | sync + async | 5 | n/a | 1 | mem |

ColumnConfig set: `{}` (default), `sort_by_tags=True`, `content_hash=True`,
`content_hash="h"`, `source=True`, `system_tags=True`, `context=True`, `meta=True`,
`all_info=True`, and `all_info` with `sort_by_tags=False`.

## Tasks

### Task 1: gate + oracle + checker + checker self-test
- [ ] Write `gate.py`, `oracle.py`, `checker.py`, `fixtures.py`.
- [ ] `test_checker.py`: build a correct source table; assert the checker passes; for each rule,
permute one column (`table.set_column(i, name, column.take(perm))`) and assert the checker
raises `AssertionError` naming that column. Run, expect PASS.
- [ ] `test_gate.py`: run the gated node for each order; assert `iter_data` order == the order.
- [ ] Commit `test(alignment): row-alignment checker and completion-order gate (ITL-1034)`.

### Task 2: node paths (reproduces ITL-1029)
- [ ] `test_nodes.py` with the full product above. Run; expect FAILURES only on rows with
`_content_hash` under `sort_by_tags` / async orders (the ITL-1029 site). Record the failing
ids in this plan.
- [ ] Commit, push, open DRAFT PR (Fixes ITL-1034), status message to coordinator.

### Task 3: streams, operators, job, run summary
- [ ] `test_streams.py`, `test_operators.py`, `test_job.py`. Any failure outside the ITL-1029 site
is a new defect: send it to the coordinator at once with a minimal repro, then fix here if a
few lines (failing-first test) or wait for the coordinator.
- [ ] Commit.

### Task 4: sweep
- [ ] Verify every suspect line of the sweep report; put the table in the PR body.

### Task 5: finish
- [ ] changelog.d/ITL-1034.md (`### Added`: the suite; no library change).
- [ ] ruff format/check, basedpyright, full `uv run pytest tests/ -n 4` (or fewer processes).
- [ ] Merge origin/main when the coordinator says the ITL-1029 fix is in; rerun; PR ready;
CodeRabbit rounds; worker_done.

## Progress notes
(Successors: append here.)

* Helpers + checker self-test done (16 pass). Sweep done (subagent). Second defect found and
sent to coordinator (msg_c718d4fe9800): FunctionPodStream resumed iterator re-enumerates
from 0 (function_pod.py:1043/1069) and drops rows; awaiting decision to fix here.
* Nov 13: suite complete (6100 tests: nodes, operators, streams, job, checker). ITL-1029
fixed on main by #344 (cases pass unmarked). ITL-1039 (#348) and ITL-1040/838 (#350)
merged; their marks removed. Remaining strict xfails name open issues only: ITL-1042,
ITL-1043, ITL-1045, ITL-1046, ITL-1050. PR #346 body holds the matrix, runtime, sweep
table and defect list. Left: CodeRabbit rounds and CI on #346, then worker_done. When
an open issue's fix merges, merge main and drop its marks (they XPASS strictly).
* Nov 13 (later): main at 5c7a058f has #351 (ITL-1050), #354 (ITL-1053), #356 (ITL-1060),
#358 (ITL-1045), #360 (ITL-1065) and #361. Merged; with --runxfail only the 60 ITL-1066
cases failed, so every other mark is gone. head now takes every configuration and must
equal as_table(cfg).slice(0, n); previews check sorted configs among the held rows.
6431 pass, 60 strict xfails (ITL-1066 only). #346 merges last; worker_done on the
coordinator's word.
* Nov 13 (later): merged main 3bc5e2af (#339), then 41798e87 (#362, ITL-1066). With
--runxfail all 60 ITL-1066 cases passed; marks removed. 6491 pass, no marks. Still to
come before #346 merges: #363 (ITL-1067), #364 (ITL-1068 warning filter), #365
(ITL-1070: GroupBy drops non-by tags).
* Nov 13 (evening): merged main 163e5ddc (#363 ITL-1067, #364 ITL-1068, #365 ITL-1070).
The suite passed unchanged. For ITL-1070, group_by_g now has data-only members and
the grouped node's members are its data columns; operator streams, operator nodes
(sync and async) and the grouped job node assert that the dropped tag n is absent from
keys(), the schema and every table. #363 and #364 needed nothing: the suite has no
all_info workaround and no warning filter. 6491 pass, no marks.
* Nov 14: CodeRabbit round on eae2391c. Stored records must be source A's record ids
once each, and `_source_y` must equal the node output's whole token (function uri
included). Operator nodes check `head` too, and node iterator reads check their
expected rows. A system-tag pair's column must descend from its source's own column:
its first hash is that source's schema hash, so two sources of one schema cannot be
told apart. Three new self-tests. Column order was declined: it is covered by
`tests/test_core/operators/test_join_tag_order.py`. 6494 pass, no marks.
* Nov 14 (later): answered 8 older review threads that had no reply. The checker now:
- looks a null data value up like any value;
- names a wrong folded record id under its own column;
- requires a system-tag pair for every source a row's data came from;
- checks #354's presentation order: nulls last, then ties by the system tags. List
tags follow Polars, where a null element inside a list comes first; GroupBy's member
sort puts nulls last at every depth. This was reported to the coordinator.
New: a job-backed `SideEffectPod` node test (`SideEffectNode.as_table` has its own
concatenation) on both orchestrators. 6503 pass, no marks.
* Nov 14 (CodeRabbit on 2926e559): the checker resolves an element no single value
explains by the member's tags (one origin: checked; none: fails), and reads all_info
inside the column config. SpiralDB previews: head(1) and head(3) under every config
across the batch boundary. Side-effect previews must deliver nothing. One strict
xfail, ITL-1091: a read of a side-effect job node after job.run() delivers every side
effect again (6 -> 12 -> 18). 6527 pass, 1 xfailed.
* Nov 14 (CodeRabbit on 647c4d45): origins are recovered source by source, and a
recovered origin's function tokens are checked. Merged main through b746c630 (#372,
#374, #367, #369, #377). ITL-1091 is fixed by #377, so the strict xfail is gone. A
side-effect job node's preview after its run now shows the rows the run passed on
(each aligned under every config) and still delivers nothing. 6530 pass, no marks.
* Nov 15 (pre-push CodeRabbit CLI review on 62959d9e, 1 finding): a derived element that
a known origin holds was skipped without its token being checked. With x present, an
exchanged `_source_p` of a shared p passed. resolve_remaining now checks it against
that origin; new self-test. #380's load path: bind + run(load_only=True), and grouped
joins the load read-backs. 6639 pass, no marks.
22 changes: 22 additions & 0 deletions superpowers/specs/itl-1034-sweep/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Repro scripts from the ITL-1034 code sweep

Evidence for issues found by the row-alignment suite and its code sweep. Run each script from
the repository root with `uv run python superpowers/specs/itl-1034-sweep/<script>`. The
results were measured on `main` at 003d2e82 (November 12, 2026).

| Script | Issue | What it shows |
|---|---|---|
| `itl1045_two_pass_as_table.py` | ITL-1045 | `FunctionPodStream.as_table` builds the table and the content hashes in two `iter_data` passes; an upstream reset in between gives the cached table the recomputed rows' hashes (0 of 3 rows carry their own hash) |
| `itl1046_index_token.py` | ITL-1046 | `Index` after `Batch` or `GroupBy` gives the element the token `"['src::x=1::x', 'src::x=2::x'][0]"` instead of the element's own `'src::x=1::x'` |
| `itl1047_same_stream_twice.py` | ITL-1047 | a binary operator given the same stream twice fails inside a job, with different internal errors on the sync and async orchestrators |
| `datagram_uuid_after_readback.py` | ITL-1060, ITL-1042 | after an async run whose pod input comes through an operator node, the function node is stale; a row read back from the store gets a fresh `datagram_uuid`, not the stored `__data_id` its own token names |
| `interleaved_reads_recompute.py` | ITL-1065 | two interleaved iterations of a pod stream behind the automatic select call the function 7 times for 6 rows; the first iteration's rows are not the stream's (a regression from #355) |
| `side_effect_stream_as_table.py` | ITL-1066 | `SideEffectPodStream.as_table` raises with `context` / `all_info` (a duplicate `_context_key`), ignores `content_hash` and `sort_by_tags` |
| `partial_read_then_head.py` | ITL-1065 | after a partial read of a pod stream fed by the automatic select, the stream is stale and `head(2)` shows no rows (a regression from #355) |
| `itl1043_loaded_operator_cache.py` | ITL-1043 | a loaded job cannot read a cached operator node: the loaded pipeline hash drops tag nullability, and the loaded content hash uses the stub source's identity |

Status on `main` at 5c7a058f (November 13, 2026): fixed are ITL-1042 and ITL-1052 (#355),
ITL-1043 (#352), ITL-1045 (#358), ITL-1046 (#353), ITL-1050 (#351), ITL-1060 (#356) and
ITL-1065 (#360); their scripts now show the fixed behaviour. ITL-1047 and ITL-1066 are not
fixed there; the suite marks only ITL-1066 (strict xfail). Later, at 41798e87, ITL-1066 is
fixed (#362) and the suite has no marks.
Loading
Loading