Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
c76abda
docs(specs): ITL-938 versioning design, inventory of hashes, labels, …
Nov 12, 2026
36adea7
docs(specs): ITL-938 axes of change, impact matrix, path-or-column te…
Nov 12, 2026
2ef72d0
docs(specs): ITL-938 add the two identity chains, source identity and…
Nov 12, 2026
e5e94b8
docs(specs): ITL-938 add system tags and source-info columns through …
Nov 12, 2026
0c57a0a
docs(specs): ITL-938 appendix A, every hash computation site and what…
Nov 12, 2026
0157ffd
docs(specs): ITL-938 appendix A wording fix
Nov 12, 2026
731f34a
docs(specs): ITL-938 record the path-or-column rule and apply it to e…
Nov 12, 2026
e4ef782
docs(specs): ITL-938 record the source identity contract; reword find…
Nov 12, 2026
def2227
docs(specs): ITL-938 core logic (schema and subject), v0.2 to v0.3 wa…
Nov 12, 2026
71fcc02
docs(specs): ITL-938 change-by-change account v0.2 to v0.3 with migra…
Nov 12, 2026
803965d
docs(specs): ITL-938 consequences of not failing on arrow_v0.1 rows
Nov 12, 2026
62d8a93
docs(specs): ITL-938 global hash version: requirement, registry, dete…
Nov 12, 2026
87fbe04
docs(specs): ITL-938 number the hash version H0 (nauticalab 61d0f18) …
Nov 12, 2026
dfb1715
docs(specs): ITL-938 H0 golden values equal v0.2.0's; relation of the…
Nov 12, 2026
62c6b75
docs(specs): ITL-938 the model: core hash version, logical types owne…
Nov 12, 2026
61f367f
docs(specs): ITL-938 built-in logical types versioned by the same mec…
Nov 12, 2026
ba4fc28
docs(specs): ITL-938 confirmed logical-type contract and global hash …
Nov 12, 2026
f3f4d31
docs(specs): ITL-938 uniform data context column on every table, layo…
Nov 12, 2026
bd7a4a6
docs(specs): ITL-938 data context std:v1:default, __data_context colu…
Nov 12, 2026
08a04ba
docs(specs): ITL-938 freeze rule, proposed data context file fields, …
Nov 12, 2026
0069b70
docs(specs): ITL-938 history of the stored function version (dropped …
Nov 12, 2026
e7ced1d
docs(specs): ITL-938 restore function version column; tier lookup fol…
Nov 12, 2026
74c92a8
docs(specs): ITL-938 label rule: every label is <algorithm>_v1 in v0.…
Nov 12, 2026
51eadf1
docs(specs): ITL-938 file hash label stays sha256 as the stated excep…
Nov 12, 2026
74d4647
docs(specs): ITL-938 labels never enter a hash computation; directory…
Nov 12, 2026
5519b9d
docs(specs): ITL-938 directory hash label is directory_v1
Nov 12, 2026
0320a50
docs(specs): ITL-938 hash label rule fully decided; paths with bare d…
Nov 12, 2026
8713c5b
docs(specs): ITL-938 data context file additions agreed; logical type…
Nov 12, 2026
2badc48
docs(specs): ITL-938 plain-column Arrow type stays an error for v0.3;…
Nov 12, 2026
4a47fa7
docs(specs): ITL-938 side-effect logs: what is stored and measured de…
Nov 12, 2026
5c16667
docs(specs): ITL-938 three more labelled hash inputs; open points on …
Nov 12, 2026
4bae79c
docs(specs): ITL-938 two kinds of side-effecting node; the function-p…
Nov 12, 2026
767c3e5
docs(specs): ITL-938 side-effect tables deferred to a dedicated follo…
Nov 12, 2026
3ec3e41
docs(specs): ITL-938 final pass: status, resolved passages, ITL-910 c…
Nov 12, 2026
70e9f27
docs(specs): ITL-938 arrow label is arrow_v1 in the cut list
Nov 12, 2026
e12c859
docs(specs): ITL-938 post-session decisions: MergeJoin intended, sour…
Nov 12, 2026
5d5ccc1
docs(specs): ITL-938 source id kind names the operation: tags_as_data…
Nov 12, 2026
6f3e767
docs(specs): ITL-938 name every input of the tags_as_data default sou…
Nov 12, 2026
5f5ff9b
docs: a source's content hash is an address, not a digest of its rows…
Nov 12, 2026
79aeec3
docs(specs): ITL-938 A.4: a tags_as_data derived source follows the r…
Nov 12, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion .zed/rules
Original file line number Diff line number Diff line change
Expand Up @@ -355,7 +355,19 @@ Function Pods: inspect data content (never tags), synthesize new values, cached

Every pipeline element has two parallel hashes:

1. content_hash() — data-inclusive. Changes when data changes. Used for deduplication.
1. content_hash() — the pipeline bound to its named sources: schema, topology and source
identity. For a source it is a stable address, not a digest of its rows (developer,
November 11, 2026: "sources content hash is meant to be a stable address and is not
necessarily going to reflect the very data content"). A root source hashes
(class name, output schema, source_id); given a source_id, changed, added or reordered
rows leave it unchanged. Row content enters only where a source puts it into its own
identity: the default source_id of the in-memory sources (a hash of the table, used when
none is given) and the elements of a ListSource. A downstream stream or node hashes its
pod's identity plus its upstreams' content hashes; a function pod's identity is its uri,
not its body. Only a datagram (one row's tags or data) has a content_hash() that is a
digest of its values; that row hash, not the node's content hash, is what results are
memoized by. The node's content hash tells apart, inside a shared table, the rows of
pipelines bound to different sources (__node_content_hash).
2. pipeline_hash() — schema + topology only. Ignores data content. Used for DB path scoping
so that different sources with identical schemas share database tables.

Expand Down
15 changes: 13 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -379,8 +379,19 @@ every emitted element came from an input row — and it keys only on tags, never

Every pipeline element has two parallel hashes:

1. **`content_hash()`** — data-inclusive. Changes when data changes. Used for deduplication
and memoization.
1. **`content_hash()`** — the pipeline bound to its named sources: schema, topology and source
identity. For a source it is a stable *address*, not a digest of its rows (developer,
November 11, 2026: "sources content hash is meant to be a stable address and is not
necessarily going to reflect the very data content"). A root source hashes
`(class name, output schema, source_id)`; given a `source_id`, changed, added or reordered
rows leave it unchanged. Row content enters only where a source puts it into its own
identity: the default `source_id` of the in-memory sources (a hash of the table, used when
none is given) and the elements of a `ListSource`. A downstream stream or node hashes its
pod's identity plus its upstreams' content hashes; a function pod's identity is its uri,
not its body. Only a datagram (one row's tags or data) has a `content_hash()` that is a
digest of its values; that row hash, not the node's content hash, is what results are
memoized by. The node's content hash tells apart, inside a shared table, the rows of
pipelines bound to different sources (`__node_content_hash`).
2. **`pipeline_hash()`** — schema + topology only. Ignores data content. Used for DB path
scoping so that different sources with identical schemas share database tables.

Expand Down
25 changes: 25 additions & 0 deletions superpowers/plans/2026-11-11-itl-938-post-session-decisions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# ITL-938: bring the design document in line with the post-session decisions

Docs only. File: `superpowers/specs/2026-11-11-itl-938-versioning-system-design.md` (PR #312).
Source of truth for each statement: PR bodies #313 to #318 (read November 11, 2026); the
coordinator's brief where no PR exists (ITL-935, ITL-953, ITL-957).

CodeRabbit on #312: review skipped (file over the size limit); no reviews, no inline comments.
Nothing to plan for.

## Edits, by decision

1. **MergeJoin (E) withdrawn; ITL-942 as in PR #315.** 5.2 row 8; 5.3 (drop the E row); 5.4
diagram (fold wording); Part 6 (what moves paths: F and N only); 7.2 row 2; Part 8 items 5, 7;
A.9 rows E, L; Part 9.
2. **Source ids.** 4.1 intro; 5.6 rule 5 + new 5a; 7.1 (label row, source identity row); 7.2
row 6a + new rows; 7.3 default-id row; Part 8 items 1a, 11; Part 9.
3. **Native containers carry no version.** 5.5 table; 7.1 logical type row; 7.2 row 4; Part 8
item 4; Part 9.
4. **Old data: prefix scan, warn when a store cannot list (PR #318).** Part 6 paths block and
"old layout" paragraph; 6.1 resolutions; 7.1; 7.2 row 3; 7.3; 7.3a rows 4, 5; Part 8 item 1;
Part 9.
5. **GroupBy / operator rule (PR #316, #317).** 5.2 new rows; 7.2 new rows; Part 9.
6. **Issue and PR numbers.** 7.2 State column; Part 8 items; header.

Then: one docs commit, push, update PR body, check CI and CodeRabbit, `worker_done`.
Loading
Loading