From 0886ffdb7aa355d5a470ec1d9ce64474f24020d9 Mon Sep 17 00:00:00 2001 From: "Edgar Y. Walker" <> Date: Wed, 11 Nov 2026 15:31:48 -0800 Subject: [PATCH 01/21] docs(plan): ITL-910 adopt starfix 0.5 investigation and plan (WIP) Co-Authored-By: Claude Opus 5.5 --- .../2026-11-11-itl-910-adopt-starfix-05.md | 149 ++++++++++++++++++ 1 file changed, 149 insertions(+) create mode 100644 superpowers/plans/2026-11-11-itl-910-adopt-starfix-05.md diff --git a/superpowers/plans/2026-11-11-itl-910-adopt-starfix-05.md b/superpowers/plans/2026-11-11-itl-910-adopt-starfix-05.md new file mode 100644 index 000000000..20c23f90e --- /dev/null +++ b/superpowers/plans/2026-11-11-itl-910-adopt-starfix-05.md @@ -0,0 +1,149 @@ +# ITL-910 — Adopt starfix 0.5.0 (hash format 2) Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use sensei:subagent-driven-development (recommended) or sensei:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Move Orcapod's Arrow hashing onto starfix 0.5.0 (hash format 2, prefix `00 00 02`), bump the +Arrow hash label `arrow_v0.1` → `arrow_v0.2`, and regenerate the golden hashes once, explicitly. + +**Architecture:** `StarfixArrowHasher` (`src/orcapod/hashing/arrow_hashers.py`) keeps calling +`ArrowDigester.hash_table` / `hash_schema` with the default (order-independent) options; only the +dependency, the label and the tests/docs/golden data change, plus whatever the new starfix rejections +require. No `ordered_columns` / `ordered_struct_fields` (developer decision 5). No legacy 0.4 mode. + +**Tech Stack:** starfix 0.5.0 (pure Python, `pyarrow>=19`), pyarrow 23.0.1 (locked; Orcapod floor already +`>=20`, so no pyarrow floor change), pytest, uv. + +Branch: `eywalker/itl-910-adopt-starfix-05-hash-format-2-in-orcapod` (Linear gitBranchName). + +--- + +## Status log (for a successor) + +- 2026-11-11: investigation done (below). Plan sent to coordinator via `ask` — WAIT for the answer + before Task 4 (golden regeneration) and before any migration-related code. + +## Investigation results + +### starfix 0.5.0 API (installed wheel, README, `arrow_digester.py`) + +- Same entry points as 0.4: `ArrowDigester(schema, ...)`, `update`, `finalize`, `hash_schema`, + `hash_record_batch`, `hash_table`, `hash_array`. New keyword-only options `ordered_columns`, + `ordered_struct_fields` (not used here). `include_metadata` unchanged. +- Every hash starts `00 00 02`; all values differ from 0.4. +- `ValueError` (was `NotImplementedError` in 0.4) for data of Null/Map/Interval/Utf8View/BinaryView, + any row count; `ValueError` for ListView/Union/RunEndEncoded; new `ValueError`s: ambiguous node + paths (duplicate names, `"a/b"` aliasing struct `a` child `b`), non-nullable field with a visible null, + duplicate metadata keys with `include_metadata`. `TypeError` for wrong container type (Orcapod + converts RecordBatch → Table first, unaffected). +- FixedSizeList data now hashes (any depth), distinct from `List`/`LargeList` of the same values. +- Canonical extension types (`arrow.uuid`, `arrow.fixed_shape_tensor`) now hash. + +### Orcapod callers + +`grep -rn starfix src`: only `src/orcapod/hashing/arrow_hashers.py` (`hash_schema`, `hash_table`). +Everything else (sources' default `source_id`, record ids, `__input_data_hash`, base/pipeline entry +ids, OperatorNode record hashes, `Data`/`Tag` content hashes, `ArrowTableHandler` for `pa.Table`, +pandas DataFrame/Series handlers) goes through `StarfixArrowHasher`. + +### Behaviour changes visible to an Orcapod user (probe: `ArrowTableSource(t, tag_columns=["k"])`, +`hasher.hash_table`, `hasher.hash_schema`, 0.4 vs 0.5) + +| Input | 0.4 | 0.5 | +|---|---|---| +| `fixed_size_list` column (top level, in struct, in list, with null slot) | `NotImplementedError` (source fails) | hashes; source constructs | +| homogeneous `tuple[T, T]` pod output (maps to `fixed_size_list`) | fails to hash | works | +| `arrow.uuid`, `arrow.fixed_shape_tensor` extension columns | `NotImplementedError` | hashes | +| `hash_schema` with null / view / interval types | `NotImplementedError` | hashes | +| **zero-row** table with a `map` column | hashes | `ValueError` | +| non-nullable field with a visible null | hashes silently | `ValueError: Non-nullable field 'x' contains nulls` | +| duplicate column names (raw hasher, e.g. a `pa.Table` value) | hashes | `ValueError: Ambiguous node path` (sources already failed with `KeyError`) | +| column `"a/b"` next to struct `a{b}` | hashes | `ValueError: Ambiguous node path 'a/b'` | +| `map` (rows), `null`, view, interval data | `NotImplementedError` | `ValueError` (same rejection, new class) | +| `CachedSource._build_merged_stream` | `except NotImplementedError` silently served from cache when live ingestion hit an unhashable type | now raises (the hashing error is no longer mistaken for "source cannot provide data") | + +ITL-848 collisions through the public interface (`ArrowTableSource.iter_data()` row content hashes): +nested list lengths `[[[1]],[[2],[3]]]` vs `[[[1],[2]],[[3]]]` collide on 0.4, differ on 0.5; null struct vs +`{"a": None}` collide on 0.4, differ on 0.5; Bool column hash depends on chunking on 0.4, not on 0.5; +non-nullable `[True]` vs `[True, False, False]` collide on 0.4, differ on 0.5. Slice offsets are canonical +on both. + +### Where the label lives + +- `src/orcapod/hashing/versioned_hashers.py:40` `_CURRENT_ARROW_HASHER_ID` +- `src/orcapod/contexts/data/v0.1.json` `arrow_hasher._config.hasher_id` (+ changelog list) +- `src/orcapod/contexts/data/schemas/context_schema.json:178` (example value) +- tests: `tests/test_hashing/test_starfix_arrow_hasher.py` (label + three pinned format-1 digests), + `tests/test_hashing/test_pandas_handlers.py` (label), `tests/test_core/operators/test_materialize_source_info.py` + (three pinned format-1 digests), `tests/test_core/operators/test_async_sync_equivalence.py`, + `tests/test_hashing/golden/cases.py` (docstring), golden data files. +- docs: `docs/reference/hashing.md` §3/§4 worked-example tables (regenerate with + `superpowers/scripts/hash_audit_example.py`); `notebooks/tutorials/01_introduction_to_orcapod.ipynb` + (outputs). Historical plans/specs under `superpowers/` are left as they are. +- Stored names: no table path embeds an Arrow hash under the default `table_scope="pipeline_hash"`. + With `table_scope="content_hash"`, the path embeds the node content hash, which moves when a source's + content hash moves. + +### Golden estimate (simulated: starfix 0.5 overlay + label patched to `arrow_v0.2`) + +| File | Entries | Changed | Unchanged | Keys added/removed | +|---|---|---|---|---| +| arrow.json | 53 | 53 | 0 | 0 | +| datagram.json | 171 | 40 | 131 | 0 | +| pipeline.json | 305 | 68 | 237 | 0 | +| schema.json | 92 | 22 | 70 | 0 | +| semantic.json | 64 | 1 (`value.dataframe.basic`, pandas via StarfixArrowHasher) | 63 | 0 | + +Every changed entry is an Arrow-derived value (row data hashes, record/entry ids, source content hashes +and default source ids, `pa.Table`/DataFrame hashes). No pipeline hash, schema hash, system-tag column +name or table path moved. + +### Migration finding (needs a decision) + +Default `table_scope` paths do not move, so a v0.3 job pointed at a v0.2 store writes into the same +tables. Observed with `DeltaTableDatabase` and the docs' `bmi_demo` job: v0.3 `run()` recomputes both rows +(old record ids miss), and a later read without `run()` returns **4 rows for 2 inputs** (v0.2 + v0.3 mixed); +before the first v0.3 run, a read returns the v0.2 rows. Stored rows carry the label +(`__input_data_hash`, `__pipeline_base_entry_id` start `b"arrow_v0.1:"` vs `b"arrow_v0.2:"`). OperatorNode +REPLAY reads all rows matching the node content hash, so a v0.3 LOG-mode write next to v0.2 rows can +duplicate too. + +## Tasks (after approval) + +### Task 1: failing tests first +- [ ] New `tests/test_hashing/test_starfix_format2.py` (public interface: `ArrowTableSource`, `iter_data`, + `content_hash`; the hasher itself via `resolve_context(None).arrow_hasher` for table-level checks): + label `arrow_v0.2` and digest prefix `00 00 02`; the four ITL-848 cases above; Bool chunking; slice/offset + and `combine_chunks` canonicality; `fixed_size_list` column hashes and differs from the same values as + `large_list`; `ArrowTableSource` with `fixed_size_list` columns constructs, keeps the type, and runs through + a function pod; pod returning `tuple[float, float]`; rejections with `ValueError` (zero-row map, + non-nullable null, duplicate names via `pa.Table` value hash, `"a/b"` alias); `arrow.uuid` column hashes. +- [ ] Update `test_starfix_arrow_hasher.py` / `test_pandas_handlers.py` label assertions to `arrow_v0.2`. +- [ ] Run, confirm failures on starfix 0.4. WIP commit. + +### Task 2: bump +- [ ] `pyproject.toml`: `starfix>=0.5.0,<0.6` (decision 6 floor); `uv lock`; report what moved in `uv.lock`. +- [ ] Label: `versioned_hashers.py`, `v0.1.json` (+ changelog line: "Hard cut (v0.3): arrow_v0.2 = starfix + hash format 2 (starfix 0.5) …"), `context_schema.json` example. +- [ ] `StarfixArrowHasher` docstring: format 2, prefix `00 00 02`, rejections. +- [ ] Re-pin the format-1 digests in `test_starfix_arrow_hasher.py` and `test_materialize_source_info.py` + (list old/new in the PR). +- [ ] Remove the generator header's "Known gaps (ITL-848)" items that are now hashable (fixed-size list, + `arrow.uuid`); add golden cases for them (adding entries is allowed). + +### Task 3: migration handling — per coordinator decision (see ask) + +### Task 4: regenerate golden (approved regeneration) +- [ ] `uv run python tests/test_hashing/golden/generate_golden_hashes.py`; compare with + `scratchpad/scripts/cmp.py` against main's files; every changed value must be Arrow-derived. + +### Task 5: docs +- [ ] CHANGELOG `[0.3.0] — unreleased` → Breaking changes and migration: starfix format 2 entry. +- [ ] `docs/reference/hashing.md` worked-example tables, prefix note; notebook outputs. +- [ ] DESIGN_ISSUES: one appended entry (resolved, Fix note). + +### Task 6: verification +- [ ] Full suite on 3.11, 3.12, 3.13, one at a time; ruff check/format; basedpyright exit code. +- [ ] Perf: 100k-row table hash, 0.4 vs 0.5; golden-test wall time before/after. + +### Task 7: PR +- [ ] Merge origin/main, push, PR (`Fixes ITL-910`, mentions ITL-848, ITL-927), CodeRabbit rounds. From 1b57f9b8094dadebd6518f8fad0162b7bef9945a Mon Sep 17 00:00:00 2001 From: "Edgar Y. Walker" <> Date: Wed, 11 Nov 2026 15:36:37 -0800 Subject: [PATCH 02/21] docs(plan): ITL-910 suite findings under starfix 0.5 (WIP) Co-Authored-By: Claude Opus 5.5 --- .../2026-11-11-itl-910-adopt-starfix-05.md | 21 ++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/superpowers/plans/2026-11-11-itl-910-adopt-starfix-05.md b/superpowers/plans/2026-11-11-itl-910-adopt-starfix-05.md index 20c23f90e..96bb966f7 100644 --- a/superpowers/plans/2026-11-11-itl-910-adopt-starfix-05.md +++ b/superpowers/plans/2026-11-11-itl-910-adopt-starfix-05.md @@ -74,7 +74,8 @@ on both. - `src/orcapod/contexts/data/schemas/context_schema.json:178` (example value) - tests: `tests/test_hashing/test_starfix_arrow_hasher.py` (label + three pinned format-1 digests), `tests/test_hashing/test_pandas_handlers.py` (label), `tests/test_core/operators/test_materialize_source_info.py` - (three pinned format-1 digests), `tests/test_core/operators/test_async_sync_equivalence.py`, + (three pinned format-1 digests), `tests/test_core/operators/test_async_sync_equivalence.py` + (`PINNED_ROW_HASHES`, `PINNED_STREAM_CONTENT_HASH`), `tests/test_hashing/golden/cases.py` (docstring), golden data files. - docs: `docs/reference/hashing.md` §3/§4 worked-example tables (regenerate with `superpowers/scripts/hash_audit_example.py`); `notebooks/tutorials/01_introduction_to_orcapod.ipynb` @@ -107,6 +108,24 @@ before the first v0.3 run, a read returns the v0.2 rows. Stored rows carry the l REPLAY reads all rows matching the node content hash, so a v0.3 LOG-mode write next to v0.2 rows can duplicate too. +### Full suite under starfix 0.5 overlay (label unchanged): 201 failed / 7400 passed / 96 skipped + +- 186 golden entries (all Arrow-derived, see above). +- Pinned format-1 digests: `test_starfix_arrow_hasher.py` (5 tests), `test_materialize_source_info.py` (1), + `test_async_sync_equivalence.py::test_issue_example_types_and_pinned_hashes`, + `test_struct_layout_identity.py::TestStructHashesUnchanged` (4 params). +- `test_group_by.py::test_falls_back_to_record_id_when_key_covers_all_tags`: member order comes from + record ids (uuid5 of the Arrow-hashed default `source_id`), so `["a","b"]` becomes `["b","a"]`. Arrow-derived. +- `test_array_v02_store.py` (2): the ITL-888 claim "an array-free pod reuses its v0.2 results" stops being + true: every function node recomputes (input data hashes move) into the same tables (mixing finding). +- `test_async_sync_equivalence.py::test_arrow_hasher_hash_schema_with_extension_type` (strict xfail, ITL-846) + now XPASSes: starfix 0.5 accepts a live extension type in `hash_schema`, but hashes it as its storage + type, so `hash_schema(Path column) == hash_schema(large_string column)`. +- Same identity loss in `hash_table` for pyarrow's canonical extension types (`arrow.uuid`, + `arrow.fixed_shape_tensor`): they are `pa.BaseExtensionType`, not `pa.ExtensionType`, so + `normalize_extension_columns` skips them, `include_metadata` stays off, and an `arrow.uuid` table hashes + equal to a `fixed_size_binary(16)` table. (On 0.4 they raised.) + ## Tasks (after approval) ### Task 1: failing tests first From b9f446548537018b5206f3602500c8ee6765a456 Mon Sep 17 00:00:00 2001 From: "Edgar Y. Walker" <> Date: Wed, 11 Nov 2026 15:39:01 -0800 Subject: [PATCH 03/21] test(hashing): failing tests for starfix hash format 2 under arrow_v0.2 (ITL-910) Co-Authored-By: Claude Opus 5.5 --- .../operators/test_async_sync_equivalence.py | 22 +- tests/test_hashing/test_starfix_format2.py | 245 ++++++++++++++++++ 2 files changed, 256 insertions(+), 11 deletions(-) create mode 100644 tests/test_hashing/test_starfix_format2.py diff --git a/tests/test_core/operators/test_async_sync_equivalence.py b/tests/test_core/operators/test_async_sync_equivalence.py index 809af1813..916509dd0 100644 --- a/tests/test_core/operators/test_async_sync_equivalence.py +++ b/tests/test_core/operators/test_async_sync_equivalence.py @@ -362,18 +362,18 @@ def test_semijoin_with_nested_columns() -> None: assert_equivalent(sync_rows(SemiJoin(), *sources), async_rows(SemiJoin(), *sources)) -@pytest.mark.xfail( - strict=True, - raises=TypeError, - reason=( - "ITL-846: ArrowHasher.hash_schema passes live extension types to starfix, " - "which cannot hash them (hash_table normalizes them first)." - ), -) def test_arrow_hasher_hash_schema_with_extension_type() -> None: - """``hash_schema`` hashes a schema with a logical (extension) column.""" - schema = pa.schema([pa.field("p", _logical_column([Path("/a")], Path).type)]) - _ = get_default_context().arrow_hasher.hash_schema(schema) # pyright: ignore[reportAttributeAccessIssue] # concrete hasher method, not on the protocol + """``hash_schema`` hashes a logical (extension) column with its identity (ITL-846). + + The extension name and metadata are part of the hash, so the schema does not hash like + one holding the extension's storage type. + """ + ext_type = _logical_column([Path("/a")], Path).type + assert isinstance(ext_type, pa.ExtensionType) + hasher = get_default_context().arrow_hasher + ext_hash = hasher.hash_schema(pa.schema([pa.field("p", ext_type)])) # pyright: ignore[reportAttributeAccessIssue] # concrete hasher method, not on the protocol + storage_hash = hasher.hash_schema(pa.schema([pa.field("p", ext_type.storage_type)])) # pyright: ignore[reportAttributeAccessIssue] # concrete hasher method, not on the protocol + assert ext_hash != storage_hash # ── Data meta columns (ITL-854) ───────────────────────────────────────────── diff --git a/tests/test_hashing/test_starfix_format2.py b/tests/test_hashing/test_starfix_format2.py new file mode 100644 index 000000000..ec21528ae --- /dev/null +++ b/tests/test_hashing/test_starfix_format2.py @@ -0,0 +1,245 @@ +"""Arrow hashing on starfix hash format 2 under the label ``arrow_v0.2`` (ITL-910). + +starfix 0.5 replaced hash format 1 (prefix ``00 00 01``) with the lossless format 2 (prefix +``00 00 02``). These tests pin what that fixes in Orcapod (the collisions of ITL-848), what it +newly accepts (``fixed_size_list`` data, ITL-927; canonical extension types) and what it newly +rejects. +""" + +from __future__ import annotations + +import pyarrow as pa +import pytest + +from orcapod.contexts import get_default_context +from orcapod.core.operators import Join +from orcapod.hashing.versioned_hashers import _CURRENT_ARROW_HASHER_ID +from orcapod.sources import ArrowTableSource +from orcapod.types import ContentHash + +FORMAT_2_PREFIX = b"\x00\x00\x02" + + +def _hash_table(table: pa.Table) -> ContentHash: + """Hash ``table`` with the default context's Arrow hasher.""" + return get_default_context().arrow_hasher.hash_table(table) + + +def _row_hashes(column: pa.Array) -> list[str]: + """Content hashes of the data of a one-column source keyed by a row tag.""" + table = pa.table( + {"k": pa.array([f"r{i}" for i in range(len(column))], pa.large_string()), "x": column} + ) + source = ArrowTableSource(table, tag_columns=["k"]) + return [data.content_hash().to_string() for _, data in source.iter_data()] + + +def _non_nullable(name: str, column: pa.Array) -> pa.Table: + """A one-column table whose field is declared non-nullable.""" + return pa.Table.from_arrays( + [column], schema=pa.schema([pa.field(name, column.type, nullable=False)]) + ) + + +# --------------------------------------------------------------------------- +# Label and format +# --------------------------------------------------------------------------- + + +class TestLabel: + def test_current_arrow_label_is_arrow_v0_2(self): + assert _CURRENT_ARROW_HASHER_ID == "arrow_v0.2" + + def test_default_context_uses_arrow_v0_2(self): + assert get_default_context().arrow_hasher.hasher_id == "arrow_v0.2" + + def test_table_hash_is_format_2(self): + result = _hash_table(pa.table({"x": pa.array([1, 2], pa.int64())})) + assert result.method == "arrow_v0.2" + assert result.digest[:3] == FORMAT_2_PREFIX + + def test_row_data_hash_carries_new_label(self): + hashes = _row_hashes(pa.array([1], pa.int64())) + assert hashes[0].startswith("arrow_v0.2:000002") + + +# --------------------------------------------------------------------------- +# ITL-848 collisions are gone +# --------------------------------------------------------------------------- + + +class TestFormat1CollisionsFixed: + def test_nested_list_lengths_are_part_of_the_row_hash(self): + list3 = pa.large_list(pa.large_list(pa.large_list(pa.int64()))) + first = _row_hashes(pa.array([[[[1]], [[2], [3]]]], list3)) + second = _row_hashes(pa.array([[[[1], [2]], [[3]]]], list3)) + assert first != second + + def test_null_struct_differs_from_struct_of_nulls(self): + struct = pa.struct([("a", pa.int64())]) + null_struct = _row_hashes(pa.array([None], struct)) + struct_of_nulls = _row_hashes(pa.array([{"a": None}], struct)) + assert null_struct != struct_of_nulls + + def test_boolean_hash_does_not_depend_on_chunking(self): + table = pa.table({"x": pa.array([True, False, True])}) + rechunked = pa.Table.from_batches(table.to_batches(max_chunksize=1)) + assert rechunked.column("x").num_chunks == 3 + assert _hash_table(table) == _hash_table(rechunked) + + def test_boolean_row_count_is_part_of_the_hash(self): + one = _non_nullable("x", pa.array([True])) + three = _non_nullable("x", pa.array([True, False, False])) + assert _hash_table(one) != _hash_table(three) + + def test_slice_offset_does_not_change_the_hash(self): + sliced = pa.table({"x": pa.array([1, 2, 3, 4], pa.int64())}).slice(1, 2) + fresh = pa.table({"x": pa.array([2, 3], pa.int64())}) + assert _hash_table(sliced) == _hash_table(fresh) + + def test_chunk_layout_does_not_change_the_hash(self): + table = pa.table({"x": pa.array(["a", None, "ccc", "dd"], pa.large_string())}) + chunked = pa.Table.from_batches(table.to_batches(max_chunksize=3)) + assert chunked.column("x").num_chunks == 2 + assert _hash_table(chunked) == _hash_table(chunked.combine_chunks()) + assert _hash_table(chunked) == _hash_table(table) + + +# --------------------------------------------------------------------------- +# fixed_size_list (ITL-927) +# --------------------------------------------------------------------------- + + +def _fsl_table() -> pa.Table: + return pa.table( + { + "k": pa.array(["a", "b"], pa.large_string()), + "v": pa.array([[1.0, 2.0], [3.0, 4.0]], pa.list_(pa.float32(), 2)), + } + ) + + +class TestFixedSizeList: + def test_fixed_size_list_column_hashes(self): + assert _hash_table(_fsl_table()).digest[:3] == FORMAT_2_PREFIX + + def test_fixed_size_list_differs_from_large_list_of_same_values(self): + fsl = pa.table({"v": pa.array([[1.0, 2.0]], pa.list_(pa.float32(), 2))}) + large = pa.table({"v": pa.array([[1.0, 2.0]], pa.large_list(pa.float32()))}) + assert _hash_table(fsl) != _hash_table(large) + + def test_nested_fixed_size_list_hashes(self): + nested = pa.table( + { + "s": pa.array( + [{"v": [1, 2]}], pa.struct([("v", pa.list_(pa.int64(), 2))]) + ), + "l": pa.array([[[1, 2]]], pa.large_list(pa.list_(pa.int64(), 2))), + } + ) + assert _hash_table(nested).digest[:3] == FORMAT_2_PREFIX + + def test_source_with_fixed_size_list_column_keeps_its_type(self): + source = ArrowTableSource(_fsl_table(), tag_columns=["k"]) + assert source.as_table().schema.field("v").type == pa.list_(pa.float32(), 2) + hashes = [data.content_hash() for _, data in source.iter_data()] + assert len(set(hashes)) == 2 + + def test_fixed_size_list_column_runs_through_a_join(self): + source = ArrowTableSource(_fsl_table(), tag_columns=["k"], infer_nullable=True) + other = ArrowTableSource( + pa.table( + { + "k": pa.array(["a", "b"], pa.large_string()), + "w": pa.array([1, 2], pa.int64()), + } + ), + tag_columns=["k"], + infer_nullable=True, + ) + joined = Join()(source, other) + table = joined.as_table() + assert table.schema.field("v").type == pa.list_(pa.float32(), 2) + assert table.column("v").to_pylist() == [[1.0, 2.0], [3.0, 4.0]] + assert len({data.content_hash() for _, data in joined.iter_data()}) == 2 + + +# --------------------------------------------------------------------------- +# Canonical Arrow extension types keep their identity +# --------------------------------------------------------------------------- + + +class TestCanonicalExtensionTypes: + def test_uuid_column_differs_from_its_storage(self): + storage = pa.array([b"0" * 16, b"1" * 16], pa.binary(16)) + uuid_table = pa.table({"u": pa.ExtensionArray.from_storage(pa.uuid(), storage)}) + storage_table = pa.table({"u": storage}) + assert _hash_table(uuid_table) != _hash_table(storage_table) + + def test_fixed_shape_tensor_differs_from_its_storage(self): + tensor_type = pa.fixed_shape_tensor(pa.float32(), [2]) + storage = pa.array([[1.0, 2.0]], pa.list_(pa.float32(), 2)) + tensor_table = pa.table( + {"t": pa.ExtensionArray.from_storage(tensor_type, storage)} + ) + assert _hash_table(tensor_table) != _hash_table(pa.table({"t": storage})) + + def test_uuid_schema_differs_from_its_storage(self): + hasher = get_default_context().arrow_hasher + uuid_schema = pa.schema([pa.field("u", pa.uuid())]) + storage_schema = pa.schema([pa.field("u", pa.binary(16))]) + assert hasher.hash_schema(uuid_schema) != hasher.hash_schema(storage_schema) # pyright: ignore[reportAttributeAccessIssue] # concrete hasher method, not on the protocol + + +# --------------------------------------------------------------------------- +# Inputs starfix 0.5 rejects +# --------------------------------------------------------------------------- + + +class TestRejections: + def test_zero_row_map_column_is_rejected(self): + table = pa.table( + {"m": pa.array([], pa.map_(pa.large_string(), pa.int64()))} + ) + with pytest.raises(ValueError, match="map"): + _hash_table(table) + + def test_non_nullable_column_holding_a_null_is_rejected(self): + table = pa.Table.from_arrays( + [pa.array(["a", "b"], pa.large_string()), pa.array([1, None], pa.int64())], + schema=pa.schema( + [ + pa.field("k", pa.large_string()), + pa.field("x", pa.int64(), nullable=False), + ] + ), + ) + with pytest.raises(ValueError, match="Non-nullable field 'x'"): + ArrowTableSource(table, tag_columns=["k"]) + + def test_duplicate_column_names_are_rejected(self): + table = pa.Table.from_arrays( + [pa.array([1]), pa.array([2])], names=["x", "x"] + ) + with pytest.raises(ValueError, match="Ambiguous node path 'x'"): + get_default_context().semantic_hasher.hash_object(table) + + def test_slash_name_aliasing_a_struct_child_is_rejected(self): + table = pa.table( + { + "k": pa.array(["a"], pa.large_string()), + "a/b": pa.array([1], pa.int64()), + "a": pa.array([{"b": 2}], pa.struct([("b", pa.int64())])), + } + ) + with pytest.raises(ValueError, match="Ambiguous node path 'a/b'"): + ArrowTableSource(table, tag_columns=["k"]) + + def test_slash_name_aliasing_nothing_is_accepted(self): + table = pa.table( + { + "k": pa.array(["a"], pa.large_string()), + "km/h": pa.array([1], pa.int64()), + } + ) + ArrowTableSource(table, tag_columns=["k"]) From ca5d8cd55ccb20668c8c6fa8c4ee68f897bad6f0 Mon Sep 17 00:00:00 2001 From: "Edgar Y. Walker" <> Date: Wed, 11 Nov 2026 15:40:08 -0800 Subject: [PATCH 04/21] feat(hashing): adopt starfix 0.5 (hash format 2) under label arrow_v0.2 (ITL-910) Bump starfix to >=0.5.0,<0.6 and the Arrow hasher label to arrow_v0.2. Hash pyarrow's canonical extension types and extension-typed schemas with their extension identity instead of as their storage type. Co-Authored-By: Claude Opus 5.5 --- pyproject.toml | 2 +- .../contexts/data/schemas/context_schema.json | 2 +- src/orcapod/contexts/data/v0.1.json | 5 ++- src/orcapod/hashing/arrow_hashers.py | 33 ++++++++++++--- src/orcapod/hashing/schema_cleaner.py | 41 +++++++++++++++++++ src/orcapod/hashing/versioned_hashers.py | 6 ++- uv.lock | 8 ++-- 7 files changed, 82 insertions(+), 15 deletions(-) diff --git a/pyproject.toml b/pyproject.toml index efd70d255..8f0e26bbf 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -19,7 +19,7 @@ dependencies = [ "graphviz>=0.21", "gitpython>=3.1.45", "universal-pathlib>=0.3.8", - "starfix~=0.4.0", + "starfix>=0.5.0,<0.6", "tzdata>=2024.1", "uuid-utils>=0.11.1", "s3fs>=2025.12.0", diff --git a/src/orcapod/contexts/data/schemas/context_schema.json b/src/orcapod/contexts/data/schemas/context_schema.json index 38c377a8f..c5f0c0f38 100644 --- a/src/orcapod/contexts/data/schemas/context_schema.json +++ b/src/orcapod/contexts/data/schemas/context_schema.json @@ -175,7 +175,7 @@ "arrow_hasher": { "_class": "orcapod.hashing.arrow_hashers.StarfixArrowHasher", "_config": { - "hasher_id": "arrow_v0.1", + "hasher_id": "arrow_v0.2", "type_converter": {"_ref": "type_converter"}, "semantic_hasher": {"_ref": "semantic_hasher"} } diff --git a/src/orcapod/contexts/data/v0.1.json b/src/orcapod/contexts/data/v0.1.json index a830d71cd..66017eb30 100644 --- a/src/orcapod/contexts/data/v0.1.json +++ b/src/orcapod/contexts/data/v0.1.json @@ -138,7 +138,7 @@ "arrow_hasher": { "_class": "orcapod.hashing.arrow_hashers.StarfixArrowHasher", "_config": { - "hasher_id": "arrow_v0.1", + "hasher_id": "arrow_v0.2", "type_converter": {"_ref": "type_converter"}, "semantic_hasher": {"_ref": "semantic_hasher"} } @@ -165,7 +165,8 @@ "Moved SpikeInterface types (LogicalSIRecording, LogicalSISorting, LogicalSIMotion, LogicalSISortingAnalyzer) and their handlers to the standalone orcapod-extension-spikeinterface package; removed all SI _optional entries from this spec and SI deps from pyproject.toml; introduced OrcapodExtension protocol and op.register_extension() for normalized extension registration (ITL-473)", "Registered SchemaHandler for orcapod.types.Schema in the context registry; moved handler dispatch before _is_structure check in hash_object so SchemaHandler takes priority over _expand_mapping for Schema objects (ITL-639)", "Registered PolarsExprHandler for polars.Expr: hashes an expression as ContentHash('polars_expr_v1', sha256 of the canonical JSON of an Orcapod-owned expression tree parsed strictly from Expr.meta.serialize(format='json')); expressions outside the supported subset (Python functions, multi-column selectors, aggregations, alias, when/then, non-finite float literals, pl.Series literals) raise ValueError naming the expression (ITL-828)", - "Hard cut (v0.3): numpy.ndarray moves to the value-based array format. LogicalNumpyArray stores extension orcapod.ndarray over struct (Orcapod dtype tag, shape, little-endian C-order packed bytes; extension metadata {\"version\":1}) and NumpyArrayHandler hashes by value with label ndarray_v1 (array_digest_v1 over the stored row). The old extension name numpy.ndarray (.npy bytes, label sha256) is no longer registered or written; v0.2 stores holding arrays are not migrated (ITL-888)" + "Hard cut (v0.3): numpy.ndarray moves to the value-based array format. LogicalNumpyArray stores extension orcapod.ndarray over struct (Orcapod dtype tag, shape, little-endian C-order packed bytes; extension metadata {\"version\":1}) and NumpyArrayHandler hashes by value with label ndarray_v1 (array_digest_v1 over the stored row). The old extension name numpy.ndarray (.npy bytes, label sha256) is no longer registered or written; v0.2 stores holding arrays are not migrated (ITL-888)", + "Hard cut (v0.3): the Arrow hasher label moves from arrow_v0.1 (starfix 0.4, hash format 1, prefix 000001) to arrow_v0.2 (starfix 0.5, hash format 2, prefix 000002). Format 2 is lossless (nested list lengths, null structs, Boolean row counts, chunking-independent) and hashes fixed-size lists; every Arrow-derived hash changes. Order options stay off; pyarrow canonical extension types (arrow.uuid, arrow.fixed_shape_tensor) and hash_schema on extension columns hash with their extension identity (ITL-910)" ] } } diff --git a/src/orcapod/hashing/arrow_hashers.py b/src/orcapod/hashing/arrow_hashers.py index 54c27a528..521fb969b 100644 --- a/src/orcapod/hashing/arrow_hashers.py +++ b/src/orcapod/hashing/arrow_hashers.py @@ -8,6 +8,7 @@ from orcapod.hashing.schema_cleaner import ( clean_schema_for_hashing, has_extension_metadata, + has_extension_type, ) from orcapod.hashing.visitors import SemanticHashingVisitor from orcapod.utils.arrow_utils import normalize_extension_columns @@ -30,8 +31,22 @@ class StarfixArrowHasher: Extension-typed columns without a registered hasher pass through with their full extension metadata intact. 2. **Starfix hashing** — ``ArrowDigester.hash_table`` produces a 35-byte - versioned SHA-256 digest that is byte-for-byte identical to the Rust - ``starfix`` crate output. + digest: the starfix hash format number ``00 00 02`` (hash format 2, + starfix 0.5) followed by a SHA-256 digest, byte-for-byte identical to + the Rust ``starfix`` crate output. The order of columns and of struct + fields is not part of the hash (starfix's order options stay off); row + order and list element order are. Extension identity + (``ARROW:extension:name`` and ``ARROW:extension:metadata``) is hashed + through starfix's ``include_metadata`` whenever the table has an + extension column, including pyarrow's built-in canonical extension types + such as ``arrow.uuid``. + + starfix raises ``ValueError`` for inputs it cannot hash: data of ``null``, + ``map``, interval and view types (whatever the row count); union and + run-end-encoded types; two schema nodes sharing a path (duplicate column + names, or a ``/`` in a name aliasing a struct child); and a non-nullable + field holding a null. Those errors propagate unchanged; they name the + column path and type. Parameters ---------- @@ -127,8 +142,14 @@ def _process_table_columns(self, table: "pa.Table | pa.RecordBatch") -> "pa.Tabl return normalize_extension_columns(intermediate) def hash_schema(self, schema: "pa.Schema") -> ContentHash: - """Hash an Arrow schema using the starfix canonical algorithm.""" - include_meta = has_extension_metadata(schema) + """Hash an Arrow schema using the starfix canonical algorithm. + + Extension-typed columns are normalized to their storage type with the + extension identity in field metadata, as ``hash_table`` does, so the + extension name and metadata are part of the hash. + """ + schema = normalize_extension_columns(schema.empty_table()).schema + include_meta = has_extension_metadata(schema) or has_extension_type(schema) if include_meta: schema = clean_schema_for_hashing(schema) digest = ArrowDigester.hash_schema(schema, include_metadata=include_meta) @@ -140,7 +161,9 @@ def hash_table(self, table: "pa.Table | pa.RecordBatch") -> ContentHash: table = pa.Table.from_batches([table]) processed_table = self._process_table_columns(table) - include_meta = has_extension_metadata(processed_table.schema) + include_meta = has_extension_metadata( + processed_table.schema + ) or has_extension_type(processed_table.schema) if include_meta: clean_schema = clean_schema_for_hashing(processed_table.schema) clean_table = pa.Table.from_arrays( diff --git a/src/orcapod/hashing/schema_cleaner.py b/src/orcapod/hashing/schema_cleaner.py index 09a55768f..2fa55086f 100644 --- a/src/orcapod/hashing/schema_cleaner.py +++ b/src/orcapod/hashing/schema_cleaner.py @@ -162,6 +162,47 @@ def clean_schema_for_hashing(schema: pa.Schema) -> pa.Schema: ) +def _contains_extension_type(arrow_type: pa.DataType) -> bool: + """Return True if ``arrow_type`` is or contains an Arrow extension type.""" + if isinstance(arrow_type, pa.BaseExtensionType): + return True + if pa.types.is_struct(arrow_type): + return any( + _contains_extension_type(arrow_type.field(i).type) + for i in range(arrow_type.num_fields) + ) + if ( + pa.types.is_list(arrow_type) + or pa.types.is_large_list(arrow_type) + or pa.types.is_fixed_size_list(arrow_type) + ): + return _contains_extension_type(arrow_type.value_type) + if pa.types.is_map(arrow_type): + return _contains_extension_type(arrow_type.key_type) or _contains_extension_type( + arrow_type.item_type + ) + if pa.types.is_dictionary(arrow_type): + return _contains_extension_type(arrow_type.value_type) + return False + + +def has_extension_type(schema: pa.Schema) -> bool: + """Return True if any field of ``schema``, at any depth, has an extension type. + + Covers Python-defined ``pa.ExtensionType`` subclasses and pyarrow's built-in canonical + extension types (``arrow.uuid``, ``arrow.fixed_shape_tensor``, ...), which derive from + ``pa.BaseExtensionType`` only. starfix hashes an extension type as its storage type and + adds its ``ARROW:extension:*`` identity only when metadata is included. + + Args: + schema: The ``pa.Schema`` to inspect. + + Returns: + True if an extension type occurs anywhere in ``schema``; False otherwise. + """ + return any(_contains_extension_type(field.type) for field in schema) + + def has_extension_metadata(schema: pa.Schema) -> bool: """Return True if ``schema`` has any ``ARROW:extension:name`` key at any depth. diff --git a/src/orcapod/hashing/versioned_hashers.py b/src/orcapod/hashing/versioned_hashers.py index c75cf7ed5..1aa1c84f4 100644 --- a/src/orcapod/hashing/versioned_hashers.py +++ b/src/orcapod/hashing/versioned_hashers.py @@ -36,8 +36,10 @@ # hashes can be distinguished from newly-computed ones. _CURRENT_SEMANTIC_HASHER_ID = "semantic_v0.1" -# The hasher_id for the Arrow hasher. -_CURRENT_ARROW_HASHER_ID = "arrow_v0.1" +# The hasher_id for the Arrow hasher. Bumped whenever the hash of any table can change, +# whatever the cause; its number is not kept equal to the starfix hash format number. +# arrow_v0.1: starfix 0.4 (hash format 1). arrow_v0.2: starfix 0.5 (hash format 2, ITL-910). +_CURRENT_ARROW_HASHER_ID = "arrow_v0.2" # --------------------------------------------------------------------------- diff --git a/uv.lock b/uv.lock index 7ea17ee90..0ab5dbb3f 100644 --- a/uv.lock +++ b/uv.lock @@ -2388,7 +2388,7 @@ requires-dist = [ { name = "redis", marker = "extra == 'all'", specifier = ">=6.2.0" }, { name = "redis", marker = "extra == 'redis'", specifier = ">=6.2.0" }, { name = "s3fs", specifier = ">=2025.12.0" }, - { name = "starfix", specifier = "~=0.4.0" }, + { name = "starfix", specifier = ">=0.5.0,<0.6" }, { name = "typer", specifier = ">=0.12" }, { name = "tzdata", specifier = ">=2024.1" }, { name = "universal-pathlib", specifier = ">=0.3.8" }, @@ -3932,14 +3932,14 @@ wheels = [ [[package]] name = "starfix" -version = "0.4.0" +version = "0.5.0" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "pyarrow" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/fc/c1/f3269cca708e13d47bf86c5459f9575e436e99d88cd29fec9e5cfd1d6003/starfix-0.4.0.tar.gz", hash = "sha256:e63711756640d4618d34c58f9d80ed7de9722f50ff806ec0ceecdcf151f26e48", size = 71807, upload-time = "2026-06-29T00:14:18.687Z" } +sdist = { url = "https://files.pythonhosted.org/packages/cb/15/f431d84042cab83680b3e857455c133bc0493c629b93fc23f28cd85b07ee/starfix-0.5.0.tar.gz", hash = "sha256:677f001fae5f0173c52ffaaf761fb96becf856ff679ab5d201208f453396a9fb", size = 225505, upload-time = "2026-11-11T23:21:04.713Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/1f/a2/b08c4c9e3e95e6e4224ad1fe857a2eaef1e473d9db6063d8151f5e8ac21f/starfix-0.4.0-py3-none-any.whl", hash = "sha256:3ec6c423eb131bfde1471a03219eea46389603753c30d90bdf3bfbe2bf8863c3", size = 10809, upload-time = "2026-06-29T00:14:17.748Z" }, + { url = "https://files.pythonhosted.org/packages/4e/f6/8b9042044b9bf520c5bac38d0630c411e66ccc47635e8c0293bd9a7d5038/starfix-0.5.0-py3-none-any.whl", hash = "sha256:6982fa4598f99e20428b76041378e3203f2894e6542f6e3ba75746bdbfea0ad2", size = 28736, upload-time = "2026-11-11T23:21:03.711Z" }, ] [[package]] From bd252f28ab2d0e688a017bc8f838779cff390904 Mon Sep 17 00:00:00 2001 From: "Edgar Y. Walker" <> Date: Wed, 11 Nov 2026 15:44:46 -0800 Subject: [PATCH 05/21] test(golden): regenerate golden hashes for starfix hash format 2 / arrow_v0.2 (ITL-910) Explicitly approved regeneration (developer decision 3, November 11, 2026): 184 Arrow-derived values change (arrow 53/53, datagram 40/171, pipeline 68/305, schema 22/92, semantic 1/64); no key added or removed among them. 21 arrow entries added: fixed-size lists, the ITL-848 collision pairs, a chunked Boolean column, canonical extension types and extension schemas. Co-Authored-By: Claude Opus 5.5 --- tests/test_hashing/golden/cases.py | 52 ++++++- tests/test_hashing/golden/data/arrow.json | 127 +++++++++------- tests/test_hashing/golden/data/datagram.json | 80 +++++----- tests/test_hashing/golden/data/pipeline.json | 140 +++++++++--------- tests/test_hashing/golden/data/schema.json | 44 +++--- tests/test_hashing/golden/data/semantic.json | 2 +- .../golden/generate_golden_hashes.py | 7 +- 7 files changed, 261 insertions(+), 191 deletions(-) diff --git a/tests/test_hashing/golden/cases.py b/tests/test_hashing/golden/cases.py index ada9870a7..2813051e7 100644 --- a/tests/test_hashing/golden/cases.py +++ b/tests/test_hashing/golden/cases.py @@ -497,6 +497,30 @@ def _arrow_tables() -> dict[str, pa.Table]: } ) tables["empty.zero_rows"] = pa.table({"a": pa.array([], pa.int64())}) + # Added with starfix hash format 2 (ITL-910): inputs format 1 could not hash or + # hashed alike (ITL-848, ITL-927). + tables["fixed_size_list.float32_2"] = pa.table( + {"a": pa.array([[0.5, 1.5], [2.5, 3.5]], pa.list_(pa.float32(), 2))} + ) + tables["fixed_size_list.null_slot"] = pa.table( + {"a": pa.array([[1, 2], None], pa.list_(pa.int64(), 2))} + ) + list3 = pa.large_list(pa.large_list(pa.large_list(pa.int64()))) + tables["collision.list_lengths_a"] = pa.table( + {"a": pa.array([[[[1]], [[2], [3]]]], list3)} + ) + tables["collision.list_lengths_b"] = pa.table( + {"a": pa.array([[[[1], [2]], [[3]]]], list3)} + ) + tables["collision.null_struct"] = pa.table( + {"a": pa.array([None], pa.struct([("x", pa.int64())]))} + ) + tables["collision.struct_of_nulls"] = pa.table( + {"a": pa.array([{"x": None}], pa.struct([("x", pa.int64())]))} + ) + tables["bool.chunked"] = pa.Table.from_batches( + tables["bool"].to_batches(max_chunksize=1) + ) return tables @@ -525,6 +549,22 @@ def _extension_tables() -> dict[str, pa.Table]: tables["extension.unregistered_passthrough"] = pa.table( {"c": pa.ExtensionArray.from_storage(GoldenOpaqueType(), storage)} ) + # pyarrow's canonical extension types, hashable with their identity since ITL-910. + tables["extension.canonical_uuid"] = pa.table( + { + "c": pa.ExtensionArray.from_storage( + pa.uuid(), pa.array([UUID_A.bytes, UUID_B.bytes], pa.binary(16)) + ) + } + ) + tables["extension.canonical_fixed_shape_tensor"] = pa.table( + { + "c": pa.ExtensionArray.from_storage( + pa.fixed_shape_tensor(pa.float32(), [2]), + pa.array([[0.5, 1.5], [2.5, 3.5]], pa.list_(pa.float32(), 2)), + ) + } + ) return tables @@ -542,14 +582,22 @@ def arrow_extension_table_hashes() -> dict[str, Any]: @case("arrow", "schema") def arrow_schema_hashes() -> dict[str, Any]: - # hash_schema is not reachable for extension-typed schemas yet (ITL-848), so only - # extension-free schemas are pinned here. hasher = _ctx().arrow_hasher return { k: hasher.hash_schema(t.schema).to_string() for k, t in _arrow_tables().items() } +@case("arrow", "extension_schema") +def arrow_extension_schema_hashes() -> dict[str, Any]: + # hash_schema hashes extension columns with their identity since ITL-910 (ITL-846). + hasher = _ctx().arrow_hasher + return { + k: hasher.hash_schema(t.schema).to_string() + for k, t in _extension_tables().items() + } + + # --------------------------------------------------------------------------- # Group: datagram — public sources carrying each logical type # --------------------------------------------------------------------------- diff --git a/tests/test_hashing/golden/data/arrow.json b/tests/test_hashing/golden/data/arrow.json index ff71bb164..5c636b5ec 100644 --- a/tests/test_hashing/golden/data/arrow.json +++ b/tests/test_hashing/golden/data/arrow.json @@ -1,55 +1,76 @@ { - "extension_table.extension.path": "arrow_v0.1:000001e32d9b28a94c4f47e33a647ed137a7316d4cf80d5bd216cc74e2ea4e46bc8d48", - "extension_table.extension.unregistered_passthrough": "arrow_v0.1:00000197e4832ff6efd7436451ab12fbbcde498ae4871ab74b13c9b80c32a4ba9ee44a", - "extension_table.extension.uuid": "arrow_v0.1:0000016efd6e3b9a4c03e2c78bc410a5e04bd5e4bfd7800e3a4054b67805c22fd6436f", - "schema.bool": "arrow_v0.1:000001e1a1ab1078fd92ded5be471806dd8fc4f7d4654dfdb7d9644942269de543e0ee", - "schema.empty.zero_rows": "arrow_v0.1:000001f31bd23e3b2489f68d11a6d3087d09fb8df563c2fb84b3084a6ddf042fe6546a", - "schema.float.float32": "arrow_v0.1:000001fcd0edfcd85a395e7a0d04b384f3cba908f87c571f7fffb6b1ac5a86958d058f", - "schema.float.float64": "arrow_v0.1:000001f68c38ed991e7d525d27dda0d5c79ad04c0a7d6ffbc61099167ac6377b475852", - "schema.layout.binary": "arrow_v0.1:0000011b8b0793ca3db4779c18d400924a60b3871633eedc7c247a97c706ad47180b9c", - "schema.layout.large_binary": "arrow_v0.1:0000011b8b0793ca3db4779c18d400924a60b3871633eedc7c247a97c706ad47180b9c", - "schema.layout.large_list": "arrow_v0.1:000001cec41debef42a67a59f89ebf943757a06619b738e0a945db6134bce4f846c8a1", - "schema.layout.large_string": "arrow_v0.1:00000191f7f5c26abcceb89a6fb399fd0dc333c743ad38a5f2a4da8b274b7e96acfe18", - "schema.layout.list": "arrow_v0.1:000001cec41debef42a67a59f89ebf943757a06619b738e0a945db6134bce4f846c8a1", - "schema.layout.string": "arrow_v0.1:00000191f7f5c26abcceb89a6fb399fd0dc333c743ad38a5f2a4da8b274b7e96acfe18", - "schema.nested.list_list": "arrow_v0.1:0000014387f6de75a16b23c9a7e9d90788dab5f2cf9bf4e891a71d5b14dd193de90424", - "schema.nested.list_struct": "arrow_v0.1:000001a257fe950b239e9d3a4da720c75744f10a405c7ad6824b744e0644a461c4da5a", - "schema.nested.struct": "arrow_v0.1:000001402dbf363e9e7ebaf5df601c1684ebab94eea5090394c38ab3c0076113f02d35", - "schema.nullability.non_nullable": "arrow_v0.1:0000013487bfb8486c6973de2ead68dde548f34da47db81157410996575a8f36791e85", - "schema.nullability.nullable": "arrow_v0.1:000001f31bd23e3b2489f68d11a6d3087d09fb8df563c2fb84b3084a6ddf042fe6546a", - "schema.nulls.int64": "arrow_v0.1:000001f31bd23e3b2489f68d11a6d3087d09fb8df563c2fb84b3084a6ddf042fe6546a", - "schema.nulls.string": "arrow_v0.1:00000191f7f5c26abcceb89a6fb399fd0dc333c743ad38a5f2a4da8b274b7e96acfe18", - "schema.order.ab": "arrow_v0.1:00000100ad4d558238e956277266aaa4960beca8b50b1bc6fb5d367fa90d01febeaecf", - "schema.order.ba": "arrow_v0.1:00000100ad4d558238e956277266aaa4960beca8b50b1bc6fb5d367fa90d01febeaecf", - "schema.order.rows_reversed": "arrow_v0.1:00000100ad4d558238e956277266aaa4960beca8b50b1bc6fb5d367fa90d01febeaecf", - "schema.width.int16": "arrow_v0.1:00000178c6959f782c8a48a2d5b2d3f29e66bf5c705bbd9a5f5e64254b6fa6ffd34a52", - "schema.width.int32": "arrow_v0.1:0000011844a51e73c25585ae05f082dabb80bfebaa16200fc06da0d6b76552db358063", - "schema.width.int64": "arrow_v0.1:000001f31bd23e3b2489f68d11a6d3087d09fb8df563c2fb84b3084a6ddf042fe6546a", - "schema.width.int8": "arrow_v0.1:000001d22fd4d64d8554ce4ab154133f31f73b2de03b55c681899ce92aeea83a708f7a", - "schema.width.uint8": "arrow_v0.1:000001c63af839f862f03281ac8fc3054dd41f42080dc323c1296f0662125f19385a49", - "table.bool": "arrow_v0.1:0000012fafed9df99235be951e30505fabd673fd8ca4e3ef8ed48fce0c5ef9ff8aacef", - "table.empty.zero_rows": "arrow_v0.1:000001d93c4003276f674c8c0cdc018db38a9cc545d7aec3a1a71f672e78facea059d7", - "table.float.float32": "arrow_v0.1:00000177212619946903d4b8c103832d9b0daad142e51e81ee334b5c00040cab8254ed", - "table.float.float64": "arrow_v0.1:000001c1d762b8d385549ea148addb9929818adb75d575f60a64156dfb0bce3b5eb19c", - "table.layout.binary": "arrow_v0.1:000001b19bcc9f2f94795bfd70390c3a62ee22a97e33db3e6d0c5b5902df8f59fd0650", - "table.layout.large_binary": "arrow_v0.1:000001b19bcc9f2f94795bfd70390c3a62ee22a97e33db3e6d0c5b5902df8f59fd0650", - "table.layout.large_list": "arrow_v0.1:00000189f2c4331bbc824253ae959dd9d76b9c1a0775f389cf087997d51c00f0f98d97", - "table.layout.large_string": "arrow_v0.1:000001590b6b8a65a49348746f5b11ad476c65d25f106becdf95bb807b1fdca8d8e88b", - "table.layout.list": "arrow_v0.1:00000189f2c4331bbc824253ae959dd9d76b9c1a0775f389cf087997d51c00f0f98d97", - "table.layout.string": "arrow_v0.1:000001590b6b8a65a49348746f5b11ad476c65d25f106becdf95bb807b1fdca8d8e88b", - "table.nested.list_list": "arrow_v0.1:000001639751855e6618fdfba13f9d4bf5305716443012b812bf81fd652a43e89a671d", - "table.nested.list_struct": "arrow_v0.1:0000013c0c348a94fa935da0e77e9a47c26c79ff726c3383ad0206da391735348af38d", - "table.nested.struct": "arrow_v0.1:0000012af9c2ad25c501bbfa488ea24088eca78480303b8b413968c33836a133458dfb", - "table.nullability.non_nullable": "arrow_v0.1:000001a55bac289fa4cb83a654a76adc0eaf9cb5a953d14c90c7afb9a3b64c1f5f51e8", - "table.nullability.nullable": "arrow_v0.1:000001d469c2c16d7e2c80bed9daf4d1bd919eaa8523edd905623eccb5fe8abed7c430", - "table.nulls.int64": "arrow_v0.1:000001604458bd26489ed611491258c0a8a7e1336772fe6b603d1948ec16bce5a7f6da", - "table.nulls.string": "arrow_v0.1:00000189124cc15f2fbfc1d2056ae127123c1408307fbf8ba05b6018ea62b49eef4d39", - "table.order.ab": "arrow_v0.1:000001fb52c5ca8baed03ac9c12f5dfd0fc7289280a4bd147200eb7cf901a1cee2dfae", - "table.order.ba": "arrow_v0.1:000001fb52c5ca8baed03ac9c12f5dfd0fc7289280a4bd147200eb7cf901a1cee2dfae", - "table.order.rows_reversed": "arrow_v0.1:00000151aa84ec1e72539ee16ed86379c9681bc6c0aee94edf56caa3f302d3d2a5dc13", - "table.width.int16": "arrow_v0.1:0000013e000f3e0b4beeba1e5ed6246ab460a93ffc370591e07e8350c16bffdf5dedd6", - "table.width.int32": "arrow_v0.1:000001d831e96457f732f785fb0963eb56a8723e18778d90b6ad83cfc6ac3062cb51ef", - "table.width.int64": "arrow_v0.1:000001d469c2c16d7e2c80bed9daf4d1bd919eaa8523edd905623eccb5fe8abed7c430", - "table.width.int8": "arrow_v0.1:00000160707f6988db37fafe9ef4506de6855cddcacba6a75f9f30875b3067da949b8d", - "table.width.uint8": "arrow_v0.1:000001dded35c6cee950b7a706dc347563e65ce3525b910ae89a111dfea033483609ac" + "extension_schema.extension.canonical_fixed_shape_tensor": "arrow_v0.2:0000022df2fb43d6457645dea99103a5ee4ca34c1728ef387685dbb73c58a43fc6a002", + "extension_schema.extension.canonical_uuid": "arrow_v0.2:00000236be584e0cd3a2668a6a580119beecc6e63496abf7dc098302bb597de10190e9", + "extension_schema.extension.path": "arrow_v0.2:0000024887bbb622b764650a2d4346f7fcd84cdc17f4a6a8a1b9a33cf0e78a782d8420", + "extension_schema.extension.unregistered_passthrough": "arrow_v0.2:000002853ab951d35ce3198dd79a53acdb002b47b213f4d7a7b7bd3f9e81e76535873f", + "extension_schema.extension.uuid": "arrow_v0.2:000002bfbbdea0a2cf9f0941aad6fbb6510bbc8b5d46f4e1bc473ddb42ba2e817dbf76", + "extension_table.extension.canonical_fixed_shape_tensor": "arrow_v0.2:0000029cdbbb5f9c731c669dceb6661878b8f0d1cccc5c6609f7e18d2f0b8691c63bf6", + "extension_table.extension.canonical_uuid": "arrow_v0.2:0000024cf6272f99fa3465a975a572a4cfa5aaaf07d1d063fda398cbbaf1dafbe8913d", + "extension_table.extension.path": "arrow_v0.2:000002f558ddf91c1c5f706827496773171d001d541cc96945e2f58420d939653efcfb", + "extension_table.extension.unregistered_passthrough": "arrow_v0.2:000002b9c5f64eb011d128c21fa0570cab2bf34915277103a843c3d2174bb14c4ad158", + "extension_table.extension.uuid": "arrow_v0.2:000002062ec7b363fa63cfbfac9085e5fbc271720d1bcfbd66018d30996915f6be39af", + "schema.bool": "arrow_v0.2:00000292f4b85e8651fe4c797b8057205fb5aa89dd8420bb6ecef56e70f91c23b06718", + "schema.bool.chunked": "arrow_v0.2:00000292f4b85e8651fe4c797b8057205fb5aa89dd8420bb6ecef56e70f91c23b06718", + "schema.collision.list_lengths_a": "arrow_v0.2:00000217b4f541f59698108a9542f81fb4551d5ed9a636f7d7e9c91e44b9ffacb1a244", + "schema.collision.list_lengths_b": "arrow_v0.2:00000217b4f541f59698108a9542f81fb4551d5ed9a636f7d7e9c91e44b9ffacb1a244", + "schema.collision.null_struct": "arrow_v0.2:000002377ba0fc7c623d903020da3c502ec41ae878327623355ec9d059e46cef008230", + "schema.collision.struct_of_nulls": "arrow_v0.2:000002377ba0fc7c623d903020da3c502ec41ae878327623355ec9d059e46cef008230", + "schema.empty.zero_rows": "arrow_v0.2:000002756a354ce60282443c464797dfb1fe280a84608545b20376d17bb62cba7e282e", + "schema.fixed_size_list.float32_2": "arrow_v0.2:00000256387ebbc501cb6fcad40accf4a1586d4cd4c97cc22a6c9f99b3103575e5c2c6", + "schema.fixed_size_list.null_slot": "arrow_v0.2:000002a91c8dfd080d7012a5495e6a913c69c3bade25caeb2cbdcb1174d7871af4b431", + "schema.float.float32": "arrow_v0.2:0000026a27b2c2276546fc46218d22a66cc44d468aa7fe5b478f6c1afe2ba648ca86ea", + "schema.float.float64": "arrow_v0.2:000002a8bb0c99c2d60fd20d091bf861eff2c889fbbd9d8f08e0a8ad4afd4a6ebb009d", + "schema.layout.binary": "arrow_v0.2:000002346ed5761cafbd7a622360039d70f05235102ebe07cdbaaa94dcc7fb99a91f65", + "schema.layout.large_binary": "arrow_v0.2:000002346ed5761cafbd7a622360039d70f05235102ebe07cdbaaa94dcc7fb99a91f65", + "schema.layout.large_list": "arrow_v0.2:000002409130499175385ea85061c49e23cc7d91be7cff0a9b4bb3eadab2dcd65e4aa6", + "schema.layout.large_string": "arrow_v0.2:000002d9023abe34e3bde2cb36f499d4eecc685252bb1e06713f0f8e04af77ef580e3b", + "schema.layout.list": "arrow_v0.2:000002409130499175385ea85061c49e23cc7d91be7cff0a9b4bb3eadab2dcd65e4aa6", + "schema.layout.string": "arrow_v0.2:000002d9023abe34e3bde2cb36f499d4eecc685252bb1e06713f0f8e04af77ef580e3b", + "schema.nested.list_list": "arrow_v0.2:0000029753856b34b9c54c1cfa002906119f22e1255fa9b944feb46c8856f395af8fdd", + "schema.nested.list_struct": "arrow_v0.2:000002cd3954fa9d6958fc1a13b0e9534b62cf76fd31dc45b3ee26ad0c054664a73b35", + "schema.nested.struct": "arrow_v0.2:0000021c6ab7531323b6ed5c4f8bceebe1a7351205355481e40a82213076e969ba8381", + "schema.nullability.non_nullable": "arrow_v0.2:000002389fdc8277c962a8352a2529d0925a0c4d915257d6a1b12d7e052574be1b89e1", + "schema.nullability.nullable": "arrow_v0.2:000002756a354ce60282443c464797dfb1fe280a84608545b20376d17bb62cba7e282e", + "schema.nulls.int64": "arrow_v0.2:000002756a354ce60282443c464797dfb1fe280a84608545b20376d17bb62cba7e282e", + "schema.nulls.string": "arrow_v0.2:000002d9023abe34e3bde2cb36f499d4eecc685252bb1e06713f0f8e04af77ef580e3b", + "schema.order.ab": "arrow_v0.2:000002b8f677fa5b62affbb33424741912b558b4465167c45a3598287211060412a743", + "schema.order.ba": "arrow_v0.2:000002b8f677fa5b62affbb33424741912b558b4465167c45a3598287211060412a743", + "schema.order.rows_reversed": "arrow_v0.2:000002b8f677fa5b62affbb33424741912b558b4465167c45a3598287211060412a743", + "schema.width.int16": "arrow_v0.2:000002c9ea066d6e1b441ad66b24a5107953dbdda78c7d25d4f4bde8fb685c4748245f", + "schema.width.int32": "arrow_v0.2:000002447a113da124fde874ccfa83a281d7053ae1606ac6fac1f90b6d0a41bf5e8845", + "schema.width.int64": "arrow_v0.2:000002756a354ce60282443c464797dfb1fe280a84608545b20376d17bb62cba7e282e", + "schema.width.int8": "arrow_v0.2:00000233ec0dccf2b00dbbd7b75e5bb06152b5a53acee7101617fbff06641e7965cb9b", + "schema.width.uint8": "arrow_v0.2:00000252e16232ff906ecf17b849d7398396d2fe973ff9ca378ffae61b93de5846d7a7", + "table.bool": "arrow_v0.2:000002dee7bb2ff6dce8ddcd5037265ab54ba5f847433b9fc2718b1e7b8c7acad1a44a", + "table.bool.chunked": "arrow_v0.2:000002dee7bb2ff6dce8ddcd5037265ab54ba5f847433b9fc2718b1e7b8c7acad1a44a", + "table.collision.list_lengths_a": "arrow_v0.2:000002295a4cad9a00a66f4c0b07e5b24145f0962f0633ca6b6995ded658cf71196827", + "table.collision.list_lengths_b": "arrow_v0.2:0000026806409bd5df9d075d56b3bbfdb6655d46d1f396e0926052cdb56cfd93edd357", + "table.collision.null_struct": "arrow_v0.2:00000238e6e122820c9902bc72be26e7c542fbc0e8f13ade44142728ac66b0b14f6a70", + "table.collision.struct_of_nulls": "arrow_v0.2:0000024ef1bfba9ecf0d650296647c90a102c4d5699ec82be53e9dab27d62e436c3cd4", + "table.empty.zero_rows": "arrow_v0.2:000002d655beec05f54e13a35a4b07c5cb5b0fae4f406e22261858cb72a6f96e3aa33f", + "table.fixed_size_list.float32_2": "arrow_v0.2:000002a963ce04c642d126d49f7babad7e389aaa077d7a937cdf38ec9ee4590a6929a7", + "table.fixed_size_list.null_slot": "arrow_v0.2:00000223564917b29cabbfdfabd08e1ebb26cad4b4c1fd1af05b62875105fbff10c5fc", + "table.float.float32": "arrow_v0.2:00000287ee09555fc05dfbac367f42c6d50de2d3e7a458444920677ff1bd62ad04da40", + "table.float.float64": "arrow_v0.2:000002d83df46319a27052e5272f72f2025c85cef0b3c8139c36940094ccc91db262e0", + "table.layout.binary": "arrow_v0.2:0000021977fb0b4aac38946863766589aa73c09e47a59bd46d2fe0bf5f8292b6e2b64a", + "table.layout.large_binary": "arrow_v0.2:0000021977fb0b4aac38946863766589aa73c09e47a59bd46d2fe0bf5f8292b6e2b64a", + "table.layout.large_list": "arrow_v0.2:0000027db606bdc3327cd8b8bb45c17ce27f5235498e5f1aca008d18e0e876218405af", + "table.layout.large_string": "arrow_v0.2:0000021aedf65c94c24afaaec8660667c76e8fc296459255d74925110c5d66612f25d9", + "table.layout.list": "arrow_v0.2:0000027db606bdc3327cd8b8bb45c17ce27f5235498e5f1aca008d18e0e876218405af", + "table.layout.string": "arrow_v0.2:0000021aedf65c94c24afaaec8660667c76e8fc296459255d74925110c5d66612f25d9", + "table.nested.list_list": "arrow_v0.2:000002d0d198475a44dc9ab4694adc8348990fca5a0e0365aaf75317a1be70479178a4", + "table.nested.list_struct": "arrow_v0.2:0000029919cc15883d26c0304782ccfeb1ac7a543a9260d9b401a4cfd56ae59f33bac3", + "table.nested.struct": "arrow_v0.2:0000022efc170a4f2578623b5525861024eb528aa2d167cf0006d7694ca971dc3fe370", + "table.nullability.non_nullable": "arrow_v0.2:000002b37da33f664f9dc4acb7851c5a8431abd457651c43aa5c2094093cc97cd33b40", + "table.nullability.nullable": "arrow_v0.2:00000236be7e40075274babcbf4ef0082a8b76252a91c085ca04e41c6ae1a718eb64a7", + "table.nulls.int64": "arrow_v0.2:0000023ba22c1f5257d8cf501925b2db2486687b2180a5b2882c8022ce3d45a56f1efd", + "table.nulls.string": "arrow_v0.2:0000026885780c872492587fc493be66f47184080b87ccbec1327e64fbe52c864d24a4", + "table.order.ab": "arrow_v0.2:000002d29b71de1ff0980cd38422829174ff8a702f4db64ec704e9d41e966bdceaeeaa", + "table.order.ba": "arrow_v0.2:000002d29b71de1ff0980cd38422829174ff8a702f4db64ec704e9d41e966bdceaeeaa", + "table.order.rows_reversed": "arrow_v0.2:000002edcc170ff123dccde69febaae0fd8a87420ac5b4d408047a18cf270223cb7e39", + "table.width.int16": "arrow_v0.2:000002c53e3f002b007a80a664f8fda9e8e11b852fe6d89f6e4883d0e7aba2499f4b7a", + "table.width.int32": "arrow_v0.2:000002ee36b64c06c9157f548f5b89d21935b40cbfa6bd06cdf283561c92523c75a8a5", + "table.width.int64": "arrow_v0.2:00000236be7e40075274babcbf4ef0082a8b76252a91c085ca04e41c6ae1a718eb64a7", + "table.width.int8": "arrow_v0.2:000002f195acdfc1ca1214df9812ef4d5d7c4dbe2ce78d2b4a6e2f6aa6f4f0cd8593d6", + "table.width.uint8": "arrow_v0.2:00000254a73782eb935f0145074c560cd638d035964249d5d2f5b3245c91f9132b7232" } diff --git a/tests/test_hashing/golden/data/datagram.json b/tests/test_hashing/golden/data/datagram.json index 5d6fa4e81..92a75dbe0 100644 --- a/tests/test_hashing/golden/data/datagram.json +++ b/tests/test_hashing/golden/data/datagram.json @@ -3,13 +3,13 @@ "arrow_source.int32.pipeline_hash": "semantic_v0.1:1d303b36f5b4f916fa158b7b557192579008f0984e6127d8832c5731797bae90", "arrow_source.int32.row[{\"k\":\"r1\"}]._tag_record_id::1d303b36f5b4f916fa158b7b557192579008f0984e6127d8832c5731797bae90": "6b49f11e03d55f29b32b06c106ded2da", "arrow_source.int32.row[{\"k\":\"r1\"}]._tag_source_id::1d303b36f5b4f916fa158b7b557192579008f0984e6127d8832c5731797bae90": "golden_arrow", - "arrow_source.int32.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.1:000001a01cd0f32e44a4839ee25caa5b0fb526dd1926b6cd9a43d717ec6a3c92364d29", + "arrow_source.int32.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.2:000002d92f4d29531f412586881fd7891dc20e057fbb83dedc711af0f9bd83b2a56d35", "arrow_source.int32.row[{\"k\":\"r2\"}]._tag_record_id::1d303b36f5b4f916fa158b7b557192579008f0984e6127d8832c5731797bae90": "c4c6f49b43e25f92a3cf173c6ceebb4d", "arrow_source.int32.row[{\"k\":\"r2\"}]._tag_source_id::1d303b36f5b4f916fa158b7b557192579008f0984e6127d8832c5731797bae90": "golden_arrow", - "arrow_source.int32.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.1:000001cbdc2fbc671d4157fa251c1772062b3c0a655fa8a5b21cbdc9b64de0d564c859", + "arrow_source.int32.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.2:00000243c87e6d05d3588edc8eb3eecc6d82902681548386747578da1a3f7953efe35b", "arrow_source.int32.row[{\"k\":\"r3\"}]._tag_record_id::1d303b36f5b4f916fa158b7b557192579008f0984e6127d8832c5731797bae90": "8401f08964385da2a8c873cc2aa3fbe5", "arrow_source.int32.row[{\"k\":\"r3\"}]._tag_source_id::1d303b36f5b4f916fa158b7b557192579008f0984e6127d8832c5731797bae90": "golden_arrow", - "arrow_source.int32.row[{\"k\":\"r3\"}].data_hash": "arrow_v0.1:0000012f75fdca845f528ca916bbfc14e1a3a6dba421d54f016c9e64f045efda579918", + "arrow_source.int32.row[{\"k\":\"r3\"}].data_hash": "arrow_v0.2:000002825e1c06b5514e562a8961019373bb97d951aac4526cd6d2695d96aa2509b326", "arrow_source.int32.system_tag_columns": [ "_tag_source_id::1d303b36f5b4f916fa158b7b557192579008f0984e6127d8832c5731797bae90", "_tag_record_id::1d303b36f5b4f916fa158b7b557192579008f0984e6127d8832c5731797bae90" @@ -18,13 +18,13 @@ "arrow_source.int64.pipeline_hash": "semantic_v0.1:1d303b36f5b4f916fa158b7b557192579008f0984e6127d8832c5731797bae90", "arrow_source.int64.row[{\"k\":\"r1\"}]._tag_record_id::1d303b36f5b4f916fa158b7b557192579008f0984e6127d8832c5731797bae90": "6b49f11e03d55f29b32b06c106ded2da", "arrow_source.int64.row[{\"k\":\"r1\"}]._tag_source_id::1d303b36f5b4f916fa158b7b557192579008f0984e6127d8832c5731797bae90": "golden_arrow", - "arrow_source.int64.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.1:000001c028af694082af2f2d902ddc2b8b21363644f3914428cb5363760cbebe521908", + "arrow_source.int64.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.2:000002784ccdd2b780948d2512ca81004840de8c31b16f446898f4e30ce7dcfbbcd4e8", "arrow_source.int64.row[{\"k\":\"r2\"}]._tag_record_id::1d303b36f5b4f916fa158b7b557192579008f0984e6127d8832c5731797bae90": "c4c6f49b43e25f92a3cf173c6ceebb4d", "arrow_source.int64.row[{\"k\":\"r2\"}]._tag_source_id::1d303b36f5b4f916fa158b7b557192579008f0984e6127d8832c5731797bae90": "golden_arrow", - "arrow_source.int64.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.1:0000017e332b258dbc5fe1f459b9d3dd514afa4dfeaae1e9c816a4f6fd4c545d691f11", + "arrow_source.int64.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.2:0000021d3e14bd65d5ca2633843f2ab29ecdf7df5626e980824695bd21aa24d86cd2fd", "arrow_source.int64.row[{\"k\":\"r3\"}]._tag_record_id::1d303b36f5b4f916fa158b7b557192579008f0984e6127d8832c5731797bae90": "8401f08964385da2a8c873cc2aa3fbe5", "arrow_source.int64.row[{\"k\":\"r3\"}]._tag_source_id::1d303b36f5b4f916fa158b7b557192579008f0984e6127d8832c5731797bae90": "golden_arrow", - "arrow_source.int64.row[{\"k\":\"r3\"}].data_hash": "arrow_v0.1:0000015c299c665cda513e3eda946aab5ae38ed0b637e47f036462361c91804656da01", + "arrow_source.int64.row[{\"k\":\"r3\"}].data_hash": "arrow_v0.2:0000023846c7aef08fc14007f95079971a9c8fc62255fa724097ba05594d335f1cd806", "arrow_source.int64.system_tag_columns": [ "_tag_source_id::1d303b36f5b4f916fa158b7b557192579008f0984e6127d8832c5731797bae90", "_tag_record_id::1d303b36f5b4f916fa158b7b557192579008f0984e6127d8832c5731797bae90" @@ -33,13 +33,13 @@ "arrow_source.int64_nullable.pipeline_hash": "semantic_v0.1:686bec5f94c7ca900d74321adabd05ed62e7562ffcee48ff6c906a4b44174a94", "arrow_source.int64_nullable.row[{\"k\":\"r1\"}]._tag_record_id::686bec5f94c7ca900d74321adabd05ed62e7562ffcee48ff6c906a4b44174a94": "6b49f11e03d55f29b32b06c106ded2da", "arrow_source.int64_nullable.row[{\"k\":\"r1\"}]._tag_source_id::686bec5f94c7ca900d74321adabd05ed62e7562ffcee48ff6c906a4b44174a94": "golden_arrow", - "arrow_source.int64_nullable.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.1:0000012081a40ec7eba71155774352b7e33349f231cd7a1ad4b4b0c1c85897ef6617c5", + "arrow_source.int64_nullable.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.2:000002521ff3d4701e70a5a30a4bb915ab2c1e06d4d4fe7b43e8b542f63c2cb2302c26", "arrow_source.int64_nullable.row[{\"k\":\"r2\"}]._tag_record_id::686bec5f94c7ca900d74321adabd05ed62e7562ffcee48ff6c906a4b44174a94": "c4c6f49b43e25f92a3cf173c6ceebb4d", "arrow_source.int64_nullable.row[{\"k\":\"r2\"}]._tag_source_id::686bec5f94c7ca900d74321adabd05ed62e7562ffcee48ff6c906a4b44174a94": "golden_arrow", - "arrow_source.int64_nullable.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.1:000001ebc7fce25941def04ca935645b0dec2d066ab0f9a902df12fa0dc7f18c84a009", + "arrow_source.int64_nullable.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.2:000002601fc387de4781312ad355ab3e8ca0f1285daeda7f5ba831bc7c9a167cf6daa8", "arrow_source.int64_nullable.row[{\"k\":\"r3\"}]._tag_record_id::686bec5f94c7ca900d74321adabd05ed62e7562ffcee48ff6c906a4b44174a94": "8401f08964385da2a8c873cc2aa3fbe5", "arrow_source.int64_nullable.row[{\"k\":\"r3\"}]._tag_source_id::686bec5f94c7ca900d74321adabd05ed62e7562ffcee48ff6c906a4b44174a94": "golden_arrow", - "arrow_source.int64_nullable.row[{\"k\":\"r3\"}].data_hash": "arrow_v0.1:000001951dc05c8c79ecd4aa8521ae9845c34c8d0b0e343c14a97bdba79157e46bb51e", + "arrow_source.int64_nullable.row[{\"k\":\"r3\"}].data_hash": "arrow_v0.2:000002565243c66b8ae12ded84d33ef6be188fda9250f3e100812d65efacd3144deeef", "arrow_source.int64_nullable.system_tag_columns": [ "_tag_source_id::686bec5f94c7ca900d74321adabd05ed62e7562ffcee48ff6c906a4b44174a94", "_tag_record_id::686bec5f94c7ca900d74321adabd05ed62e7562ffcee48ff6c906a4b44174a94" @@ -48,13 +48,13 @@ "arrow_source.int8.pipeline_hash": "semantic_v0.1:1d303b36f5b4f916fa158b7b557192579008f0984e6127d8832c5731797bae90", "arrow_source.int8.row[{\"k\":\"r1\"}]._tag_record_id::1d303b36f5b4f916fa158b7b557192579008f0984e6127d8832c5731797bae90": "6b49f11e03d55f29b32b06c106ded2da", "arrow_source.int8.row[{\"k\":\"r1\"}]._tag_source_id::1d303b36f5b4f916fa158b7b557192579008f0984e6127d8832c5731797bae90": "golden_arrow", - "arrow_source.int8.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.1:000001760d63148f60206156a8c6224b0f78ded56f5b7982e6211196b4a330a07b311d", + "arrow_source.int8.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.2:000002b17bad7455bf81d473552afdb3ad02a6c6dda0b630b3ba0347c717c419fb489f", "arrow_source.int8.row[{\"k\":\"r2\"}]._tag_record_id::1d303b36f5b4f916fa158b7b557192579008f0984e6127d8832c5731797bae90": "c4c6f49b43e25f92a3cf173c6ceebb4d", "arrow_source.int8.row[{\"k\":\"r2\"}]._tag_source_id::1d303b36f5b4f916fa158b7b557192579008f0984e6127d8832c5731797bae90": "golden_arrow", - "arrow_source.int8.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.1:0000012622e7bf473c8c58db20b86b66fbc8a070cf4e7731339a0b2103f9d68a00f643", + "arrow_source.int8.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.2:000002ff7e5abaafd6df855a3d15271da4f3c4d63e10e3aa190237ff8dc76925762cdd", "arrow_source.int8.row[{\"k\":\"r3\"}]._tag_record_id::1d303b36f5b4f916fa158b7b557192579008f0984e6127d8832c5731797bae90": "8401f08964385da2a8c873cc2aa3fbe5", "arrow_source.int8.row[{\"k\":\"r3\"}]._tag_source_id::1d303b36f5b4f916fa158b7b557192579008f0984e6127d8832c5731797bae90": "golden_arrow", - "arrow_source.int8.row[{\"k\":\"r3\"}].data_hash": "arrow_v0.1:000001c502c37a181f7c3e19bf6486dc69d141b3b35d14c742c854ac39c8f8f9964fee", + "arrow_source.int8.row[{\"k\":\"r3\"}].data_hash": "arrow_v0.2:000002fa70cfa37eaab0c4e54cc5fe972b6001d9e2db1814160b335fb1d78df7e1169b", "arrow_source.int8.system_tag_columns": [ "_tag_source_id::1d303b36f5b4f916fa158b7b557192579008f0984e6127d8832c5731797bae90", "_tag_record_id::1d303b36f5b4f916fa158b7b557192579008f0984e6127d8832c5731797bae90" @@ -63,13 +63,13 @@ "arrow_source.large_string.pipeline_hash": "semantic_v0.1:b58c50c5ff232cdb71eb0272547485074c9784600f5a34218434f7c3de419e8e", "arrow_source.large_string.row[{\"k\":\"r1\"}]._tag_record_id::b58c50c5ff232cdb71eb0272547485074c9784600f5a34218434f7c3de419e8e": "6b49f11e03d55f29b32b06c106ded2da", "arrow_source.large_string.row[{\"k\":\"r1\"}]._tag_source_id::b58c50c5ff232cdb71eb0272547485074c9784600f5a34218434f7c3de419e8e": "golden_arrow", - "arrow_source.large_string.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.1:000001b4407c40d9132743421fec4649ee47bd485dd294d5e5d0c178345df08cecb387", + "arrow_source.large_string.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.2:0000027dc69926b790513bbe98295d34f942eb71ed1a12b4aab259220fd0489bf4fb6c", "arrow_source.large_string.row[{\"k\":\"r2\"}]._tag_record_id::b58c50c5ff232cdb71eb0272547485074c9784600f5a34218434f7c3de419e8e": "c4c6f49b43e25f92a3cf173c6ceebb4d", "arrow_source.large_string.row[{\"k\":\"r2\"}]._tag_source_id::b58c50c5ff232cdb71eb0272547485074c9784600f5a34218434f7c3de419e8e": "golden_arrow", - "arrow_source.large_string.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.1:000001cd356426fd64cb49847fc1e14449cc9893d210b4388f3ccfa1d6052fef5f5ddc", + "arrow_source.large_string.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.2:000002e2668873da695349ec66d49132a1443f22972c7abb108643145169a8e45a7dc1", "arrow_source.large_string.row[{\"k\":\"r3\"}]._tag_record_id::b58c50c5ff232cdb71eb0272547485074c9784600f5a34218434f7c3de419e8e": "8401f08964385da2a8c873cc2aa3fbe5", "arrow_source.large_string.row[{\"k\":\"r3\"}]._tag_source_id::b58c50c5ff232cdb71eb0272547485074c9784600f5a34218434f7c3de419e8e": "golden_arrow", - "arrow_source.large_string.row[{\"k\":\"r3\"}].data_hash": "arrow_v0.1:000001e0c4e88cf1c63c94e17cdcfca663841a9c103ca9ef471c2108002d0b4c690d61", + "arrow_source.large_string.row[{\"k\":\"r3\"}].data_hash": "arrow_v0.2:000002b08595f56b1cd4a87cb0bb7658c9defbae43f2a9c02c37b5b781bb2d69bf0d89", "arrow_source.large_string.system_tag_columns": [ "_tag_source_id::b58c50c5ff232cdb71eb0272547485074c9784600f5a34218434f7c3de419e8e", "_tag_record_id::b58c50c5ff232cdb71eb0272547485074c9784600f5a34218434f7c3de419e8e" @@ -78,13 +78,13 @@ "arrow_source.string.pipeline_hash": "semantic_v0.1:b58c50c5ff232cdb71eb0272547485074c9784600f5a34218434f7c3de419e8e", "arrow_source.string.row[{\"k\":\"r1\"}]._tag_record_id::b58c50c5ff232cdb71eb0272547485074c9784600f5a34218434f7c3de419e8e": "6b49f11e03d55f29b32b06c106ded2da", "arrow_source.string.row[{\"k\":\"r1\"}]._tag_source_id::b58c50c5ff232cdb71eb0272547485074c9784600f5a34218434f7c3de419e8e": "golden_arrow", - "arrow_source.string.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.1:000001b4407c40d9132743421fec4649ee47bd485dd294d5e5d0c178345df08cecb387", + "arrow_source.string.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.2:0000027dc69926b790513bbe98295d34f942eb71ed1a12b4aab259220fd0489bf4fb6c", "arrow_source.string.row[{\"k\":\"r2\"}]._tag_record_id::b58c50c5ff232cdb71eb0272547485074c9784600f5a34218434f7c3de419e8e": "c4c6f49b43e25f92a3cf173c6ceebb4d", "arrow_source.string.row[{\"k\":\"r2\"}]._tag_source_id::b58c50c5ff232cdb71eb0272547485074c9784600f5a34218434f7c3de419e8e": "golden_arrow", - "arrow_source.string.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.1:000001cd356426fd64cb49847fc1e14449cc9893d210b4388f3ccfa1d6052fef5f5ddc", + "arrow_source.string.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.2:000002e2668873da695349ec66d49132a1443f22972c7abb108643145169a8e45a7dc1", "arrow_source.string.row[{\"k\":\"r3\"}]._tag_record_id::b58c50c5ff232cdb71eb0272547485074c9784600f5a34218434f7c3de419e8e": "8401f08964385da2a8c873cc2aa3fbe5", "arrow_source.string.row[{\"k\":\"r3\"}]._tag_source_id::b58c50c5ff232cdb71eb0272547485074c9784600f5a34218434f7c3de419e8e": "golden_arrow", - "arrow_source.string.row[{\"k\":\"r3\"}].data_hash": "arrow_v0.1:000001e0c4e88cf1c63c94e17cdcfca663841a9c103ca9ef471c2108002d0b4c690d61", + "arrow_source.string.row[{\"k\":\"r3\"}].data_hash": "arrow_v0.2:000002b08595f56b1cd4a87cb0bb7658c9defbae43f2a9c02c37b5b781bb2d69bf0d89", "arrow_source.string.system_tag_columns": [ "_tag_source_id::b58c50c5ff232cdb71eb0272547485074c9784600f5a34218434f7c3de419e8e", "_tag_record_id::b58c50c5ff232cdb71eb0272547485074c9784600f5a34218434f7c3de419e8e" @@ -93,10 +93,10 @@ "dataclass.pipeline_hash": "semantic_v0.1:c16022c03e2749f6a4f1d7ac2b255e87d1ccce6895c5328f2489103558b2b14c", "dataclass.row[{\"k\":\"r1\"}]._tag_record_id::c16022c03e2749f6a4f1d7ac2b255e87d1ccce6895c5328f2489103558b2b14c": "26db93a5eab45b0485c2699148b0600d", "dataclass.row[{\"k\":\"r1\"}]._tag_source_id::c16022c03e2749f6a4f1d7ac2b255e87d1ccce6895c5328f2489103558b2b14c": "golden_dataclass", - "dataclass.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.1:0000015e306dd402af59e45e6f5a9113e71e691f96006cb8cf63246504800071050f55", + "dataclass.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.2:0000027b15d8db313a05d7a3f0af1cfa788fd730bb3d47eacb201de61f4c24b9a2f0bd", "dataclass.row[{\"k\":\"r2\"}]._tag_record_id::c16022c03e2749f6a4f1d7ac2b255e87d1ccce6895c5328f2489103558b2b14c": "71d5c6ce2f7b5b5b9dfdc7a61a7df366", "dataclass.row[{\"k\":\"r2\"}]._tag_source_id::c16022c03e2749f6a4f1d7ac2b255e87d1ccce6895c5328f2489103558b2b14c": "golden_dataclass", - "dataclass.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.1:000001b088edbc3232e6d836e6036a75d5c0a0d83864166d21a16ad113608e21afe445", + "dataclass.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.2:000002abd8873e34ca7ee3f967b31d5fc3e56013d151495ae59d9a53f856f81deacc85", "dataclass.system_tag_columns": [ "_tag_source_id::c16022c03e2749f6a4f1d7ac2b255e87d1ccce6895c5328f2489103558b2b14c", "_tag_record_id::c16022c03e2749f6a4f1d7ac2b255e87d1ccce6895c5328f2489103558b2b14c" @@ -105,10 +105,10 @@ "dataframe.pipeline_hash": "semantic_v0.1:38f74931045e006f752450c2d26dbf9096c5bff65fbe952591dec83b04296a14", "dataframe.row[{\"k\":\"r1\"}]._tag_record_id::38f74931045e006f752450c2d26dbf9096c5bff65fbe952591dec83b04296a14": "3b7c66de1eb65a05a236c53ce3a3e292", "dataframe.row[{\"k\":\"r1\"}]._tag_source_id::38f74931045e006f752450c2d26dbf9096c5bff65fbe952591dec83b04296a14": "golden_dataframe", - "dataframe.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.1:00000108d551603b5a5df12a0c1ac3f7d5b3caa1108525058b0e450c02ae93e94e358f", + "dataframe.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.2:00000229c5394be73a9c86795235a5f408835ded8167d2b05e00a114ab89760aebbee2", "dataframe.row[{\"k\":\"r2\"}]._tag_record_id::38f74931045e006f752450c2d26dbf9096c5bff65fbe952591dec83b04296a14": "a075d0a40203532b92170507912ff6e6", "dataframe.row[{\"k\":\"r2\"}]._tag_source_id::38f74931045e006f752450c2d26dbf9096c5bff65fbe952591dec83b04296a14": "golden_dataframe", - "dataframe.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.1:00000134950bb185ada91e996752020cd29a1dff584382c019be51e4f0ef15560e868b", + "dataframe.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.2:0000024dc4bbb7a657d7ba9af692ed778b268a1ed8a3ebf4c7243362a5c6072af4a451", "dataframe.system_tag_columns": [ "_tag_source_id::38f74931045e006f752450c2d26dbf9096c5bff65fbe952591dec83b04296a14", "_tag_record_id::38f74931045e006f752450c2d26dbf9096c5bff65fbe952591dec83b04296a14" @@ -117,10 +117,10 @@ "list_path.pipeline_hash": "semantic_v0.1:3d12eb0f29844205335f9ffd7843afc13c308dece8bae64b9f94d980c690e020", "list_path.row[{\"k\":\"r1\"}]._tag_record_id::3d12eb0f29844205335f9ffd7843afc13c308dece8bae64b9f94d980c690e020": "059c77251e845bafb5a3839bf547a2f5", "list_path.row[{\"k\":\"r1\"}]._tag_source_id::3d12eb0f29844205335f9ffd7843afc13c308dece8bae64b9f94d980c690e020": "golden_list_path", - "list_path.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.1:00000135fc611ef9ad245d4906a4d4d8da286dafc50f184443aaecefdf4fbd63e65433", + "list_path.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.2:000002cc2ece0ae8ac648486e63890a8d4b354e1276bd85ca6575ce7ec8d8085928dd3", "list_path.row[{\"k\":\"r2\"}]._tag_record_id::3d12eb0f29844205335f9ffd7843afc13c308dece8bae64b9f94d980c690e020": "35b02f96d67b500db2ae69b3138302b8", "list_path.row[{\"k\":\"r2\"}]._tag_source_id::3d12eb0f29844205335f9ffd7843afc13c308dece8bae64b9f94d980c690e020": "golden_list_path", - "list_path.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.1:0000018efe66b052bc46996b623c89fbc73f8e2ba005fa27d2388989fbfdbd053624f8", + "list_path.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.2:000002cdaddd554ca8557405f6b0dc07b8a1b706b116ac336d86d6da402fa99ddea137", "list_path.system_tag_columns": [ "_tag_source_id::3d12eb0f29844205335f9ffd7843afc13c308dece8bae64b9f94d980c690e020", "_tag_record_id::3d12eb0f29844205335f9ffd7843afc13c308dece8bae64b9f94d980c690e020" @@ -129,10 +129,10 @@ "ndarray.pipeline_hash": "semantic_v0.1:be37e32a759adc87f88657caf69a48238722f512a9c80892ab023739e4ebfda5", "ndarray.row[{\"k\":\"r1\"}]._tag_record_id::be37e32a759adc87f88657caf69a48238722f512a9c80892ab023739e4ebfda5": "9caca45075165a87bf8320fef620e949", "ndarray.row[{\"k\":\"r1\"}]._tag_source_id::be37e32a759adc87f88657caf69a48238722f512a9c80892ab023739e4ebfda5": "golden_ndarray", - "ndarray.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.1:000001c80cacf9821f15fd18dfc7f6649268fcffefeaad1449c81cdfa716a8738f8d6e", + "ndarray.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.2:000002e79fea6f79a5a7e617db9cc2a749f627de86be7128c084616e397c5a04d28d49", "ndarray.row[{\"k\":\"r2\"}]._tag_record_id::be37e32a759adc87f88657caf69a48238722f512a9c80892ab023739e4ebfda5": "992e0c0ae0e957df944a73440a6f6451", "ndarray.row[{\"k\":\"r2\"}]._tag_source_id::be37e32a759adc87f88657caf69a48238722f512a9c80892ab023739e4ebfda5": "golden_ndarray", - "ndarray.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.1:000001ba0bc856108ff6198b30e7d1f5fc78bdbca5f1790c108b0a35f1d735b5e1b896", + "ndarray.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.2:000002922f3a9172b4d736387286cf2dd4d1121d85238403d9c8c350785a03809cbb3b", "ndarray.system_tag_columns": [ "_tag_source_id::be37e32a759adc87f88657caf69a48238722f512a9c80892ab023739e4ebfda5", "_tag_record_id::be37e32a759adc87f88657caf69a48238722f512a9c80892ab023739e4ebfda5" @@ -141,10 +141,10 @@ "nested.pipeline_hash": "semantic_v0.1:938fabe22ed56031d43337723a8c8478ae6010fdb3594e9dc8c14d7d98b6df09", "nested.row[{\"k\":\"r1\"}]._tag_record_id::938fabe22ed56031d43337723a8c8478ae6010fdb3594e9dc8c14d7d98b6df09": "22fdf98f207f580e80aa48ea45e7d81b", "nested.row[{\"k\":\"r1\"}]._tag_source_id::938fabe22ed56031d43337723a8c8478ae6010fdb3594e9dc8c14d7d98b6df09": "golden_nested", - "nested.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.1:000001c31188a44c589ce3daf646ca787cf70383531e48ea6864e540be884a15d180bf", + "nested.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.2:00000261f569412bb12da248eaef81f93ee80ab2c93978f737fa604cc11309276f6f40", "nested.row[{\"k\":\"r2\"}]._tag_record_id::938fabe22ed56031d43337723a8c8478ae6010fdb3594e9dc8c14d7d98b6df09": "20a95fcca8955b96b501e71105cfcd7c", "nested.row[{\"k\":\"r2\"}]._tag_source_id::938fabe22ed56031d43337723a8c8478ae6010fdb3594e9dc8c14d7d98b6df09": "golden_nested", - "nested.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.1:000001a7658c283bcd7a1c6d933a5dba77b96c6457e8384a5a8d58a151a6acec619469", + "nested.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.2:000002f4312647b9ca61759ae335c7b0639d667709ae8f797a7564e31bdf9a454bc5a1", "nested.system_tag_columns": [ "_tag_source_id::938fabe22ed56031d43337723a8c8478ae6010fdb3594e9dc8c14d7d98b6df09", "_tag_record_id::938fabe22ed56031d43337723a8c8478ae6010fdb3594e9dc8c14d7d98b6df09" @@ -153,10 +153,10 @@ "optional.pipeline_hash": "semantic_v0.1:2b2731c4ac73505b88c0146d11eb7c77d6e6c701f08da676440e28d9f6d845a0", "optional.row[{\"k\":\"r1\"}]._tag_record_id::2b2731c4ac73505b88c0146d11eb7c77d6e6c701f08da676440e28d9f6d845a0": "4e41e4e151be5a88b2f275893f537118", "optional.row[{\"k\":\"r1\"}]._tag_source_id::2b2731c4ac73505b88c0146d11eb7c77d6e6c701f08da676440e28d9f6d845a0": "golden_optional", - "optional.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.1:000001f910e5b3279111cf4734db9a4669b90b16aa1cecba678ef8364d7b6b28615487", + "optional.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.2:000002f774b1bc717e2ddf7bacc9707ccb64b05ef3935a68fcb064980cb7134e672c1e", "optional.row[{\"k\":\"r2\"}]._tag_record_id::2b2731c4ac73505b88c0146d11eb7c77d6e6c701f08da676440e28d9f6d845a0": "5b3fa9a72962574690a8e28a8f04224a", "optional.row[{\"k\":\"r2\"}]._tag_source_id::2b2731c4ac73505b88c0146d11eb7c77d6e6c701f08da676440e28d9f6d845a0": "golden_optional", - "optional.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.1:0000010d55a89a30404439bfd8b5fb204fda1e0aafeaefc0e9c625b524221d032f0bca", + "optional.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.2:0000027714fb655c044c9c124dfcb5b6ee5895e2ff00b3edd051990817bdb01e6bea34", "optional.system_tag_columns": [ "_tag_source_id::2b2731c4ac73505b88c0146d11eb7c77d6e6c701f08da676440e28d9f6d845a0", "_tag_record_id::2b2731c4ac73505b88c0146d11eb7c77d6e6c701f08da676440e28d9f6d845a0" @@ -165,10 +165,10 @@ "path.pipeline_hash": "semantic_v0.1:bf2528f33e2c72825ecca9151c2302d77df0e6e7544e955e56101468f9b2bd09", "path.row[{\"k\":\"r1\"}]._tag_record_id::bf2528f33e2c72825ecca9151c2302d77df0e6e7544e955e56101468f9b2bd09": "8995b9653892518d975b74e051ddff38", "path.row[{\"k\":\"r1\"}]._tag_source_id::bf2528f33e2c72825ecca9151c2302d77df0e6e7544e955e56101468f9b2bd09": "golden_path", - "path.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.1:00000135bf11f7d6830036c99e7445af70c2d668909f9000362d8ede899e435e18a8fc", + "path.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.2:000002a230080e1f03f22543bcd61b9956710b80b51a14fc6b74c0bdc994e3e82b4cd1", "path.row[{\"k\":\"r2\"}]._tag_record_id::bf2528f33e2c72825ecca9151c2302d77df0e6e7544e955e56101468f9b2bd09": "e2adbc7a08965c6db075bab848287b62", "path.row[{\"k\":\"r2\"}]._tag_source_id::bf2528f33e2c72825ecca9151c2302d77df0e6e7544e955e56101468f9b2bd09": "golden_path", - "path.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.1:0000015b4f8156dc62851f96532b141710b02eb0d2d9e558578002c8daa84669c00339", + "path.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.2:00000234b6166fa805040e49bada1a6fad4b8c406173b1df7d3aa99db1de764196de36", "path.system_tag_columns": [ "_tag_source_id::bf2528f33e2c72825ecca9151c2302d77df0e6e7544e955e56101468f9b2bd09", "_tag_record_id::bf2528f33e2c72825ecca9151c2302d77df0e6e7544e955e56101468f9b2bd09" @@ -177,10 +177,10 @@ "pydantic.pipeline_hash": "semantic_v0.1:aeb2c7a55a50242b1c5916876b4004e77108a32ac2b97bd974184ad4091ed967", "pydantic.row[{\"k\":\"r1\"}]._tag_record_id::aeb2c7a55a50242b1c5916876b4004e77108a32ac2b97bd974184ad4091ed967": "cb9bbf1a138d5c848e86f99bab7c0acb", "pydantic.row[{\"k\":\"r1\"}]._tag_source_id::aeb2c7a55a50242b1c5916876b4004e77108a32ac2b97bd974184ad4091ed967": "golden_pydantic", - "pydantic.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.1:0000012280b233c56771fa4f901eff64090067273d718ac8d36b881b78d79d4128ab15", + "pydantic.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.2:000002459c1d7ed12f91d7f292eafe1514160775ff885d92151dcf0da2ecf256dbbf1a", "pydantic.row[{\"k\":\"r2\"}]._tag_record_id::aeb2c7a55a50242b1c5916876b4004e77108a32ac2b97bd974184ad4091ed967": "10c1dbaba95a5c5fa7c14661231224eb", "pydantic.row[{\"k\":\"r2\"}]._tag_source_id::aeb2c7a55a50242b1c5916876b4004e77108a32ac2b97bd974184ad4091ed967": "golden_pydantic", - "pydantic.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.1:00000138d77ff36d7b2f3c326a128b325d8219e31f55be9264a21abf51638993059c7e", + "pydantic.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.2:000002ce57007b0730b902132be534f05b27828f588bea325de4d54876721c6c5c7463", "pydantic.system_tag_columns": [ "_tag_source_id::aeb2c7a55a50242b1c5916876b4004e77108a32ac2b97bd974184ad4091ed967", "_tag_record_id::aeb2c7a55a50242b1c5916876b4004e77108a32ac2b97bd974184ad4091ed967" @@ -189,10 +189,10 @@ "scalars.pipeline_hash": "semantic_v0.1:1fdf145196d8e5dd7b38887cc8798325147df1f5c6f2af80952488a76f9aec07", "scalars.row[{\"k\":\"r1\"}]._tag_record_id::1fdf145196d8e5dd7b38887cc8798325147df1f5c6f2af80952488a76f9aec07": "dd4b595c8e6554f7ae4ca854e411dfa8", "scalars.row[{\"k\":\"r1\"}]._tag_source_id::1fdf145196d8e5dd7b38887cc8798325147df1f5c6f2af80952488a76f9aec07": "golden_scalars", - "scalars.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.1:00000162a09a0c2dd7d8e7c09355e4c4d3b1992e26b0d6a53b706da57ea9117183eb7c", + "scalars.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.2:000002c8161def16592dc28c2dfa2cff5084c6d1bf397667bf401453b85f1636b29896", "scalars.row[{\"k\":\"r2\"}]._tag_record_id::1fdf145196d8e5dd7b38887cc8798325147df1f5c6f2af80952488a76f9aec07": "ffc8945f5a4053d79c69860fd298d6e0", "scalars.row[{\"k\":\"r2\"}]._tag_source_id::1fdf145196d8e5dd7b38887cc8798325147df1f5c6f2af80952488a76f9aec07": "golden_scalars", - "scalars.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.1:000001d49094978c1867f7432b53ac342c9bf81f9e5cb636a8a7e8c5090027eb9b6f70", + "scalars.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.2:0000020528b8069a7f49fd5905ecc79c4ff589d0ec471e0274bffdd5a51ae7756655f2", "scalars.system_tag_columns": [ "_tag_source_id::1fdf145196d8e5dd7b38887cc8798325147df1f5c6f2af80952488a76f9aec07", "_tag_record_id::1fdf145196d8e5dd7b38887cc8798325147df1f5c6f2af80952488a76f9aec07" @@ -201,10 +201,10 @@ "set_uuid.pipeline_hash": "semantic_v0.1:060b8a8e22b65b3cf6cde2ee5fc3ae4eedd840b377a2e071ec5e99cbb8a913ae", "set_uuid.row[{\"k\":\"r1\"}]._tag_record_id::060b8a8e22b65b3cf6cde2ee5fc3ae4eedd840b377a2e071ec5e99cbb8a913ae": "2d3990a5b84f5cf0a5c846c448a25267", "set_uuid.row[{\"k\":\"r1\"}]._tag_source_id::060b8a8e22b65b3cf6cde2ee5fc3ae4eedd840b377a2e071ec5e99cbb8a913ae": "golden_set_uuid", - "set_uuid.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.1:0000011eb69965d474d220990b2895cc181cf2938ebbc5cc8ae96f5eead708129c4198", + "set_uuid.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.2:000002648eddf1e15a4ed2e1ef619eb2d072950ba44f29ad23c935f8241e845cea0e81", "set_uuid.row[{\"k\":\"r2\"}]._tag_record_id::060b8a8e22b65b3cf6cde2ee5fc3ae4eedd840b377a2e071ec5e99cbb8a913ae": "6ce8daa1eb235f2cb041e30468ff396a", "set_uuid.row[{\"k\":\"r2\"}]._tag_source_id::060b8a8e22b65b3cf6cde2ee5fc3ae4eedd840b377a2e071ec5e99cbb8a913ae": "golden_set_uuid", - "set_uuid.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.1:000001fc38bc3b27de6a06637c17cf1c24119d2d13782bc2a78080596dfae657ff5d0b", + "set_uuid.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.2:000002ec023f0196ddaaa81bf359b83f34e17b39554ede462359bb9f030977c4a005f8", "set_uuid.system_tag_columns": [ "_tag_source_id::060b8a8e22b65b3cf6cde2ee5fc3ae4eedd840b377a2e071ec5e99cbb8a913ae", "_tag_record_id::060b8a8e22b65b3cf6cde2ee5fc3ae4eedd840b377a2e071ec5e99cbb8a913ae" @@ -213,10 +213,10 @@ "uuid.pipeline_hash": "semantic_v0.1:33b9e5f9945a83a66c7e77d79a188ec3dc9bbb2c64820c88ab47ffc361296581", "uuid.row[{\"k\":\"r1\"}]._tag_record_id::33b9e5f9945a83a66c7e77d79a188ec3dc9bbb2c64820c88ab47ffc361296581": "c17eccb33f355b1e9d6878f55b081026", "uuid.row[{\"k\":\"r1\"}]._tag_source_id::33b9e5f9945a83a66c7e77d79a188ec3dc9bbb2c64820c88ab47ffc361296581": "golden_uuid", - "uuid.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.1:000001e05de561e03537d1d2a793c43b38a9a7eae680f223a0396113a4137917faa8ca", + "uuid.row[{\"k\":\"r1\"}].data_hash": "arrow_v0.2:00000228d36dcdde1b6e7ff9ea8439b7d31c857f2df240443250ddec8cab124a896222", "uuid.row[{\"k\":\"r2\"}]._tag_record_id::33b9e5f9945a83a66c7e77d79a188ec3dc9bbb2c64820c88ab47ffc361296581": "adfcc16e72e55d30884f9feaf02caef2", "uuid.row[{\"k\":\"r2\"}]._tag_source_id::33b9e5f9945a83a66c7e77d79a188ec3dc9bbb2c64820c88ab47ffc361296581": "golden_uuid", - "uuid.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.1:0000014c189ef879ddf91cfe2ee92808d90ebf835de50dcb6a84c468545d5fe61c4ff8", + "uuid.row[{\"k\":\"r2\"}].data_hash": "arrow_v0.2:00000236a95b5f9e34143f912dcf0b7da9f7531c7fca0120c93237b053897f3e11ca0f", "uuid.system_tag_columns": [ "_tag_source_id::33b9e5f9945a83a66c7e77d79a188ec3dc9bbb2c64820c88ab47ffc361296581", "_tag_record_id::33b9e5f9945a83a66c7e77d79a188ec3dc9bbb2c64820c88ab47ffc361296581" diff --git a/tests/test_hashing/golden/data/pipeline.json b/tests/test_hashing/golden/data/pipeline.json index d5b5257ce..5b36b2577 100644 --- a/tests/test_hashing/golden/data/pipeline.json +++ b/tests/test_hashing/golden/data/pipeline.json @@ -3,10 +3,10 @@ "batch.pipeline_hash": "semantic_v0.1:60c337773086050f28d338eb45ebda97ab278426b7b491c996b951c31d71a078", "batch.row[{\"subject\":[\"s1\",\"s2\"]}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "0843c16c912c561eb8920aa92fc21c85", "batch.row[{\"subject\":[\"s1\",\"s2\"]}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "5f85d65fa9a90d1a4266f0b684b4a2c2a040cf7835bd14dfd37cde20fc635efb", - "batch.row[{\"subject\":[\"s1\",\"s2\"]}].data_hash": "arrow_v0.1:0000010819e767e277e136c495f98e9bc5da5d252b5a2b83cdbdec869994e5ba033508", + "batch.row[{\"subject\":[\"s1\",\"s2\"]}].data_hash": "arrow_v0.2:0000020b638d3ccc763f2a88e2ade9b759bb8641c55073185d4839047a3fc91276dc7d", "batch.row[{\"subject\":[\"s3\"]}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "47519d093cfc58878d21a35188b10c78", "batch.row[{\"subject\":[\"s3\"]}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "e223d8ec1e6450dae88d4eb042479e5fa6104e6fd30b15add076db936b290367", - "batch.row[{\"subject\":[\"s3\"]}].data_hash": "arrow_v0.1:0000014957b9ee25e0ea51c695f25e5fc85b74946b3b9c4e391f88bb8d4a4734694a4e", + "batch.row[{\"subject\":[\"s3\"]}].data_hash": "arrow_v0.2:000002928bbf5d31fadf74ea3118af85c0d2273f77d53274f492e63a955520e4fac208", "batch.system_tag_columns": [ "_tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956", "_tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956" @@ -15,33 +15,33 @@ "column_selection.drop_data.pipeline_hash": "semantic_v0.1:d4de6c0b8a694ef32b07cba1fcc682e22f0b73c27dd7f3a4f6b398188bd5255c", "column_selection.drop_data.row[{\"subject\":\"s1\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "e8e02084627f5e1aad83839e421538e8", "column_selection.drop_data.row[{\"subject\":\"s1\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "subjects", - "column_selection.drop_data.row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.1:000001ee27bd12d66af74db02ecec14119254c263796aa8551e3d3249dc0025f22be17", + "column_selection.drop_data.row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.2:000002c734791b7452bb2affc60031491e7541bb53fb971cf3ef8b9af919eb0bf1f56e", "column_selection.drop_data.row[{\"subject\":\"s2\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "144bb37f3c295e16919d770641bb8c65", "column_selection.drop_data.row[{\"subject\":\"s2\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "subjects", - "column_selection.drop_data.row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.1:00000133552dc58a31ff558a0bf035031e2228ec7981bb85c39ab22ae0aaad86efdb5f", + "column_selection.drop_data.row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.2:00000252e4f32a133cd6a8813f816e08b6bde1cdb467baa95d026fc41c5672b6b98110", "column_selection.drop_data.row[{\"subject\":\"s3\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "d9391c2dcea750c496e0895ad301b30d", "column_selection.drop_data.row[{\"subject\":\"s3\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "subjects", - "column_selection.drop_data.row[{\"subject\":\"s3\"}].data_hash": "arrow_v0.1:000001966c5e58c797ab3329316dffd36c3dcf09a3e0a400fc4511114e1e99f10939ea", + "column_selection.drop_data.row[{\"subject\":\"s3\"}].data_hash": "arrow_v0.2:0000025f33a2f5bf3e956edfb7ca628b6871f52b7bbf788d5a3774ddffe300d7d4aff5", "column_selection.drop_data.system_tag_columns": [ "_tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956", "_tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956" ], "column_selection.drop_tag.content_hash": "semantic_v0.1:adc2bbbc40dfe518e580cc332638c7d7cdad2d170a392649ea72c38a97d21db0", "column_selection.drop_tag.pipeline_hash": "semantic_v0.1:ddfedb2b384d0d6d5e5236f60aaedc02182660a9518c3e6d93f2535cd02c7c30", - "column_selection.drop_tag.row[{\"cohort\":\"a\"}].data_hash": "arrow_v0.1:00000133552dc58a31ff558a0bf035031e2228ec7981bb85c39ab22ae0aaad86efdb5f", - "column_selection.drop_tag.row[{\"cohort\":\"b\"}].data_hash": "arrow_v0.1:000001966c5e58c797ab3329316dffd36c3dcf09a3e0a400fc4511114e1e99f10939ea", + "column_selection.drop_tag.row[{\"cohort\":\"a\"}].data_hash": "arrow_v0.2:00000252e4f32a133cd6a8813f816e08b6bde1cdb467baa95d026fc41c5672b6b98110", + "column_selection.drop_tag.row[{\"cohort\":\"b\"}].data_hash": "arrow_v0.2:0000025f33a2f5bf3e956edfb7ca628b6871f52b7bbf788d5a3774ddffe300d7d4aff5", "column_selection.drop_tag.system_tag_columns": [], "column_selection.select_data.content_hash": "semantic_v0.1:4d5b0e7c758f28ad95439e0910a4f0fcc8d6916953ad4516bd33f4bdeebf45e0", "column_selection.select_data.pipeline_hash": "semantic_v0.1:390c5e93d5a1cc266a2f62461a5c7e706a801d5cb9d3ab045176540b47db04ad", "column_selection.select_data.row[{\"subject\":\"s1\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "e8e02084627f5e1aad83839e421538e8", "column_selection.select_data.row[{\"subject\":\"s1\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "subjects", - "column_selection.select_data.row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.1:0000017cb817a7c2cfd6c716987b09fea2cc10609b9c4b2c854116bdc7a3ede1e625ea", + "column_selection.select_data.row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.2:00000281f4d4a4e7742a2cae9d733cb30453503892bf97d5d5695d5b491df60c9af327", "column_selection.select_data.row[{\"subject\":\"s2\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "144bb37f3c295e16919d770641bb8c65", "column_selection.select_data.row[{\"subject\":\"s2\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "subjects", - "column_selection.select_data.row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.1:00000124d0ecb4bf9d093b60b0947eaff30ac930b9b3b429f06d1b8848185b7e0fb1f8", + "column_selection.select_data.row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.2:0000022420dc43cfa6c4d3682f044a90a1a222554a5d2fd5bc7ebf015f830d3e18869f", "column_selection.select_data.row[{\"subject\":\"s3\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "d9391c2dcea750c496e0895ad301b30d", "column_selection.select_data.row[{\"subject\":\"s3\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "subjects", - "column_selection.select_data.row[{\"subject\":\"s3\"}].data_hash": "arrow_v0.1:000001810e6ae31eebcd4f2bfadeb2472fce6a5d52d063b10cf286713e2ecd2fdec977", + "column_selection.select_data.row[{\"subject\":\"s3\"}].data_hash": "arrow_v0.2:000002805d579510905b591a2d3748d388531edbef8cec4da1cef5f18abe1a77b21238", "column_selection.select_data.system_tag_columns": [ "_tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956", "_tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956" @@ -50,10 +50,10 @@ "column_selection.select_tag.pipeline_hash": "semantic_v0.1:7ee4cf0256ecc2c9150b67a01a71d0a6a1368f452b872bede59b60ae6fcd8073", "column_selection.select_tag.row[{\"cohort\":\"a\"}]._tag_record_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4": "37d45a37e6255cd78bb62c4c9d047ca5", "column_selection.select_tag.row[{\"cohort\":\"a\"}]._tag_source_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4": "cohorts", - "column_selection.select_tag.row[{\"cohort\":\"a\"}].data_hash": "arrow_v0.1:00000133552dc58a31ff558a0bf035031e2228ec7981bb85c39ab22ae0aaad86efdb5f", + "column_selection.select_tag.row[{\"cohort\":\"a\"}].data_hash": "arrow_v0.2:00000252e4f32a133cd6a8813f816e08b6bde1cdb467baa95d026fc41c5672b6b98110", "column_selection.select_tag.row[{\"cohort\":\"b\"}]._tag_record_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4": "d0cc03b6594d51788075066f5c81deb0", "column_selection.select_tag.row[{\"cohort\":\"b\"}]._tag_source_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4": "cohorts", - "column_selection.select_tag.row[{\"cohort\":\"b\"}].data_hash": "arrow_v0.1:000001966c5e58c797ab3329316dffd36c3dcf09a3e0a400fc4511114e1e99f10939ea", + "column_selection.select_tag.row[{\"cohort\":\"b\"}].data_hash": "arrow_v0.2:0000025f33a2f5bf3e956edfb7ca628b6871f52b7bbf788d5a3774ddffe300d7d4aff5", "column_selection.select_tag.system_tag_columns": [ "_tag_source_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4", "_tag_record_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4" @@ -62,10 +62,10 @@ "group_by.pipeline_hash": "semantic_v0.1:5ab1cb5d5ce9b641d19045bae0d94bbb9e15703bebfe37ec57fd13b141d2a0f0", "group_by.row[{\"cohort\":\"a\"}]._tag_record_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4": "7d212ee5c358519c9e4729f4a0898aad", "group_by.row[{\"cohort\":\"a\"}]._tag_source_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4": "bab61585da6894512c5947710f0c0d4bb46e3f0fcd511a088a4c76780f1ca5ae", - "group_by.row[{\"cohort\":\"a\"}].data_hash": "arrow_v0.1:0000018910e185c56ec7890dcba74a0553f06e355191c27a97a423dbbc32bea2441149", + "group_by.row[{\"cohort\":\"a\"}].data_hash": "arrow_v0.2:0000022b0f148a444e75daddc97429afe196d896fc41169dfe6fe9408438fde809c3ac", "group_by.row[{\"cohort\":\"b\"}]._tag_record_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4": "b7f2f8f7412a54fdb03cd96b6b0b1643", "group_by.row[{\"cohort\":\"b\"}]._tag_source_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4": "90ffe0efe9214e2770ce632ae3146e4bb33e9d28568b55b7dd67f07862836010", - "group_by.row[{\"cohort\":\"b\"}].data_hash": "arrow_v0.1:000001ba6adaea3c56bf0be427961819e0491e678c6dc6486999ab5025d9074ec9bbd4", + "group_by.row[{\"cohort\":\"b\"}].data_hash": "arrow_v0.2:0000028b04ecd69f72beab1905bf77d8f6300e916e6a6ce3a3db61b8dd15d641a213f0", "group_by.system_tag_columns": [ "_tag_source_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4", "_tag_record_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4" @@ -74,10 +74,10 @@ "index_pick.index.pipeline_hash": "semantic_v0.1:375b6916dd1d0c46e5553b471809ef66120dc092a19192a1f7b052e2481299c1", "index_pick.index.row[{\"subject\":\"s1\"}]._tag_record_id::2ae63653af6a57129ca7c6edfa8aa02c0511f59a8ee4acb678e223bb6302e22a": "dd65b531e3d957a0ae3f2fba11e021d5", "index_pick.index.row[{\"subject\":\"s1\"}]._tag_source_id::2ae63653af6a57129ca7c6edfa8aa02c0511f59a8ee4acb678e223bb6302e22a": "lists", - "index_pick.index.row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.1:00000189f1942fbc40737ff78f6428f7be73a5dc531b1426c075ae3d5bc790b4ef8fcd", + "index_pick.index.row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.2:0000025f5739a1fc427e583ba103ab5a7c1a5b868aadb554a722a1c0bd1d2081e4a34f", "index_pick.index.row[{\"subject\":\"s2\"}]._tag_record_id::2ae63653af6a57129ca7c6edfa8aa02c0511f59a8ee4acb678e223bb6302e22a": "00c12388e8c4522893b029b39a8ec7aa", "index_pick.index.row[{\"subject\":\"s2\"}]._tag_source_id::2ae63653af6a57129ca7c6edfa8aa02c0511f59a8ee4acb678e223bb6302e22a": "lists", - "index_pick.index.row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.1:00000164b528c22b7183231910091256b6cb1d63bef2ded631aa330114a2fef997ed8c", + "index_pick.index.row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.2:000002d9f79b11b0d6928476289f10dc1ffce27ef6af09affa82b079560b96435f62eb", "index_pick.index.system_tag_columns": [ "_tag_source_id::2ae63653af6a57129ca7c6edfa8aa02c0511f59a8ee4acb678e223bb6302e22a", "_tag_record_id::2ae63653af6a57129ca7c6edfa8aa02c0511f59a8ee4acb678e223bb6302e22a" @@ -86,10 +86,10 @@ "index_pick.pick.pipeline_hash": "semantic_v0.1:054e63991c65cb241b7ad88d011c4ef14ab3062b8f08642c0c39ddc7466d90a5", "index_pick.pick.row[{\"subject\":\"s1\"}]._tag_record_id::2ae63653af6a57129ca7c6edfa8aa02c0511f59a8ee4acb678e223bb6302e22a": "dd65b531e3d957a0ae3f2fba11e021d5", "index_pick.pick.row[{\"subject\":\"s1\"}]._tag_source_id::2ae63653af6a57129ca7c6edfa8aa02c0511f59a8ee4acb678e223bb6302e22a": "lists", - "index_pick.pick.row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.1:000001380f07142b3132f93c7dfd5ef23532dbc35e83a7ddf3bd28ffbdc3405eb14045", + "index_pick.pick.row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.2:000002d3aa3df31c715ed9534811e3b3ded71a017ea8132b501c69ed3ad5b78411d423", "index_pick.pick.row[{\"subject\":\"s2\"}]._tag_record_id::2ae63653af6a57129ca7c6edfa8aa02c0511f59a8ee4acb678e223bb6302e22a": "00c12388e8c4522893b029b39a8ec7aa", "index_pick.pick.row[{\"subject\":\"s2\"}]._tag_source_id::2ae63653af6a57129ca7c6edfa8aa02c0511f59a8ee4acb678e223bb6302e22a": "lists", - "index_pick.pick.row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.1:00000157b70714f9f88f91649e6d135d91610267c66aef90f7552a03354fe3a8ffbdb9", + "index_pick.pick.row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.2:00000218fa0f5bdf561a92ce32cecef15970f979890cf7c44fd9b8a0c8766e7c730dc2", "index_pick.pick.system_tag_columns": [ "_tag_source_id::2ae63653af6a57129ca7c6edfa8aa02c0511f59a8ee4acb678e223bb6302e22a", "_tag_record_id::2ae63653af6a57129ca7c6edfa8aa02c0511f59a8ee4acb678e223bb6302e22a" @@ -100,12 +100,12 @@ "job.node[adjust].row[{\"subject\":\"s1\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956:1": "e8e02084627f5e1aad83839e421538e8", "job.node[adjust].row[{\"subject\":\"s1\"}]._tag_source_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec:0": "scans", "job.node[adjust].row[{\"subject\":\"s1\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956:1": "subjects", - "job.node[adjust].row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.1:0000014a3871996b0dc2563b8f3979740245bde22aac53376bba5592849cebc035b864", + "job.node[adjust].row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.2:000002df3a95f3b9eddaf078096dee1388e410637a1a4ce3dd8028296b1a274c1292f2", "job.node[adjust].row[{\"subject\":\"s2\"}]._tag_record_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec:0": "42d239bc04015d1aaded030d9e02850f", "job.node[adjust].row[{\"subject\":\"s2\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956:1": "144bb37f3c295e16919d770641bb8c65", "job.node[adjust].row[{\"subject\":\"s2\"}]._tag_source_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec:0": "scans", "job.node[adjust].row[{\"subject\":\"s2\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956:1": "subjects", - "job.node[adjust].row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.1:0000014c738e65c53a408dfd0e017a32dd96504847f5ce08b13213b96168262a97e264", + "job.node[adjust].row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.2:0000023565098d1a88ecd75dbc8ddc4900dc8304e066dc0c889114a2a41e2fe495fc30", "job.node[adjust].system_tag_columns": [ "_tag_source_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec:0", "_tag_record_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec:0", @@ -118,12 +118,12 @@ "job.node[joined].row[{\"subject\":\"s1\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956:1": "e8e02084627f5e1aad83839e421538e8", "job.node[joined].row[{\"subject\":\"s1\"}]._tag_source_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec:0": "scans", "job.node[joined].row[{\"subject\":\"s1\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956:1": "subjects", - "job.node[joined].row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.1:00000130ab270ba209296b0933d7df77e7712b75b0e1cc5f9344775e30a29c9f4cc882", + "job.node[joined].row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.2:000002c945ad08138f51298940c60333852ef5191eb16077d96105386d44840c0ccfdf", "job.node[joined].row[{\"subject\":\"s2\"}]._tag_record_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec:0": "42d239bc04015d1aaded030d9e02850f", "job.node[joined].row[{\"subject\":\"s2\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956:1": "144bb37f3c295e16919d770641bb8c65", "job.node[joined].row[{\"subject\":\"s2\"}]._tag_source_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec:0": "scans", "job.node[joined].row[{\"subject\":\"s2\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956:1": "subjects", - "job.node[joined].row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.1:00000186afa42ffc71c45174cd8b17e5a938cdb4d20d29a99d653af922ed86c4cf0e4d", + "job.node[joined].row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.2:0000026b8369e60be10e43dd72a0be4e8d93a992f827cd6baab33909331dcdaeea8b64", "job.node[joined].system_tag_columns": [ "_tag_source_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec:0", "_tag_record_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec:0", @@ -134,10 +134,10 @@ "job.node[scans].pipeline_hash": "semantic_v0.1:2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec", "job.node[scans].row[{\"subject\":\"s1\"}]._tag_record_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec": "45c5e6aa24d25f21a15c4d1016b857d3", "job.node[scans].row[{\"subject\":\"s1\"}]._tag_source_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec": "scans", - "job.node[scans].row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.1:000001ac375e92d49c253f99d8cdbb55487634eeb95fec0d05ef5e9515b59ac74b78d1", + "job.node[scans].row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.2:000002ef2b37a1979bd734849c241456c50e96644db0ba5a7b25ecb84cee278e6686f2", "job.node[scans].row[{\"subject\":\"s2\"}]._tag_record_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec": "42d239bc04015d1aaded030d9e02850f", "job.node[scans].row[{\"subject\":\"s2\"}]._tag_source_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec": "scans", - "job.node[scans].row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.1:000001f7fa6c7bb04912cca05f5a979e9bdecab132fd134dba8f18bf379836955c857b", + "job.node[scans].row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.2:00000210b14e22145ce1d5f71e190ade4ca03ba523ad98d8749ea7beef3c15dd749c20", "job.node[scans].system_tag_columns": [ "_tag_source_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec", "_tag_record_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec" @@ -146,24 +146,24 @@ "job.node[subjects].pipeline_hash": "semantic_v0.1:f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956", "job.node[subjects].row[{\"subject\":\"s1\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "e8e02084627f5e1aad83839e421538e8", "job.node[subjects].row[{\"subject\":\"s1\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "subjects", - "job.node[subjects].row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.1:00000137de028aa69769a5d29c9003e923af9a0b081ff4067b2eb08af4ef943e4939b6", + "job.node[subjects].row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.2:000002d1c5b49de38ef6efc552c2a47b8ab59dd267f8eb55d91814edee99b1b52acc65", "job.node[subjects].row[{\"subject\":\"s2\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "144bb37f3c295e16919d770641bb8c65", "job.node[subjects].row[{\"subject\":\"s2\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "subjects", - "job.node[subjects].row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.1:000001584cc7c090776151311c6768ba98c02c0235250c6685493d3207138ef8d8b484", + "job.node[subjects].row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.2:000002338ce8710f5ecb156a72c8446651adb7b0c21635d0aa92d298f09da56faed0f8", "job.node[subjects].row[{\"subject\":\"s3\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "d9391c2dcea750c496e0895ad301b30d", "job.node[subjects].row[{\"subject\":\"s3\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "subjects", - "job.node[subjects].row[{\"subject\":\"s3\"}].data_hash": "arrow_v0.1:000001562e4c70b60df10ed1243d0d9129beff719b867cb16e18d654f375d361e64c51", + "job.node[subjects].row[{\"subject\":\"s3\"}].data_hash": "arrow_v0.2:0000023401f1ce40c8cfdd6e4f343d7ffdcd51c1f0521f44cf6030e27e1ca0da154616", "job.node[subjects].system_tag_columns": [ "_tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956", "_tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956" ], "job.table[golden_job/_result/golden_adjust/semantic_v0.1:5cd978b399b203b46d0d8a7d395dcbb592d63aaeeefbd3da97f59149202d48f3/v0/python.function.v0/rdb_v1].input_data_hashes": [ - "arrow_v0.1:00000130ab270ba209296b0933d7df77e7712b75b0e1cc5f9344775e30a29c9f4cc882", - "arrow_v0.1:00000186afa42ffc71c45174cd8b17e5a938cdb4d20d29a99d653af922ed86c4cf0e4d" + "arrow_v0.2:0000026b8369e60be10e43dd72a0be4e8d93a992f827cd6baab33909331dcdaeea8b64", + "arrow_v0.2:000002c945ad08138f51298940c60333852ef5191eb16077d96105386d44840c0ccfdf" ], "job.table[golden_job/golden_adjust/semantic_v0.1:5cd978b399b203b46d0d8a7d395dcbb592d63aaeeefbd3da97f59149202d48f3/v0/python.function.v0/schema:semantic_v0.1:b61a42486c29af296f5c2f1d8129f20b61e023d492747e397a50756cb153aff8/pdb_v1].record_ids": [ - "arrow_v0.1:00000137cf61ccacb0c63080e74a9bc00a90d5c415a8f89f295323068742ef3088c7b9", - "arrow_v0.1:000001cb65d6834af1be3e6bc180d9ffa57241031847c38afe6decfb454e9cf6004e30" + "arrow_v0.2:000002118147ab4ef2d7e048cc8bb2deba63027eefaeee0a65d4d2d4873ba3a60a9739", + "arrow_v0.2:000002c5733660533bd081d5c22a0138c066d7e5869cb0acad1b30edf3b39a9b701e01" ], "job.table_paths": [ "golden_job/_result/golden_adjust/semantic_v0.1:5cd978b399b203b46d0d8a7d395dcbb592d63aaeeefbd3da97f59149202d48f3/v0/python.function.v0/rdb_v1", @@ -175,12 +175,12 @@ "join_pod.join.row[{\"subject\":\"s1\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956:1": "e8e02084627f5e1aad83839e421538e8", "join_pod.join.row[{\"subject\":\"s1\"}]._tag_source_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec:0": "scans", "join_pod.join.row[{\"subject\":\"s1\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956:1": "subjects", - "join_pod.join.row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.1:00000130ab270ba209296b0933d7df77e7712b75b0e1cc5f9344775e30a29c9f4cc882", + "join_pod.join.row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.2:000002c945ad08138f51298940c60333852ef5191eb16077d96105386d44840c0ccfdf", "join_pod.join.row[{\"subject\":\"s2\"}]._tag_record_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec:0": "42d239bc04015d1aaded030d9e02850f", "join_pod.join.row[{\"subject\":\"s2\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956:1": "144bb37f3c295e16919d770641bb8c65", "join_pod.join.row[{\"subject\":\"s2\"}]._tag_source_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec:0": "scans", "join_pod.join.row[{\"subject\":\"s2\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956:1": "subjects", - "join_pod.join.row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.1:00000186afa42ffc71c45174cd8b17e5a938cdb4d20d29a99d653af922ed86c4cf0e4d", + "join_pod.join.row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.2:0000026b8369e60be10e43dd72a0be4e8d93a992f827cd6baab33909331dcdaeea8b64", "join_pod.join.system_tag_columns": [ "_tag_source_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec:0", "_tag_record_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec:0", @@ -193,12 +193,12 @@ "join_pod.pod.row[{\"subject\":\"s1\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956:1": "e8e02084627f5e1aad83839e421538e8", "join_pod.pod.row[{\"subject\":\"s1\"}]._tag_source_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec:0": "scans", "join_pod.pod.row[{\"subject\":\"s1\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956:1": "subjects", - "join_pod.pod.row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.1:0000014a3871996b0dc2563b8f3979740245bde22aac53376bba5592849cebc035b864", + "join_pod.pod.row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.2:000002df3a95f3b9eddaf078096dee1388e410637a1a4ce3dd8028296b1a274c1292f2", "join_pod.pod.row[{\"subject\":\"s2\"}]._tag_record_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec:0": "42d239bc04015d1aaded030d9e02850f", "join_pod.pod.row[{\"subject\":\"s2\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956:1": "144bb37f3c295e16919d770641bb8c65", "join_pod.pod.row[{\"subject\":\"s2\"}]._tag_source_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec:0": "scans", "join_pod.pod.row[{\"subject\":\"s2\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956:1": "subjects", - "join_pod.pod.row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.1:0000014c738e65c53a408dfd0e017a32dd96504847f5ce08b13213b96168262a97e264", + "join_pod.pod.row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.2:0000023565098d1a88ecd75dbc8ddc4900dc8304e066dc0c889114a2a41e2fe495fc30", "join_pod.pod.system_tag_columns": [ "_tag_source_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec:0", "_tag_record_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec:0", @@ -209,13 +209,13 @@ "map_data.pipeline_hash": "semantic_v0.1:968f2ba68b526db0d6a5f0ad80f746f5cb57d2fe24588d9c374cf17d229d83e1", "map_data.row[{\"subject\":\"s1\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "e8e02084627f5e1aad83839e421538e8", "map_data.row[{\"subject\":\"s1\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "subjects", - "map_data.row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.1:0000012b54015317016ea374e1b6f4b14a7ba6454a4a320bdd1a66d56081d9b4e1e327", + "map_data.row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.2:0000022ad1363967c26697bb148943f0b193f748f21da43e4ba5f3763197a04bf53289", "map_data.row[{\"subject\":\"s2\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "144bb37f3c295e16919d770641bb8c65", "map_data.row[{\"subject\":\"s2\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "subjects", - "map_data.row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.1:000001bf869237358aa3c91a042d4c6171c7eff1b6089ec739096d20313b4c32f8edfc", + "map_data.row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.2:000002a61fcaac02c3fd261c22c4fab3c89182aef1912fb4a19ae1c01af9cf69b4b6b6", "map_data.row[{\"subject\":\"s3\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "d9391c2dcea750c496e0895ad301b30d", "map_data.row[{\"subject\":\"s3\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "subjects", - "map_data.row[{\"subject\":\"s3\"}].data_hash": "arrow_v0.1:000001e87c392dee12edd21fe5080442060fcd8d9a50627e70b90b25865ff7361bf893", + "map_data.row[{\"subject\":\"s3\"}].data_hash": "arrow_v0.2:000002ef21c37efbb54c93fff65d16c5222fed1364ebf8ab9a49758b064f3edd83f3a1", "map_data.system_tag_columns": [ "_tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956", "_tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956" @@ -226,12 +226,12 @@ "merge_join.row[{\"subject\":\"s1\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956::eba863719d1f29e059571a7c159a0aeaeed1858760612e5b471011b3f7843d7e:1": "e8e02084627f5e1aad83839e421538e8", "merge_join.row[{\"subject\":\"s1\"}]._tag_source_id::ceea3411fcbfdce34ac8306ac3b5a8d234839abc6addf6bcd2fe28f1c974081b::ceea3411fcbfdce34ac8306ac3b5a8d234839abc6addf6bcd2fe28f1c974081b:0": "reweigh", "merge_join.row[{\"subject\":\"s1\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956::eba863719d1f29e059571a7c159a0aeaeed1858760612e5b471011b3f7843d7e:1": "subjects", - "merge_join.row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.1:000001ec09b4a15ec5e06d5d5be991cb2865b48ead20dac75c66d16a4d8472de7cb4e3", + "merge_join.row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.2:000002654a49c9ec7661bbbb88c7695033d0082d20abafa31fcfa9112e39834de7cc5d", "merge_join.row[{\"subject\":\"s3\"}]._tag_record_id::ceea3411fcbfdce34ac8306ac3b5a8d234839abc6addf6bcd2fe28f1c974081b::ceea3411fcbfdce34ac8306ac3b5a8d234839abc6addf6bcd2fe28f1c974081b:0": "e7fcc4debbb656f5b186d21f53897b41", "merge_join.row[{\"subject\":\"s3\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956::eba863719d1f29e059571a7c159a0aeaeed1858760612e5b471011b3f7843d7e:1": "d9391c2dcea750c496e0895ad301b30d", "merge_join.row[{\"subject\":\"s3\"}]._tag_source_id::ceea3411fcbfdce34ac8306ac3b5a8d234839abc6addf6bcd2fe28f1c974081b::ceea3411fcbfdce34ac8306ac3b5a8d234839abc6addf6bcd2fe28f1c974081b:0": "reweigh", "merge_join.row[{\"subject\":\"s3\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956::eba863719d1f29e059571a7c159a0aeaeed1858760612e5b471011b3f7843d7e:1": "subjects", - "merge_join.row[{\"subject\":\"s3\"}].data_hash": "arrow_v0.1:0000013aeb0e14ea94b020f971939368a6b7948b5f9713b0f2cf2c1ddbe43a57ce6e53", + "merge_join.row[{\"subject\":\"s3\"}].data_hash": "arrow_v0.2:000002b78038f0da00d8b6e29ff96251d4d12632f5c978e5fa58dc118e53e582f90c96", "merge_join.system_tag_columns": [ "_tag_source_id::ceea3411fcbfdce34ac8306ac3b5a8d234839abc6addf6bcd2fe28f1c974081b::ceea3411fcbfdce34ac8306ac3b5a8d234839abc6addf6bcd2fe28f1c974081b:0", "_tag_record_id::ceea3411fcbfdce34ac8306ac3b5a8d234839abc6addf6bcd2fe28f1c974081b::ceea3411fcbfdce34ac8306ac3b5a8d234839abc6addf6bcd2fe28f1c974081b:0", @@ -242,13 +242,13 @@ "pod.pipeline_hash": "semantic_v0.1:f6e833fa312653a09ad67ca8f8d447630badf1ad1cfe6f298d822fbc231243d2", "pod.row[{\"subject\":\"s1\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "e8e02084627f5e1aad83839e421538e8", "pod.row[{\"subject\":\"s1\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "subjects", - "pod.row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.1:00000155d85a85806e3601d4eea978d885d32a45e3fcf6d018d37cbbcf140d077fe21a", + "pod.row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.2:0000024da91c65d3ebacee016a9080299e31887b2aabe961f00c475443c6930693e22a", "pod.row[{\"subject\":\"s2\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "144bb37f3c295e16919d770641bb8c65", "pod.row[{\"subject\":\"s2\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "subjects", - "pod.row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.1:0000017db75343362e61ae6e3a5dcce30fa838e5be6c908cf1c18629a6905bedc10587", + "pod.row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.2:0000021afa3e9774d8a28522ff03f4981ecb0b290e3cefc0f602fe11edf4d3430d103e", "pod.row[{\"subject\":\"s3\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "d9391c2dcea750c496e0895ad301b30d", "pod.row[{\"subject\":\"s3\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "subjects", - "pod.row[{\"subject\":\"s3\"}].data_hash": "arrow_v0.1:000001a3b408003eb5481eec125ba73ba0190c72fee13f147953cc4e850afbafc36c1c", + "pod.row[{\"subject\":\"s3\"}].data_hash": "arrow_v0.2:000002c2a1324b1334c4c49e87c175c8ad33ea00a325fea75e441edb4108c611994179", "pod.system_tag_columns": [ "_tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956", "_tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956" @@ -257,10 +257,10 @@ "polars_filter.pipeline_hash": "semantic_v0.1:c3495e32f466d812eefc8a1af06ffb3a94e4de51a0bee266c03fc1f1539f4fb3", "polars_filter.row[{\"cohort\":\"a\",\"subject\":\"s1\"}]._tag_record_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4": "b47fb712e72a55f4920c3fc7db2f75f5", "polars_filter.row[{\"cohort\":\"a\",\"subject\":\"s1\"}]._tag_source_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4": "cohorts", - "polars_filter.row[{\"cohort\":\"a\",\"subject\":\"s1\"}].data_hash": "arrow_v0.1:000001ee27bd12d66af74db02ecec14119254c263796aa8551e3d3249dc0025f22be17", + "polars_filter.row[{\"cohort\":\"a\",\"subject\":\"s1\"}].data_hash": "arrow_v0.2:000002c734791b7452bb2affc60031491e7541bb53fb971cf3ef8b9af919eb0bf1f56e", "polars_filter.row[{\"cohort\":\"a\",\"subject\":\"s2\"}]._tag_record_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4": "37d45a37e6255cd78bb62c4c9d047ca5", "polars_filter.row[{\"cohort\":\"a\",\"subject\":\"s2\"}]._tag_source_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4": "cohorts", - "polars_filter.row[{\"cohort\":\"a\",\"subject\":\"s2\"}].data_hash": "arrow_v0.1:00000133552dc58a31ff558a0bf035031e2228ec7981bb85c39ab22ae0aaad86efdb5f", + "polars_filter.row[{\"cohort\":\"a\",\"subject\":\"s2\"}].data_hash": "arrow_v0.2:00000252e4f32a133cd6a8813f816e08b6bde1cdb467baa95d026fc41c5672b6b98110", "polars_filter.system_tag_columns": [ "_tag_source_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4", "_tag_record_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4" @@ -269,10 +269,10 @@ "polars_filter_expr.expr.pipeline_hash": "semantic_v0.1:365702de4f28d94a1d119f12fd83b6b15b09ad7ed9aaf3e412ec9ca2c8392fcc", "polars_filter_expr.expr.row[{\"cohort\":\"a\",\"subject\":\"s1\"}]._tag_record_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4": "b47fb712e72a55f4920c3fc7db2f75f5", "polars_filter_expr.expr.row[{\"cohort\":\"a\",\"subject\":\"s1\"}]._tag_source_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4": "cohorts", - "polars_filter_expr.expr.row[{\"cohort\":\"a\",\"subject\":\"s1\"}].data_hash": "arrow_v0.1:000001ee27bd12d66af74db02ecec14119254c263796aa8551e3d3249dc0025f22be17", + "polars_filter_expr.expr.row[{\"cohort\":\"a\",\"subject\":\"s1\"}].data_hash": "arrow_v0.2:000002c734791b7452bb2affc60031491e7541bb53fb971cf3ef8b9af919eb0bf1f56e", "polars_filter_expr.expr.row[{\"cohort\":\"a\",\"subject\":\"s2\"}]._tag_record_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4": "37d45a37e6255cd78bb62c4c9d047ca5", "polars_filter_expr.expr.row[{\"cohort\":\"a\",\"subject\":\"s2\"}]._tag_source_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4": "cohorts", - "polars_filter_expr.expr.row[{\"cohort\":\"a\",\"subject\":\"s2\"}].data_hash": "arrow_v0.1:00000133552dc58a31ff558a0bf035031e2228ec7981bb85c39ab22ae0aaad86efdb5f", + "polars_filter_expr.expr.row[{\"cohort\":\"a\",\"subject\":\"s2\"}].data_hash": "arrow_v0.2:00000252e4f32a133cd6a8813f816e08b6bde1cdb467baa95d026fc41c5672b6b98110", "polars_filter_expr.expr.system_tag_columns": [ "_tag_source_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4", "_tag_record_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4" @@ -281,10 +281,10 @@ "polars_filter_expr.mixed.pipeline_hash": "semantic_v0.1:145c27c659a765361d8fbe7b8831928621e8aac954dbda45bbcbd0251f36e27c", "polars_filter_expr.mixed.row[{\"cohort\":\"a\",\"subject\":\"s1\"}]._tag_record_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4": "b47fb712e72a55f4920c3fc7db2f75f5", "polars_filter_expr.mixed.row[{\"cohort\":\"a\",\"subject\":\"s1\"}]._tag_source_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4": "cohorts", - "polars_filter_expr.mixed.row[{\"cohort\":\"a\",\"subject\":\"s1\"}].data_hash": "arrow_v0.1:000001ee27bd12d66af74db02ecec14119254c263796aa8551e3d3249dc0025f22be17", + "polars_filter_expr.mixed.row[{\"cohort\":\"a\",\"subject\":\"s1\"}].data_hash": "arrow_v0.2:000002c734791b7452bb2affc60031491e7541bb53fb971cf3ef8b9af919eb0bf1f56e", "polars_filter_expr.mixed.row[{\"cohort\":\"a\",\"subject\":\"s2\"}]._tag_record_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4": "37d45a37e6255cd78bb62c4c9d047ca5", "polars_filter_expr.mixed.row[{\"cohort\":\"a\",\"subject\":\"s2\"}]._tag_source_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4": "cohorts", - "polars_filter_expr.mixed.row[{\"cohort\":\"a\",\"subject\":\"s2\"}].data_hash": "arrow_v0.1:00000133552dc58a31ff558a0bf035031e2228ec7981bb85c39ab22ae0aaad86efdb5f", + "polars_filter_expr.mixed.row[{\"cohort\":\"a\",\"subject\":\"s2\"}].data_hash": "arrow_v0.2:00000252e4f32a133cd6a8813f816e08b6bde1cdb467baa95d026fc41c5672b6b98110", "polars_filter_expr.mixed.system_tag_columns": [ "_tag_source_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4", "_tag_record_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4" @@ -295,7 +295,7 @@ "polars_filter_expr_job.node[adjust].row[{\"subject\":\"s2\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956::53252e3314d5992ea18c9a7886b9673a191d0933c83da9b1d8926e09ab8962fd:1": "144bb37f3c295e16919d770641bb8c65", "polars_filter_expr_job.node[adjust].row[{\"subject\":\"s2\"}]._tag_source_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec:0": "scans", "polars_filter_expr_job.node[adjust].row[{\"subject\":\"s2\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956::53252e3314d5992ea18c9a7886b9673a191d0933c83da9b1d8926e09ab8962fd:1": "subjects", - "polars_filter_expr_job.node[adjust].row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.1:0000014c738e65c53a408dfd0e017a32dd96504847f5ce08b13213b96168262a97e264", + "polars_filter_expr_job.node[adjust].row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.2:0000023565098d1a88ecd75dbc8ddc4900dc8304e066dc0c889114a2a41e2fe495fc30", "polars_filter_expr_job.node[adjust].system_tag_columns": [ "_tag_source_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec:0", "_tag_record_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec:0", @@ -308,7 +308,7 @@ "polars_filter_expr_job.node[joined].row[{\"subject\":\"s2\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956::53252e3314d5992ea18c9a7886b9673a191d0933c83da9b1d8926e09ab8962fd:1": "144bb37f3c295e16919d770641bb8c65", "polars_filter_expr_job.node[joined].row[{\"subject\":\"s2\"}]._tag_source_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec:0": "scans", "polars_filter_expr_job.node[joined].row[{\"subject\":\"s2\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956::53252e3314d5992ea18c9a7886b9673a191d0933c83da9b1d8926e09ab8962fd:1": "subjects", - "polars_filter_expr_job.node[joined].row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.1:00000186afa42ffc71c45174cd8b17e5a938cdb4d20d29a99d653af922ed86c4cf0e4d", + "polars_filter_expr_job.node[joined].row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.2:0000026b8369e60be10e43dd72a0be4e8d93a992f827cd6baab33909331dcdaeea8b64", "polars_filter_expr_job.node[joined].system_tag_columns": [ "_tag_source_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec:0", "_tag_record_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec:0", @@ -319,10 +319,10 @@ "polars_filter_expr_job.node[older].pipeline_hash": "semantic_v0.1:53252e3314d5992ea18c9a7886b9673a191d0933c83da9b1d8926e09ab8962fd", "polars_filter_expr_job.node[older].row[{\"subject\":\"s2\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "144bb37f3c295e16919d770641bb8c65", "polars_filter_expr_job.node[older].row[{\"subject\":\"s2\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "subjects", - "polars_filter_expr_job.node[older].row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.1:000001584cc7c090776151311c6768ba98c02c0235250c6685493d3207138ef8d8b484", + "polars_filter_expr_job.node[older].row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.2:000002338ce8710f5ecb156a72c8446651adb7b0c21635d0aa92d298f09da56faed0f8", "polars_filter_expr_job.node[older].row[{\"subject\":\"s3\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "d9391c2dcea750c496e0895ad301b30d", "polars_filter_expr_job.node[older].row[{\"subject\":\"s3\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "subjects", - "polars_filter_expr_job.node[older].row[{\"subject\":\"s3\"}].data_hash": "arrow_v0.1:000001562e4c70b60df10ed1243d0d9129beff719b867cb16e18d654f375d361e64c51", + "polars_filter_expr_job.node[older].row[{\"subject\":\"s3\"}].data_hash": "arrow_v0.2:0000023401f1ce40c8cfdd6e4f343d7ffdcd51c1f0521f44cf6030e27e1ca0da154616", "polars_filter_expr_job.node[older].system_tag_columns": [ "_tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956", "_tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956" @@ -331,10 +331,10 @@ "polars_filter_expr_job.node[scans].pipeline_hash": "semantic_v0.1:2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec", "polars_filter_expr_job.node[scans].row[{\"subject\":\"s1\"}]._tag_record_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec": "45c5e6aa24d25f21a15c4d1016b857d3", "polars_filter_expr_job.node[scans].row[{\"subject\":\"s1\"}]._tag_source_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec": "scans", - "polars_filter_expr_job.node[scans].row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.1:000001ac375e92d49c253f99d8cdbb55487634eeb95fec0d05ef5e9515b59ac74b78d1", + "polars_filter_expr_job.node[scans].row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.2:000002ef2b37a1979bd734849c241456c50e96644db0ba5a7b25ecb84cee278e6686f2", "polars_filter_expr_job.node[scans].row[{\"subject\":\"s2\"}]._tag_record_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec": "42d239bc04015d1aaded030d9e02850f", "polars_filter_expr_job.node[scans].row[{\"subject\":\"s2\"}]._tag_source_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec": "scans", - "polars_filter_expr_job.node[scans].row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.1:000001f7fa6c7bb04912cca05f5a979e9bdecab132fd134dba8f18bf379836955c857b", + "polars_filter_expr_job.node[scans].row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.2:00000210b14e22145ce1d5f71e190ade4ca03ba523ad98d8749ea7beef3c15dd749c20", "polars_filter_expr_job.node[scans].system_tag_columns": [ "_tag_source_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec", "_tag_record_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec" @@ -343,22 +343,22 @@ "polars_filter_expr_job.node[subjects].pipeline_hash": "semantic_v0.1:f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956", "polars_filter_expr_job.node[subjects].row[{\"subject\":\"s1\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "e8e02084627f5e1aad83839e421538e8", "polars_filter_expr_job.node[subjects].row[{\"subject\":\"s1\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "subjects", - "polars_filter_expr_job.node[subjects].row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.1:00000137de028aa69769a5d29c9003e923af9a0b081ff4067b2eb08af4ef943e4939b6", + "polars_filter_expr_job.node[subjects].row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.2:000002d1c5b49de38ef6efc552c2a47b8ab59dd267f8eb55d91814edee99b1b52acc65", "polars_filter_expr_job.node[subjects].row[{\"subject\":\"s2\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "144bb37f3c295e16919d770641bb8c65", "polars_filter_expr_job.node[subjects].row[{\"subject\":\"s2\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "subjects", - "polars_filter_expr_job.node[subjects].row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.1:000001584cc7c090776151311c6768ba98c02c0235250c6685493d3207138ef8d8b484", + "polars_filter_expr_job.node[subjects].row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.2:000002338ce8710f5ecb156a72c8446651adb7b0c21635d0aa92d298f09da56faed0f8", "polars_filter_expr_job.node[subjects].row[{\"subject\":\"s3\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "d9391c2dcea750c496e0895ad301b30d", "polars_filter_expr_job.node[subjects].row[{\"subject\":\"s3\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "subjects", - "polars_filter_expr_job.node[subjects].row[{\"subject\":\"s3\"}].data_hash": "arrow_v0.1:000001562e4c70b60df10ed1243d0d9129beff719b867cb16e18d654f375d361e64c51", + "polars_filter_expr_job.node[subjects].row[{\"subject\":\"s3\"}].data_hash": "arrow_v0.2:0000023401f1ce40c8cfdd6e4f343d7ffdcd51c1f0521f44cf6030e27e1ca0da154616", "polars_filter_expr_job.node[subjects].system_tag_columns": [ "_tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956", "_tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956" ], "polars_filter_expr_job.table[golden_filter_job/_result/golden_adjust/semantic_v0.1:5cd978b399b203b46d0d8a7d395dcbb592d63aaeeefbd3da97f59149202d48f3/v0/python.function.v0/rdb_v1].input_data_hashes": [ - "arrow_v0.1:00000186afa42ffc71c45174cd8b17e5a938cdb4d20d29a99d653af922ed86c4cf0e4d" + "arrow_v0.2:0000026b8369e60be10e43dd72a0be4e8d93a992f827cd6baab33909331dcdaeea8b64" ], "polars_filter_expr_job.table[golden_filter_job/golden_adjust/semantic_v0.1:5cd978b399b203b46d0d8a7d395dcbb592d63aaeeefbd3da97f59149202d48f3/v0/python.function.v0/schema:semantic_v0.1:1bf083d50625083700e69f699b90f9f01c9c1e93d56e1a56feb4ad1506005e93/pdb_v1].record_ids": [ - "arrow_v0.1:000001ea2218fa4e752319925270feb64dd91210fd24cc94e1fe0faed39c8345774c8c" + "arrow_v0.2:0000022ad2e7cf689d57072ff3ed2627943d06158df2dfff7c59a978eefcfc4dd4300c" ], "polars_filter_expr_job.table_paths": [ "golden_filter_job/_result/golden_adjust/semantic_v0.1:5cd978b399b203b46d0d8a7d395dcbb592d63aaeeefbd3da97f59149202d48f3/v0/python.function.v0/rdb_v1", @@ -368,10 +368,10 @@ "semi_join.pipeline_hash": "semantic_v0.1:822ca17d6b35230006b80ecd32d046192ed22378cd995365b9b0ac6e49f7224a", "semi_join.row[{\"subject\":\"s1\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "e8e02084627f5e1aad83839e421538e8", "semi_join.row[{\"subject\":\"s1\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "subjects", - "semi_join.row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.1:00000137de028aa69769a5d29c9003e923af9a0b081ff4067b2eb08af4ef943e4939b6", + "semi_join.row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.2:000002d1c5b49de38ef6efc552c2a47b8ab59dd267f8eb55d91814edee99b1b52acc65", "semi_join.row[{\"subject\":\"s2\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "144bb37f3c295e16919d770641bb8c65", "semi_join.row[{\"subject\":\"s2\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "subjects", - "semi_join.row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.1:000001584cc7c090776151311c6768ba98c02c0235250c6685493d3207138ef8d8b484", + "semi_join.row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.2:000002338ce8710f5ecb156a72c8446651adb7b0c21635d0aa92d298f09da56faed0f8", "semi_join.system_tag_columns": [ "_tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956", "_tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956" @@ -380,13 +380,13 @@ "sources.cohorts.pipeline_hash": "semantic_v0.1:13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4", "sources.cohorts.row[{\"cohort\":\"a\",\"subject\":\"s1\"}]._tag_record_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4": "b47fb712e72a55f4920c3fc7db2f75f5", "sources.cohorts.row[{\"cohort\":\"a\",\"subject\":\"s1\"}]._tag_source_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4": "cohorts", - "sources.cohorts.row[{\"cohort\":\"a\",\"subject\":\"s1\"}].data_hash": "arrow_v0.1:000001ee27bd12d66af74db02ecec14119254c263796aa8551e3d3249dc0025f22be17", + "sources.cohorts.row[{\"cohort\":\"a\",\"subject\":\"s1\"}].data_hash": "arrow_v0.2:000002c734791b7452bb2affc60031491e7541bb53fb971cf3ef8b9af919eb0bf1f56e", "sources.cohorts.row[{\"cohort\":\"a\",\"subject\":\"s2\"}]._tag_record_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4": "37d45a37e6255cd78bb62c4c9d047ca5", "sources.cohorts.row[{\"cohort\":\"a\",\"subject\":\"s2\"}]._tag_source_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4": "cohorts", - "sources.cohorts.row[{\"cohort\":\"a\",\"subject\":\"s2\"}].data_hash": "arrow_v0.1:00000133552dc58a31ff558a0bf035031e2228ec7981bb85c39ab22ae0aaad86efdb5f", + "sources.cohorts.row[{\"cohort\":\"a\",\"subject\":\"s2\"}].data_hash": "arrow_v0.2:00000252e4f32a133cd6a8813f816e08b6bde1cdb467baa95d026fc41c5672b6b98110", "sources.cohorts.row[{\"cohort\":\"b\",\"subject\":\"s3\"}]._tag_record_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4": "d0cc03b6594d51788075066f5c81deb0", "sources.cohorts.row[{\"cohort\":\"b\",\"subject\":\"s3\"}]._tag_source_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4": "cohorts", - "sources.cohorts.row[{\"cohort\":\"b\",\"subject\":\"s3\"}].data_hash": "arrow_v0.1:000001966c5e58c797ab3329316dffd36c3dcf09a3e0a400fc4511114e1e99f10939ea", + "sources.cohorts.row[{\"cohort\":\"b\",\"subject\":\"s3\"}].data_hash": "arrow_v0.2:0000025f33a2f5bf3e956edfb7ca628b6871f52b7bbf788d5a3774ddffe300d7d4aff5", "sources.cohorts.system_tag_columns": [ "_tag_source_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4", "_tag_record_id::13d8efe7433022d8f328bbd9ea2cf66397ef44167932c3b08fb43153bc1be1d4" @@ -395,10 +395,10 @@ "sources.scans.pipeline_hash": "semantic_v0.1:2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec", "sources.scans.row[{\"subject\":\"s1\"}]._tag_record_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec": "45c5e6aa24d25f21a15c4d1016b857d3", "sources.scans.row[{\"subject\":\"s1\"}]._tag_source_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec": "scans", - "sources.scans.row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.1:000001ac375e92d49c253f99d8cdbb55487634eeb95fec0d05ef5e9515b59ac74b78d1", + "sources.scans.row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.2:000002ef2b37a1979bd734849c241456c50e96644db0ba5a7b25ecb84cee278e6686f2", "sources.scans.row[{\"subject\":\"s2\"}]._tag_record_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec": "42d239bc04015d1aaded030d9e02850f", "sources.scans.row[{\"subject\":\"s2\"}]._tag_source_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec": "scans", - "sources.scans.row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.1:000001f7fa6c7bb04912cca05f5a979e9bdecab132fd134dba8f18bf379836955c857b", + "sources.scans.row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.2:00000210b14e22145ce1d5f71e190ade4ca03ba523ad98d8749ea7beef3c15dd749c20", "sources.scans.system_tag_columns": [ "_tag_source_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec", "_tag_record_id::2f6020c8bd0f792547e5060b0fcbf2f2629eaaf14ffb4be67a050bd30bb414ec" @@ -407,13 +407,13 @@ "sources.subjects.pipeline_hash": "semantic_v0.1:f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956", "sources.subjects.row[{\"subject\":\"s1\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "e8e02084627f5e1aad83839e421538e8", "sources.subjects.row[{\"subject\":\"s1\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "subjects", - "sources.subjects.row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.1:00000137de028aa69769a5d29c9003e923af9a0b081ff4067b2eb08af4ef943e4939b6", + "sources.subjects.row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.2:000002d1c5b49de38ef6efc552c2a47b8ab59dd267f8eb55d91814edee99b1b52acc65", "sources.subjects.row[{\"subject\":\"s2\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "144bb37f3c295e16919d770641bb8c65", "sources.subjects.row[{\"subject\":\"s2\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "subjects", - "sources.subjects.row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.1:000001584cc7c090776151311c6768ba98c02c0235250c6685493d3207138ef8d8b484", + "sources.subjects.row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.2:000002338ce8710f5ecb156a72c8446651adb7b0c21635d0aa92d298f09da56faed0f8", "sources.subjects.row[{\"subject\":\"s3\"}]._tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "d9391c2dcea750c496e0895ad301b30d", "sources.subjects.row[{\"subject\":\"s3\"}]._tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956": "subjects", - "sources.subjects.row[{\"subject\":\"s3\"}].data_hash": "arrow_v0.1:000001562e4c70b60df10ed1243d0d9129beff719b867cb16e18d654f375d361e64c51", + "sources.subjects.row[{\"subject\":\"s3\"}].data_hash": "arrow_v0.2:0000023401f1ce40c8cfdd6e4f343d7ffdcd51c1f0521f44cf6030e27e1ca0da154616", "sources.subjects.system_tag_columns": [ "_tag_source_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956", "_tag_record_id::f004fc93e6b885270cb7be1a7e8e368a81c824e524081a3bc1bd3a23cb102956" diff --git a/tests/test_hashing/golden/data/schema.json b/tests/test_hashing/golden/data/schema.json index b0fba5589..007896af3 100644 --- a/tests/test_hashing/golden/data/schema.json +++ b/tests/test_hashing/golden/data/schema.json @@ -46,58 +46,58 @@ "source_schema_hash.record_id_column": "_tag_record_id::a3e81ce4fbd701a0fb97cae134f236953f7f66a8e7b7f4741339b45fe9f485ba", "source_schema_hash.schema_hash": "a3e81ce4fbd701a0fb97cae134f236953f7f66a8e7b7f4741339b45fe9f485ba", "source_schema_hash.source_id_column": "_tag_source_id::a3e81ce4fbd701a0fb97cae134f236953f7f66a8e7b7f4741339b45fe9f485ba", - "source_stream_builder.default_source_id.content_hash": "semantic_v0.1:7eafdaffcb2ac49d760748516b005f13cdab23008421690ade60c691ed7199ea", + "source_stream_builder.default_source_id.content_hash": "semantic_v0.1:167c1d63d67f2fa1998069d2470f6051338e187b615e6c85f79e636dd588b2e2", "source_stream_builder.default_source_id.pipeline_hash": "semantic_v0.1:f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367", - "source_stream_builder.default_source_id.row[{\"subject\":\"s1\"}]._tag_record_id::f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367": "71c09013eb5f5f5582ad41da30acbcf7", - "source_stream_builder.default_source_id.row[{\"subject\":\"s1\"}]._tag_source_id::f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367": "000001630ee60cef9dfe3cce7a6782fa30ab89d235ba2de5934430582a04969396ab0a", - "source_stream_builder.default_source_id.row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.1:00000128b75b53ced7f9752b31affee90b8cd666b24b0802f3de19dff3f742d6636bb8", - "source_stream_builder.default_source_id.row[{\"subject\":\"s2\"}]._tag_record_id::f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367": "e2960b661cf05e14874582345bdde8e8", - "source_stream_builder.default_source_id.row[{\"subject\":\"s2\"}]._tag_source_id::f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367": "000001630ee60cef9dfe3cce7a6782fa30ab89d235ba2de5934430582a04969396ab0a", - "source_stream_builder.default_source_id.row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.1:000001a39a4d7b517d56c4c323c744f782e70521e4d759af0f4ff4aedc806c1a191297", - "source_stream_builder.default_source_id.row[{\"subject\":\"s3\"}]._tag_record_id::f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367": "5d4deb7fe5675941942e2b590adaf240", - "source_stream_builder.default_source_id.row[{\"subject\":\"s3\"}]._tag_source_id::f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367": "000001630ee60cef9dfe3cce7a6782fa30ab89d235ba2de5934430582a04969396ab0a", - "source_stream_builder.default_source_id.row[{\"subject\":\"s3\"}].data_hash": "arrow_v0.1:0000010364967ccec6827a7b2dd8ef76352ce8aed48ccd414bae0ddf1e5b46b0dfc40c", + "source_stream_builder.default_source_id.row[{\"subject\":\"s1\"}]._tag_record_id::f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367": "2d25e22e5ed45f2ebcd9ba0e574ce4ab", + "source_stream_builder.default_source_id.row[{\"subject\":\"s1\"}]._tag_source_id::f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367": "000002e12eafa1dc72203ca9da1ca5d064031683cf089f47bad40744ecd2c923d0fde3", + "source_stream_builder.default_source_id.row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.2:00000289f61a7619845a37a72864e1da3c74a3aff25d66aa6c1d4b6d3874e84ae12e85", + "source_stream_builder.default_source_id.row[{\"subject\":\"s2\"}]._tag_record_id::f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367": "7a85d0e335bf539d990068a86fb4409e", + "source_stream_builder.default_source_id.row[{\"subject\":\"s2\"}]._tag_source_id::f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367": "000002e12eafa1dc72203ca9da1ca5d064031683cf089f47bad40744ecd2c923d0fde3", + "source_stream_builder.default_source_id.row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.2:0000029f0b00a74de3d3f453d7f4dec0ddf7b3dfa05a6cbedec40a31a038f7d764adcb", + "source_stream_builder.default_source_id.row[{\"subject\":\"s3\"}]._tag_record_id::f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367": "174dcf5eb0c05b08967518af5d71cd7b", + "source_stream_builder.default_source_id.row[{\"subject\":\"s3\"}]._tag_source_id::f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367": "000002e12eafa1dc72203ca9da1ca5d064031683cf089f47bad40744ecd2c923d0fde3", + "source_stream_builder.default_source_id.row[{\"subject\":\"s3\"}].data_hash": "arrow_v0.2:0000026b8c4f7e93665f79d39dce50db9d7f494ee1e49a34cd03f226b604f2a6e6427f", "source_stream_builder.default_source_id.schema_hash": "f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367", - "source_stream_builder.default_source_id.source_id": "000001630ee60cef9dfe3cce7a6782fa30ab89d235ba2de5934430582a04969396ab0a", + "source_stream_builder.default_source_id.source_id": "000002e12eafa1dc72203ca9da1ca5d064031683cf089f47bad40744ecd2c923d0fde3", "source_stream_builder.default_source_id.system_tag_columns": [ "_tag_source_id::f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367", "_tag_record_id::f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367" ], - "source_stream_builder.default_source_id.table_hash": "arrow_v0.1:000001630ee60cef9dfe3cce7a6782fa30ab89d235ba2de5934430582a04969396ab0a", - "source_stream_builder.named_source_id.content_hash": "semantic_v0.1:f424ce4643b9e4dd37573e30c019354c66d7361265828482115a6a18cb74f7c7", + "source_stream_builder.default_source_id.table_hash": "arrow_v0.2:000002e12eafa1dc72203ca9da1ca5d064031683cf089f47bad40744ecd2c923d0fde3", + "source_stream_builder.named_source_id.content_hash": "semantic_v0.1:69b4b3098babd295b560941edfc51ecc92cb1a49d68c7614a3172449f09cab3e", "source_stream_builder.named_source_id.pipeline_hash": "semantic_v0.1:f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367", "source_stream_builder.named_source_id.row[{\"subject\":\"s1\"}]._tag_record_id::f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367": "3657ba768edb5ccda83740fcb80aa959", "source_stream_builder.named_source_id.row[{\"subject\":\"s1\"}]._tag_source_id::f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367": "golden_source", - "source_stream_builder.named_source_id.row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.1:00000128b75b53ced7f9752b31affee90b8cd666b24b0802f3de19dff3f742d6636bb8", + "source_stream_builder.named_source_id.row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.2:00000289f61a7619845a37a72864e1da3c74a3aff25d66aa6c1d4b6d3874e84ae12e85", "source_stream_builder.named_source_id.row[{\"subject\":\"s2\"}]._tag_record_id::f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367": "bce9b135c1675095ac3a1e7099490014", "source_stream_builder.named_source_id.row[{\"subject\":\"s2\"}]._tag_source_id::f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367": "golden_source", - "source_stream_builder.named_source_id.row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.1:000001a39a4d7b517d56c4c323c744f782e70521e4d759af0f4ff4aedc806c1a191297", + "source_stream_builder.named_source_id.row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.2:0000029f0b00a74de3d3f453d7f4dec0ddf7b3dfa05a6cbedec40a31a038f7d764adcb", "source_stream_builder.named_source_id.row[{\"subject\":\"s3\"}]._tag_record_id::f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367": "9ac1e81621a25135b681f367e7a3b98a", "source_stream_builder.named_source_id.row[{\"subject\":\"s3\"}]._tag_source_id::f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367": "golden_source", - "source_stream_builder.named_source_id.row[{\"subject\":\"s3\"}].data_hash": "arrow_v0.1:0000010364967ccec6827a7b2dd8ef76352ce8aed48ccd414bae0ddf1e5b46b0dfc40c", + "source_stream_builder.named_source_id.row[{\"subject\":\"s3\"}].data_hash": "arrow_v0.2:0000026b8c4f7e93665f79d39dce50db9d7f494ee1e49a34cd03f226b604f2a6e6427f", "source_stream_builder.named_source_id.schema_hash": "f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367", "source_stream_builder.named_source_id.source_id": "golden_source", "source_stream_builder.named_source_id.system_tag_columns": [ "_tag_source_id::f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367", "_tag_record_id::f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367" ], - "source_stream_builder.named_source_id.table_hash": "arrow_v0.1:000001630ee60cef9dfe3cce7a6782fa30ab89d235ba2de5934430582a04969396ab0a", - "source_stream_builder.record_id_column.content_hash": "semantic_v0.1:6983755521b8c4ca1771dab9fd20de2d92ce24a15438cca78378fb098d57d87d", + "source_stream_builder.named_source_id.table_hash": "arrow_v0.2:000002e12eafa1dc72203ca9da1ca5d064031683cf089f47bad40744ecd2c923d0fde3", + "source_stream_builder.record_id_column.content_hash": "semantic_v0.1:0bbf3af4ed09b723aa345c2de91cbf5845771a5860d319c775388aae5751d482", "source_stream_builder.record_id_column.pipeline_hash": "semantic_v0.1:f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367", "source_stream_builder.record_id_column.row[{\"subject\":\"s1\"}]._tag_record_id::f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367": "2c42f741f6ab5ba29ec12fddfed561f8", "source_stream_builder.record_id_column.row[{\"subject\":\"s1\"}]._tag_source_id::f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367": "golden_source", - "source_stream_builder.record_id_column.row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.1:00000128b75b53ced7f9752b31affee90b8cd666b24b0802f3de19dff3f742d6636bb8", + "source_stream_builder.record_id_column.row[{\"subject\":\"s1\"}].data_hash": "arrow_v0.2:00000289f61a7619845a37a72864e1da3c74a3aff25d66aa6c1d4b6d3874e84ae12e85", "source_stream_builder.record_id_column.row[{\"subject\":\"s2\"}]._tag_record_id::f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367": "8df4a32981b850a9915c8a5715376d1b", "source_stream_builder.record_id_column.row[{\"subject\":\"s2\"}]._tag_source_id::f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367": "golden_source", - "source_stream_builder.record_id_column.row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.1:000001a39a4d7b517d56c4c323c744f782e70521e4d759af0f4ff4aedc806c1a191297", + "source_stream_builder.record_id_column.row[{\"subject\":\"s2\"}].data_hash": "arrow_v0.2:0000029f0b00a74de3d3f453d7f4dec0ddf7b3dfa05a6cbedec40a31a038f7d764adcb", "source_stream_builder.record_id_column.row[{\"subject\":\"s3\"}]._tag_record_id::f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367": "f015aeed0c0e5c14a3280d0f3e828564", "source_stream_builder.record_id_column.row[{\"subject\":\"s3\"}]._tag_source_id::f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367": "golden_source", - "source_stream_builder.record_id_column.row[{\"subject\":\"s3\"}].data_hash": "arrow_v0.1:0000010364967ccec6827a7b2dd8ef76352ce8aed48ccd414bae0ddf1e5b46b0dfc40c", + "source_stream_builder.record_id_column.row[{\"subject\":\"s3\"}].data_hash": "arrow_v0.2:0000026b8c4f7e93665f79d39dce50db9d7f494ee1e49a34cd03f226b604f2a6e6427f", "source_stream_builder.record_id_column.schema_hash": "f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367", "source_stream_builder.record_id_column.source_id": "golden_source", "source_stream_builder.record_id_column.system_tag_columns": [ "_tag_source_id::f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367", "_tag_record_id::f639932798b21a97cbf494c5b56b76142293dd248b775ecdcfdce639b4177367" ], - "source_stream_builder.record_id_column.table_hash": "arrow_v0.1:000001630ee60cef9dfe3cce7a6782fa30ab89d235ba2de5934430582a04969396ab0a" + "source_stream_builder.record_id_column.table_hash": "arrow_v0.2:000002e12eafa1dc72203ca9da1ca5d064031683cf089f47bad40744ecd2c923d0fde3" } diff --git a/tests/test_hashing/golden/data/semantic.json b/tests/test_hashing/golden/data/semantic.json index 4180e3648..dc68a6bff 100644 --- a/tests/test_hashing/golden/data/semantic.json +++ b/tests/test_hashing/golden/data/semantic.json @@ -32,7 +32,7 @@ "value.bytearray.binary": "semantic_v0.1:d891b13714f4aec76114361fe22ebfb829f952b88c9ad49c4afa16896612ee75", "value.bytes.binary": "semantic_v0.1:d891b13714f4aec76114361fe22ebfb829f952b88c9ad49c4afa16896612ee75", "value.bytes.empty": "semantic_v0.1:12ae32cb1ec02d01eda3581b127c1fee3b0dc53572ed6baf239721a03d82e126", - "value.dataframe.basic": "arrow_v0.1:000001b0e17dcec7cd73f09d83119c75f39a9638089533cbf9cc478de4a79e3082cdb9", + "value.dataframe.basic": "arrow_v0.2:000002bac72e84776fc6d62d3928f88ec11672873c07bca9de0cc5a7689cd9675800dc", "value.dict.ab": "semantic_v0.1:43258cff783fe7036d8a43033f830adfc60ec037382473548ac742b888292777", "value.dict.ba": "semantic_v0.1:43258cff783fe7036d8a43033f830adfc60ec037382473548ac742b888292777", "value.dict.empty": "semantic_v0.1:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a", diff --git a/tests/test_hashing/golden/generate_golden_hashes.py b/tests/test_hashing/golden/generate_golden_hashes.py index 0aa7446a5..9e86f5c24 100644 --- a/tests/test_hashing/golden/generate_golden_hashes.py +++ b/tests/test_hashing/golden/generate_golden_hashes.py @@ -29,9 +29,10 @@ process, whatever ``PYTHONHASHSEED`` is. A value that differs by interpreter is a defect to fix, never something to pin per version. -Known gaps (ITL-848), not pinned until fixed: ``hash_schema`` on a schema with an extension -type, ``null``-typed columns, the ``arrow.uuid`` canonical extension type, and map and -fixed-size-list values. +Not pinned, because starfix rejects their data (ITL-848): ``null``-typed and map columns. +Since starfix 0.5 (hash format 2, label ``arrow_v0.2``, ITL-910), ``hash_schema`` on extension +columns, the canonical ``arrow.uuid`` and ``arrow.fixed_shape_tensor`` types, and +fixed-size-list values are hashable and pinned. Comparing releases ------------------ From a5ceb0ab7cdcc91da9168d27bf0b9cc45564a26d Mon Sep 17 00:00:00 2001 From: "Edgar Y. Walker" <> Date: Wed, 11 Nov 2026 15:44:46 -0800 Subject: [PATCH 06/21] test: re-pin format-1 digests and record-id order in starfix format 2 (ITL-910) Co-Authored-By: Claude Opus 5.5 --- .../operators/test_async_sync_equivalence.py | 11 +++++---- tests/test_core/operators/test_group_by.py | 5 ++-- .../operators/test_materialize_source_info.py | 15 ++++++++---- tests/test_hashing/test_pandas_handlers.py | 4 ++-- .../test_hashing/test_starfix_arrow_hasher.py | 24 +++++++++---------- .../test_struct_layout_identity.py | 21 ++++++++-------- 6 files changed, 44 insertions(+), 36 deletions(-) diff --git a/tests/test_core/operators/test_async_sync_equivalence.py b/tests/test_core/operators/test_async_sync_equivalence.py index 916509dd0..5bec5adda 100644 --- a/tests/test_core/operators/test_async_sync_equivalence.py +++ b/tests/test_core/operators/test_async_sync_equivalence.py @@ -546,15 +546,16 @@ def _narrow_source(column: str, source_id: str) -> ArrowTableSource: # Recorded from the sync path on main (ab544982) before the ITL-837 fix. The sync -# path must not change; the async path must now produce the same values. +# path must not change; the async path must now produce the same values. Row hashes +# re-recorded in starfix hash format 2 (arrow_v0.2, ITL-910); the stream hashes did not move. PINNED_ROW_HASHES = { 1: ( - "arrow_v0.1:000001515f1eb3674534e0226bbe2aada98adb5137a0f8829ee815fdb994809ac001fa", - "arrow_v0.1:00000119d2a6630f08dd30ef49c4a298862b7d129d2169d84963ea4f801eb8248e99e4", + "arrow_v0.2:0000027e1a4693c531d4411fafa68a1ac623a0ed3f04a6402de0fb8f040f0aba717ae8", + "arrow_v0.2:0000025c4232f42a6012b104827b2c4731e143af6044e010e9a37ae5ffa82fe3ae7153", ), 2: ( - "arrow_v0.1:000001ea3d721d986f7382996ef41ded3d1a43baf41a543fde471b20c71190b45dbf98", - "arrow_v0.1:000001fa7bf20e72872ffc6860a21d75ad93c38bd54753fa9eb325502acc783b693d43", + "arrow_v0.2:000002e3b4cd9f773e0263ba270d251b90058702fa282e129feebdbc428d832080752e", + "arrow_v0.2:00000218de95b54e8987999a339fe1726957da66219b529c4699be853334833e71f60e", ), } PINNED_STREAM_CONTENT_HASH = ( diff --git a/tests/test_core/operators/test_group_by.py b/tests/test_core/operators/test_group_by.py index 5519c94ef..bdf1ec73d 100644 --- a/tests/test_core/operators/test_group_by.py +++ b/tests/test_core/operators/test_group_by.py @@ -122,13 +122,14 @@ def test_falls_back_to_record_id_when_key_covers_all_tags(self): record_id is a uuid5 of source content, so the order it imposes is not human-predictable but IS stable across processes. Asserted exactly -- sorting the result before comparing would make this blind to the very - ordering it exists to pin down. + ordering it exists to pin down. The default source_id is an Arrow table + hash, so this order moved with starfix hash format 2 (ITL-910). """ table = pa.table({"subject": ["G", "G"], "path": ["b", "a"]}) src = ArrowTableSource(table, tag_columns=["subject"], infer_nullable=True) out = GroupBy(by=["subject"]).process(src) assert len(out.as_table()) == 1 - assert out.as_table().column("path").to_pylist()[0] == ["a", "b"] + assert out.as_table().column("path").to_pylist()[0] == ["b", "a"] def test_record_id_breaks_ties_between_duplicate_tags(self): """Duplicate tag tuples must not fall back to raw emission order.""" diff --git a/tests/test_core/operators/test_materialize_source_info.py b/tests/test_core/operators/test_materialize_source_info.py index 2ed6791b2..c9acb337a 100644 --- a/tests/test_core/operators/test_materialize_source_info.py +++ b/tests/test_core/operators/test_materialize_source_info.py @@ -564,6 +564,9 @@ def _record_ids(node: OperatorJobNode) -> dict[str, bytes]: class TestIdentityPinned: """Values recorded on ``main`` at ab544982, before the fix. + Re-recorded under starfix hash format 2 (``arrow_v0.2``, ITL-910): the node content + hash and the record hashes are Arrow-derived; the pipeline hash is not. + The node's identity, the output data and the row-0 record hash (whose source-info token was already right) are unchanged; only rows whose stored source-info token was wrong get a new record hash. @@ -584,7 +587,7 @@ def test_operator_node_identity_and_row_zero_record_hash(self): "semantic_v0.1:1f533cfa7293bad96debd5cda622e04bc39d72970ce6827dcd6e890a7428823c" ) assert node.content_hash().to_string() == ( - "semantic_v0.1:6dc7f9c42cf4d0351d8a577bb2c72d69aadbca97e1c31d0cfc62d6a8087f3038" + "semantic_v0.1:36a4b61a5ed690ffa95ff31ba0c19a43e31b5b2ac51e1e5848d4b184e4b9ccf7" ) ids = _record_ids(node) assert ids["a"] == ROW_ZERO_RECORD_ID @@ -605,16 +608,18 @@ def test_operator_node_identity_and_row_zero_record_hash(self): } -# ``_record_hash`` of rows a, b, c stored by the pinned node on ab544982 (pre-fix). +# ``_record_hash`` of rows a, b, c stored by the pinned node before the fix (ab544982). # Row a's source-info token was already right, so the fix leaves its record hash. +# Re-recorded in starfix hash format 2 (ITL-910) by reproducing the pre-fix broadcast +# with ``_broadcast_row_zero``. ROW_ZERO_RECORD_ID = ( - b"000001688856047a9f37112172e612834209fb07f2cf4be5db30f0fa3b7059db85905e" + b"000002016f74f387a07874a088c8c44f1b85ff70a395680158b6a87520085c95439ec4" ) PRE_FIX_RECORD_IDS = frozenset( { ROW_ZERO_RECORD_ID, - b"0000016f23c92ad43ef01bb80e438f16da66d02354998351430b3237d938a340943039", - b"00000138599ee77c1f3dc62bf02a117150b32db27c6582db885cda2d28b7cef3964657", + b"000002de01a6a62bba3ec3ec7cb9c64a8bf4a7da8f0ab959fca01dcfd2d162f91e9cf6", + b"00000270703c99569d9ffb213ac4a8825f880e6ffe25eb1b30019da118752a73996acc", } ) diff --git a/tests/test_hashing/test_pandas_handlers.py b/tests/test_hashing/test_pandas_handlers.py index f6af7faf0..cf6417236 100644 --- a/tests/test_hashing/test_pandas_handlers.py +++ b/tests/test_hashing/test_pandas_handlers.py @@ -24,7 +24,7 @@ def test_returns_content_hash(self, arrow_hasher): df = pd.DataFrame({"a": [1.0, 2.0], "b": [3, 4]}) result = PandasDataFrameHandler(arrow_hasher).handle(df, hasher=None) assert isinstance(result, ContentHash) - assert result.method == "arrow_v0.1" + assert result.method == "arrow_v0.2" def test_same_content_same_hash(self, arrow_hasher): from orcapod.hashing.semantic_hashing.builtin_handlers import ( @@ -95,7 +95,7 @@ def test_returns_content_hash(self, arrow_hasher): s = pd.Series([1.0, 2.0, 3.0], name="x") result = PandasSeriesHandler(arrow_hasher).handle(s, hasher=None) assert isinstance(result, ContentHash) - assert result.method == "arrow_v0.1" + assert result.method == "arrow_v0.2" def test_same_content_same_hash(self, arrow_hasher): from orcapod.hashing.semantic_hashing.builtin_handlers import ( diff --git a/tests/test_hashing/test_starfix_arrow_hasher.py b/tests/test_hashing/test_starfix_arrow_hasher.py index 330a7c85b..8b046816e 100644 --- a/tests/test_hashing/test_starfix_arrow_hasher.py +++ b/tests/test_hashing/test_starfix_arrow_hasher.py @@ -38,7 +38,7 @@ # Helpers # --------------------------------------------------------------------------- -HASHER_ID = "arrow_v0.1" +HASHER_ID = "arrow_v0.2" _DIGEST_LEN = 35 # 3 version bytes + 32 SHA-256 bytes @@ -107,7 +107,7 @@ def test_golden_value(self): h = _make_hasher().hash_schema(schema) assert ( h.digest.hex() - == "000001d676ef0263a8e0e7500b1c97033993dbe445172ca0f9e7577b3994bfa6224b4c" + == "000002e9eb1c825a8a882b565ef45a2ec832c10e31ad067736d2a6cb8a9403ff91c33f" ), ( f"Schema golden digest changed — was the starfix algorithm updated? " f"Got: {h.digest.hex()}" @@ -126,10 +126,10 @@ def test_nullability_golden_values(self): s_nullable = pa.schema([pa.field("x", pa.int32(), nullable=True)]) s_non_nullable = pa.schema([pa.field("x", pa.int32(), nullable=False)]) assert _make_hasher().hash_schema(s_nullable).digest.hex() == ( - "000001b8005339e69df64cda60f9ff9c98caa264092a7b666c3f7f85a2bfed20bae3db" + "000002fa30e1a356fea8d3a5602a6fa1d49a43349a840f65ead55463f5ead55ea64e9a" ) assert _make_hasher().hash_schema(s_non_nullable).digest.hex() == ( - "00000179353568a71430411e8108ee02d425800f6d5054d9b2baa871cad90f3e06422a" + "0000022571a7dcd462c26761f5b5353ccbd0c8d80655b462c3bb622560ec0d590a5662" ) @@ -254,7 +254,7 @@ def test_golden_value_table(self): h = _make_hasher().hash_table(table) assert ( h.digest.hex() - == "0000010cd7fe5462420b84f03a06925374e528817a3b72319e679a17e7380964878791" + == "00000277215cd88b3019240dec61d8c4bed8bf6fd05e48d5684e1e7d3acc957f54102c" ), ( f"Table golden digest changed — was the starfix algorithm updated? " f"Got: {h.digest.hex()}" @@ -306,9 +306,9 @@ def test_hasher_id_matches_current_constant(self): hasher = get_versioned_semantic_arrow_hasher() assert hasher.hasher_id == _CURRENT_ARROW_HASHER_ID - def test_current_hasher_id_is_arrow_v0_1(self): + def test_current_hasher_id_is_arrow_v0_2(self): """Pin the current version constant to detect accidental version drifts.""" - assert _CURRENT_ARROW_HASHER_ID == "arrow_v0.1" + assert _CURRENT_ARROW_HASHER_ID == "arrow_v0.2" # --------------------------------------------------------------------------- @@ -327,7 +327,7 @@ def test_v01_context_hasher_id(self): from orcapod.contexts import resolve_context ctx = resolve_context("v0.1") - assert ctx.arrow_hasher.hasher_id == "arrow_v0.1" + assert ctx.arrow_hasher.hasher_id == "arrow_v0.2" def test_context_hash_table_functional(self): from orcapod.contexts import resolve_context @@ -350,7 +350,7 @@ def test_context_hash_table_functional(self): class TestHashSchemaExtensionAware: def test_extension_free_golden_value_unchanged(self): """Stability invariant: extension-free schema hash is byte-identical to - the pre-v0.3.0 golden value (include_metadata=False is the default).""" + the starfix format-2 (arrow_v0.2) golden value (include_metadata=False is the default).""" schema = pa.schema( [ pa.field("id", pa.int32(), nullable=False), @@ -360,7 +360,7 @@ def test_extension_free_golden_value_unchanged(self): h = _make_hasher().hash_schema(schema) assert ( h.digest.hex() - == "000001d676ef0263a8e0e7500b1c97033993dbe445172ca0f9e7577b3994bfa6224b4c" + == "000002e9eb1c825a8a882b565ef45a2ec832c10e31ad067736d2a6cb8a9403ff91c33f" ), ( f"Stability invariant broken — extension-free golden digest changed. Got: {h.digest.hex()}" ) @@ -442,7 +442,7 @@ def test_unrelated_metadata_does_not_affect_hash(self): class TestHashTableExtensionAware: def test_extension_free_table_golden_value_unchanged(self): """Stability invariant: extension-free table hash is byte-identical to - the pre-v0.3.0 golden value.""" + the starfix format-2 (arrow_v0.2) golden value.""" table = pa.table( { "id": pa.array([1, 2, 3], type=pa.int32()), @@ -453,7 +453,7 @@ def test_extension_free_table_golden_value_unchanged(self): h = _make_hasher().hash_table(table) assert ( h.digest.hex() - == "0000010cd7fe5462420b84f03a06925374e528817a3b72319e679a17e7380964878791" + == "00000277215cd88b3019240dec61d8c4bed8bf6fd05e48d5684e1e7d3acc957f54102c" ), ( f"Stability invariant broken — extension-free table golden digest changed. Got: {h.digest.hex()}" ) diff --git a/tests/test_semantic_types/test_struct_layout_identity.py b/tests/test_semantic_types/test_struct_layout_identity.py index 45fc9133f..1dcf7e3be 100644 --- a/tests/test_semantic_types/test_struct_layout_identity.py +++ b/tests/test_semantic_types/test_struct_layout_identity.py @@ -50,38 +50,39 @@ def _tables() -> dict[str, pa.Table]: _FLAT_HASHES: dict[str, object] = { - "stream_content": "90dab8a6e5b7b03c83ef20fbfdd4040410f05f9e20d8ca2f64d6f2ef374b8e99", + "stream_content": "1362a2951a4ce959504f5a33d325766ef6c35e021d166be54b2f358b8e47e97d", "stream_pipeline": "34209011cc4048b6425d8b01c9c11e20dfc27a6f45b40a9948389bced49a76e4", "source_content": "7263fb2ff4346ac025631fd34c7204971478d83165084f9a748e61acc5e9f919", "source_pipeline": "34209011cc4048b6425d8b01c9c11e20dfc27a6f45b40a9948389bced49a76e4", "data_rows": [ - "0000010aa7ec467ed05cf6156b4c537186edb548c2e0e61f09d66dde96d9f644aee318", - "000001df354be2d86566d498a075294b6cbd403e4566dac064c6306787245d7ecf8004", + "000002a3b465dfc79ea3f32b39f55417f4c2372d7a3cc7ad63aef00aa9108b21b76b9c", + "000002e540dc3b59a01d14edc26eebff96453fb1de9018c351676cef8d1f67ee3346b2", ], } -# Hashes on main (365a5f74), before the converter change. +# Hashes on main (365a5f74), before the converter change; the stream content and per-row +# data hashes re-recorded in starfix hash format 2 (arrow_v0.2, ITL-910). MAIN_HASHES: dict[str, dict[str, object]] = { "small": _FLAT_HASHES, "large": _FLAT_HASHES, "nested": { - "stream_content": "9e7d62a473a0e47a8ca74ad950245a3256248a7ed8994645198d82e4d6b33902", + "stream_content": "d88d09bb6c288a29cd0edbf62826bf8d9afb6b2c6b9d320aa248c4496b7d63f3", "stream_pipeline": "f8745264f2422bf76eaec06120aa00b510cafa0ee3abdfac3d5396a614afb3a1", "source_content": "2ee725a65a6050703c4164f901ac06e2137a0e2c71f7cf7b636423a1e1ac2a9b", "source_pipeline": "f8745264f2422bf76eaec06120aa00b510cafa0ee3abdfac3d5396a614afb3a1", "data_rows": [ - "0000010f2cedeebe0599b55b059f918c148c3c0056dd9848d7fdecfc1eba992c23962f", - "000001618d4b5959d543202f49171b6c2b89bc5fef30b9bd157c03a13458b3ae882c8c", + "000002b734209953036fae7ee935c232f7949f4874dc22dca2af4ecfa2882e5ddaf457", + "0000024d11adcf6077042679df69776a748d71f2818fc47042913d83fa1fbebca67ad9", ], }, "list_of_struct": { - "stream_content": "276d8ec2a45a4e62be7496c9ce33687558647c5369c0828671f51f357a4fb6ea", + "stream_content": "407a24e60bf74b666560f868d193dc35efcaf531d8d357bd7e414b861436336d", "stream_pipeline": "85b8d45d79762e42d1edf56c3853379193c21d5879a30166fcf3bc3c8e56a733", "source_content": "19a5d5780cd84af633704022bf8f7a80ba72555533390ee5e69c487f4ec253cf", "source_pipeline": "85b8d45d79762e42d1edf56c3853379193c21d5879a30166fcf3bc3c8e56a733", "data_rows": [ - "00000151b3c44eaf49ed64174b9b47f89a779cd478c6874cbcac5a7df049d33ae1a7f9", - "000001dbb51ff7212e3b1dbe16f90d8d3133c5b8743d76d823fd9c903f6c9d2538eca5", + "000002c262cfe3995378fe521abbc19b8cd14f10f626b5e5b1259a20f6d4277677fb87", + "000002188277589a87f65b49271c1a2cf461ebed48ddd2fd3cb943198f420fced6862a", ], }, } From f5433ffd6f2e675a63c592b2158784d0c12bee56 Mon Sep 17 00:00:00 2001 From: "Edgar Y. Walker" <> Date: Wed, 11 Nov 2026 15:46:27 -0800 Subject: [PATCH 07/21] docs: starfix hash format 2 / arrow_v0.2 in CHANGELOG, hashing reference and DESIGN_ISSUES (ITL-910, WIP) Co-Authored-By: Claude Opus 5.5 --- CHANGELOG.md | 33 ++++++++++++++++++++ DESIGN_ISSUES.md | 23 ++++++++++++++ docs/reference/hashing.md | 64 +++++++++++++++++++++++++++++++++------ 3 files changed, 111 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index dfdc26d3b..eec006848 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -47,6 +47,39 @@ v0.3, or keep using v0.2 for an existing store. A migration is planned after v0. the MRO, so a registered `np.ndarray` subclass wins over `np.ndarray` for itself and its subclasses, whatever the registration order. +#### Arrow hashing moves to starfix hash format 2: every Arrow-derived hash changes (ITL-910) + +**Orcapod now requires `starfix>=0.5.0,<0.6`, and the Arrow hash label moves from `arrow_v0.1` +to `arrow_v0.2`.** starfix 0.4 (hash format 1) lost information for some inputs, so different +data could hash the same (ITL-848): the lengths of nested lists (`[[[1]], [[2], [3]]]` and +`[[[1], [2]], [[3]]]`), a null struct and a struct whose fields are all null, and Boolean +columns, whose hash also depended on how the table was split into chunks. starfix 0.5 (hash +format 2, prefix `00 00 02`) is lossless; this is a fix, so the old values are not kept. + +- **What moves.** Every hash computed from an Arrow table: per-row data hashes, default + `source_id`s and the `record_id`s and source content hashes built from them, function + input-data hashes, pipeline base and entry ids, operator and cached-source record hashes, + invocation hashes, and the hashes of `pa.Table`, pandas `DataFrame` and `Series` values. The + hash label in front of each (`arrow_v0.2:000002...`) shows the new format. Pipeline hashes, + schema hashes, system-tag column names and table paths under the default + `table_scope="pipeline_hash"` do not move. Paths that embed a content hash + (`table_scope="content_hash"`, `CachedSource` tables) move with it. Where a `GroupBy` or + `Batch` orders members by `record_id`, the order of members can change too. +- **Results stored by 0.2.0.** STORED-DATA-PARAGRAPH (ITL-910 D1, pending) +- **Now hashable.** `fixed_size_list` columns, at any depth (ITL-927): a source or operator + carrying one works, and its type is kept. A `fixed_size_list[n]` hashes differently from a + `large_list` with the same values. pyarrow's canonical extension types (`arrow.uuid`, + `arrow.fixed_shape_tensor`) hash with their extension name and metadata, so an `arrow.uuid` + column never hashes like the `fixed_size_binary(16)` column holding the same bytes, and + `StarfixArrowHasher.hash_schema` keeps the identity of extension columns (ITL-846). +- **Now refused** with `ValueError` naming the column path and type, where starfix 0.4 hashed + silently: a non-nullable field holding a null; a zero-row table with a `map` column; two + schema nodes sharing a path (duplicate column names, for example in a `pa.Table` value, or a + column `"a/b"` next to a struct column `a` with a child `b`). Data of `null`, `map`, + interval, `string_view` and `binary_view` columns, union and run-end-encoded types are + still refused, now with `ValueError` instead of `NotImplementedError`. +- Column order and struct field order are still not part of an Arrow hash. + #### A changed column type is refused when appending to a Delta table (ITL-884) Before writing to an existing `DeltaTableDatabase` table, the flush now compares the new rows' diff --git a/DESIGN_ISSUES.md b/DESIGN_ISSUES.md index 695708a20..910936f32 100644 --- a/DESIGN_ISSUES.md +++ b/DESIGN_ISSUES.md @@ -2442,3 +2442,26 @@ results. Known compromise: the struct's children are nullable at the Arrow level, because Polars drops nested non-null flags on its way back to Arrow (ITL-842). The logical type enforces non-null instead. Tighten under a new storage version once nested nullability survives Polars. + +--- + +## `src/orcapod/hashing/arrow_hashers.py` — Arrow hash format (starfix) + +### HS2 — starfix hash format 1 collided on nested lists, null structs and Booleans +**Status:** in progress +**Severity:** high +**Issue:** ITL-910 (fixes the starfix side of ITL-848; ITL-927) + +`StarfixArrowHasher` used starfix 0.4 (hash format 1, label `arrow_v0.1`), which lost +information for some inputs, so different data reached the same row hash: nested-list lengths +(`[[[1]], [[2], [3]]]` and `[[[1], [2]], [[3]]]`), a null struct and a struct whose fields are +all null, and non-nullable Boolean columns (`[True]` and `[True, False, False]`); a Boolean +column's hash also depended on chunking. starfix 0.4 also could not hash `fixed_size_list` +data, so a source with such a column failed at construction (ITL-927). + +**Fix:** starfix 0.5 (hash format 2, prefix `00 00 02`) under the new label `arrow_v0.2` +(developer's label rule: the label moves whenever any table hash can change). Every +Arrow-derived hash moves; the golden files were regenerated once, explicitly. Order options +stay off. pyarrow's canonical extension types and `hash_schema` on extension columns hash with +their extension identity instead of as their storage type. Open: what a v0.3 node does with +rows a v0.2 store holds under `arrow_v0.1` in the same table (ITL-910 decision D1). diff --git a/docs/reference/hashing.md b/docs/reference/hashing.md index 13ddcf73a..8cdb2c3a5 100644 --- a/docs/reference/hashing.md +++ b/docs/reference/hashing.md @@ -132,24 +132,24 @@ each suffix is the stream's `pipeline_hash()` (sites 1b and 6). | Site | Row | Value (hex) | |------|-----|-------------| -| 7 | `alice` `base_entry_id` | `6172726f775f76302e313a00000113f967f1187ac2982e65b6afbf324d264d50a6efe3357a35f09b8a2b32454d61` | -| 7 | `bob` `base_entry_id` | `6172726f775f76302e313a000001b6162a6faa76b5e065e89849acc09f41edfb5bb1a7c7ea626fd2920de6046586` | -| 8 | `alice` `pipeline_entry_id` (idx=0) | `6172726f775f76302e313a0000016d7853d1b6b3dc62063d3982911c3f18b8a719e63cfd41429fff970137c52d9c` | -| 8 | `bob` `pipeline_entry_id` (idx=0) | `6172726f775f76302e313a000001244dc0d59c23ec5d4a19b972a2a98f410054da09f839115841cefdf7f4851f63` | +| 7 | `alice` `base_entry_id` | `6172726f775f76302e323a000002d147e6dfdc31690391034ed592bb75d4c4407034504165d168717c0779b64d3f` | +| 7 | `bob` `base_entry_id` | `6172726f775f76302e323a000002250edea78c481fa87d262c5938c9288a5538a17a10636b54a6ea83a18c77eb82` | +| 8 | `alice` `pipeline_entry_id` (idx=0) | `6172726f775f76302e323a0000028f3588b0c592a1ef02e782927fd355fa4e7887597c75d350a428647f9fc81cd9` | +| 8 | `bob` `pipeline_entry_id` (idx=0) | `6172726f775f76302e323a0000026122df3637ef917d0f6fb996572dfab4c8b92a6539f97430054c0b2271363acc` | -> The hex prefix `6172726f775f76302e31` decodes to the ASCII string `arrow_v0.1` — the +> The hex prefix `6172726f775f76302e32` decodes to the ASCII string `arrow_v0.2` — the > `method` portion of the `b"{method}:{digest}"` format described in §3. #### §4 — Side-Effect Record ID & Invocation Hash | Site | Row | Value | |------|-----|-------| -| 10 | `alice` `invocation_hash` | `semantic_v0.1:1f71386f47538f7b5ca97554104469650b8ec5103a824cbe811a7b776913da91::arrow_v0.1:0000016d7853d1b6b3dc62063d3982911c3f18b8a719e63cfd41429fff970137c52d9c` | -| 10 | `bob` `invocation_hash` | `semantic_v0.1:1f71386f47538f7b5ca97554104469650b8ec5103a824cbe811a7b776913da91::arrow_v0.1:000001244dc0d59c23ec5d4a19b972a2a98f410054da09f839115841cefdf7f4851f63` | +| 10 | `alice` `invocation_hash` | `semantic_v0.1:1f71386f47538f7b5ca97554104469650b8ec5103a824cbe811a7b776913da91::arrow_v0.2:0000028f3588b0c592a1ef02e782927fd355fa4e7887597c75d350a428647f9fc81cd9` | +| 10 | `bob` `invocation_hash` | `semantic_v0.1:1f71386f47538f7b5ca97554104469650b8ec5103a824cbe811a7b776913da91::arrow_v0.2:0000026122df3637ef917d0f6fb996572dfab4c8b92a6539f97430054c0b2271363acc` | > Both rows share the same `semantic_v0.1:1f71386f...` component — this is the > `SideEffectPodStream.pipeline_hash()`, which depends only on the pod and its upstream -> topology, not on row content. The `arrow_v0.1:...` component is the per-row `record_id` +> topology, not on row content. The `arrow_v0.2:...` component is the per-row `record_id` > from site 9, which differs per row. #### §5 — Data Function URI Hash @@ -334,7 +334,7 @@ topology suffix, encoding the full join lineage into each column name. | Field | Value | |---|---| | **Inputs** | Full raw Arrow table (all columns, before any system tag injection) | -| **Algorithm** | `StarfixArrowHasher.hash_table(table).to_hex(path_n_char)` — versioned SHA-256 via the `starfix` Rust crate | +| **Algorithm** | `StarfixArrowHasher.hash_table(table).to_hex(path_n_char)` — starfix hash format 2: the 3-byte format number `00 00 02` followed by a SHA-256 digest (`starfix` 0.5, byte-identical to the Rust crate) | | **Output format** | Truncated hex `str` (length = `path_n_char`) | | **Uniqueness guarantee** | Unique per raw table content; changes if any cell value changes | | **Known exclusions** | Used **only** as a fallback when no explicit `source_id` is provided. A user-supplied `source_id` bypasses this computation entirely. | @@ -618,3 +618,49 @@ print(hasher.hash_object(np.arange(6, dtype=np.float32).reshape(2, 3)).to_string print(hasher.hash_object(np.array(2.5)).to_string()) # ndarray_v1:47f182d862f5dd597d88836deddaed609ed94490827302e75e8364e877633cae ``` + +## 9. Arrow Table Hash (`arrow_v0.2`) + +Every Arrow-derived hash in the sites above (row data hashes, `record_id`s built from a +default `source_id`, base and pipeline entry ids, operator and cached-source record hashes, +`pa.Table` and pandas values) comes from `StarfixArrowHasher.hash_table`, which returns +`ContentHash(method="arrow_v0.2", digest=D)`. `D` is 35 bytes: the starfix hash format +number `00 00 02` followed by a SHA-256 digest, computed by `starfix` 0.5 (hash format 2) and +byte-identical to the Rust `starfix` crate. The byte layout is specified in starfix's +`docs/byte-layout-spec.md`. + +**Label rule.** The label `arrow_vX.Y` moves whenever the hash of any table can change, +whatever the cause. Its number is not kept equal to the starfix format number; the data +context's changelog records which starfix format each label uses. + +| Label | starfix | Format prefix | Orcapod | +|---|---|---|---| +| `arrow_v0.1` | 0.4 | `00 00 01` | up to v0.2 | +| `arrow_v0.2` | 0.5 | `00 00 02` | v0.3 | + +**What is part of the hash.** Column names, Arrow types (with large and non-large layouts of +strings, binaries and lists hashing alike), nullability, every value, row order, list element +order, list lengths at every depth, null structs (distinct from structs whose fields are all +null) and the row count. Column order and struct field order are not part of the hash. How a +table is split into chunks or slices is not part of it either. A `fixed_size_list[n]` +column hashes differently from a `large_list` column with the same values. Extension +columns are hashed as their storage type plus their extension name and metadata, including +pyarrow's canonical extension types (`arrow.uuid`, `arrow.fixed_shape_tensor`); only +`ARROW:extension:*` metadata is hashed, all other metadata is ignored. + +**What cannot be hashed.** starfix raises `ValueError`, naming the column path and type, for: +data of `null`, `map`, interval, `string_view` and `binary_view` columns (whatever the row +count; their schemas hash); union and run-end-encoded types; two schema nodes sharing a path +(duplicate column names, or a name containing `/` that aliases a struct child, such as a +column `"a/b"` next to a struct column `a` with a child `b`); and a non-nullable field holding +a null. A source whose table has such a column fails when it is constructed. + +```python +import pyarrow as pa +from orcapod.contexts import get_default_context + +hasher = get_default_context().arrow_hasher +digest = hasher.hash_table(pa.table({"x": pa.array([1, 2, 3], pa.int64())})) +print(digest.method, digest.digest[:3].hex()) +# arrow_v0.2 000002 +``` From bf8784fa2010e7188129c515c2467e9bc5884e8d Mon Sep 17 00:00:00 2001 From: "Edgar Y. Walker" <> Date: Wed, 11 Nov 2026 15:47:48 -0800 Subject: [PATCH 08/21] style: ruff format and type the format-2 tests (ITL-910) Co-Authored-By: Claude Opus 5.5 --- src/orcapod/hashing/schema_cleaner.py | 6 +++--- tests/test_hashing/test_starfix_format2.py | 21 ++++++++++++--------- 2 files changed, 15 insertions(+), 12 deletions(-) diff --git a/src/orcapod/hashing/schema_cleaner.py b/src/orcapod/hashing/schema_cleaner.py index 2fa55086f..a3021c811 100644 --- a/src/orcapod/hashing/schema_cleaner.py +++ b/src/orcapod/hashing/schema_cleaner.py @@ -178,9 +178,9 @@ def _contains_extension_type(arrow_type: pa.DataType) -> bool: ): return _contains_extension_type(arrow_type.value_type) if pa.types.is_map(arrow_type): - return _contains_extension_type(arrow_type.key_type) or _contains_extension_type( - arrow_type.item_type - ) + return _contains_extension_type( + arrow_type.key_type + ) or _contains_extension_type(arrow_type.item_type) if pa.types.is_dictionary(arrow_type): return _contains_extension_type(arrow_type.value_type) return False diff --git a/tests/test_hashing/test_starfix_format2.py b/tests/test_hashing/test_starfix_format2.py index ec21528ae..44bba9489 100644 --- a/tests/test_hashing/test_starfix_format2.py +++ b/tests/test_hashing/test_starfix_format2.py @@ -8,11 +8,15 @@ from __future__ import annotations +from typing import Any + import pyarrow as pa import pytest from orcapod.contexts import get_default_context from orcapod.core.operators import Join + +# Non-public on purpose: the label constant itself is under test. from orcapod.hashing.versioned_hashers import _CURRENT_ARROW_HASHER_ID from orcapod.sources import ArrowTableSource from orcapod.types import ContentHash @@ -25,16 +29,19 @@ def _hash_table(table: pa.Table) -> ContentHash: return get_default_context().arrow_hasher.hash_table(table) -def _row_hashes(column: pa.Array) -> list[str]: +def _row_hashes(column: pa.Array[Any]) -> list[str]: """Content hashes of the data of a one-column source keyed by a row tag.""" table = pa.table( - {"k": pa.array([f"r{i}" for i in range(len(column))], pa.large_string()), "x": column} + { + "k": pa.array([f"r{i}" for i in range(len(column))], pa.large_string()), + "x": column, + } ) source = ArrowTableSource(table, tag_columns=["k"]) return [data.content_hash().to_string() for _, data in source.iter_data()] -def _non_nullable(name: str, column: pa.Array) -> pa.Table: +def _non_nullable(name: str, column: pa.Array[Any]) -> pa.Table: """A one-column table whose field is declared non-nullable.""" return pa.Table.from_arrays( [column], schema=pa.schema([pa.field(name, column.type, nullable=False)]) @@ -198,9 +205,7 @@ def test_uuid_schema_differs_from_its_storage(self): class TestRejections: def test_zero_row_map_column_is_rejected(self): - table = pa.table( - {"m": pa.array([], pa.map_(pa.large_string(), pa.int64()))} - ) + table = pa.table({"m": pa.array([], pa.map_(pa.large_string(), pa.int64()))}) with pytest.raises(ValueError, match="map"): _hash_table(table) @@ -218,9 +223,7 @@ def test_non_nullable_column_holding_a_null_is_rejected(self): ArrowTableSource(table, tag_columns=["k"]) def test_duplicate_column_names_are_rejected(self): - table = pa.Table.from_arrays( - [pa.array([1]), pa.array([2])], names=["x", "x"] - ) + table = pa.Table.from_arrays([pa.array([1]), pa.array([2])], names=["x", "x"]) with pytest.raises(ValueError, match="Ambiguous node path 'x'"): get_default_context().semantic_hasher.hash_object(table) From 6dfb456c2cb2c687f61ff13a456a00476ed9fbfe Mon Sep 17 00:00:00 2001 From: "Edgar Y. Walker" <> Date: Wed, 11 Nov 2026 15:53:39 -0800 Subject: [PATCH 09/21] docs(plan): ITL-910 D1 design (explicit failure on foreign hash labels) Co-Authored-By: Claude Opus 5.5 --- .../2026-11-11-itl-910-adopt-starfix-05.md | 23 +++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/superpowers/plans/2026-11-11-itl-910-adopt-starfix-05.md b/superpowers/plans/2026-11-11-itl-910-adopt-starfix-05.md index 96bb966f7..97b835e5f 100644 --- a/superpowers/plans/2026-11-11-itl-910-adopt-starfix-05.md +++ b/superpowers/plans/2026-11-11-itl-910-adopt-starfix-05.md @@ -166,3 +166,26 @@ duplicate too. ### Task 7: PR - [ ] Merge origin/main, push, PR (`Fixes ITL-910`, mentions ITL-848, ITL-927), CodeRabbit rounds. + +## D1 = D (developer, 2026-11-11): fail explicitly on rows hashed under another label + +Design (tests first): +- New public `orcapod.errors.StoreHashLabelError(StoreError)` (always fails the run, whatever + `error_policy`): attributes `record_path`, `found_label`, `current_label`; message names the table, + both labels and the remedy (fresh store, or keep the version that wrote it; migration tool planned). + No opt-out (`ignore_schema` does not disable it). +- Helpers in `src/orcapod/hashing/stored_hash_labels.py` return what they find (no raising): + `foreign_arrow_label(values, current_label)` for `b"