diff --git a/CHANGELOG.md b/CHANGELOG.md index c237135..9bf6237 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,9 +1,6 @@ # Changelog -## Unreleased (proposed 0.5.0) - -The package version comes from the git tag at release (hatch-vcs); nothing in the source -pins it to 0.5.0. +## 0.5.0 (2026-11-11) ### Changed — hash format 2 (every hash changes) @@ -56,6 +53,15 @@ buffers: at 1M rows, `list` takes about 20 ms instead of an estimated 90 (canonical forms; a Python-defined type's serializer must reproduce the stored bytes); see the README section "Extension types". - Golden vectors `tests/golden/format2_v0.5.json`, generated by Rust and asserted in full. +- **`FixedSizeList` data** (ITL-927), byte-identical to the Rust crate: record batches, + streams and `hash_array` accept `FixedSizeList` data (any size, also 0; at any depth; + dictionary-encoded; as extension storage such as `arrow.fixed_shape_tensor`). + `FixedSizeList` is a different type from `List` / `LargeList`, from + `FixedSizeList` (`m != n`) and from `FixedSizeBinary(n)`: equal values hash + differently. A fixed-size list node's entry is its validity if nullable and nothing else (no + lengths, as `n` is in the schema); its element node's elements are the `n` items of every + effectively valid slot. Items under null slots and child positions past the last slot never + affect the hash. Pinned by the 73 `fsl_*` golden vectors and 34 relations. ### Changed (breaking) @@ -64,6 +70,7 @@ buffers: at 1M rows, `list` takes about 20 ms instead of an estimated 90 pyarrow version only from 19 on. - In every mode, `ValueError` is raised for schemas with duplicate or `/`-aliased field paths; for `ListView`, `LargeListView`, `Union` and `RunEndEncoded` types; for hashing data of - `Null`, `FixedSizeList`, `Map`, `Interval` and view types (previously `NotImplementedError` - or accepted); for a non-nullable field holding a null that no enclosing struct null explains; - and for a streamed batch whose schema differs from the digester's. + `Null`, `Map`, `Interval` and view types, also as the items of a `FixedSizeList` + (previously `NotImplementedError` or accepted); for a non-nullable field holding a null that + no enclosing struct null explains; and for a streamed batch whose schema differs from the + digester's. diff --git a/README.md b/README.md index ca17b31..0b0e45c 100644 --- a/README.md +++ b/README.md @@ -64,7 +64,7 @@ Keyword-only, all `False` by default, the same names as the Rust `HasherConfig` | Option | Effect | |---|---| | `ordered_columns` | The order of the top-level columns is part of the hash. | -| `ordered_struct_fields` | The order of struct fields, at every depth (struct children, list elements, map keys and values, dictionary values), is part of the hash. | +| `ordered_struct_fields` | The order of struct fields, at every depth (struct children, list and fixed-size list elements, map keys and values, dictionary values), is part of the hash. | | `include_metadata` | Schema- and field-level Arrow metadata is part of the hash. A schema without metadata hashes the same either way. | ```python @@ -77,13 +77,22 @@ digest = ArrowDigester.hash_table(table, ordered_columns=True, ordered_struct_fi DataFrame, or a `pa.StructArray` whose type `struct` differs from `struct`. - Hashes taken with different order options never coincide, even when order cannot matter. - With `include_metadata`, metadata is keyed by each field's node path (column `c`, struct - child `c/x`, list element `c/`, map key and value `m//key` and `m//value`). Extension types + child `c/x`, list or fixed-size list element `c/`, map key and value `m//key` and `m//value`). Extension types are hashed as their storage type; with `include_metadata` their `ARROW:extension:name` and `ARROW:extension:metadata` are hashed too (see [Extension types](#extension-types)). - `hash_array` has no `include_metadata` on purpose: an array carries no schema or field metadata. It accepts a `ChunkedArray` (hashed like the combined array). - A digester is not thread-safe; a batch rejected by `update` leaves it unchanged. +### Fixed-size lists + +`FixedSizeList` data is hashed (any size `n`, also 0; at any depth; dictionary-encoded; +as the storage of extension types such as `arrow.fixed_shape_tensor`). It is its own type: it +never hashes like `List` / `LargeList` with the same values, like a fixed-size list of +another size, or like `FixedSizeBinary(n)`. The items under a null slot (or under a slot +masked by a null struct) never affect the hash, and neither do child positions past the last +slot; a null of a non-nullable item there is accepted, a visible one is rejected. + ### Extension types With `include_metadata`, an extension type's field metadata is hashed **as stored**, including @@ -117,9 +126,9 @@ rebuilds the type from them, so starfix-python hashes what pyarrow can observe: within a struct, or a name containing `/` that aliases another node (a column `"a/b"` next to a struct column `a` with a child `b`). A `/` that aliases nothing (`"km/h"`) is fine; - `ListView`, `LargeListView`, `Union` and `RunEndEncoded` types; -- hashing data of a schema that contains `Null`, `FixedSizeList`, `Map`, `Interval`, - `Utf8View` or `BinaryView` (their schemas hash; their data does not), whatever the number of - rows; +- hashing data of a schema that contains `Null`, `Map`, `Interval`, `Utf8View` or `BinaryView` + (their schemas hash; their data does not), also inside a fixed-size list, whatever the number + of rows; - a non-nullable field holding a null that no enclosing struct null explains; - with `include_metadata`: a duplicate metadata key on a field or the schema, or explicit `ARROW:extension:*` keys that conflict with the field's registered extension type; @@ -144,8 +153,10 @@ crate has the same restriction. Hashing takes time linear in the data size and works on Arrow buffers. Leaf values are fed to the digest in bounded chunks without copying the payload, but auxiliary memory is not bounded: it grows with the number of visited elements and list items (validity bitmaps, list lengths, -list parent indices), and the values of a dictionary of struct or list values are materialized -(decoded with `take`) before they are traversed. Fixed-width, Boolean and list columns hash at tens of millions of rows per second. +list parent indices), and the values of a dictionary of struct, list or fixed-size list values +are materialized (decoded with `take`) before they are traversed. A fixed-size list column +without null slots is hashed from one zero-copy slice of its child; with null slots, its items +are selected in chunks of about one million items. Fixed-width, Boolean and list columns hash at tens of millions of rows per second. String and binary columns hash at about 3 to 5 million rows per second, because each value is visited in Python. Each `update` call also costs a few microseconds per column, to check the batch's schema. starfix-python is a pure-Python implementation; the Rust crate is faster on diff --git a/docs/metamorphic/specs/2026-11-11-fixed-size-list-python.md b/docs/metamorphic/specs/2026-11-11-fixed-size-list-python.md new file mode 100644 index 0000000..d729160 --- /dev/null +++ b/docs/metamorphic/specs/2026-11-11-fixed-size-list-python.md @@ -0,0 +1,65 @@ +# FixedSizeList data in starfix-python (ITL-927) + +**Date:** 2026-11-11 +**Issue:** ITL-927 +**Repos:** nauticalab/starfix (Rust, authoritative), nauticalab/starfix-python (this repo) +**Status:** Layout confirmed by the maintainer on 2026-11-11; ships in 0.5.0. + +## Source of truth + +The layout is starfix's `docs/byte-layout-spec.md` (Section 3.2, Example 3b) and its design +`docs/metamorphic/specs/2026-11-11-fixed-size-list-design.md`. The 73 `fsl_*` vectors and 34 +relations of `tests/golden/format2_v0.5.json` are generated by Rust and copied here byte for +byte. This note records only the Python-specific decisions. + +## Layout (summary) + +- A fixed-size-list node's entry is its effective validity if the field is nullable and empty + otherwise: no lengths digest (`n` is in the schema digest) and no values. +- Its element node `p/` gets the `n` items of every effectively valid slot, in order. Items of + slots that are null (own validity, or masked by a null enclosing struct) and child positions + past `len * n` are never visited. Struct masking restarts below the list, as for List. +- A null of a non-nullable element under a null slot is accepted; a visible one is rejected. + +## Implementation (`src/starfix/_format2.py`) + +- `_Entry` kind `fixed_size_list`: validity only. +- `_visit_fixed_size_list_items`: slot `i` covers + `arr.values[(arr.offset + i) * n, (arr.offset + i + 1) * n)`, because pyarrow's + `FixedSizeListArray.values` **ignores the array offset**. The items of all slots are one + zero-copy slice of the child. + - No null slot (`present is None`) or `n = 0`: the slice is visited once, without any + selection. No per-item index, mask or copy is built (pinned by + `test_fixed_size_list_without_nulls_builds_no_item_index`). + - Otherwise slots are visited in chunks of about `_ITEM_BUDGET` (1M) items. A chunk without + elements is skipped; a chunk of elements only is visited without a selection; any other + chunk is visited with the slot bits repeated `n` times as the item selection, which the + existing select-aware code applies to every node below (validity, non-nullable check, + lengths, values). +- The repeated bits come from `take(bits, list_parent_indices(FixedSizeList[n]))` over + a validity-free list of nulls: `list_parent_indices` on the real array **skips null slots** + (pyarrow 19 to 26), and `flatten()` skips own-null slots but not struct-masked ones, so + neither is used on the hashed array. +- Leaf values below are hashed by the existing code: fixed-width values through the zero-copy + unsigned-integer / `FixedSizeBinary` view, so no kernel of the logical type is needed (pyarrow + 19 and 20 have no `filter` kernel for decimal32 / decimal64). +- A dictionary of fixed-size lists is decoded per row with `take` before it is traversed (as + for lists and structs); hidden child nulls are not validated. + +## Tests + +- `tests/test_golden_parity_format2.py`: all 496 vectors and 148 relations, no skips. The IPC + payloads are read without full validation (the hidden-null vectors hold nulls of + non-nullable fields under null slots). +- `tests/test_format2.py`: spec Example 3b byte for byte (record batch and `hash_array`), the + non-nullable entry, `n = 0`, slices and trailing child positions on locally built arrays + (IPC normalizes both, so golden vectors cannot pin them), struct masking at two depths with + a different offset at every level, non-nullable slots and items, dictionary of fixed-size + lists, chunked item selection at any chunk size. +- `tests/test_format2_type_matrix.py`: fixed-size lists (`n = 0, 1, 2`, a null slot, trailing + child positions) of every supported leaf type, nested in list / struct / dictionary and + containing struct / list / fixed-size list, each compared with a reference holding other + content under the null slot and no trailing positions, through `hash_array`, record batch + and stream, sliced and chunked. CI runs it on pyarrow 19.0.0, 20.0.0 and the latest. +- A differential run against the Rust branch (random fixed-size-list tables via IPC) is + reported on the PR; it is not part of the repository. diff --git a/src/starfix/_format2.py b/src/starfix/_format2.py index e3599be..cc5d82d 100644 --- a/src/starfix/_format2.py +++ b/src/starfix/_format2.py @@ -46,6 +46,7 @@ def _require_little_endian() -> None: _U64 = struct.Struct(" int: @@ -422,7 +423,14 @@ class _Entry: def __init__(self, field: pa.Field) -> None: t = _logical(field.type) - self.kind = "struct" if pa.types.is_struct(t) else "list" if _is_list(t) else "leaf" + if pa.types.is_struct(t): + self.kind = "struct" + elif _is_list(t): + self.kind = "list" + elif pa.types.is_fixed_size_list(t): + self.kind = "fixed_size_list" # Section 3.2: validity only (n is in the schema) + else: + self.kind = "leaf" self.validity = _Bits() if field.nullable else None self.lengths = hashlib.sha256() if self.kind == "list" else None self.values = hashlib.sha256() if self.kind == "leaf" else None @@ -663,9 +671,9 @@ def _visit( """Feed the elements of the node `field` at `path` into its entry (Section 3.2). `arr` holds the node's candidate elements; `select` (None: all) marks which of them are - elements at all: below a list, the items of valid list values (items of null or masked - list values are not visited). `enclosing` is the AND of the enclosing struct validities - at this list level (None: all valid). Value payloads (string/binary bytes, list + elements at all: below a list or fixed-size list, the items of valid list values (items + of null or masked list values are not visited). `enclosing` is the AND of the enclosing + struct validities at this list level (None: all valid). Value payloads (string/binary bytes, list children) are not copied: list children are zero-copy slices of the original values with a selection mask. Per-element metadata is still allocated: validity bitmaps, list lengths cast to u64, and list_parent_indices (8 bytes per list item). @@ -721,13 +729,57 @@ def _visit( _visit(arr.type.value_field, items, path + "/", None, entries, item_select) return + if entry.kind == "fixed_size_list": + _visit_fixed_size_list_items(arr, present, path + "/", entries) + return + _hash_values(arr, present, entry.values) +def _visit_fixed_size_list_items(arr: pa.FixedSizeListArray, present: pa.Array | None, path: str, + entries: dict) -> None: + """The element node of a fixed-size list (Section 3.2): the n items of every slot marked + in `present` (None: all), in order. Items of other slots and child positions past + len * n are not elements and are never visited. + + pyarrow's ``FixedSizeListArray.values`` ignores the array offset, so slot i covers + ``values[(offset + i) * n, (offset + i + 1) * n)``. The items are visited zero-copy as + one slice of the child when all slots are elements (no per-item index or mask is built); + otherwise the slots are visited in chunks of about _ITEM_BUDGET items (entries are + streams, so chunks hash like one pass), each chunk with the slot bits repeated n times as + the item selection, skipping chunks without any element and selecting nothing in chunks + of elements only. As for List, a fixed-size-list boundary resets struct masking. + """ + n = arr.type.list_size + element = arr.type.value_field + items = arr.values.slice(arr.offset * n, len(arr) * n) + if present is None or n == 0: + _visit(element, items, path, None, entries) + return + slots = max(1, _ITEM_BUDGET // n) + for start in range(0, len(arr), slots): + chosen = present.slice(start, slots) + count = len(chosen) + valid = chosen.true_count + if valid == 0: + continue + item_select = None + if valid < count: + item_select = _repeat_each(chosen, n) + _visit(element, items.slice(start * n, count * n), path, None, entries, item_select) + + +def _repeat_each(bits: pa.Array, n: int) -> pa.Array: + """Every bit of `bits` n times in a row. The parent indices come from a FixedSizeList of + nulls without validity (list_parent_indices skips null fixed-size-list slots).""" + slots = pa.FixedSizeListArray.from_arrays(pa.nulls(len(bits) * n), n) + return pc.take(bits, pc.list_parent_indices(slots)) + + def _is_unsupported_data(t: pa.DataType) -> bool: """Section 8: types whose schema hashes but whose data does not.""" return ( - pa.types.is_null(t) or pa.types.is_fixed_size_list(t) or pa.types.is_map(t) + pa.types.is_null(t) or pa.types.is_map(t) or str(t) in _INTERVALS or str(t) in ("string_view", "binary_view") ) diff --git a/src/starfix/arrow_digester.py b/src/starfix/arrow_digester.py index 415ab49..cdb641c 100644 --- a/src/starfix/arrow_digester.py +++ b/src/starfix/arrow_digester.py @@ -29,9 +29,9 @@ class ArrowDigester: Raises ``ValueError`` (in every mode) for: schemas in which two nodes share a path (duplicate names, or a name containing ``/`` that aliases another node); ListView, LargeListView, Union and RunEndEncoded types; hashing data of a schema containing - Null, FixedSizeList, Map, Interval or view types (their schemas hash), whatever the - number of rows; a non-nullable field holding a null that no enclosing struct null - explains; duplicate metadata keys when ``include_metadata`` is on; and a streamed + Null, Map, Interval or view types (their schemas hash; also inside a FixedSizeList), + whatever the number of rows; a non-nullable field holding a null that no enclosing + struct null explains; duplicate metadata keys when ``include_metadata`` is on; and a streamed batch whose schema differs from the digester's. Raises ``TypeError`` for a wrong container type (for example a Table passed to ``hash_record_batch``). diff --git a/tests/golden/format2_v0.5.json b/tests/golden/format2_v0.5.json index 8d7af9e..5f32da5 100644 --- a/tests/golden/format2_v0.5.json +++ b/tests/golden/format2_v0.5.json @@ -4232,6 +4232,736 @@ "ordered_struct_fields": false, "ipc_b64": "/////3gBAAAQAAAAAAAKAAwACgAJAAQACgAAAAgAAAAAAQQAyP///wQAAAABAAAAFAAAABAAGAAUAA4ADwAEABAACAAQAAAAOAAAACwAAAAAAAEVEAAAAAwBAAAIAAgAAAAEAAgAAAAEAAAAUP///wgAAAAAAAABAQAAAAgAAACQ////wP///xgAAAAMAAAAAAABFcQAAAABAAAACAAAALD////g////GAAAAAwAAAAAAAEVmAAAAAEAAAAYAAAA0P///xAAFAAQAA4ADwAEAAAACAAQAAAAHAAAAAwAAAAAAAEVXAAAAAEAAAAcAAAABAAEAAQAAAAQABQAEAAAAA8ABAAAAAgAEAAAABgAAAAgAAAAAAAAAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAAEAAAAaXRlbQAAAAAEAAAAaXRlbQAAAAAEAAAAaXRlbQAAAAAEAAAAaXRlbQAAAAABAAAAeAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////3gBAAAQAAAADAAYABYAFQAEAAgADAAAABwAAACAAgAAAAAAAAAAAAAAAgQACAAKAAAABAAIAAAAEAAAAAAACgAYAAwACAAEAAoAAABsAAAAEAAAAAMAAAAAAAAAAAAAAAUAAAADAAAAAAAAAAEAAAAAAAAAAwAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAAAAAADAAAAAAAAAAEAAAAAAAAAAwAAAAAAAAABAAAAAAAAAAAAAAAKAAAAAAAAAAAAAAABAAAAAAAAAEAAAAAAAAAAIAAAAAAAAACAAAAAAAAAAAEAAAAAAAAAwAAAAAAAAAAgAAAAAAAAAAABAAAAAAAAAQAAAAAAAABAAQAAAAAAACAAAAAAAAAAgAEAAAAAAAABAAAAAAAAAMABAAAAAAAAIAAAAAAAAAAAAgAAAAAAAAEAAAAAAAAAQAIAAAAAAAAMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAACAAAAAAAAAAMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAIAAAAAAAAAAwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAgAAAAAAAAADAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAACAAAAAAAAAAMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAGAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAcAAAAIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////+4AAAAEAAAAAwAGgAYABcABAAIAAwAAAAgAAAAgAAAAAAAAAAAAAAAAAAAAwQACgAYAAwACAAEAAoAAAAsAAAAEAAAAAEAAAAAAAAAAAAAAAEAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAIAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAACAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAACwAAAAQAAAAAQAAAAAAAAAAAAAAAQAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAAAAAAAAAAAAAQAAAAAAAABAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/////uAAAABAAAAAMABoAGAAXAAQACAAMAAAAIAAAAIAAAAAAAAAAAAAAAAAAAAMEAAoAGAAMAAgABAAKAAAALAAAABAAAAABAAAAAAAAAAAAAAABAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAACAAAAAAAAAAAAAAABAAAAAAAAAEAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////8AAAAA", "expected_hash": "000002372a0e5b060491779a0d6e3405bbfa0326c1484fa0b6ddfd1ccab02915e585dc" + }, + { + "id": "fsl_spec_example_rb_f0", + "description": "f: FixedSizeList? rows [[1, null], null, [3, 4]]; child [1, null, 7, 8, 3, 4] (spec worked example)", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA1P///yAAAAAMAAAAAAABEGQAAAABAAAAJAAAAAAABgAIAAQABgAAAAIAAAAQABQAEAAOAA8ABAAAAAgAEAAAABgAAAAgAAAAAAABAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAAEAAAAaXRlbQAAAAABAAAAZgAAAAAAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAADAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAAAwAAAAAAAAAAAAAAAgAAAAMAAAAAAAAAAQAAAAAAAAAGAAAAAAAAAAEAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAABAAAAAAAAAIAAAAAAAAAAGAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD0AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAcAAAAIAAAAAwAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "000002e0b5ce37cf5204e24c40101db9cfae979021334d99aea08e0408f8174f841f3b" + }, + { + "id": "fsl_spec_example_hidden_null_rb_f0", + "description": "same rows; the null slot covers child [null, null]", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA1P///yAAAAAMAAAAAAABEGQAAAABAAAAJAAAAAAABgAIAAQABgAAAAIAAAAQABQAEAAOAA8ABAAAAAgAEAAAABgAAAAgAAAAAAABAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAAEAAAAaXRlbQAAAAABAAAAZgAAAAAAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAADAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAAAwAAAAAAAAAAAAAAAgAAAAMAAAAAAAAAAQAAAAAAAAAGAAAAAAAAAAMAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAABAAAAAAAAAIAAAAAAAAAAGAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAwAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "000002e0b5ce37cf5204e24c40101db9cfae979021334d99aea08e0408f8174f841f3b" + }, + { + "id": "fsl_spec_example_hidden_zero_rb_f0", + "description": "same rows; the null slot covers child [0, 0]", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA1P///yAAAAAMAAAAAAABEGQAAAABAAAAJAAAAAAABgAIAAQABgAAAAIAAAAQABQAEAAOAA8ABAAAAAgAEAAAABgAAAAgAAAAAAABAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAAEAAAAaXRlbQAAAAABAAAAZgAAAAAAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAADAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAAAwAAAAAAAAAAAAAAAgAAAAMAAAAAAAAAAQAAAAAAAAAGAAAAAAAAAAEAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAABAAAAAAAAAIAAAAAAAAAAGAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD0AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAwAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "000002e0b5ce37cf5204e24c40101db9cfae979021334d99aea08e0408f8174f841f3b" + }, + { + "id": "fsl_spec_example_visible_rb_f0", + "description": "f: FixedSizeList? rows [[1, null], null, [3, 5]] (one visible item differs)", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA1P///yAAAAAMAAAAAAABEGQAAAABAAAAJAAAAAAABgAIAAQABgAAAAIAAAAQABQAEAAOAA8ABAAAAAgAEAAAABgAAAAgAAAAAAABAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAAEAAAAaXRlbQAAAAABAAAAZgAAAAAAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAADAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAAAwAAAAAAAAAAAAAAAgAAAAMAAAAAAAAAAQAAAAAAAAAGAAAAAAAAAAEAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAABAAAAAAAAAIAAAAAAAAAAGAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD0AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAcAAAAIAAAAAwAAAAUAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "0000026aa9cb7d87f1e792e2d3fe3eebdbdf19f05de42824f370f0ea123008d5c32e66" + }, + { + "id": "fsl_spec_example_as_list_rb_f0", + "description": "f: List? rows [[1, null], null, [3, 4]] (same values as a variable-size list)", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA3P///xwAAAAMAAAAAAABDFwAAAABAAAAHAAAAAQABAAEAAAAEAAUABAADgAPAAQAAAAIABAAAAAYAAAAIAAAAAAAAQIcAAAACAAMAAQACwAIAAAAIAAAAAAAAAEAAAAABAAAAGl0ZW0AAAAAAQAAAGYAAAAAAAAAAAAAAAAAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAAAAAQAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAAAwAAAAAAAAAAAAAAAgAAAAMAAAAAAAAAAQAAAAAAAAAEAAAAAAAAAAEAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAAQAAAAAAAAAIAAAAAAAAAAAQAAAAAAAADAAAAAAAAAABAAAAAAAAAABQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAAAAgAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAANAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAADAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////8AAAAA", + "expected_hash": "000002eacfe7035fe13127173192e27c5a044ed2866ff040d88a2872e084396e95cf40" + }, + { + "id": "fsl_spec_example_as_large_list_rb_f0", + "description": "f: LargeList? rows [[1, null], null, [3, 4]]", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA3P///xwAAAAMAAAAAAABFVwAAAABAAAAHAAAAAQABAAEAAAAEAAUABAADgAPAAQAAAAIABAAAAAYAAAAIAAAAAAAAQIcAAAACAAMAAQACwAIAAAAIAAAAAAAAAEAAAAABAAAAGl0ZW0AAAAAAQAAAGYAAAAAAAAAAAAAAAAAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAAAAAQAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAAAwAAAAAAAAAAAAAAAgAAAAMAAAAAAAAAAQAAAAAAAAAEAAAAAAAAAAEAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAAgAAAAAAAAAIAAAAAAAAAAAQAAAAAAAADAAAAAAAAAABAAAAAAAAAABQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAAAAAAACAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAANAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAADAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////8AAAAA", + "expected_hash": "000002eacfe7035fe13127173192e27c5a044ed2866ff040d88a2872e084396e95cf40" + }, + { + "id": "fsl_spec_example_rb_f3", + "description": "spec worked example, ordered_columns + ordered_struct_fields", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": true, + "ordered_struct_fields": true, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA1P///yAAAAAMAAAAAAABEGQAAAABAAAAJAAAAAAABgAIAAQABgAAAAIAAAAQABQAEAAOAA8ABAAAAAgAEAAAABgAAAAgAAAAAAABAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAAEAAAAaXRlbQAAAAABAAAAZgAAAAAAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAADAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAAAwAAAAAAAAAAAAAAAgAAAAMAAAAAAAAAAQAAAAAAAAAGAAAAAAAAAAEAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAABAAAAAAAAAIAAAAAAAAAAGAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD0AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAcAAAAIAAAAAwAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "00000292d582a3316ec9b4cf876d417b356dc41413cd190a46eb8df4d492da02c33ca1" + }, + { + "id": "fsl_spec_example_schema_f0", + "description": "schema f: FixedSizeList?", + "method": "hash_schema", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA1P///yAAAAAMAAAAAAABEGQAAAABAAAAJAAAAAAABgAIAAQABgAAAAIAAAAQABQAEAAOAA8ABAAAAAgAEAAAABgAAAAgAAAAAAABAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAAEAAAAaXRlbQAAAAABAAAAZgAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "0000026f5192f92fc173927559a21ec8ef3d64508fb5d2817384510bb678f30b5b3c59" + }, + { + "id": "fsl_spec_example_as_list_schema_f0", + "description": "schema f: List?", + "method": "hash_schema", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA3P///xwAAAAMAAAAAAABDFwAAAABAAAAHAAAAAQABAAEAAAAEAAUABAADgAPAAQAAAAIABAAAAAYAAAAIAAAAAAAAQIcAAAACAAMAAQACwAIAAAAIAAAAAAAAAEAAAAABAAAAGl0ZW0AAAAAAQAAAGYAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "00000254160c7da2238e36f0d5d7ad94ab4cc06310bfa855df5a4d2d8a4788ea284031" + }, + { + "id": "fsl_spec_example_array_f0", + "description": "hash_array FixedSizeList [[1, null], null, [3, 4]], child [1, null, 7, 8, 3, 4]", + "method": "hash_array", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA1P///yAAAAAMAAAAAAABEGQAAAABAAAAJAAAAAAABgAIAAQABgAAAAIAAAAQABQAEAAOAA8ABAAAAAgAEAAAABgAAAAgAAAAAAABAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAAEAAAAaXRlbQAAAAAFAAAAYXJyYXkAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAADAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAAAwAAAAAAAAAAAAAAAgAAAAMAAAAAAAAAAQAAAAAAAAAGAAAAAAAAAAEAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAABAAAAAAAAAIAAAAAAAAAAGAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD0AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAcAAAAIAAAAAwAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "000002643049e32b09a1afd9ef25cc247bd4591b93812aeafb01dcf40e3b67931b86a8" + }, + { + "id": "fsl_spec_example_hidden_null_array_f0", + "description": "hash_array, the null slot covers child [null, null]", + "method": "hash_array", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA1P///yAAAAAMAAAAAAABEGQAAAABAAAAJAAAAAAABgAIAAQABgAAAAIAAAAQABQAEAAOAA8ABAAAAAgAEAAAABgAAAAgAAAAAAABAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAAEAAAAaXRlbQAAAAAFAAAAYXJyYXkAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAADAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAAAwAAAAAAAAAAAAAAAgAAAAMAAAAAAAAAAQAAAAAAAAAGAAAAAAAAAAMAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAABAAAAAAAAAIAAAAAAAAAAGAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAwAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "000002643049e32b09a1afd9ef25cc247bd4591b93812aeafb01dcf40e3b67931b86a8" + }, + { + "id": "fsl_spec_example_as_list_array_f0", + "description": "hash_array List [[1, null], null, [3, 4]]", + "method": "hash_array", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA3P///xwAAAAMAAAAAAABDFwAAAABAAAAHAAAAAQABAAEAAAAEAAUABAADgAPAAQAAAAIABAAAAAYAAAAIAAAAAAAAQIcAAAACAAMAAQACwAIAAAAIAAAAAAAAAEAAAAABAAAAGl0ZW0AAAAABQAAAGFycmF5AAAAAAAAAAAAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAAAAAQAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAAAwAAAAAAAAAAAAAAAgAAAAMAAAAAAAAAAQAAAAAAAAAEAAAAAAAAAAEAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAAQAAAAAAAAAIAAAAAAAAAAAQAAAAAAAADAAAAAAAAAABAAAAAAAAAABQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAAAAgAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAANAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAADAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////8AAAAA", + "expected_hash": "00000207fb5ee6844f08f08e223aafff6244df94a0a9844dfb2395a676584715374eb1" + }, + { + "id": "fsl_root_array_f0", + "description": "hash_array FixedSizeList [[1, 2], [3, 4]] (root is always nullable)", + "method": "hash_array", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA1P///yAAAAAMAAAAAAAAEGQAAAABAAAAJAAAAAAABgAIAAQABgAAAAIAAAAQABQAEAAAAA8ABAAAAAgAEAAAABgAAAAgAAAAAAAAAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAAEAAAAaXRlbQAAAAAFAAAAYXJyYXkAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAADAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAAAgAAAAAAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAABAAAAAAAAAIAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAgAAAAMAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "0000027c88e88899a3d5fcd1d8613505d7f1a694797ff9bf6ebf3721ff9911a86601bb" + }, + { + "id": "fsl_root_as_list_array_f0", + "description": "hash_array List [[1, 2], [3, 4]]", + "method": "hash_array", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA3P///xwAAAAMAAAAAAAADFwAAAABAAAAHAAAAAQABAAEAAAAEAAUABAAAAAPAAQAAAAIABAAAAAYAAAAIAAAAAAAAAIcAAAACAAMAAQACwAIAAAAIAAAAAAAAAEAAAAABAAAAGl0ZW0AAAAABQAAAGFycmF5AAAAAAAAAAAAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAAAAAQAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAAAgAAAAAAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAAMAAAAAAAAAIAAAAAAAAAAAQAAAAAAAADAAAAAAAAAABAAAAAAAAAA/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAIAAAADAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////8AAAAA", + "expected_hash": "000002360ce4e41c1edd4a844c734b44f3125fdf5191af2e3be8d2163cbd43e53a70f9" + }, + { + "id": "fsl_size1_rb_f0", + "description": "f: FixedSizeList non-null rows [[1], [2], [3], [4]] (same child values)", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA1P///yAAAAAMAAAAAAAAEGQAAAABAAAAJAAAAAAABgAIAAQABgAAAAEAAAAQABQAEAAAAA8ABAAAAAgAEAAAABgAAAAgAAAAAAAAAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAAEAAAAaXRlbQAAAAABAAAAZgAAAAAAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAADAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAABAAAAAAAAAAAAAAAAgAAAAQAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAABAAAAAAAAAIAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAgAAAAMAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "0000029651fc8d4305d33693aaa4c90cbfe51d5e6a6b6c55fe9cfc1cc948841614c525" + }, + { + "id": "fsl_size2_rb_f0", + "description": "f: FixedSizeList non-null rows [[1, 2], [3, 4]] (same child values)", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA1P///yAAAAAMAAAAAAAAEGQAAAABAAAAJAAAAAAABgAIAAQABgAAAAIAAAAQABQAEAAAAA8ABAAAAAgAEAAAABgAAAAgAAAAAAAAAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAAEAAAAaXRlbQAAAAABAAAAZgAAAAAAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAADAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAAAgAAAAAAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAABAAAAAAAAAIAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAgAAAAMAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "0000028be2f0225fe8a574a5bb4835101a019111e4187dfe1457796471a866eecbfc2b" + }, + { + "id": "fsl_size4_rb_f0", + "description": "f: FixedSizeList non-null rows [[1, 2, 3, 4]] (same child values)", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA1P///yAAAAAMAAAAAAAAEGQAAAABAAAAJAAAAAAABgAIAAQABgAAAAQAAAAQABQAEAAAAA8ABAAAAAgAEAAAABgAAAAgAAAAAAAAAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAAEAAAAaXRlbQAAAAABAAAAZgAAAAAAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAADAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAAAQAAAAAAAAAAAAAAAgAAAAEAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAABAAAAAAAAAIAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAgAAAAMAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "000002f35a7ed29e361bd7fa4753e6b2179947a3e894f254dd71dda864b362218b89e3" + }, + { + "id": "fsl_size2_empty_stream_f0", + "description": "stream with no batches, f: FixedSizeList non-null", + "method": "hash_stream", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA1P///yAAAAAMAAAAAAAAEGQAAAABAAAAJAAAAAAABgAIAAQABgAAAAIAAAAQABQAEAAAAA8ABAAAAAgAEAAAABgAAAAgAAAAAAAAAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAAEAAAAaXRlbQAAAAABAAAAZgAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "0000027028e220dbd2834f765179d5bf58e7c4f4f3101e8481f51e8ea4c28e1077fd70" + }, + { + "id": "fsl_size2_empty_array_f0", + "description": "hash_array of an empty FixedSizeList", + "method": "hash_array", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA1P///yAAAAAMAAAAAAAAEGQAAAABAAAAJAAAAAAABgAIAAQABgAAAAIAAAAQABQAEAAAAA8ABAAAAAgAEAAAABgAAAAgAAAAAAAAAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAAEAAAAaXRlbQAAAAAFAAAAYXJyYXkAAAAAAAAA/////7gAAAAQAAAAAAAKAA4ADAALAAQACgAAABQAAAAAAAADBAAKAAwAAAAIAAQACgAAADAAAAAEAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "000002d8fd64d09a1bd5f22332895ca9c706f0ec4ad10d69dd1ff0faaedbc8e36d0652" + }, + { + "id": "fsl_size3_empty_stream_f0", + "description": "stream with no batches, f: FixedSizeList non-null", + "method": "hash_stream", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA1P///yAAAAAMAAAAAAAAEGQAAAABAAAAJAAAAAAABgAIAAQABgAAAAMAAAAQABQAEAAAAA8ABAAAAAgAEAAAABgAAAAgAAAAAAAAAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAAEAAAAaXRlbQAAAAABAAAAZgAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "000002b568b63ad4ce77ca91b15e97eec812687622c55a2dfe8af9b651949ed80d2fa1" + }, + { + "id": "fsl_size3_empty_array_f0", + "description": "hash_array of an empty FixedSizeList", + "method": "hash_array", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA1P///yAAAAAMAAAAAAAAEGQAAAABAAAAJAAAAAAABgAIAAQABgAAAAMAAAAQABQAEAAAAA8ABAAAAAgAEAAAABgAAAAgAAAAAAAAAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAAEAAAAaXRlbQAAAAAFAAAAYXJyYXkAAAAAAAAA/////7gAAAAQAAAAAAAKAA4ADAALAAQACgAAABQAAAAAAAADBAAKAAwAAAAIAAQACgAAADAAAAAEAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "0000027d6ef8046cd823d06dad5a1dcd5736e5a8de7282e01af040a99156d8784bc65b" + }, + { + "id": "fsl_fsb2_rb_f0", + "description": "f: FixedSizeBinary(2) non-null rows [0x0102, 0x0304]", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////3gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAUAAAAEAAUABAAAAAPAAQAAAAIABAAAAAcAAAADAAAAAAAAA8YAAAAAAAAAAAABgAIAAQABgAAAAIAAAABAAAAZgAAAAAAAAD/////uAAAABAAAAAMABoAGAAXAAQACAAMAAAAIAAAAIAAAAAAAAAAAAAAAAAAAAMEAAoAGAAMAAgABAAKAAAALAAAABAAAAACAAAAAAAAAAAAAAABAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAACAAAAAAAAAAAAAAABAAAAAAAAAEAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQIDBAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////8AAAAA", + "expected_hash": "00000286b04c26856082882c8d9d12c322b40a7cb3f5f5162890e8b83697c0c93ec394" + }, + { + "id": "fsl_uint8x2_rb_f0", + "description": "f: FixedSizeList non-null rows [[1, 2], [3, 4]]", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAUAAAAEAAUABAAAAAPAAQAAAAIABAAAAAYAAAADAAAAAAAABBYAAAAAQAAABwAAADS////AgAAABAAFgAQAAAADwAEAAAACAAQAAAAGAAAABwAAAAAAAACGAAAAAAABgAIAAQABgAAAAgAAAAAAAAABAAAAGl0ZW0AAAAAAQAAAGYAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAADAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAAAgAAAAAAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAABAAAAAAAAAIAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAgMEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "000002c561070a4d370e740b0217060ef7757a08cb4f438a30183a01fc7b2c38839c1b" + }, + { + "id": "fsl_fsb2_array_f0", + "description": "hash_array FixedSizeBinary(2) [0x0102, 0x0304]", + "method": "hash_array", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////3gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAUAAAAEAAUABAAAAAPAAQAAAAIABAAAAAcAAAADAAAAAAAAA8YAAAAAAAAAAAABgAIAAQABgAAAAIAAAAFAAAAYXJyYXkAAAD/////uAAAABAAAAAMABoAGAAXAAQACAAMAAAAIAAAAIAAAAAAAAAAAAAAAAAAAAMEAAoAGAAMAAgABAAKAAAALAAAABAAAAACAAAAAAAAAAAAAAABAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAACAAAAAAAAAAAAAAABAAAAAAAAAEAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQIDBAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////8AAAAA", + "expected_hash": "00000232ae5ffe26182bea3d4974b5cdc9cbccf7864d80581caa699bc9cab24d16640d" + }, + { + "id": "fsl_uint8x2_array_f0", + "description": "hash_array FixedSizeList [[1, 2], [3, 4]]", + "method": "hash_array", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAUAAAAEAAUABAAAAAPAAQAAAAIABAAAAAYAAAADAAAAAAAABBYAAAAAQAAABwAAADS////AgAAABAAFgAQAAAADwAEAAAACAAQAAAAGAAAABwAAAAAAAACGAAAAAAABgAIAAQABgAAAAgAAAAAAAAABAAAAGl0ZW0AAAAABQAAAGFycmF5AAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAADAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAAAgAAAAAAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAABAAAAAAAAAIAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAgMEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "0000024591c47787b895d96db3bd77376d8359e5cb53949f97cffb0ddbd1ee4942a2e0" + }, + { + "id": "fsl_fsl_hidden_values_rb_f0", + "description": "f: FixedSizeList? non-null, 2>? rows [[[1, 2], [3, 4]], null, [[5, 6], [7, 8]]]; the null slot covers [[9, 9], [9, 9]]", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "//////gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAUAAAAEAAUABAADgAPAAQAAAAIABAAAAAYAAAADAAAAAAAARCUAAAAAQAAAAwAAADa////AgAAANT///8gAAAADAAAAAAAABBkAAAAAQAAACQAAAAAAAYACAAEAAYAAAACAAAAEAAUABAAAAAPAAQAAAAIABAAAAAYAAAAIAAAAAAAAAIcAAAACAAMAAQACwAIAAAAIAAAAAAAAAEAAAAABAAAAGl0ZW0AAAAABAAAAGl0ZW0AAAAAAQAAAGYAAAAAAAAAAAAAAP/////4AAAAEAAAAAwAGgAYABcABAAIAAwAAAAgAAAAAAEAAAAAAAAAAAAAAAAAAwQACgAYAAwACAAEAAoAAABMAAAAEAAAAAMAAAAAAAAAAAAAAAMAAAADAAAAAAAAAAEAAAAAAAAABgAAAAAAAAAAAAAAAAAAAAwAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAQAAAAAAAABAAAAAAAAAAAEAAAAAAAAAgAAAAAAAAAACAAAAAAAAAMAAAAAAAAAAMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAFAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP//AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAgAAAAMAAAAEAAAACQAAAAkAAAAJAAAACQAAAAUAAAAGAAAABwAAAAgAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "000002201c9b4ec3e2c9a6e719b90a6324cb8d4fcde760fc8aa5abc134f74ba5fadd78" + }, + { + "id": "fsl_fsl_hidden_nulls_rb_f0", + "description": "same rows; the null slot covers [null, [null, null]] (nulls of non-nullable fields, hidden)", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "//////gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAUAAAAEAAUABAADgAPAAQAAAAIABAAAAAYAAAADAAAAAAAARCUAAAAAQAAAAwAAADa////AgAAANT///8gAAAADAAAAAAAABBkAAAAAQAAACQAAAAAAAYACAAEAAYAAAACAAAAEAAUABAAAAAPAAQAAAAIABAAAAAYAAAAIAAAAAAAAAIcAAAACAAMAAQACwAIAAAAIAAAAAAAAAEAAAAABAAAAGl0ZW0AAAAABAAAAGl0ZW0AAAAAAQAAAGYAAAAAAAAAAAAAAP/////4AAAAEAAAAAwAGgAYABcABAAIAAwAAAAgAAAAAAEAAAAAAAAAAAAAAAAAAwQACgAYAAwACAAEAAoAAABMAAAAEAAAAAMAAAAAAAAAAAAAAAMAAAADAAAAAAAAAAEAAAAAAAAABgAAAAAAAAABAAAAAAAAAAwAAAAAAAAABAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAQAAAAAAAABAAAAAAAAAAAEAAAAAAAAAgAAAAAAAAAACAAAAAAAAAMAAAAAAAAAAMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAFAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAOwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA8PAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAgAAAAMAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAUAAAAGAAAABwAAAAgAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "000002201c9b4ec3e2c9a6e719b90a6324cb8d4fcde760fc8aa5abc134f74ba5fadd78" + }, + { + "id": "fsl_fsl_hidden_values_array_f0", + "description": "hash_array of the fsl_fsl_hidden_values column", + "method": "hash_array", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "//////gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAUAAAAEAAUABAADgAPAAQAAAAIABAAAAAYAAAADAAAAAAAARCUAAAAAQAAAAwAAADa////AgAAANT///8gAAAADAAAAAAAABBkAAAAAQAAACQAAAAAAAYACAAEAAYAAAACAAAAEAAUABAAAAAPAAQAAAAIABAAAAAYAAAAIAAAAAAAAAIcAAAACAAMAAQACwAIAAAAIAAAAAAAAAEAAAAABAAAAGl0ZW0AAAAABAAAAGl0ZW0AAAAABQAAAGFycmF5AAAAAAAAAP/////4AAAAEAAAAAwAGgAYABcABAAIAAwAAAAgAAAAAAEAAAAAAAAAAAAAAAAAAwQACgAYAAwACAAEAAoAAABMAAAAEAAAAAMAAAAAAAAAAAAAAAMAAAADAAAAAAAAAAEAAAAAAAAABgAAAAAAAAAAAAAAAAAAAAwAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAQAAAAAAAABAAAAAAAAAAAEAAAAAAAAAgAAAAAAAAAACAAAAAAAAAMAAAAAAAAAAMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAFAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP//AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAgAAAAMAAAAEAAAACQAAAAkAAAAJAAAACQAAAAUAAAAGAAAABwAAAAgAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "000002d520da471db6f269fe2796b9f367c6e4e35c0ee6ec125ff6185757243fe558b6" + }, + { + "id": "fsl_fsl_hidden_nulls_array_f0", + "description": "hash_array of the fsl_fsl_hidden_nulls column", + "method": "hash_array", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "//////gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAUAAAAEAAUABAADgAPAAQAAAAIABAAAAAYAAAADAAAAAAAARCUAAAAAQAAAAwAAADa////AgAAANT///8gAAAADAAAAAAAABBkAAAAAQAAACQAAAAAAAYACAAEAAYAAAACAAAAEAAUABAAAAAPAAQAAAAIABAAAAAYAAAAIAAAAAAAAAIcAAAACAAMAAQACwAIAAAAIAAAAAAAAAEAAAAABAAAAGl0ZW0AAAAABAAAAGl0ZW0AAAAABQAAAGFycmF5AAAAAAAAAP/////4AAAAEAAAAAwAGgAYABcABAAIAAwAAAAgAAAAAAEAAAAAAAAAAAAAAAAAAwQACgAYAAwACAAEAAoAAABMAAAAEAAAAAMAAAAAAAAAAAAAAAMAAAADAAAAAAAAAAEAAAAAAAAABgAAAAAAAAABAAAAAAAAAAwAAAAAAAAABAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAQAAAAAAAABAAAAAAAAAAAEAAAAAAAAAgAAAAAAAAAACAAAAAAAAAMAAAAAAAAAAMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAFAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAOwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA8PAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAgAAAAMAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAUAAAAGAAAABwAAAAgAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "000002d520da471db6f269fe2796b9f367c6e4e35c0ee6ec125ff6185757243fe558b6" + }, + { + "id": "fsl_struct_masked_values_rb_f0", + "description": "s: Struct non-null>? rows [{x: [1, 2]}, null]; the null row's x is [7, 8]", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "//////gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAUAAAAEAAUABAADgAPAAQAAAAIABAAAAAcAAAADAAAAAAAAQ2QAAAAAQAAAAwAAAAEAAQABAAAANT///8gAAAADAAAAAAAABBkAAAAAQAAACQAAAAAAAYACAAEAAYAAAACAAAAEAAUABAAAAAPAAQAAAAIABAAAAAYAAAAIAAAAAAAAAIcAAAACAAMAAQACwAIAAAAIAAAAAAAAAEAAAAABAAAAGl0ZW0AAAAAAQAAAHgAAAABAAAAcwAAAAAAAAAAAAAAAAAAAP/////4AAAAEAAAAAwAGgAYABcABAAIAAwAAAAgAAAAAAEAAAAAAAAAAAAAAAAAAwQACgAYAAwACAAEAAoAAABMAAAAEAAAAAIAAAAAAAAAAAAAAAMAAAACAAAAAAAAAAEAAAAAAAAAAgAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAQAAAAAAAABAAAAAAAAAAAEAAAAAAAAAgAAAAAAAAAABAAAAAAAAAMAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAgAAAAcAAAAIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "000002687f08043d5259daef1c74e55d21186dc28d134854e898e61ceba357ff102402" + }, + { + "id": "fsl_struct_masked_nulls_rb_f0", + "description": "same rows; the null row's x is [null, null] (non-nullable item, masked)", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "//////gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAUAAAAEAAUABAADgAPAAQAAAAIABAAAAAcAAAADAAAAAAAAQ2QAAAAAQAAAAwAAAAEAAQABAAAANT///8gAAAADAAAAAAAABBkAAAAAQAAACQAAAAAAAYACAAEAAYAAAACAAAAEAAUABAAAAAPAAQAAAAIABAAAAAYAAAAIAAAAAAAAAIcAAAACAAMAAQACwAIAAAAIAAAAAAAAAEAAAAABAAAAGl0ZW0AAAAAAQAAAHgAAAABAAAAcwAAAAAAAAAAAAAAAAAAAP/////4AAAAEAAAAAwAGgAYABcABAAIAAwAAAAgAAAAAAEAAAAAAAAAAAAAAAAAAwQACgAYAAwACAAEAAoAAABMAAAAEAAAAAIAAAAAAAAAAAAAAAMAAAACAAAAAAAAAAEAAAAAAAAAAgAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAAgAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAQAAAAAAAABAAAAAAAAAAAEAAAAAAAAAgAAAAAAAAAABAAAAAAAAAMAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "000002687f08043d5259daef1c74e55d21186dc28d134854e898e61ceba357ff102402" + }, + { + "id": "fsl_struct2_masked_values_rb_f0", + "description": "s: Struct non-null> non-null>? rows [null, {x: {x: [3, 4]}}]; the null row's list is [7, 8]", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////zgBAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAUAAAAEAAUABAADgAPAAQAAAAIABAAAAAYAAAADAAAAAAAAQ24AAAAAQAAAAgAAADg////sP///xwAAAAMAAAAAAAADZAAAAABAAAADAAAAAQABAAEAAAA1P///yAAAAAMAAAAAAAAEGQAAAABAAAAJAAAAAAABgAIAAQABgAAAAIAAAAQABQAEAAAAA8ABAAAAAgAEAAAABgAAAAgAAAAAAAAAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAAEAAAAaXRlbQAAAAABAAAAeAAAAAEAAAB4AAAAAQAAAHMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/////+AAAABAAAAAMABoAGAAXAAQACAAMAAAAIAAAAEABAAAAAAAAAAAAAAAAAAMEAAoAGAAMAAgABAAKAAAAXAAAABAAAAACAAAAAAAAAAAAAAAEAAAAAgAAAAAAAAABAAAAAAAAAAIAAAAAAAAAAAAAAAAAAAACAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAFAAAAAAAAAAAAAAABAAAAAAAAAEAAAAAAAAAAAQAAAAAAAACAAAAAAAAAAAEAAAAAAAAAwAAAAAAAAAABAAAAAAAAAAABAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAcAAAAIAAAAAwAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/////AAAAAA==", + "expected_hash": "00000267b28e34c0fe8ad42f53689a2b2d14ac0ca15c9db3dd9540d225cc3f16a79911" + }, + { + "id": "fsl_struct2_masked_nulls_rb_f0", + "description": "same rows; the null row's list is [null, null]", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////zgBAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAUAAAAEAAUABAADgAPAAQAAAAIABAAAAAYAAAADAAAAAAAAQ24AAAAAQAAAAgAAADg////sP///xwAAAAMAAAAAAAADZAAAAABAAAADAAAAAQABAAEAAAA1P///yAAAAAMAAAAAAAAEGQAAAABAAAAJAAAAAAABgAIAAQABgAAAAIAAAAQABQAEAAAAA8ABAAAAAgAEAAAABgAAAAgAAAAAAAAAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAAEAAAAaXRlbQAAAAABAAAAeAAAAAEAAAB4AAAAAQAAAHMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/////+AAAABAAAAAMABoAGAAXAAQACAAMAAAAIAAAAEABAAAAAAAAAAAAAAAAAAMEAAoAGAAMAAgABAAKAAAAXAAAABAAAAACAAAAAAAAAAAAAAAEAAAAAgAAAAAAAAABAAAAAAAAAAIAAAAAAAAAAAAAAAAAAAACAAAAAAAAAAAAAAAAAAAABAAAAAAAAAACAAAAAAAAAAAAAAAFAAAAAAAAAAAAAAABAAAAAAAAAEAAAAAAAAAAAQAAAAAAAACAAAAAAAAAAAEAAAAAAAAAwAAAAAAAAAABAAAAAAAAAAABAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAwAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/////AAAAAA==", + "expected_hash": "00000267b28e34c0fe8ad42f53689a2b2d14ac0ca15c9db3dd9540d225cc3f16a79911" + }, + { + "id": "fsl_zero_rb_f0", + "description": "f: FixedSizeList? rows [[], [], []] (no validity buffer)", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA3P///xwAAAAMAAAAAAABEFwAAAABAAAAHAAAAAQABAAEAAAAEAAUABAADgAPAAQAAAAIABAAAAAYAAAAIAAAAAAAAQIcAAAACAAMAAQACwAIAAAAIAAAAAAAAAEAAAAABAAAAGl0ZW0AAAAAAQAAAGYAAAAAAAAAAAAAAAAAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAABAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAAAwAAAAAAAAAAAAAAAgAAAAMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////8AAAAA", + "expected_hash": "000002e483c3fb0d2e81c5a70c91f964b012ec65d15621314f0fedbf98f8016c725dd1" + }, + { + "id": "fsl_zero_two_rows_rb_f0", + "description": "f: FixedSizeList? rows [[], []]", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA3P///xwAAAAMAAAAAAABEFwAAAABAAAAHAAAAAQABAAEAAAAEAAUABAADgAPAAQAAAAIABAAAAAYAAAAIAAAAAAAAQIcAAAACAAMAAQACwAIAAAAIAAAAAAAAAEAAAAABAAAAGl0ZW0AAAAAAQAAAGYAAAAAAAAAAAAAAAAAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAABAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAAAgAAAAAAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////8AAAAA", + "expected_hash": "000002975fb4d6d8723de6394f1712088709ee5f8bff7a84d1044d74b62e751901f229" + }, + { + "id": "fsl_zero_null_middle_rb_f0", + "description": "f: FixedSizeList? rows [[], null, []]", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA3P///xwAAAAMAAAAAAABEFwAAAABAAAAHAAAAAQABAAEAAAAEAAUABAADgAPAAQAAAAIABAAAAAYAAAAIAAAAAAAAQIcAAAACAAMAAQACwAIAAAAIAAAAAAAAAEAAAAABAAAAGl0ZW0AAAAAAQAAAGYAAAAAAAAAAAAAAAAAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAABAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAAAwAAAAAAAAAAAAAAAgAAAAMAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////8AAAAA", + "expected_hash": "00000249d70c393a4ab1f790dd0629842b97a6cf56f6394da9c2ae6bb7c6bff88869ba" + }, + { + "id": "fsl_zero_null_first_rb_f0", + "description": "f: FixedSizeList? rows [null, [], []]", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA3P///xwAAAAMAAAAAAABEFwAAAABAAAAHAAAAAQABAAEAAAAEAAUABAADgAPAAQAAAAIABAAAAAYAAAAIAAAAAAAAQIcAAAACAAMAAQACwAIAAAAIAAAAAAAAAEAAAAABAAAAGl0ZW0AAAAAAQAAAGYAAAAAAAAAAAAAAAAAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAABAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAAAwAAAAAAAAAAAAAAAgAAAAMAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////8AAAAA", + "expected_hash": "000002e389065abd460a0d2da7a88f75c0dc123973b7ed5535953c7d31671ebafff691" + }, + { + "id": "fsl_zero_non_null_rb_f0", + "description": "f: FixedSizeList non-null rows [[], [], []]", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAUAAAAEAAUABAAAAAPAAQAAAAIABAAAAAcAAAADAAAAAAAABBcAAAAAQAAABwAAAAEAAQABAAAABAAFAAQAA4ADwAEAAAACAAQAAAAGAAAACAAAAAAAAECHAAAAAgADAAEAAsACAAAACAAAAAAAAABAAAAAAQAAABpdGVtAAAAAAEAAABmAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAABAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAAAwAAAAAAAAAAAAAAAgAAAAMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////8AAAAA", + "expected_hash": "00000238e3bc63d6f025bc27f0db13ff5368891f28b4c19fe0c0d976abe4feaa6931d9" + }, + { + "id": "fsl_zero_all_valid_rb_f0", + "description": "f: FixedSizeList? rows [[], [], []] with an all-valid validity buffer", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA3P///xwAAAAMAAAAAAABEFwAAAABAAAAHAAAAAQABAAEAAAAEAAUABAADgAPAAQAAAAIABAAAAAYAAAAIAAAAAAAAQIcAAAACAAMAAQACwAIAAAAIAAAAAAAAAEAAAAABAAAAGl0ZW0AAAAAAQAAAGYAAAAAAAAAAAAAAAAAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAABAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAAAwAAAAAAAAAAAAAAAgAAAAMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////8AAAAA", + "expected_hash": "000002e483c3fb0d2e81c5a70c91f964b012ec65d15621314f0fedbf98f8016c725dd1" + }, + { + "id": "fsl_zero_null_middle_array_f0", + "description": "hash_array FixedSizeList [[], null, []]", + "method": "hash_array", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA3P///xwAAAAMAAAAAAABEFwAAAABAAAAHAAAAAQABAAEAAAAEAAUABAADgAPAAQAAAAIABAAAAAYAAAAIAAAAAAAAQIcAAAACAAMAAQACwAIAAAAIAAAAAAAAAEAAAAABAAAAGl0ZW0AAAAABQAAAGFycmF5AAAAAAAAAAAAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAABAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAAAwAAAAAAAAAAAAAAAgAAAAMAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////8AAAAA", + "expected_hash": "000002a3b57ae600389d2f8ce85f1a6192c5891cd5cfc66026b213736ae7139b4daac6" + }, + { + "id": "fsl_zero_null_first_array_f0", + "description": "hash_array FixedSizeList [null, [], []]", + "method": "hash_array", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA3P///xwAAAAMAAAAAAABEFwAAAABAAAAHAAAAAQABAAEAAAAEAAUABAADgAPAAQAAAAIABAAAAAYAAAAIAAAAAAAAQIcAAAACAAMAAQACwAIAAAAIAAAAAAAAAEAAAAABAAAAGl0ZW0AAAAABQAAAGFycmF5AAAAAAAAAAAAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAABAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAAAwAAAAAAAAAAAAAAAgAAAAMAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////8AAAAA", + "expected_hash": "000002411eca15baf35db127e727597640a8fd79910f78820f9134c60dfeb95d2bfcbb" + }, + { + "id": "fsl_zero_array_f0", + "description": "hash_array FixedSizeList [[], [], []]", + "method": "hash_array", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "//////gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAUAAAAEAAUABAAAAAPAAQAAAAIABAAAAAcAAAADAAAAAAAABBcAAAAAQAAABwAAAAEAAQABAAAABAAFAAQAA4ADwAEAAAACAAQAAAAGAAAACAAAAAAAAECHAAAAAgADAAEAAsACAAAACAAAAAAAAABAAAAAAQAAABpdGVtAAAAAAUAAABhcnJheQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////+4AAAAEAAAAAwAGgAYABcABAAIAAwAAAAgAAAAQAAAAAAAAAAAAAAAAAAAAwQACgAYAAwACAAEAAoAAAA8AAAAEAAAAAMAAAAAAAAAAAAAAAIAAAADAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADAAAAAAAAAAAAAAABAAAAAAAAAEAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/////AAAAAA==", + "expected_hash": "000002bdf7e1e55de31e2727bb16d90e92b45cca9b43ef3096bf6cadbf49d34f6c5788" + }, + { + "id": "fsl_in_null_list_hidden_rb_f0", + "description": "l: List?>? rows [[[1, 2]], null, [[3, 4]]]; the null list value covers a null slot over [null, null]", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "//////gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA3P///xwAAAAMAAAAAAABDKQAAAABAAAAHAAAAAQABAAEAAAAEAAUABAADgAPAAQAAAAIABAAAAAgAAAADAAAAAAAARBkAAAAAQAAACQAAAAAAAYACAAEAAYAAAACAAAAEAAUABAAAAAPAAQAAAAIABAAAAAYAAAAIAAAAAAAAAIcAAAACAAMAAQACwAIAAAAIAAAAAAAAAEAAAAABAAAAGl0ZW0AAAAABAAAAGl0ZW0AAAAAAQAAAGwAAAAAAAAAAAAAAP/////4AAAAEAAAAAwAGgAYABcABAAIAAwAAAAgAAAAQAEAAAAAAAAAAAAAAAAAAwQACgAYAAwACAAEAAoAAABMAAAAEAAAAAMAAAAAAAAAAAAAAAMAAAADAAAAAAAAAAEAAAAAAAAAAwAAAAAAAAABAAAAAAAAAAYAAAAAAAAAAgAAAAAAAAAAAAAABQAAAAAAAAAAAAAAAQAAAAAAAABAAAAAAAAAABAAAAAAAAAAgAAAAAAAAAABAAAAAAAAAMAAAAAAAAAAAQAAAAAAAAAAAQAAAAAAABgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAFAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEAAAACAAAAAwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAUAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAzAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAIAAAAAAAAAAAAAAAMAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////8AAAAA", + "expected_hash": "0000021e1ecfc99a033b00e24f700b119b3d40ccb06fc0ced08ca5742e348a895c11dc" + }, + { + "id": "fsl_in_null_list_values_rb_f0", + "description": "same rows; the null list value covers a valid slot [9, 9]", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "//////gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA3P///xwAAAAMAAAAAAABDKQAAAABAAAAHAAAAAQABAAEAAAAEAAUABAADgAPAAQAAAAIABAAAAAgAAAADAAAAAAAARBkAAAAAQAAACQAAAAAAAYACAAEAAYAAAACAAAAEAAUABAAAAAPAAQAAAAIABAAAAAYAAAAIAAAAAAAAAIcAAAACAAMAAQACwAIAAAAIAAAAAAAAAEAAAAABAAAAGl0ZW0AAAAABAAAAGl0ZW0AAAAAAQAAAGwAAAAAAAAAAAAAAP/////4AAAAEAAAAAwAGgAYABcABAAIAAwAAAAgAAAAQAEAAAAAAAAAAAAAAAAAAwQACgAYAAwACAAEAAoAAABMAAAAEAAAAAMAAAAAAAAAAAAAAAMAAAADAAAAAAAAAAEAAAAAAAAAAwAAAAAAAAAAAAAAAAAAAAYAAAAAAAAAAAAAAAAAAAAAAAAABQAAAAAAAAAAAAAAAQAAAAAAAABAAAAAAAAAABAAAAAAAAAAgAAAAAAAAAABAAAAAAAAAMAAAAAAAAAAAQAAAAAAAAAAAQAAAAAAABgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAFAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEAAAACAAAAAwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAIAAAAJAAAACQAAAAMAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////8AAAAA", + "expected_hash": "0000021e1ecfc99a033b00e24f700b119b3d40ccb06fc0ced08ca5742e348a895c11dc" + }, + { + "id": "fsl_in_null_list_clean_rb_f0", + "description": "same rows; the null list value is empty", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "//////gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA3P///xwAAAAMAAAAAAABDKQAAAABAAAAHAAAAAQABAAEAAAAEAAUABAADgAPAAQAAAAIABAAAAAgAAAADAAAAAAAARBkAAAAAQAAACQAAAAAAAYACAAEAAYAAAACAAAAEAAUABAAAAAPAAQAAAAIABAAAAAYAAAAIAAAAAAAAAIcAAAACAAMAAQACwAIAAAAIAAAAAAAAAEAAAAABAAAAGl0ZW0AAAAABAAAAGl0ZW0AAAAAAQAAAGwAAAAAAAAAAAAAAP/////4AAAAEAAAAAwAGgAYABcABAAIAAwAAAAgAAAAQAEAAAAAAAAAAAAAAAAAAwQACgAYAAwACAAEAAoAAABMAAAAEAAAAAMAAAAAAAAAAAAAAAMAAAADAAAAAAAAAAEAAAAAAAAAAgAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAABQAAAAAAAAAAAAAAAQAAAAAAAABAAAAAAAAAABAAAAAAAAAAgAAAAAAAAAABAAAAAAAAAMAAAAAAAAAAAQAAAAAAAAAAAQAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAFAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAEAAAABAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAIAAAADAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////8AAAAA", + "expected_hash": "0000021e1ecfc99a033b00e24f700b119b3d40ccb06fc0ced08ca5742e348a895c11dc" + }, + { + "id": "fsl_fresh_rb_f0", + "description": "f: FixedSizeList? rows [[1, 2], null, [3, null], [5, 6]]", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA1P///yAAAAAMAAAAAAABEGQAAAABAAAAJAAAAAAABgAIAAQABgAAAAIAAAAQABQAEAAOAA8ABAAAAAgAEAAAABgAAAAgAAAAAAABAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAAEAAAAaXRlbQAAAAABAAAAZgAAAAAAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAADAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAABAAAAAAAAAAAAAAAAgAAAAQAAAAAAAAAAQAAAAAAAAAIAAAAAAAAAAMAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAABAAAAAAAAAIAAAAAAAAAAIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAANMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAgAAAAAAAAAAAAAAAwAAAAAAAAAFAAAABgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "0000028903b59c39883b372acdae00899925994ea65157e1f1cfa7d262b972360d0b4f" + }, + { + "id": "fsl_sliced_rb_f0", + "description": "the same rows as a slice (offset 1) of [[0, 0], [1, 2], null, [3, null], [5, 6], [9, 9]] (in memory; the IPC encoding has offset 0)", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA1P///yAAAAAMAAAAAAABEGQAAAABAAAAJAAAAAAABgAIAAQABgAAAAIAAAAQABQAEAAOAA8ABAAAAAgAEAAAABgAAAAgAAAAAAABAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAAEAAAAaXRlbQAAAAABAAAAZgAAAAAAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAADAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAABAAAAAAAAAAAAAAAAgAAAAQAAAAAAAAAAQAAAAAAAAAIAAAAAAAAAAIAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAABAAAAAAAAAIAAAAAAAAAAIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAANcAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAgAAAAcAAAAAAAAAAwAAAAAAAAAFAAAABgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "0000028903b59c39883b372acdae00899925994ea65157e1f1cfa7d262b972360d0b4f" + }, + { + "id": "fsl_split_stream_f0", + "description": "the same rows in batches of 1, 0, 2 and 1 rows", + "method": "hash_stream", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA1P///yAAAAAMAAAAAAABEGQAAAABAAAAJAAAAAAABgAIAAQABgAAAAIAAAAQABQAEAAOAA8ABAAAAAgAEAAAABgAAAAgAAAAAAABAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAAEAAAAaXRlbQAAAAABAAAAZgAAAAAAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAADAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAAAQAAAAAAAAAAAAAAAgAAAAEAAAAAAAAAAAAAAAAAAAACAAAAAAAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAABAAAAAAAAAIAAAAAAAAAACAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////7gAAAAQAAAAAAAKAA4ADAALAAQACgAAABQAAAAAAAADBAAKAAwAAAAIAAQACgAAADAAAAAEAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAADAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAAAgAAAAAAAAAAAAAAAgAAAAIAAAAAAAAAAQAAAAAAAAAEAAAAAAAAAAMAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAABAAAAAAAAAIAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAADAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAAAQAAAAAAAAAAAAAAAgAAAAEAAAAAAAAAAAAAAAAAAAACAAAAAAAAAAAAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAABAAAAAAAAAIAAAAAAAAAACAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAFAAAABgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "0000028903b59c39883b372acdae00899925994ea65157e1f1cfa7d262b972360d0b4f" + }, + { + "id": "fsl_trailing_child_rb_f0", + "description": "the same rows over a child with two extra trailing positions [8, null] (in memory; the IPC encoding drops them)", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA1P///yAAAAAMAAAAAAABEGQAAAABAAAAJAAAAAAABgAIAAQABgAAAAIAAAAQABQAEAAOAA8ABAAAAAgAEAAAABgAAAAgAAAAAAABAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAAEAAAAaXRlbQAAAAABAAAAZgAAAAAAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAADAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAABAAAAAAAAAAAAAAAAgAAAAQAAAAAAAAAAQAAAAAAAAAIAAAAAAAAAAMAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAABAAAAAAAAAIAAAAAAAAAAIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAANMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAgAAAAAAAAAAAAAAAwAAAAAAAAAFAAAABgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "0000028903b59c39883b372acdae00899925994ea65157e1f1cfa7d262b972360d0b4f" + }, + { + "id": "fsl_fresh_array_f0", + "description": "hash_array FixedSizeList [[1, 2], null, [3, null], [5, 6]]", + "method": "hash_array", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA1P///yAAAAAMAAAAAAABEGQAAAABAAAAJAAAAAAABgAIAAQABgAAAAIAAAAQABQAEAAOAA8ABAAAAAgAEAAAABgAAAAgAAAAAAABAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAAEAAAAaXRlbQAAAAAFAAAAYXJyYXkAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAADAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAABAAAAAAAAAAAAAAAAgAAAAQAAAAAAAAAAQAAAAAAAAAIAAAAAAAAAAMAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAABAAAAAAAAAIAAAAAAAAAAIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAANMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAgAAAAAAAAAAAAAAAwAAAAAAAAAFAAAABgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "000002f2067ba338e70ae05dd194278ac9d9f9cb35687d103fb7b686a885e21353bf32" + }, + { + "id": "fsl_sliced_array_f0", + "description": "hash_array of the same values as a slice (offset 1; in memory, the IPC encoding has offset 0)", + "method": "hash_array", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA1P///yAAAAAMAAAAAAABEGQAAAABAAAAJAAAAAAABgAIAAQABgAAAAIAAAAQABQAEAAOAA8ABAAAAAgAEAAAABgAAAAgAAAAAAABAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAAEAAAAaXRlbQAAAAAFAAAAYXJyYXkAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAADAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAABAAAAAAAAAAAAAAAAgAAAAQAAAAAAAAAAQAAAAAAAAAIAAAAAAAAAAIAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAABAAAAAAAAAIAAAAAAAAAAIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAANcAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAgAAAAcAAAAAAAAAAwAAAAAAAAAFAAAABgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "000002f2067ba338e70ae05dd194278ac9d9f9cb35687d103fb7b686a885e21353bf32" + }, + { + "id": "fsl_struct_fresh_rb_f0", + "description": "s: Struct?>? rows [{x: [1, 2]}, {x: null}, {x: [3, null]}, null]", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "//////gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAAsP///xwAAAAMAAAAAAABDZAAAAABAAAADAAAAAQABAAEAAAA1P///yAAAAAMAAAAAAABEGQAAAABAAAAJAAAAAAABgAIAAQABgAAAAIAAAAQABQAEAAOAA8ABAAAAAgAEAAAABgAAAAgAAAAAAABAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAAEAAAAaXRlbQAAAAABAAAAeAAAAAEAAABzAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/////4AAAAEAAAAAwAGgAYABcABAAIAAwAAAAgAAAAAAEAAAAAAAAAAAAAAAAAAwQACgAYAAwACAAEAAoAAABMAAAAEAAAAAQAAAAAAAAAAAAAAAMAAAAEAAAAAAAAAAEAAAAAAAAABAAAAAAAAAABAAAAAAAAAAgAAAAAAAAAAwAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAQAAAAAAAABAAAAAAAAAAAEAAAAAAAAAgAAAAAAAAAABAAAAAAAAAMAAAAAAAAAAIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAHAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAANMAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAgAAAAAAAAAAAAAAAwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "000002662527eb07b89cca6bf5633bff7418da1d0b9d4d36b5fcff56fb5abd6189695c" + }, + { + "id": "fsl_struct_sliced_rb_f0", + "description": "the same rows as a slice (offset 1) of a struct whose masked row holds [5, 6] (in memory; the IPC encoding has offset 0)", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "//////gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAAsP///xwAAAAMAAAAAAABDZAAAAABAAAADAAAAAQABAAEAAAA1P///yAAAAAMAAAAAAABEGQAAAABAAAAJAAAAAAABgAIAAQABgAAAAIAAAAQABQAEAAOAA8ABAAAAAgAEAAAABgAAAAgAAAAAAABAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAAEAAAAaXRlbQAAAAABAAAAeAAAAAEAAABzAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP/////4AAAAEAAAAAwAGgAYABcABAAIAAwAAAAgAAAAAAEAAAAAAAAAAAAAAAAAAwQACgAYAAwACAAEAAoAAABMAAAAEAAAAAQAAAAAAAAAAAAAAAMAAAAEAAAAAAAAAAEAAAAAAAAABAAAAAAAAAABAAAAAAAAAAgAAAAAAAAAAgAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAQAAAAAAAABAAAAAAAAAAAEAAAAAAAAAgAAAAAAAAAABAAAAAAAAAMAAAAAAAAAAIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAHAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAANcAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAgAAAAcAAAAAAAAAAwAAAAAAAAAFAAAABgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "000002662527eb07b89cca6bf5633bff7418da1d0b9d4d36b5fcff56fb5abd6189695c" + }, + { + "id": "fsl_dict_rb_f0", + "description": "f: Dictionary>? keys [2, null, 1, 0, 2] over [[1, 2], null, [3, 4]] (the null value hides [null, 7])", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "//////gAAAAQAAAAAAAKAAwACgAJAAQACgAAAAgAAAAAAQQAyP///wQAAAABAAAAFAAAABAAGAAUAA4ADwAEABAACAAQAAAAQAAAACwAAAAAAAEQEAAAAIAAAAAIAAgAAAAEAAgAAAAEAAAAuP///wgAAAAAAAABAQAAACQAAAAAAAYACAAEAAYAAAACAAAAEAAUABAAAAAPAAQAAAAIABAAAAAYAAAAIAAAAAAAAAIcAAAACAAMAAQACwAIAAAAIAAAAAAAAAEAAAAABAAAAGl0ZW0AAAAAAQAAAGYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////+4AAAAEAAAAAwAGAAWABUABAAIAAwAAAAcAAAAwAAAAAAAAAAAAAAAAAIEAAgACgAAAAQACAAAABAAAAAAAAoAGAAMAAgABAAKAAAAPAAAABAAAAADAAAAAAAAAAAAAAACAAAAAwAAAAAAAAABAAAAAAAAAAYAAAAAAAAAAQAAAAAAAAAAAAAAAwAAAAAAAAAAAAAAAQAAAAAAAABAAAAAAAAAAAEAAAAAAAAAgAAAAAAAAAAYAAAAAAAAAAUAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA7AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAIAAAAAAAAABwAAAAMAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////+4AAAAEAAAAAwAGgAYABcABAAIAAwAAAAgAAAAgAAAAAAAAAAAAAAAAAAAAwQACgAYAAwACAAEAAoAAAAsAAAAEAAAAAUAAAAAAAAAAAAAAAEAAAAFAAAAAAAAAAEAAAAAAAAAAAAAAAIAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAAFAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB0AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAEAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "000002b9af6bfccb94c7e9c28dba24a36942241d15b61aad394a137a52bcbb2d20f13b" + }, + { + "id": "fsl_dict_plain_rb_f0", + "description": "f: FixedSizeList? rows [[3, 4], null, null, [1, 2], [3, 4]]", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "//////gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAUAAAAEAAUABAADgAPAAQAAAAIABAAAAAgAAAADAAAAAAAARBkAAAAAQAAACQAAAAAAAYACAAEAAYAAAACAAAAEAAUABAAAAAPAAQAAAAIABAAAAAYAAAAIAAAAAAAAAIcAAAACAAMAAQACwAIAAAAIAAAAAAAAAEAAAAABAAAAGl0ZW0AAAAAAQAAAGYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////+4AAAAEAAAAAwAGgAYABcABAAIAAwAAAAgAAAAwAAAAAAAAAAAAAAAAAAAAwQACgAYAAwACAAEAAoAAAA8AAAAEAAAAAUAAAAAAAAAAAAAAAIAAAAFAAAAAAAAAAIAAAAAAAAACgAAAAAAAAAEAAAAAAAAAAAAAAADAAAAAAAAAAAAAAABAAAAAAAAAEAAAAAAAAAAAgAAAAAAAACAAAAAAAAAACgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABkAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADDAwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAwAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAIAAAADAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////8AAAAA", + "expected_hash": "000002b9af6bfccb94c7e9c28dba24a36942241d15b61aad394a137a52bcbb2d20f13b" + }, + { + "id": "fsl_dict_split_stream_f0", + "description": "the dictionary column in single-row batches", + "method": "hash_stream", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "//////gAAAAQAAAAAAAKAAwACgAJAAQACgAAAAgAAAAAAQQAyP///wQAAAABAAAAFAAAABAAGAAUAA4ADwAEABAACAAQAAAAQAAAACwAAAAAAAEQEAAAAIAAAAAIAAgAAAAEAAgAAAAEAAAAuP///wgAAAAAAAABAQAAACQAAAAAAAYACAAEAAYAAAACAAAAEAAUABAAAAAPAAQAAAAIABAAAAAYAAAAIAAAAAAAAAIcAAAACAAMAAQACwAIAAAAIAAAAAAAAAEAAAAABAAAAGl0ZW0AAAAAAQAAAGYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////+4AAAAEAAAAAwAGAAWABUABAAIAAwAAAAcAAAAwAAAAAAAAAAAAAAAAAIEAAgACgAAAAQACAAAABAAAAAAAAoAGAAMAAgABAAKAAAAPAAAABAAAAADAAAAAAAAAAAAAAACAAAAAwAAAAAAAAABAAAAAAAAAAYAAAAAAAAAAQAAAAAAAAAAAAAAAwAAAAAAAAAAAAAAAQAAAAAAAABAAAAAAAAAAAEAAAAAAAAAgAAAAAAAAAAYAAAAAAAAAAUAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA7AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAIAAAAAAAAABwAAAAMAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////+4AAAAEAAAAAwAGgAYABcABAAIAAwAAAAgAAAAgAAAAAAAAAAAAAAAAAAAAwQACgAYAAwACAAEAAoAAAAsAAAAEAAAAAEAAAAAAAAAAAAAAAEAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAIAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAACAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAACwAAAAQAAAAAQAAAAAAAAAAAAAAAQAAAAEAAAAAAAAAAQAAAAAAAAAAAAAAAgAAAAAAAAAAAAAAAQAAAAAAAABAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/////uAAAABAAAAAMABoAGAAXAAQACAAMAAAAIAAAAIAAAAAAAAAAAAAAAAAAAAMEAAoAGAAMAAgABAAKAAAALAAAABAAAAABAAAAAAAAAAAAAAABAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAACAAAAAAAAAAAAAAABAAAAAAAAAEAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////+4AAAAEAAAAAwAGgAYABcABAAIAAwAAAAgAAAAgAAAAAAAAAAAAAAAAAAAAwQACgAYAAwACAAEAAoAAAAsAAAAEAAAAAEAAAAAAAAAAAAAAAEAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAIAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAACAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAACwAAAAQAAAAAQAAAAAAAAAAAAAAAQAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAAAAAAAAAAAAAQAAAAAAAABAAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/////AAAAAA==", + "expected_hash": "000002b9af6bfccb94c7e9c28dba24a36942241d15b61aad394a137a52bcbb2d20f13b" + }, + { + "id": "fsl_dict_array_f0", + "description": "hash_array of the dictionary of fixed-size lists", + "method": "hash_array", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "//////gAAAAQAAAAAAAKAAwACgAJAAQACgAAAAgAAAAAAQQAyP///wQAAAABAAAAFAAAABAAGAAUAA4ADwAEABAACAAQAAAAQAAAACwAAAAAAAEQEAAAAIAAAAAIAAgAAAAEAAgAAAAEAAAAuP///wgAAAAAAAABAQAAACQAAAAAAAYACAAEAAYAAAACAAAAEAAUABAAAAAPAAQAAAAIABAAAAAYAAAAIAAAAAAAAAIcAAAACAAMAAQACwAIAAAAIAAAAAAAAAEAAAAABAAAAGl0ZW0AAAAABQAAAGFycmF5AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////+4AAAAEAAAAAwAGAAWABUABAAIAAwAAAAcAAAAwAAAAAAAAAAAAAAAAAIEAAgACgAAAAQACAAAABAAAAAAAAoAGAAMAAgABAAKAAAAPAAAABAAAAADAAAAAAAAAAAAAAACAAAAAwAAAAAAAAABAAAAAAAAAAYAAAAAAAAAAQAAAAAAAAAAAAAAAwAAAAAAAAAAAAAAAQAAAAAAAABAAAAAAAAAAAEAAAAAAAAAgAAAAAAAAAAYAAAAAAAAAAUAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA7AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAIAAAAAAAAABwAAAAMAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////+4AAAAEAAAAAwAGgAYABcABAAIAAwAAAAgAAAAgAAAAAAAAAAAAAAAAAAAAwQACgAYAAwACAAEAAoAAAAsAAAAEAAAAAUAAAAAAAAAAAAAAAEAAAAFAAAAAAAAAAEAAAAAAAAAAAAAAAIAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAAFAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAB0AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAEAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "0000020031c23bbc21fad30e1a96d6295c08cce21197108d8690d70b4cd5203fc9d01e" + }, + { + "id": "fsl_dict_plain_array_f0", + "description": "hash_array of the plain fixed-size lists", + "method": "hash_array", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "//////gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAUAAAAEAAUABAADgAPAAQAAAAIABAAAAAgAAAADAAAAAAAARBkAAAAAQAAACQAAAAAAAYACAAEAAYAAAACAAAAEAAUABAAAAAPAAQAAAAIABAAAAAYAAAAIAAAAAAAAAIcAAAACAAMAAQACwAIAAAAIAAAAAAAAAEAAAAABAAAAGl0ZW0AAAAABQAAAGFycmF5AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////+4AAAAEAAAAAwAGgAYABcABAAIAAwAAAAgAAAAwAAAAAAAAAAAAAAAAAAAAwQACgAYAAwACAAEAAoAAAA8AAAAEAAAAAUAAAAAAAAAAAAAAAIAAAAFAAAAAAAAAAIAAAAAAAAACgAAAAAAAAAEAAAAAAAAAAAAAAADAAAAAAAAAAAAAAABAAAAAAAAAEAAAAAAAAAAAgAAAAAAAACAAAAAAAAAACgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABkAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADDAwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAwAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAIAAAADAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////8AAAAA", + "expected_hash": "0000020031c23bbc21fad30e1a96d6295c08cce21197108d8690d70b4cd5203fc9d01e" + }, + { + "id": "fsl_of_list_rb_f0", + "description": "f: FixedSizeList?, 2> non-null rows [[[1, 2], []], [[3], null]]", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "//////gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAUAAAAEAAUABAAAAAPAAQAAAAIABAAAAAgAAAADAAAAAAAABCUAAAAAQAAABQAAAAAAAYACAAEAAYAAAACAAAA3P///xwAAAAMAAAAAAABDFwAAAABAAAAHAAAAAQABAAEAAAAEAAUABAADgAPAAQAAAAIABAAAAAYAAAAIAAAAAAAAQIcAAAACAAMAAQACwAIAAAAIAAAAAAAAAEAAAAABAAAAGl0ZW0AAAAABAAAAGl0ZW0AAAAAAQAAAGYAAAAAAAAAAAAAAP/////4AAAAEAAAAAwAGgAYABcABAAIAAwAAAAgAAAAQAEAAAAAAAAAAAAAAAAAAwQACgAYAAwACAAEAAoAAABMAAAAEAAAAAIAAAAAAAAAAAAAAAMAAAACAAAAAAAAAAAAAAAAAAAABAAAAAAAAAABAAAAAAAAAAMAAAAAAAAAAAAAAAAAAAAAAAAABQAAAAAAAAAAAAAAAQAAAAAAAABAAAAAAAAAAAEAAAAAAAAAgAAAAAAAAAAUAAAAAAAAAMAAAAAAAAAAAQAAAAAAAAAAAQAAAAAAAAwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAAAAAgAAAAMAAAADAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAIAAAADAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////8AAAAA", + "expected_hash": "000002efea83f2f01528068f375e95e29dd669e295073ae8168b28432722e312466759" + }, + { + "id": "fsl_of_struct_ab_rb_f0", + "description": "f: FixedSizeList?, 2>? rows [[{a: 1, b: true}, {a: null, b: false}], [null, {a: 4, b: false}]] (struct children in the order of the id)", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////zgBAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAAeP///yAAAAAMAAAAAAABEMgAAAABAAAAFAAAAAAABgAIAAQABgAAAAIAAACk////HAAAAAwAAAAAAAENkAAAAAIAAABUAAAAGAAAANT///8QABQAEAAAAA8ABAAAAAgAEAAAABgAAAAMAAAAAAAABhAAAAAAAAAABAAEAAQAAAABAAAAYgAAABAAFAAQAA4ADwAEAAAACAAQAAAAGAAAACAAAAAAAAECHAAAAAgADAAEAAsACAAAACAAAAAAAAABAAAAAAEAAABhAAAABAAAAGl0ZW0AAAAAAQAAAGYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/////+AAAABAAAAAMABoAGAAXAAQACAAMAAAAIAAAAIABAAAAAAAAAAAAAAAAAAMEAAoAGAAMAAgABAAKAAAAXAAAABAAAAACAAAAAAAAAAAAAAAEAAAAAgAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAAQAAAAAAAAAEAAAAAAAAAAEAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAGAAAAAAAAAAAAAAABAAAAAAAAAEAAAAAAAAAAAQAAAAAAAACAAAAAAAAAAAEAAAAAAAAAwAAAAAAAAAAQAAAAAAAAAAABAAAAAAAAAQAAAAAAAABAAQAAAAAAAAEAAAAAAAAA/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAsAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAANAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAADAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAFAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "0000027b61753dc51614f0517215dac6cb3530682029c58a0239594f4a3b6b4f146996" + }, + { + "id": "fsl_of_struct_ba_rb_f0", + "description": "f: FixedSizeList?, 2>? rows [[{a: 1, b: true}, {a: null, b: false}], [null, {a: 4, b: false}]] (struct children in the order of the id)", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////zgBAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAAsP///yAAAAAMAAAAAAABEMgAAAABAAAAFAAAAAAABgAIAAQABgAAAAIAAADc////HAAAAAwAAAAAAAENkAAAAAIAAABgAAAAGAAAAJD///8QABQAEAAOAA8ABAAAAAgAEAAAABgAAAAgAAAAAAABAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAABAAAAYQAAABAAFAAQAAAADwAEAAAACAAQAAAAGAAAAAwAAAAAAAAGEAAAAAAAAAAEAAQABAAAAAEAAABiAAAABAAAAGl0ZW0AAAAAAQAAAGYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/////+AAAABAAAAAMABoAGAAXAAQACAAMAAAAIAAAAIABAAAAAAAAAAAAAAAAAAMEAAoAGAAMAAgABAAKAAAAXAAAABAAAAACAAAAAAAAAAAAAAAEAAAAAgAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAAQAAAAAAAAAEAAAAAAAAAAAAAAAAAAAABAAAAAAAAAABAAAAAAAAAAAAAAAGAAAAAAAAAAAAAAABAAAAAAAAAEAAAAAAAAAAAQAAAAAAAACAAAAAAAAAAAEAAAAAAAAAwAAAAAAAAAABAAAAAAAAAAABAAAAAAAAAQAAAAAAAABAAQAAAAAAABAAAAAAAAAA/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAsAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA0AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAMAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "0000027b61753dc51614f0517215dac6cb3530682029c58a0239594f4a3b6b4f146996" + }, + { + "id": "fsl_of_struct_ab_rb_f2", + "description": "f: FixedSizeList?, 2>? rows [[{a: 1, b: true}, {a: null, b: false}], [null, {a: 4, b: false}]] (struct children in the order of the id)", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": true, + "ipc_b64": "/////zgBAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAAeP///yAAAAAMAAAAAAABEMgAAAABAAAAFAAAAAAABgAIAAQABgAAAAIAAACk////HAAAAAwAAAAAAAENkAAAAAIAAABUAAAAGAAAANT///8QABQAEAAAAA8ABAAAAAgAEAAAABgAAAAMAAAAAAAABhAAAAAAAAAABAAEAAQAAAABAAAAYgAAABAAFAAQAA4ADwAEAAAACAAQAAAAGAAAACAAAAAAAAECHAAAAAgADAAEAAsACAAAACAAAAAAAAABAAAAAAEAAABhAAAABAAAAGl0ZW0AAAAAAQAAAGYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/////+AAAABAAAAAMABoAGAAXAAQACAAMAAAAIAAAAIABAAAAAAAAAAAAAAAAAAMEAAoAGAAMAAgABAAKAAAAXAAAABAAAAACAAAAAAAAAAAAAAAEAAAAAgAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAAQAAAAAAAAAEAAAAAAAAAAEAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAGAAAAAAAAAAAAAAABAAAAAAAAAEAAAAAAAAAAAQAAAAAAAACAAAAAAAAAAAEAAAAAAAAAwAAAAAAAAAAQAAAAAAAAAAABAAAAAAAAAQAAAAAAAABAAQAAAAAAAAEAAAAAAAAA/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAsAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAANAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAAAAAADAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAFAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "000002621a460c82d10512ab9c4c4fc101e67194ad038ecd5037f1639852f1f2d7d489" + }, + { + "id": "fsl_of_struct_ba_rb_f2", + "description": "f: FixedSizeList?, 2>? rows [[{a: 1, b: true}, {a: null, b: false}], [null, {a: 4, b: false}]] (struct children in the order of the id)", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": true, + "ipc_b64": "/////zgBAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAAsP///yAAAAAMAAAAAAABEMgAAAABAAAAFAAAAAAABgAIAAQABgAAAAIAAADc////HAAAAAwAAAAAAAENkAAAAAIAAABgAAAAGAAAAJD///8QABQAEAAOAA8ABAAAAAgAEAAAABgAAAAgAAAAAAABAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAABAAAAYQAAABAAFAAQAAAADwAEAAAACAAQAAAAGAAAAAwAAAAAAAAGEAAAAAAAAAAEAAQABAAAAAEAAABiAAAABAAAAGl0ZW0AAAAAAQAAAGYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/////+AAAABAAAAAMABoAGAAXAAQACAAMAAAAIAAAAIABAAAAAAAAAAAAAAAAAAMEAAoAGAAMAAgABAAKAAAAXAAAABAAAAACAAAAAAAAAAAAAAAEAAAAAgAAAAAAAAAAAAAAAAAAAAQAAAAAAAAAAQAAAAAAAAAEAAAAAAAAAAAAAAAAAAAABAAAAAAAAAABAAAAAAAAAAAAAAAGAAAAAAAAAAAAAAABAAAAAAAAAEAAAAAAAAAAAQAAAAAAAACAAAAAAAAAAAEAAAAAAAAAwAAAAAAAAAABAAAAAAAAAAABAAAAAAAAAQAAAAAAAABAAQAAAAAAABAAAAAAAAAA/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAsAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA0AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAMAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "0000027ae631ecad5e30c197df8519f629b0af5ca4e02c0517aa5c6268c5354a53a404" + }, + { + "id": "fsl_of_dict_rb_f0", + "description": "f: FixedSizeList?, 2> non-null rows [[\"a\", null], [\"b\", \"a\"]]", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "//////gAAAAQAAAAAAAKAAwACgAJAAQACgAAAAgAAAAAAQQAjP///wQAAAABAAAAFAAAABAAFAAQAAAADwAEAAAACAAQAAAAIAAAAAwAAAAAAAAQgAAAAAEAAAAkAAAAAAAGAAgABAAGAAAAAgAAABAAGAAUAA4ADwAEABAACAAQAAAAQAAAADQAAAAAAAEFEAAAADQAAAAIAAgAAAAEAAgAAAAMAAAACAAMAAQACwAIAAAACAAAAAAAAAEAAAAABAAEAAQAAAAEAAAAaXRlbQAAAAABAAAAZgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////+4AAAAEAAAAAwAGAAWABUABAAIAAwAAAAcAAAAwAAAAAAAAAAAAAAAAAIEAAgACgAAAAQACAAAABAAAAAAAAoAGAAMAAgABAAKAAAALAAAABAAAAACAAAAAAAAAAAAAAABAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAADAAAAAAAAAAAAAAABAAAAAAAAAEAAAAAAAAAADAAAAAAAAACAAAAAAAAAAAIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAYWIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////+4AAAAEAAAAAwAGgAYABcABAAIAAwAAAAgAAAAwAAAAAAAAAAAAAAAAAAAAwQACgAYAAwACAAEAAoAAAA8AAAAEAAAAAIAAAAAAAAAAAAAAAIAAAACAAAAAAAAAAAAAAAAAAAABAAAAAAAAAABAAAAAAAAAAAAAAADAAAAAAAAAAAAAAABAAAAAAAAAEAAAAAAAAAAAQAAAAAAAACAAAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAANAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////8AAAAA", + "expected_hash": "000002f7dce18f584e4a5065a944cb166772f2eebd52b3c7418273ef70611db6b143a3" + }, + { + "id": "fsl_of_dict_plain_rb_f0", + "description": "f: FixedSizeList non-null rows [[\"a\", null], [\"b\", \"a\"]]", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////7gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAUAAAAEAAUABAAAAAPAAQAAAAIABAAAAAgAAAADAAAAAAAABBYAAAAAQAAACQAAAAAAAYACAAEAAYAAAACAAAAEAAUABAADgAPAAQAAAAIABAAAAAYAAAADAAAAAAAAQUQAAAAAAAAAAQABAAEAAAABAAAAGl0ZW0AAAAAAQAAAGYAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAAAAAQAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAAAgAAAAAAAAAAAAAAAgAAAAIAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAEAAAAAAAAAAAAAAAQAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAABAAAAAAAAAIAAAAAAAAAAFAAAAAAAAADAAAAAAAAAAAMAAAAAAAAA/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA0AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAEAAAACAAAAAwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAYWJhAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////8AAAAA", + "expected_hash": "000002f7dce18f584e4a5065a944cb166772f2eebd52b3c7418273ef70611db6b143a3" + }, + { + "id": "fsl_in_list_rb_f0", + "description": "l: List?> non-null rows [[[1, 2], null], [], [[5, 6]]]", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "//////gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAAoP///xwAAAAMAAAAAAAADKQAAAABAAAAHAAAAAQABAAEAAAAEAAUABAADgAPAAQAAAAIABAAAAAgAAAADAAAAAAAARBkAAAAAQAAACQAAAAAAAYACAAEAAYAAAACAAAAEAAUABAAAAAPAAQAAAAIABAAAAAYAAAAIAAAAAAAAAIcAAAACAAMAAQACwAIAAAAIAAAAAAAAAEAAAAABAAAAGl0ZW0AAAAABAAAAGl0ZW0AAAAAAQAAAGwAAAAAAAAAAAAAAP/////4AAAAEAAAAAwAGgAYABcABAAIAAwAAAAgAAAAQAEAAAAAAAAAAAAAAAAAAwQACgAYAAwACAAEAAoAAABMAAAAEAAAAAMAAAAAAAAAAAAAAAMAAAADAAAAAAAAAAAAAAAAAAAAAwAAAAAAAAABAAAAAAAAAAYAAAAAAAAAAAAAAAAAAAAAAAAABQAAAAAAAAAAAAAAAQAAAAAAAABAAAAAAAAAABAAAAAAAAAAgAAAAAAAAAABAAAAAAAAAMAAAAAAAAAAAQAAAAAAAAAAAQAAAAAAABgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAIAAAACAAAAAwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAUAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAIAAAADAAAABAAAAAUAAAAGAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////8AAAAA", + "expected_hash": "0000024ab5d3ba62dafc469507177166de69f94ab5ef3756b04f03b41129836d6d1d81" + }, + { + "id": "fsl_in_struct_rb_f0", + "description": "s: Struct?>? rows [{x: [1, 2]}, {x: null}, null]", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "//////gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAEAAAA3P///xwAAAAMAAAAAAABDaAAAAABAAAAHAAAAAQABAAEAAAAEAAUABAADgAPAAQAAAAIABAAAAAgAAAADAAAAAAAARBkAAAAAQAAACQAAAAAAAYACAAEAAYAAAACAAAAEAAUABAAAAAPAAQAAAAIABAAAAAYAAAAIAAAAAAAAAIcAAAACAAMAAQACwAIAAAAIAAAAAAAAAEAAAAABAAAAGl0ZW0AAAAAAQAAAHgAAAABAAAAcwAAAAAAAAAAAAAAAAAAAP/////4AAAAEAAAAAwAGgAYABcABAAIAAwAAAAgAAAAAAEAAAAAAAAAAAAAAAAAAwQACgAYAAwACAAEAAoAAABMAAAAEAAAAAMAAAAAAAAAAAAAAAMAAAADAAAAAAAAAAEAAAAAAAAAAwAAAAAAAAABAAAAAAAAAAYAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAQAAAAAAAABAAAAAAAAAAAEAAAAAAAAAgAAAAAAAAAABAAAAAAAAAMAAAAAAAAAAGAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAgAAAAMAAAAEAAAABQAAAAYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "000002f2d94e3f09c1c7d98ec186eab0cf40c2a82de1c101ae15961603dd59ead32c1b" + }, + { + "id": "fsl_meta_element_rb_f0_im1", + "description": "f: FixedSizeList non-null rows [[1, 2]], include_metadata (path f/)", + "method": "hash_record_batch", + "include_metadata": true, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "//////gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAUAAAAEAAUABAAAAAPAAQAAAAIABAAAAAgAAAADAAAAAAAABCcAAAAAQAAACgAAAAAAAYACgAEAAYAAAACAAAAAAASABgAFAASABMACAAAAAwABAASAAAAOAAAABgAAAAgAAAAAAABAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAAEAAAAaXRlbQAAAAABAAAADAAAAAgADAAIAAQACAAAAAgAAAAMAAAAAQAAAHMAAAAEAAAAdW5pdAAAAAABAAAAZgAAAP////+4AAAAEAAAAAwAGgAYABcABAAIAAwAAAAgAAAAwAAAAAAAAAAAAAAAAAAAAwQACgAYAAwACAAEAAoAAAA8AAAAEAAAAAEAAAAAAAAAAAAAAAIAAAABAAAAAAAAAAAAAAAAAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAADAAAAAAAAAAAAAAABAAAAAAAAAEAAAAAAAAAAAQAAAAAAAACAAAAAAAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////8AAAAA", + "expected_hash": "000002d46bffc33bc77fe9411f3ae50d20f8af821c388bc06e87b0edfb220f38dddec6" + }, + { + "id": "fsl_meta_element_rb_f0", + "description": "the same batch without include_metadata", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "//////gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAUAAAAEAAUABAAAAAPAAQAAAAIABAAAAAgAAAADAAAAAAAABCcAAAAAQAAACgAAAAAAAYACgAEAAYAAAACAAAAAAASABgAFAASABMACAAAAAwABAASAAAAOAAAABgAAAAgAAAAAAABAhwAAAAIAAwABAALAAgAAAAgAAAAAAAAAQAAAAAEAAAAaXRlbQAAAAABAAAADAAAAAgADAAIAAQACAAAAAgAAAAMAAAAAQAAAHMAAAAEAAAAdW5pdAAAAAABAAAAZgAAAP////+4AAAAEAAAAAwAGgAYABcABAAIAAwAAAAgAAAAwAAAAAAAAAAAAAAAAAAAAwQACgAYAAwACAAEAAoAAAA8AAAAEAAAAAEAAAAAAAAAAAAAAAIAAAABAAAAAAAAAAAAAAAAAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAADAAAAAAAAAAAAAAABAAAAAAAAAEAAAAAAAAAAAQAAAAAAAACAAAAAAAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////8AAAAA", + "expected_hash": "000002c8180924f720e3ef2c503e37b289b4f4f2b27c4843fcc0abe746865d8f292800" + }, + { + "id": "fsl_no_meta_rb_f0", + "description": "f: FixedSizeList non-null rows [[1, 2]] (no metadata)", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "//////gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAUAAAAEAAUABAAAAAPAAQAAAAIABAAAAAgAAAADAAAAAAAABBkAAAAAQAAACQAAAAAAAYACAAEAAYAAAACAAAAEAAUABAADgAPAAQAAAAIABAAAAAYAAAAIAAAAAAAAQIcAAAACAAMAAQACwAIAAAAIAAAAAAAAAEAAAAABAAAAGl0ZW0AAAAAAQAAAGYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////+4AAAAEAAAAAwAGgAYABcABAAIAAwAAAAgAAAAwAAAAAAAAAAAAAAAAAAAAwQACgAYAAwACAAEAAoAAAA8AAAAEAAAAAEAAAAAAAAAAAAAAAIAAAABAAAAAAAAAAAAAAAAAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAADAAAAAAAAAAAAAAABAAAAAAAAAEAAAAAAAAAAAQAAAAAAAACAAAAAAAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAQAAAAIAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////8AAAAA", + "expected_hash": "000002c8180924f720e3ef2c503e37b289b4f4f2b27c4843fcc0abe746865d8f292800" + }, + { + "id": "fsl_ext_tensor_rb_f0_im1", + "description": "t: arrow.fixed_shape_tensor {\"shape\":[2,2]} over FixedSizeList? rows [[1, 2, null, 4]], include_metadata", + "method": "hash_record_batch", + "include_metadata": true, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////3gBAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAYAAAAAAASABgAFAASABMACAAAAAwABAASAAAAeAAAACAAAAAMAAAAAAABEGAAAAABAAAAJAAAAAAABgAIAAQABgAAAAQAAAAQABYAEAAOAA8ABAAAAAgAEAAAABgAAAAcAAAAAAABAxgAAAAAAAYACAAGAAYAAAAAAAEAAAAAAAQAAABpdGVtAAAAAAEAAAB0AAAAAgAAAFgAAAAEAAAAuP///wgAAAAkAAAAGAAAAGFycm93LmZpeGVkX3NoYXBlX3RlbnNvcgAAAAAUAAAAQVJST1c6ZXh0ZW5zaW9uOm5hbWUAAAAACAAMAAgABAAIAAAACAAAABgAAAAPAAAAeyJzaGFwZSI6WzIsMl19ABgAAABBUlJPVzpleHRlbnNpb246bWV0YWRhdGEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAADAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAAAQAAAAAAAAAAAAAAAgAAAAEAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAEAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAABAAAAAAAAAIAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAsAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAIA/AAAAQAAAAAAAAIBAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "000002d7785502ea6e57ffa371d47b87421a536786dc9417a775b74d94b2f951d78e37" + }, + { + "id": "fsl_ext_tensor_rb_f0", + "description": "the same tensor batch without include_metadata", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "/////3gBAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAYAAAAAAASABgAFAASABMACAAAAAwABAASAAAAeAAAACAAAAAMAAAAAAABEGAAAAABAAAAJAAAAAAABgAIAAQABgAAAAQAAAAQABYAEAAOAA8ABAAAAAgAEAAAABgAAAAcAAAAAAABAxgAAAAAAAYACAAGAAYAAAAAAAEAAAAAAAQAAABpdGVtAAAAAAEAAAB0AAAAAgAAAFgAAAAEAAAAuP///wgAAAAkAAAAGAAAAGFycm93LmZpeGVkX3NoYXBlX3RlbnNvcgAAAAAUAAAAQVJST1c6ZXh0ZW5zaW9uOm5hbWUAAAAACAAMAAgABAAIAAAACAAAABgAAAAPAAAAeyJzaGFwZSI6WzIsMl19ABgAAABBUlJPVzpleHRlbnNpb246bWV0YWRhdGEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////7gAAAAQAAAADAAaABgAFwAEAAgADAAAACAAAADAAAAAAAAAAAAAAAAAAAADBAAKABgADAAIAAQACgAAADwAAAAQAAAAAQAAAAAAAAAAAAAAAgAAAAEAAAAAAAAAAAAAAAAAAAAEAAAAAAAAAAEAAAAAAAAAAAAAAAMAAAAAAAAAAAAAAAEAAAAAAAAAQAAAAAAAAAABAAAAAAAAAIAAAAAAAAAAEAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAsAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAIA/AAAAQAAAAAAAAIBAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA/////wAAAAA=", + "expected_hash": "000002e1f341f5fd5958c1c42a1c8bd02b2c327c860dd37529c84ba68e18301ff18435" + }, + { + "id": "fsl_tensor_storage_rb_f0", + "description": "t: FixedSizeList? rows [[1, 2, null, 4]] (the storage alone)", + "method": "hash_record_batch", + "include_metadata": false, + "ordered_columns": false, + "ordered_struct_fields": false, + "ipc_b64": "//////gAAAAQAAAAAAAKAAwACgAJAAQACgAAABAAAAAAAQQACAAIAAAABAAIAAAABAAAAAEAAAAUAAAAEAAUABAADgAPAAQAAAAIABAAAAAgAAAADAAAAAAAARBgAAAAAQAAACQAAAAAAAYACAAEAAYAAAAEAAAAEAAWABAADgAPAAQAAAAIABAAAAAYAAAAHAAAAAAAAQMYAAAAAAAGAAgABgAGAAAAAAABAAAAAAAEAAAAaXRlbQAAAAABAAAAdAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////+4AAAAEAAAAAwAGgAYABcABAAIAAwAAAAgAAAAwAAAAAAAAAAAAAAAAAAAAwQACgAYAAwACAAEAAoAAAA8AAAAEAAAAAEAAAAAAAAAAAAAAAIAAAABAAAAAAAAAAAAAAAAAAAABAAAAAAAAAABAAAAAAAAAAAAAAADAAAAAAAAAAAAAAABAAAAAAAAAEAAAAAAAAAAAQAAAAAAAACAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAALAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACAPwAAAEAAAAAAAACAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////8AAAAA", + "expected_hash": "000002e1f341f5fd5958c1c42a1c8bd02b2c327c860dd37529c84ba68e18301ff18435" } ], "relations": [ @@ -5219,6 +5949,288 @@ "nested_hidden_depth4_large_dict_array_f0" ], "why": "dictionary encoding and list widths preserve array hashes when null descendants are hidden" + }, + { + "kind": "equal", + "ids": [ + "fsl_spec_example_rb_f0", + "fsl_spec_example_hidden_null_rb_f0", + "fsl_spec_example_hidden_zero_rb_f0" + ], + "why": "child positions under a null fixed-size-list slot are not elements" + }, + { + "kind": "equal", + "ids": [ + "fsl_spec_example_array_f0", + "fsl_spec_example_hidden_null_array_f0" + ], + "why": "child positions under a null fixed-size-list slot are not elements (hash_array)" + }, + { + "kind": "different", + "ids": [ + "fsl_spec_example_rb_f0", + "fsl_spec_example_visible_rb_f0" + ], + "why": "visible fixed-size-list items are hashed" + }, + { + "kind": "equal", + "ids": [ + "fsl_spec_example_as_list_rb_f0", + "fsl_spec_example_as_large_list_rb_f0" + ], + "why": "List and LargeList are the same type" + }, + { + "kind": "different", + "ids": [ + "fsl_spec_example_rb_f0", + "fsl_spec_example_as_list_rb_f0" + ], + "why": "FixedSizeList and List are different types, also for equal values (record batch)" + }, + { + "kind": "different", + "ids": [ + "fsl_spec_example_schema_f0", + "fsl_spec_example_as_list_schema_f0" + ], + "why": "FixedSizeList and List are different types (schema)" + }, + { + "kind": "different", + "ids": [ + "fsl_spec_example_array_f0", + "fsl_spec_example_as_list_array_f0" + ], + "why": "FixedSizeList and List are different types, also for equal values (hash_array)" + }, + { + "kind": "different", + "ids": [ + "fsl_spec_example_rb_f0", + "fsl_spec_example_rb_f3" + ], + "why": "order options change the header" + }, + { + "kind": "different", + "ids": [ + "fsl_root_array_f0", + "fsl_root_as_list_array_f0" + ], + "why": "FixedSizeList and List are different types (hash_array root)" + }, + { + "kind": "different", + "ids": [ + "fsl_size1_rb_f0", + "fsl_size2_rb_f0", + "fsl_size4_rb_f0" + ], + "why": "FixedSizeList sizes are different types" + }, + { + "kind": "different", + "ids": [ + "fsl_size2_empty_stream_f0", + "fsl_size3_empty_stream_f0" + ], + "why": "FixedSizeList<_, 2> and FixedSizeList<_, 3> are different types (no data)" + }, + { + "kind": "different", + "ids": [ + "fsl_size2_empty_array_f0", + "fsl_size3_empty_array_f0" + ], + "why": "FixedSizeList sizes are different types (hash_array)" + }, + { + "kind": "different", + "ids": [ + "fsl_fsb2_rb_f0", + "fsl_uint8x2_rb_f0" + ], + "why": "FixedSizeBinary(n) and FixedSizeList are different types" + }, + { + "kind": "different", + "ids": [ + "fsl_fsb2_array_f0", + "fsl_uint8x2_array_f0" + ], + "why": "FixedSizeBinary(n) and FixedSizeList are different types (hash_array)" + }, + { + "kind": "equal", + "ids": [ + "fsl_fsl_hidden_values_rb_f0", + "fsl_fsl_hidden_nulls_rb_f0" + ], + "why": "inner slots under a null outer slot are not elements, whatever they hold" + }, + { + "kind": "equal", + "ids": [ + "fsl_fsl_hidden_values_array_f0", + "fsl_fsl_hidden_nulls_array_f0" + ], + "why": "inner slots under a null outer slot are not elements (hash_array)" + }, + { + "kind": "equal", + "ids": [ + "fsl_struct_masked_values_rb_f0", + "fsl_struct_masked_nulls_rb_f0" + ], + "why": "a fixed-size-list slot under a null struct is effectively null: its items are hidden" + }, + { + "kind": "equal", + "ids": [ + "fsl_struct2_masked_values_rb_f0", + "fsl_struct2_masked_nulls_rb_f0" + ], + "why": "struct masking two levels up hides the fixed-size-list items" + }, + { + "kind": "different", + "ids": [ + "fsl_zero_rb_f0", + "fsl_zero_two_rows_rb_f0", + "fsl_zero_null_middle_rb_f0", + "fsl_zero_null_first_rb_f0", + "fsl_zero_non_null_rb_f0" + ], + "why": "zero-size lists: row count, validity and nullability are the information" + }, + { + "kind": "equal", + "ids": [ + "fsl_zero_rb_f0", + "fsl_zero_all_valid_rb_f0" + ], + "why": "an all-valid validity buffer equals none" + }, + { + "kind": "different", + "ids": [ + "fsl_zero_null_middle_array_f0", + "fsl_zero_null_first_array_f0", + "fsl_zero_array_f0" + ], + "why": "zero-size lists that differ only in validity are different (hash_array)" + }, + { + "kind": "equal", + "ids": [ + "fsl_in_null_list_hidden_rb_f0", + "fsl_in_null_list_values_rb_f0", + "fsl_in_null_list_clean_rb_f0" + ], + "why": "fixed-size lists under a null List value are not elements" + }, + { + "kind": "equal", + "ids": [ + "fsl_fresh_rb_f0", + "fsl_sliced_rb_f0", + "fsl_split_stream_f0", + "fsl_trailing_child_rb_f0" + ], + "why": "slices, batch splits and trailing child positions do not change the hash" + }, + { + "kind": "equal", + "ids": [ + "fsl_fresh_array_f0", + "fsl_sliced_array_f0" + ], + "why": "a slice hashes like the logical value (hash_array)" + }, + { + "kind": "equal", + "ids": [ + "fsl_struct_fresh_rb_f0", + "fsl_struct_sliced_rb_f0" + ], + "why": "a sliced struct of fixed-size lists hashes like the logical value" + }, + { + "kind": "equal", + "ids": [ + "fsl_dict_rb_f0", + "fsl_dict_plain_rb_f0", + "fsl_dict_split_stream_f0" + ], + "why": "a dictionary of fixed-size lists is its decoded values" + }, + { + "kind": "equal", + "ids": [ + "fsl_dict_array_f0", + "fsl_dict_plain_array_f0" + ], + "why": "a dictionary of fixed-size lists is its decoded values (hash_array)" + }, + { + "kind": "equal", + "ids": [ + "fsl_of_struct_ab_rb_f0", + "fsl_of_struct_ba_rb_f0" + ], + "why": "struct field order inside a fixed-size list is ignored by default" + }, + { + "kind": "different", + "ids": [ + "fsl_of_struct_ab_rb_f2", + "fsl_of_struct_ba_rb_f2" + ], + "why": "ordered_struct_fields applies inside a fixed-size list" + }, + { + "kind": "equal", + "ids": [ + "fsl_of_dict_rb_f0", + "fsl_of_dict_plain_rb_f0" + ], + "why": "a dictionary item is its decoded values" + }, + { + "kind": "equal", + "ids": [ + "fsl_meta_element_rb_f0", + "fsl_no_meta_rb_f0" + ], + "why": "without include_metadata element metadata is not hashed" + }, + { + "kind": "different", + "ids": [ + "fsl_meta_element_rb_f0_im1", + "fsl_no_meta_rb_f0" + ], + "why": "with include_metadata element metadata (path f/) is hashed" + }, + { + "kind": "equal", + "ids": [ + "fsl_ext_tensor_rb_f0", + "fsl_tensor_storage_rb_f0" + ], + "why": "an extension type hashes as its storage without include_metadata" + }, + { + "kind": "different", + "ids": [ + "fsl_ext_tensor_rb_f0_im1", + "fsl_tensor_storage_rb_f0" + ], + "why": "with include_metadata the extension keys are hashed" } ] } diff --git a/tests/test_format2.py b/tests/test_format2.py index 891a41e..c0901cc 100644 --- a/tests/test_format2.py +++ b/tests/test_format2.py @@ -511,9 +511,11 @@ def test_streaming_check() -> None: @pytest.mark.parametrize( "t", - [pa.null(), pa.list_(pa.int8(), 2), pa.map_(pa.utf8(), pa.int8()), pa.month_day_nano_interval(), - pa.struct([pa.field("n", pa.null())]), pa.list_(pa.null())], - ids=["null", "fixed_size_list", "map", "interval", "nested_null", "list_of_null"], + [pa.null(), pa.map_(pa.utf8(), pa.int8()), pa.month_day_nano_interval(), + pa.struct([pa.field("n", pa.null())]), pa.list_(pa.null()), + pa.list_(pa.null(), 2), pa.list_(pa.map_(pa.utf8(), pa.int8()), 2), pa.list_(pa.string_view(), 0)], + ids=["null", "map", "interval", "nested_null", "list_of_null", + "fixed_size_list_of_null", "fixed_size_list_of_map", "fixed_size_list_of_view"], ) def test_unsupported_data_rejected_whatever_the_row_count(t) -> None: """Spec 7 item 3: rejected before any data is hashed, also with zero rows / no batches.""" @@ -1045,3 +1047,234 @@ def test_dictionary_of_struct_leaf_null_hidden_under_null_outer_struct(child_val assert f2.hash_record_batch( pa.record_batch([dict_arr], schema=pa.schema([pa.field("d", dict_arr.type)])), **options ) == f2.hash_record_batch(pa.record_batch([plain], schema=pa.schema([pa.field("d", outer_t)])), **options), options + + +# ── FixedSizeList data (ITL-927) ──────────────────────────────────────── +# +# Spec 3.2: a fixed-size-list node's entry is its (effective) validity if nullable and empty +# otherwise (no lengths digest, no values). Its element node gets the n items of every +# effectively valid slot; items under null or struct-masked slots and child positions beyond +# len * n are not elements. Cross-language values: the fsl_* golden vectors. + +_FSL_I32 = b'{"FixedSizeList":[{"data_type":"Int32","nullable":true},2]}' + + +def _fsl(child: pa.Array, n: int, valid: list[bool] | None, length: int, *, offset: int = 0, + item_nullable: bool = True) -> pa.Array: + """A FixedSizeList array built from buffers: `length` slots at `offset` + over `child` (which may hold more positions than (offset + length) * n).""" + t = pa.list_(pa.field("item", child.type, nullable=item_nullable), n) + validity = None + if valid is not None: + bits = sum(1 << i for i, ok in enumerate(valid) if ok) + validity = pa.py_buffer(bits.to_bytes((len(valid) + 7) // 8 or 1, "little")) + return pa.Array.from_buffers(t, length, [validity], offset=offset, children=[child]) + + +def test_fixed_size_list_spec_example_3b_bytes() -> None: + """Spec Example 3b: rows [[1, null], null, [3, 4]], child [1, null, 7, 8, 3, 4].""" + arr = _fsl(pa.array([1, None, 7, 8, 3, 4], pa.int32()), 2, [True, False, True], 3) + assert arr.to_pylist() == [[1, None], None, [3, 4]] + batch = pa.record_batch([arr], schema=pa.schema([pa.field("f", arr.type, nullable=True)])) + final = sha( + hr(), + sd(b'{"f":{"data_type":' + _FSL_I32 + b',"nullable":true}}'), + u64(3), + u64(3) + b"\x05", # entry "f": validity [1,0,1], no lengths + u64(4) + b"\x0d", # entry "f/": validity [1,0,1,1] (items of slots 0 and 2 only) + sha(i32(1, 3, 4)), + ) + assert f2.hash_record_batch(batch) == V + final + # The hash_array root is always nullable; its element node is "/". + assert f2.hash_array(arr) == V + sha(ha(), _FSL_I32, u64(3), u64(3) + b"\x05", u64(4) + b"\x0d", sha(i32(1, 3, 4))) + + +def test_fixed_size_list_non_nullable_entry_is_empty() -> None: + """A non-nullable fixed-size list contributes no item at all; n is in the schema only.""" + t = pa.list_(pa.field("item", pa.int32(), nullable=False), 2) + arr = pa.array([[1, 2], [3, 4]], t) + schema = pa.schema([pa.field("f", t, nullable=False)]) + final = sha( + hr(), + sd(b'{"f":{"data_type":{"FixedSizeList":[{"data_type":"Int32","nullable":false},2]},"nullable":false}}'), + u64(2), + sha(i32(1, 2, 3, 4)), # entry "f" is empty; entry "f/" is the values digest + ) + assert f2.hash_record_batch(pa.record_batch([arr], schema=schema)) == V + final + + +def test_fixed_size_list_size_zero_bytes() -> None: + """n = 0: the element subtree has no elements; row count and validity carry everything.""" + arr = _fsl(pa.array([], pa.int32()), 0, [True, False, True], 3) + assert arr.to_pylist() == [[], None, []] + t_json = b'{"FixedSizeList":[{"data_type":"Int32","nullable":true},0]}' + empty_child = u64(0) + sha(b"") # validity with bit_count 0, values digest of nothing + assert f2.hash_array(arr) == V + sha(ha(), t_json, u64(3), u64(3) + b"\x05", empty_child) + schema = pa.schema([pa.field("f", arr.type, nullable=False)]) + no_nulls = _fsl(pa.array([], pa.int32()), 0, None, 3) + final = sha(hr(), sd(b'{"f":{"data_type":' + t_json + b',"nullable":false}}'), u64(3), empty_child) + assert f2.hash_record_batch(pa.record_batch([no_nulls], schema=schema)) == V + final + assert f2.hash_record_batch(pa.record_batch([no_nulls.slice(1)], schema=schema)) != V + final + + +def test_fixed_size_list_slices_and_trailing_child_positions() -> None: + """Slot i covers child [(offset + i) * n, (offset + i + 1) * n): pyarrow's .values ignores the + array offset. Positions past len * n are not elements. (IPC normalizes both away, so the + golden vectors cannot pin this; the fresh array is pinned by fsl_fresh_rb_f0.)""" + fresh = _fsl(pa.array([1, 2, 7, 7, 3, None, 5, 6], pa.int32()), 2, [True, False, True, True], 4) + assert fresh.to_pylist() == [[1, 2], None, [3, None], [5, 6]] + # Physical slots: 0 (before the offset), 1..4 (the rows), 5 and two more child positions + # (trailing); the child array itself starts at offset 3. + child = pa.array([42, 42, 42, 0, 0, 1, 2, 7, 7, 3, None, 5, 6, 9, 9, 8, None], pa.int32()).slice(3) + sliced = _fsl(child, 2, [True, True, False, True, True, True], 4, offset=1) + assert sliced.offset == 1 and sliced.to_pylist() == fresh.to_pylist() + trailing = _fsl(pa.array([1, 2, 7, 7, 3, None, 5, 6, 8, None], pa.int32()), 2, [True, False, True, True], 4) + schema = pa.schema([pa.field("f", fresh.type)]) + expected = f2.hash_record_batch(pa.record_batch([fresh], schema=schema)) + items = u64(6) + bytes([0b110111]) # entry "f/": items of slots 0, 2, 3 + assert expected == V + sha(hr(), sd(b'{"f":{"data_type":' + _FSL_I32 + b',"nullable":true}}'), u64(4), + u64(4) + b"\x0d", items, sha(i32(1, 2, 3, 5, 6))) + for arr in (sliced, trailing, fresh.slice(1).slice(0, 0), fresh.slice(2)): + got = f2.hash_record_batch(pa.record_batch([arr], schema=schema)) + same = f2.hash_record_batch(pa.record_batch([pa.array(arr.to_pylist(), fresh.type)], schema=schema)) + assert got == same + assert f2.hash_record_batch(pa.record_batch([sliced], schema=schema)) == expected + assert f2.hash_record_batch(pa.record_batch([trailing], schema=schema)) == expected + assert f2.hash_array(sliced) == f2.hash_array(trailing) == f2.hash_array(fresh) + d = f2.Digester(schema) + for start, stop in ((0, 1), (1, 1), (1, 3), (3, 4)): + d.update(pa.record_batch([sliced.slice(start, stop - start)], schema=schema)) + assert d.finalize() == expected + + +def test_fixed_size_list_struct_masking_two_depths_on_sliced_arrays() -> None: + """s: Struct?>?>?; the FSL's own validity and both + struct validities disagree, every level sits at a different offset, and hidden child + positions hold junk. Logical rows: s = [1,0,1,1,1], t = [1,1,0,1,1], f = [0,1,1,1,1]; the + effectively valid slots are rows 3 and 4: [3, null] and [5, 6].""" + f_type = pa.list_(pa.field("item", pa.int32(), nullable=True), 2) + t_type = pa.struct([pa.field("f", f_type, nullable=True)]) + s_type = pa.struct([pa.field("t", t_type, nullable=True)]) + + def bitmap(valid: list[bool]) -> pa.Buffer: + return pa.py_buffer(sum(1 << i for i, ok in enumerate(valid) if ok).to_bytes((len(valid) + 7) // 8, "little")) + + # f_full: 10 slots; rows 5..9 are logical rows 0..4, and two trailing child positions + # follow. Its values child itself has offset 1. + f_full_valid = [True] * 5 + [False, True, True, True, True] + junk = {5: [11, None], 6: [None, 12], 7: [13, 14]} # hidden under f's own null, s's null, t's null + data = [*range(10)] + junk[5] + junk[6] + junk[7] + [3, None, 5, 6, 77, None] + values = pa.array([99, *data], pa.int32()).slice(1) + f_full = pa.Array.from_buffers(f_type, 10, [bitmap(f_full_valid)], children=[values]) + f_child = f_full.slice(3) # offset 3; t row k is f_full row 3 + k + t_valid = [True, False, True, True, False, True, True] # t rows 2..6 are logical rows 0..4 + t = pa.Array.from_buffers(t_type, 7, [bitmap(t_valid)], children=[f_child]) + s = pa.Array.from_buffers(s_type, 5, [bitmap([False, False, True, False, True, True, True])], offset=2, + children=[t]) + assert s.to_pylist() == [{"t": {"f": None}}, None, {"t": None}, {"t": {"f": [3, None]}}, {"t": {"f": [5, 6]}}] + + fresh = pa.array(s.to_pylist(), s_type) + schema = pa.schema([pa.field("s", s_type, nullable=True)]) + json_ = (b'{"s":{"data_type":{"Struct":[{"data_type":{"Struct":[{"data_type":' + _FSL_I32 + + b',"name":"f","nullable":true}]},"name":"t","nullable":true}]},"nullable":true}}') + expected = V + sha( + hr(), sd(json_), u64(5), + u64(5) + b"\x1d", # s: [1,0,1,1,1] + u64(5) + b"\x19", # s/t: [1,0,0,1,1] + u64(5) + b"\x18", # s/t/f: [0,0,0,1,1] + u64(4) + b"\x0d", # s/t/f/: [1,0,1,1] + sha(i32(3, 5, 6)), + ) + for arr in (s, fresh): + assert f2.hash_record_batch(pa.record_batch([arr], schema=schema)) == expected + d = f2.Digester(schema) + for row in range(5): + d.update(pa.record_batch([arr.slice(row, 1)], schema=schema)) + assert d.finalize() == expected + assert f2.hash_array(s) == f2.hash_array(fresh) + + +def test_fixed_size_list_non_nullable_slots_and_items() -> None: + """Spec 7 item 4: a visible null (slot of a non-nullable FSL, or item of a non-nullable + element) is rejected; nulls hidden under a null slot are accepted and ignored.""" + t = pa.list_(pa.field("item", pa.int32(), nullable=False), 2) + nullable = pa.schema([pa.field("f", t, nullable=True)]) + hidden = _fsl(pa.array([1, 2, None, None, 3, 4], pa.int32()), 2, [True, False, True], 3, item_nullable=False) + clean = _fsl(pa.array([1, 2, 0, 0, 3, 4], pa.int32()), 2, [True, False, True], 3, item_nullable=False) + assert f2.hash_record_batch(pa.record_batch([hidden], schema=nullable)) == f2.hash_record_batch( + pa.record_batch([clean], schema=nullable)) + assert f2.hash_array(hidden) == f2.hash_array(clean) + + visible = _fsl(pa.array([1, None, 0, 0, 3, 4], pa.int32()), 2, [True, False, True], 3, item_nullable=False) + with pytest.raises(ValueError, match="Non-nullable field 'f/' contains nulls"): + f2.hash_record_batch(pa.record_batch([visible], schema=nullable)) + with pytest.raises(ValueError, match="Non-nullable field '/' contains nulls"): + f2.hash_array(visible) + non_nullable = pa.schema([pa.field("f", t, nullable=False)]) + with pytest.raises(ValueError, match="Non-nullable field 'f' contains nulls"): + f2.hash_record_batch(pa.record_batch([clean], schema=non_nullable)) + # A rejected batch leaves the digester unchanged. + d = f2.Digester(nullable) + d.update(pa.record_batch([clean], schema=nullable)) + with pytest.raises(ValueError): + d.update(pa.record_batch([visible], schema=nullable)) + assert d.finalize() == f2.hash_record_batch(pa.record_batch([clean], schema=nullable)) + + +def test_fixed_size_list_differs_from_list_and_from_other_sizes() -> None: + """Maintainer decision: a fixed-size list is not type-identical to a variable-size list + (List and LargeList are: both are LargeList in the canonical form).""" + rows = [[1, 2], [3, 4]] + as_list, as_large_list = (pa.array(rows, t) for t in (pa.list_(pa.int32()), pa.large_list(pa.int32()))) + assert f2.hash_array(as_list) == f2.hash_array(as_large_list) + hashes = {f2.hash_array(pa.array(rows, t)) for t in (pa.list_(pa.int32(), 2), pa.list_(pa.int32()))} + hashes.add(f2.hash_array(pa.array([b"\x01\x02", b"\x03\x04"], pa.binary(2)))) + hashes.add(f2.hash_array(pa.FixedSizeListArray.from_arrays(pa.array([1, 2, 3, 4], pa.int32()), 4))) + hashes.add(f2.hash_array(pa.FixedSizeListArray.from_arrays(pa.array([1, 2, 3, 4], pa.int32()), 1))) + assert len(hashes) == 5 + schema_hashes = {f2.hash_schema(pa.schema([pa.field("f", t)])) + for t in (pa.list_(pa.int32(), 2), pa.list_(pa.int32(), 3), pa.list_(pa.int32()))} + assert len(schema_hashes) == 3 + + +def test_dictionary_of_fixed_size_list_equals_plain() -> None: + t = pa.list_(pa.field("item", pa.int32(), nullable=False), 2) + values = _fsl(pa.array([1, 2, None, None, 3, 4], pa.int32()), 2, [True, False, True], 3, item_nullable=False) + keys = pa.array([2, None, 1, 0, 2], pa.int8()) + dict_arr = pa.DictionaryArray.from_arrays(keys, values) + plain = pa.array([[3, 4], None, None, [1, 2], [3, 4]], t) + assert f2.hash_array(dict_arr) == f2.hash_array(plain) + for arr in (dict_arr, dict_arr.slice(1)): + ref = plain if arr is dict_arr else plain.slice(1) + assert f2.hash_record_batch(pa.record_batch([arr], names=["f"])) == f2.hash_record_batch( + pa.record_batch([ref], names=["f"])) + + +@pytest.mark.parametrize("budget", [1, 2, 3, 5, 64]) +def test_fixed_size_list_items_visited_in_bounded_chunks(budget, monkeypatch) -> None: + """Slots with nulls are visited in chunks of about _ITEM_BUDGET items; any chunk size gives + the hash of one pass (the reference is computed with the default budget).""" + t = pa.list_(pa.field("item", pa.list_(pa.utf8()), nullable=True), 3) + rows = [[["a"], None, []], None, [["b", None], ["c"], None], None, None, [[], [], ["d"]], [None, None, None]] + arr = pa.array(rows, t) + nested = pa.array([rows, None, rows[2:]], pa.list_(t)) # List> + expected = [f2.hash_array(a) for a in (arr, arr.slice(2), nested)] + monkeypatch.setattr(f2, "_ITEM_BUDGET", budget) + assert [f2.hash_array(a) for a in (arr, arr.slice(2), nested)] == expected + + +def test_fixed_size_list_without_nulls_builds_no_item_index(monkeypatch) -> None: + """Without null slots the child is one zero-copy slice: no per-item index or mask.""" + def forbidden(*_args, **_kwargs): + raise AssertionError("per-item index built on the null-free path") + + arr = pa.FixedSizeListArray.from_arrays(pa.array(range(4 * 384), pa.float32()), 384) + expected = f2.hash_array(arr) + batch = pa.record_batch([arr], names=["lfp"]) + expected_rb = f2.hash_record_batch(batch) + monkeypatch.setattr(f2, "_repeat_each", forbidden) + monkeypatch.setattr(f2.pc, "list_parent_indices", forbidden) + monkeypatch.setattr(f2.pc, "take", forbidden) + monkeypatch.setattr(f2.pc, "filter", forbidden) + assert f2.hash_array(arr) == expected + assert f2.hash_record_batch(batch) == expected_rb diff --git a/tests/test_format2_type_matrix.py b/tests/test_format2_type_matrix.py index 1c386bc..e0dd080 100644 --- a/tests/test_format2_type_matrix.py +++ b/tests/test_format2_type_matrix.py @@ -5,8 +5,10 @@ ``hash_array``, ``hash_record_batch`` and the streaming digester for every supported leaf type, with and without nulls, contiguous, sliced at a non-zero offset, split into chunks / batches, dictionary-encoded (null keys, valid keys to null values, nested dictionaries), -and nested under list / large_list / struct / dictionary. CI runs it on pyarrow 19.0.0 -and on the latest pyarrow. +and nested under list / large_list / struct / dictionary / fixed-size list (n = 0, 1, 2, +with a null slot and trailing child positions; fixed-size lists inside list / struct / +dictionary and lists / structs / fixed-size lists inside them). CI runs it on pyarrow +19.0.0, 20.0.0 and the latest pyarrow. Expected values: @@ -223,6 +225,34 @@ def outer(child: pa.Array) -> pa.Array: return outer +def _fsl_slots(n: int, child: pa.Array) -> int: + return len(child) if n == 0 else max(0, (len(child) - 1) // n) + + +def _as_fsl(n: int) -> Callable[[pa.Array], pa.Array]: + """FixedSizeList<_, n> over the child, built from buffers: slot 1 is null and hides its n + items, and at least one child position trails the last slot (n = 0: all of them).""" + def wrap(child: pa.Array) -> pa.Array: + slots = _fsl_slots(n, child) + t = pa.list_(pa.field("item", child.type, nullable=True), n) + return pa.Array.from_buffers(t, slots, [_validity(slots, (1,) if slots > 1 else ())], children=[child]) + return wrap + + +def _as_fsl_clean(n: int) -> Callable[[pa.Array], pa.Array]: + """The logical value of _as_fsl(n)(child) with other physical content: nulls under the + null slot and no trailing child positions (concatenated slices, no compute kernel).""" + def wrap(child: pa.Array) -> pa.Array: + slots = _fsl_slots(n, child) + if slots > 1 and n: + items = pa.concat_arrays([child.slice(0, n), pa.nulls(n, child.type), child.slice(2 * n, (slots - 2) * n)]) + else: + items = child.slice(0, slots * n) + t = pa.list_(pa.field("item", child.type, nullable=True), n) + return pa.Array.from_buffers(t, slots, [_validity(slots, (1,) if slots > 1 else ())], children=[items]) + return wrap + + _WRAPPERS: dict[str, tuple[Callable[[pa.Array], pa.Array], Callable[[pa.Array], pa.Array]]] = { # name: (wrapper, reference wrapper of the plain child) "leaf": (lambda a: a, lambda a: a), @@ -237,6 +267,21 @@ def outer(child: pa.Array) -> pa.Array: _as_dict_of(lambda a: _as_list(True)(_as_struct(a))), _plain_dict_of(lambda a: _as_list(True)(_as_struct(a))), ), + # FixedSizeList (ITL-927); each reference holds other content under null slots and no + # trailing child positions, so hidden content is checked for every leaf type as well. + "fsl[2]": (_as_fsl(2), _as_fsl_clean(2)), + "fsl[1]": (_as_fsl(1), _as_fsl_clean(1)), + "fsl[0]": (_as_fsl(0), _as_fsl_clean(0)), + "fsl": (lambda a: _as_fsl(2)(_as_struct(a)), lambda a: _as_fsl_clean(2)(_as_struct(a))), + "fsl": (lambda a: _as_fsl(1)(_as_list(True)(a)), lambda a: _as_fsl_clean(1)(_as_list(True)(a))), + "fsl": (lambda a: _as_fsl(2)(_as_fsl(2)(a)), lambda a: _as_fsl_clean(2)(_as_fsl_clean(2)(a))), + "list": (lambda a: _as_list(False)(_as_fsl(2)(a)), lambda a: _as_list(False)(_as_fsl_clean(2)(a))), + "struct": (lambda a: _as_struct(_as_fsl(2)(a)), lambda a: _as_struct(_as_fsl_clean(2)(a))), + "dict": (_as_dict_of(_as_fsl(2)), _plain_dict_of(_as_fsl_clean(2))), + "dict>": ( + _as_dict_of(lambda a: _as_fsl(1)(_as_struct(a))), + _plain_dict_of(lambda a: _as_fsl_clean(1)(_as_struct(a))), + ), } diff --git a/tests/test_golden_parity_format2.py b/tests/test_golden_parity_format2.py index cbbe146..3781247 100644 --- a/tests/test_golden_parity_format2.py +++ b/tests/test_golden_parity_format2.py @@ -65,8 +65,8 @@ def _hash(vector: dict[str, Any]) -> str: # Counts of the final golden file; the file is append-only, so they can only grow. -_MIN_VECTORS = 423 -_MIN_RELATIONS = 114 +_MIN_VECTORS = 496 +_MIN_RELATIONS = 148 def test_fixture_is_well_formed() -> None: