Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,13 @@ version; patch releases remain backward compatible.

## Unreleased

- `decode_binary_value` and `encode_binary_value` handle an `Array` or `Map`
run of zero-width `Tuple()` values in constant time per tuple node instead of
once per element. Before, a small nested value such as
`Array(Tuple(Tuple(Tuple())))` could drive work far beyond its input size.
The 1,000,000-element cap per container is unchanged.
- `decode_binary_value` returns `BinaryValueError::Invalid` for a caller-built
`FixedString(0)` or `QBit(T, 0)` type, matching `encode_binary_value`.
- `ch-tcp-proto`: support compressed client-fed INSERT. Upload blocks and the
empty upload terminator use LZ4 frames, with at most 1 MiB of plaintext per
frame. Server schema blocks use the existing compressed receive path.
Expand Down
17 changes: 17 additions & 0 deletions CODEC_CONTRACT.md
Original file line number Diff line number Diff line change
Expand Up @@ -2490,6 +2490,23 @@ aggregate descriptor forms. The server writes SimpleAggregateFunction with
`0x2e` and AggregateFunction with `0x25`; AggregateFunction value encoding
remains unsupported.

A zero-width single value is `Tuple()` or a Tuple whose elements are all
zero-width, through aliases and LowCardinality. It has no value bytes, so an
`Array` or `Map` count of them is not backed by input. Both helpers keep the
1,000,000-element cap on such a count per container. Within the cap,
`decode_binary_value` appends the whole run in one step per tuple node, and
`encode_binary_value` writes the count after checking each child tuple length
once. An empty run does no work in either direction. Encode runs the per-row
checks from the first short row, or from the start when the run's columns carry
a tuple validity bitmap, a LowCardinality dictionary, or a shape mismatch, so
its errors match per-row encoding. `Nullable`, `Array`, `Map`,
Comment thread
joe-clickhouse marked this conversation as resolved.
Resolved
and `Nothing` elements are not zero-width here: each value reads at least one
byte, and a non-null `Nothing` is `Unsupported`. `encode_binary_value` also
caps a run of `Nullable` zero-width values at 1,000,000, so a longer decoded
run does not encode again. `decode_binary_value` rejects
`FixedString(0)` and `QBit(T, 0)` as `Invalid`, since neither type parser can
produce them.

For a top-level SharedVariant or JSON shared cell, strip the top-level
`Nullable(T)` to `T`, or `LowCardinality(Nullable(T))` to `LowCardinality(T)`,
before writing both the descriptor and value. Route null rows as Dynamic NULL
Expand Down
14 changes: 14 additions & 0 deletions scripts/gen_fixtures.sh
Original file line number Diff line number Diff line change
Expand Up @@ -535,6 +535,18 @@ FORMAT Native
SQL
)

# Zero-width Array(Tuple()) values in Dynamic SharedVariant cells. With
# max_types=0 every value goes to SharedVariant as a binary descriptor plus one
# serializeBinary value, where Tuple() is zero bytes.
zero_width_dynamic_query=$(
cat <<'SQL'
SELECT
CAST(arrayMap(x -> tuple(), range(79)), 'Dynamic(max_types=0)') AS v,
CAST([arrayMap(x -> tuple(), range(13)), []], 'Dynamic(max_types=0)') AS w
FORMAT Native
SQL
)

capture() {
local name="$1"
local query="$2"
Expand Down Expand Up @@ -569,4 +581,6 @@ capture "json_string_rev0.native" "${json_aux_query}" \
capture "json_flattened_rev0.native" "${json_aux_query}" \
--data-urlencode "output_format_native_use_flattened_dynamic_and_json_serialization=1"

capture "zero_width_dynamic_rev0.native" "${zero_width_dynamic_query}"

echo "Wrote fixtures to ${fixture_dir}"
Loading
Loading