You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[coalesce] Add specialized DictionaryArray path for push_batch_with_indices #11940
#11743 adds a specialized in-progress coalescer for byte dictionaries, but the indexed-copy path introduced by #11174 will still use the default InProgressArray::copy_rows_by_indices implementation. That implementation calls take to materialize an intermediate dictionary before copying it into the coalescer.
Describe the solution you'd like
Add copy_rows_by_indices support to InProgressDictionaryArray for Dictionary<K, Utf8 | LargeUtf8 | Binary | LargeBinary>.
For a pointer-identical values array, gather the selected primitive keys and validity directly into the in-progress buffers while reusing the values Arc. This should avoid the intermediate take allocation while preserving repeated and out-of-order indices, null keys, bounds validation, and target-batch chunking. Inputs with a different values mapping must continue to use a correct fallback.
Use the dictionary take cases in coalesce_kernels to compare the specialized path with the existing materialized path for both shared and changing values arrays.
Activity
Rich-T-kid commentedon Oct 27, 2026
take