diff --git a/_posts/2026-11-16-arrow-go-18.9.0.md b/_posts/2026-11-16-arrow-go-18.9.0.md new file mode 100644 index 000000000000..6535a7330a2b --- /dev/null +++ b/_posts/2026-11-16-arrow-go-18.9.0.md @@ -0,0 +1,312 @@ +--- +layout: post +title: "Apache Arrow Go 18.9.0 Release" +date: "2026-11-16 00:00:00" +author: pmc +categories: [release] +--- + + +The Apache Arrow team is pleased to announce the v18.9.0 release of Apache Arrow Go. +This minor release covers 152 commits from 27 distinct contributors. + +## Contributors +```console +$ git shortlog -sn v18.8.0..v18.9.0 + 45 Minh Vu + 37 Harrison Crosse + 31 Matt Topol + 6 singhpratech + 3 Sankalp Thakur + 2 Animesh Kumar + 2 Colton Loftus + 2 Edward Shen + 2 Samuele Resca + 2 Steve O'Brien + 2 Stochastic + 2 Willem Jan + 2 autoCAT + 2 matanper + 1 Arnaud + 1 Arnold Wakim + 1 Ayaan + 1 Jake Thomas + 1 Kyon + 1 Madan Kumar + 1 Nirajan Sah + 1 Ondřej Pavela + 1 Tobias Pütz + 1 hp + 1 peterolkhov + 1 prtkgaur + 1 ump45nose +``` + +## Highlights + +- Go 1.26 is now the minimum supported Go version [#1326](https://github.com/apache/arrow-go/pull/1326) + +### Arrow + +#### Features + +- Added a scalar aggregate framework to the compute package, starting with the `count` and `sum` kernels [#1336](https://github.com/apache/arrow-go/pull/1336) +- Added the `starts_with` compute function, including `string_view`/`binary_view` inputs, `IgnoreCase`, and the Substrait `starts_with` mapping used by +iceberg-go prefix row filters [#1448](https://github.com/apache/arrow-go/pull/1448), [#1467](https://github.com/apache/arrow-go/pull/1467), [#1468](https://github.com/apache/arrow-go/pull/1468) +- Added `SortKey.FloatTotalOrder` to sort floats by IEEE 754 totalOrder, distinguishing `-0`/`+0` and signed NaNs [#1430](https://github.com/apache/arrow-go/pull/1430) +- Added `Decimal32`, `Decimal64`, `StringView` and `BinaryView` scalar types [#1501](https://github.com/apache/arrow-go/pull/1501), [#1490](https://github.com/apache/arrow-go/pull/1490) +- Added `ipc.WithZstdLevel` to choose the zstd compression level of IPC writers [#1376](https://github.com/apache/arrow-go/pull/1376) +- Added `Reader.ContinueFrom` to resume an IPC reader on a stream that arrives as several blobs, a pattern that stopped working in v18.8.0 [#1346](https://github.com/apache/arrow-go/pull/1346) +- The Flight SQL `database/sql` driver now accepts `grpc+tcp://`, `grpc+tls://` and `grpc+unix://` DSNs [#1420](https://github.com/apache/arrow-go/pull/1420) +- The JSON and CSV readers expose their input offset to help locate decoding errors [#1342](https://github.com/apache/arrow-go/pull/1342) + +#### Bug Fixes + +- `is_null`, `is_not_null` and `not` returned wrong masks for sliced arrays, dictionaries with null values, and null, union and run-end encoded arrays, so +filters could silently keep or drop the wrong rows. `is_null` now also honors `NullOptions.NanIsNull` [#1456](https://github.com/apache/arrow-go/pull/1456), +[#1458](https://github.com/apache/arrow-go/pull/1458), [#1461](https://github.com/apache/arrow-go/pull/1461), [#1465](https://github.com/apache/arrow-go/pull/1465), [#1488](https://github.com/apache/arrow-go/pull/1488) +- `NewDatumWithoutOwning` now returns a non-owning Datum; previously `Release()` freed the caller's memory [#1377](https://github.com/apache/arrow-go/pull/1377) +- `and_kleene` and `or_kleene` returned `false` instead of null when `ExecCtx.ChunkSize` was smaller than the input [#1306](https://github.com/apache/arrow-go/pull/1306) +- A cancelled `CallFunction` no longer returns a result with a nil error, and the default `NumParallel` follows the cgroup-aware `GOMAXPROCS` in +containers [#1410](https://github.com/apache/arrow-go/pull/1410), [#1384](https://github.com/apache/arrow-go/pull/1384) +- Fixed generated assembly that clobbered the frame pointer or Go's reserved registers, breaking profilers and frame-pointer unwinders, in the amd64 compute +kernels and the Parquet bit-unpacking and Bloom filter kernels [#1432](https://github.com/apache/arrow-go/pull/1432), [#1300](https://github.com/apache/arrow-go/pull/1300), [#1379](https://github.com/apache/arrow-go/pull/1379) +- Decimal rescaling no longer panics or returns wrong values, infinite decimal strings are rejected, and scales past `MaxScale` fail clearly [#1307](https://github.com/apache/arrow-go/pull/1307), +[#1409](https://github.com/apache/arrow-go/pull/1409), [#1513](https://github.com/apache/arrow-go/pull/1513), [#1514](https://github.com/apache/arrow-go/pull/1514), [#1515](https://github.com/apache/arrow-go/pull/1515) +- The C Data interface exports extension arrays as their storage and imports zero-length fixed-size lists, such as those returned by DataFusion [#1526](https://github.com/apache/arrow-go/pull/1526), +[#1308](https://github.com/apache/arrow-go/pull/1308) +- `arrow.TypeEqual` now compares `LargeListType` like `ListType`, so `array.Concatenate` no longer rejects identical large list types [#1440](https://github.com/apache/arrow-go/pull/1440) +- `NewMappedFileReader` no longer panics on dictionary fields, and Flight SQL `ExecuteIngest` sends the schema for empty inputs [#1365](https://github.com/apache/arrow-go/pull/1365), +[#1433](https://github.com/apache/arrow-go/pull/1433) +- The JSON record reader no longer swallows an unexpected EOF on truncated input [#1341](https://github.com/apache/arrow-go/pull/1341) + +#### Performance Improvements + +Performance remains a major theme: 51 of the merged pull requests are dedicated +optimizations. Unless noted otherwise, the figures below are the PR-reported +benchmarks on an Apple M1 Pro. + +- Bulk array comparisons: comparing an array without a validity bitmap against one with a materialized bitmap is up to 414x faster for booleans (8.8ms to 21.3us +over 1M values) and 15.3x for int64, and all-valid list equality over 65,536 rows drops from 17.8ms to 0.197ms [#1327](https://github.com/apache/arrow-go/pull/1327), +[#1330](https://github.com/apache/arrow-go/pull/1330), [#1347](https://github.com/apache/arrow-go/pull/1347) +- SIMD filtering now covers 8-, 16- and 64-bit types: 1M-element alternating int64 filters go from 3.97ms to 0.45ms (8.7x), int16 from 4.23ms to 225us, and +8-bit alternating masks run 22-24x faster. AVX2 take-index compaction adds up to 4.5x on AMD EPYC [#1311](https://github.com/apache/arrow-go/pull/1311), +[#1349](https://github.com/apache/arrow-go/pull/1349), [#1350](https://github.com/apache/arrow-go/pull/1350), [#1348](https://github.com/apache/arrow-go/pull/1348) +- `SortIndices` radix-sorts single-chunk, null-free integer columns: 65,536 random int64 values sort in 1.31ms instead of 12.3ms [#1391](https://github.com/apache/arrow-go/pull/1391) +- Dictionary-heavy workloads: `unique` over 256 chunks with changing dictionaries is 36.6x faster (60.0ms to 1.64ms) with 98.5% fewer bytes allocated, +concatenating chunks that share a dictionary drops from 82.4us to 3.7us, and builders reuse unchanged dictionaries, cutting the per-row cost of +small-batch dictionary reads through `pqarrow` by up to 93% [#1395](https://github.com/apache/arrow-go/pull/1395), [#1403](https://github.com/apache/arrow-go/pull/1403), [#1444](https://github.com/apache/arrow-go/pull/1444) +- Bitmap utilities: `AppendBools` takes 88% less time (geometric mean), unaligned bitmap operations are up to 11.4x faster, and misaligned bitmap equality and +unaligned transfers improve 6.5x and 5.4x (geometric means) [#1402](https://github.com/apache/arrow-go/pull/1402), [#1437](https://github.com/apache/arrow-go/pull/1437), +[#1414](https://github.com/apache/arrow-go/pull/1414), [#1416](https://github.com/apache/arrow-go/pull/1416) +- IPC zstd compression: pooled decoders cut decompression time by 69-78% for buffers up to 256KiB per column, and pooled encoders take a writer-per-record +workload from 246.4us to 4.5us and from about 2.2MiB to 7.1KiB allocated per op (Apple M2 Pro) [#1331](https://github.com/apache/arrow-go/pull/1331), [#1429](https://github.com/apache/arrow-go/pull/1429) +- Full-file CSV reads stream into builders: 1.69x faster with 71% fewer bytes allocated [#1393](https://github.com/apache/arrow-go/pull/1393) + +### Parquet + +#### Features + +- Added opt-in ALP (Adaptive Lossless floating-Point) encoding for FLOAT and DOUBLE columns, interoperable with Arrow C++. Writers enable it with +`WithALPEncoding`; reading needs no option [#1289](https://github.com/apache/arrow-go/pull/1289) +- Added an optional `parquet/encryption` package with KMS support, so files encrypted by pyarrow's `CryptoFactory` (PKMT1 key material, single or double +wrapping) can be read through `KmsKeyRetriever` [#1478](https://github.com/apache/arrow-go/pull/1478) +- Write geospatial statistics (bounding box and geometry types) for GEOMETRY column chunks, and `null_count` statistics for GEOMETRY and GEOGRAPHY [#1504](https://github.com/apache/arrow-go/pull/1504), +[#1533](https://github.com/apache/arrow-go/pull/1533) + +#### Bug Fixes + +- Encryption interoperability: read encrypted files written by parquet-mr (Spark, Hive, Iceberg on parquet-java), preserve `AES_GCM_CTR_V1` in plaintext +footers, and return an error instead of panicking on columns without a key [#1390](https://github.com/apache/arrow-go/pull/1390), [#1412](https://github.com/apache/arrow-go/pull/1412), [#1406](https://github.com/apache/arrow-go/pull/1406) +- `pqarrow` wrote wrong data for sliced `FixedSizeList` columns and for `FixedSizeList` columns spanning more than one row group [#1382](https://github.com/apache/arrow-go/pull/1382) +- Encoders no longer carry stale state across pages: RLE boolean columns repeated earlier values on later pages, plain boolean and delta encoders wrote +pre-`Reset` state, and all-null `DELTA_BINARY_PACKED` pages were padded with zeros [#1539](https://github.com/apache/arrow-go/pull/1539), [#1545](https://github.com/apache/arrow-go/pull/1545), +[#1546](https://github.com/apache/arrow-go/pull/1546), [#1400](https://github.com/apache/arrow-go/pull/1400) +- Statistics: V2 data pages no longer report zero nulls without page statistics, byte array statistics written from Arrow count empty strings, and +`UpdateSpaced` counts non-null values from the validity bitmap [#1543](https://github.com/apache/arrow-go/pull/1543), [#1527](https://github.com/apache/arrow-go/pull/1527), [#1367](https://github.com/apache/arrow-go/pull/1367) +- Fixed a memory leak in DELTA decoders, and dictionary-encoded boolean columns now return an error instead of panicking [#1394](https://github.com/apache/arrow-go/pull/1394), +[#1547](https://github.com/apache/arrow-go/pull/1547) + +#### Performance Improvements + +- Reading dictionary-encoded columns allocates the dictionary up front again, fixing an allocation regression introduced in v18.8.0 [#1473](https://github.com/apache/arrow-go/pull/1473) +- Bloom filter hashing pools its scratch buffers: hashing 64 values drops from 5.52us to 311ns, and dictionary Bloom filter writes take up to 32% less time +(Intel Core Ultra 7 155H) [#1378](https://github.com/apache/arrow-go/pull/1378) +- Level and boolean decoding: nullable boolean bitmap expansion takes 85% less time (geometric mean), 1-bit packed levels decode in 55-60% less time, all-true and +all-false RLE booleans decode 66-98% faster (amd64), and narrow RLE literal levels encode 2.25-2.61x faster [#1405](https://github.com/apache/arrow-go/pull/1405), +[#1404](https://github.com/apache/arrow-go/pull/1404), [#1417](https://github.com/apache/arrow-go/pull/1417), [#1419](https://github.com/apache/arrow-go/pull/1419) +- Dictionary encoding: indices are encoded in RLE batches (65,536 constant indices drop from 187.7us to 26.0us), and FLBA and Int96 dictionary insertion is +29-33% faster with up to 50% fewer allocations [#1328](https://github.com/apache/arrow-go/pull/1328), [#1312](https://github.com/apache/arrow-go/pull/1312) +- DELTA encodings: constant `DELTA_BINARY_PACKED` blocks decode 2.1x faster, `DELTA_LENGTH_BYTE_ARRAY` decodes 1.62-2.00x and discards 2.43-2.74x faster, +`DELTA_BYTE_ARRAY` prefix matching is up to 4.7x faster (AMD EPYC), and its discard path drops from 65,535 allocations to none [#1396](https://github.com/apache/arrow-go/pull/1396), +[#1415](https://github.com/apache/arrow-go/pull/1415), [#1388](https://github.com/apache/arrow-go/pull/1388), [#1321](https://github.com/apache/arrow-go/pull/1321) +- Reads: optional columns use dense decoding for batches without nulls (up to 45% faster per row on AMD EPYC), and serial reads build struct child readers +without goroutines, 37-43% faster (Apple M4 Max) [#1447](https://github.com/apache/arrow-go/pull/1447), [#1434](https://github.com/apache/arrow-go/pull/1434) + +## Changelog + +### Selected Changes + +A selection of the user-facing changes in this release. See the full changelog link +below for every change in the release, including automated dependency updates. + +#### Arrow + +* feat(arrow/ipc): add Reader.ContinueFrom for streams split across blobs by @zeroshade in [#1346](https://github.com/apache/arrow-go/pull/1346) +* feat(arrow/ipc): add WithZstdLevel option for IPC writer by @Asthenia0412 in [#1376](https://github.com/apache/arrow-go/pull/1376) +* feat(arrow/flight/flightsql/driver): accept grpc+tcp, grpc+tls and grpc+unix schemes by @nirajan1111 in [#1420](https://github.com/apache/arrow-go/pull/1420) +* fix: expose internal offsets in json/csv readers by @Willem-J-an in [#1342](https://github.com/apache/arrow-go/pull/1342) +* feat(arrow/scalar): add StringView and BinaryView scalars by @hcrosse in [#1490](https://github.com/apache/arrow-go/pull/1490) +* feat(arrow/scalar): add Decimal32 and Decimal64 scalars by @hcrosse in [#1501](https://github.com/apache/arrow-go/pull/1501) +* fix(arrow): support zero-length fixed-size lists by @jakthom in [#1308](https://github.com/apache/arrow-go/pull/1308) +* fix(array/record): bldr stop ignore unexpected EOF by @Willem-J-an in [#1341](https://github.com/apache/arrow-go/pull/1341) +* fix(arrow/ipc): open mapped files with dictionary fields by @hpaliwal8 in [#1365](https://github.com/apache/arrow-go/pull/1365) +* fix(arrow): handle dense union slices and empty validation by @skyon0522-ai in [#1385](https://github.com/apache/arrow-go/pull/1385) +* fix(arrow/array): validate bufferless run-end encoded arrays by @zeroshade in [#1421](https://github.com/apache/arrow-go/pull/1421) +* fix(arrow/flight/flightsql): send schema for empty ingest by @arnoldwakim in [#1433](https://github.com/apache/arrow-go/pull/1433) +* fix(arrow): compare LargeListType like ListType in TypeEqual by @paveon in [#1440](https://github.com/apache/arrow-go/pull/1440) +* fix(arrow/bitutil): preserve overlapping bitmap transfers by @fallintoplace in [#1389](https://github.com/apache/arrow-go/pull/1389) +* fix(arrow/decimal): widen Decimal64 rescale to 64 bits by @winklemad in [#1307](https://github.com/apache/arrow-go/pull/1307) +* fix(arrow/decimal): handle native word widths in big integer conversions by @zeroshade in [#1409](https://github.com/apache/arrow-go/pull/1409) +* fix(arrow/decimal): correct rescaling results and report cast errors by @hcrosse in [#1513](https://github.com/apache/arrow-go/pull/1513) +* fix(arrow/decimal): reject infinite decimal strings and bound extreme exponents by @hcrosse in [#1514](https://github.com/apache/arrow-go/pull/1514) +* fix(arrow/decimal): fail clearly for Decimal128 and Decimal256 scales past MaxScale by @hcrosse in [#1515](https://github.com/apache/arrow-go/pull/1515) +* fix(arrow/cdata): export extension arrays as their storage by @hcrosse in [#1526](https://github.com/apache/arrow-go/pull/1526) +* fix(arrow/cdata): export an empty view sizes buffer as the zero region by @hcrosse in [#1544](https://github.com/apache/arrow-go/pull/1544) +* fix(arrow/array): validate extension arrays by their storage type by @hcrosse in [#1487](https://github.com/apache/arrow-go/pull/1487) +* fix(arrow/array): keep the storage dictionary in NewExtensionArrayWithStorage by @hcrosse in [#1540](https://github.com/apache/arrow-go/pull/1540) +* fix(arrow/csv): preserve braces in list values by @peterolkhov in [#1499](https://github.com/apache/arrow-go/pull/1499) +* fix(arrow/csv): preserve lists holding a single empty string by @zeroshade in [#1505](https://github.com/apache/arrow-go/pull/1505) +* fix(arrow/csv): keep list items equal to the null value distinct from nulls by @zeroshade in [#1508](https://github.com/apache/arrow-go/pull/1508) + +#### Compute + +* feat(arrow/compute): scalar aggregate framework with the count and sum kernels by @singhpratech in [#1336](https://github.com/apache/arrow-go/pull/1336) +* feat(arrow/compute): add IEEE 754 total-order float sorting option by @zeroshade in [#1430](https://github.com/apache/arrow-go/pull/1430) +* feat(arrow/compute): add starts_with kernel by @hcrosse in [#1448](https://github.com/apache/arrow-go/pull/1448) +* feat(arrow/compute): support string_view and binary_view in starts_with by @hcrosse in [#1467](https://github.com/apache/arrow-go/pull/1467) +* feat(arrow/compute): support IgnoreCase in starts_with by @hcrosse in [#1468](https://github.com/apache/arrow-go/pull/1468) +* fix(compute): detect temporal rounding overflow by @fallintoplace in [#1127](https://github.com/apache/arrow-go/pull/1127) +* fix(compute/exec): do not carry a cached null count across ArraySpan.SetSlice by @singhpratech in [#1306](https://github.com/apache/arrow-go/pull/1306) +* fix(arrow/compute): allow casting nullable struct fields to non-nullable when no nulls by @sankalpsthakur in [#1363](https://github.com/apache/arrow-go/pull/1363) +* fix(arrow/compute): fix NewDatumWithoutOwning ownership semantics by @Asthenia0412 in [#1377](https://github.com/apache/arrow-go/pull/1377) +* fix(arrow/compute): respect GOMAXPROCS for default NumParallel by @samueleresca in [#1384](https://github.com/apache/arrow-go/pull/1384) +* fix(arrow/compute): do not return a result from a cancelled call by @zeroshade in [#1410](https://github.com/apache/arrow-go/pull/1410) +* fix(arrow/compute): correct amd64 kernel assembly ABI by @zeroshade in [#1432](https://github.com/apache/arrow-go/pull/1432) +* fix(arrow/compute): honor the input offset in is_not_null by @hcrosse in [#1456](https://github.com/apache/arrow-go/pull/1456) +* fix(arrow/compute): register is_null, is_not_null and is_nan as unary functions by @hcrosse in [#1457](https://github.com/apache/arrow-go/pull/1457) +* fix(arrow/compute): honor the input offset in not by @hcrosse in [#1458](https://github.com/apache/arrow-go/pull/1458) +* fix(arrow/compute): treat null dictionary values as null in is_null and is_not_null by @hcrosse in [#1461](https://github.com/apache/arrow-go/pull/1461) +* fix(arrow/compute): handle null, union and run-end encoded arrays in is_null and is_not_null by @hcrosse in [#1465](https://github.com/apache/arrow-go/pull/1465) +* fix(arrow/compute): honor NullOptions.NanIsNull in is_null by @hcrosse in [#1488](https://github.com/apache/arrow-go/pull/1488) +* fix(arrow/compute/exprs): accept string and binary dictionary columns in field references by @hcrosse in [#1449](https://github.com/apache/arrow-go/pull/1449) +* fix(arrow/compute/exprs): fix offsets, nulls and ownership of nested field references by @zeroshade in [#1474](https://github.com/apache/arrow-go/pull/1474) +* fix(arrow/compute): use int16 kernels for casts to int16 by @hcrosse in [#1503](https://github.com/apache/arrow-go/pull/1503) + +#### Parquet + +* feat(parquet): add ALP encoding for FLOAT and DOUBLE columns by @prtkgaur in [#1289](https://github.com/apache/arrow-go/pull/1289) +* feat(parquet/encryption): KMS support for reading PKMT1 key material by @EdwardShen125 in [#1478](https://github.com/apache/arrow-go/pull/1478) +* feat(parquet/file): Write geospatial parquet row group statistics for GEOMETRY column chunks by @C-Loftus in [#1504](https://github.com/apache/arrow-go/pull/1504) +* fix(parquet/file): write `null_count` statistics for GEOMETRY and GEOGRAPHY column chunks by @C-Loftus in [#1533](https://github.com/apache/arrow-go/pull/1533) +* fix(parquet/utils): make unpack32 NEON profiler-safe by @zeroshade in [#1300](https://github.com/apache/arrow-go/pull/1300) +* fix(parquet/metadata): stop bloom filter asm clobbering the frame pointer by @zeroshade in [#1379](https://github.com/apache/arrow-go/pull/1379) +* fix(parquet/variant): prevent metadata offset cursor overflow by @matanper in [#1290](https://github.com/apache/arrow-go/pull/1290) +* fix(parquet/variant): return "" for an empty long-form string by @CaptainAni187 in [#1353](https://github.com/apache/arrow-go/pull/1353) +* fix(parquet/metadata): count non-null values from the validity bitmap in UpdateSpaced by @CaptainAni187 in [#1367](https://github.com/apache/arrow-go/pull/1367) +* fix(parquet/pqarrow): honor offset and null slots when writing fixed-size lists by @zeroshade in [#1382](https://github.com/apache/arrow-go/pull/1382) +* fix(parquet/file): read encrypted page headers written by parquet-mr by @sankalpsthakur in [#1390](https://github.com/apache/arrow-go/pull/1390) +* fix(parquet/internal/encoding): do not leak delta bit width buffers from decoders by @zeroshade in [#1394](https://github.com/apache/arrow-go/pull/1394) +* fix(parquet/internal/encoding): reset delta bit pack encoder state after flush by @zeroshade in [#1400](https://github.com/apache/arrow-go/pull/1400) +* fix(parquet): return an error instead of panicking on a hidden encrypted column by @ump45nose in [#1406](https://github.com/apache/arrow-go/pull/1406) +* fix(parquet): preserve CTR algorithm in plaintext footers by @autoCAT6 in [#1412](https://github.com/apache/arrow-go/pull/1412) +* fix(parquet/pqarrow): release partial readers when building a field reader fails by @elsteveogrande in [#1435](https://github.com/apache/arrow-go/pull/1435) +* fix(parquet): release writer buffers when a column chunk flush fails by @twuebi in [#1506](https://github.com/apache/arrow-go/pull/1506) +* fix(parquet/metadata): count empty strings in byte array statistics from Arrow by @hcrosse in [#1527](https://github.com/apache/arrow-go/pull/1527) +* fix(parquet/compress): read to the end of a Brotli stream that fills dst by @hcrosse in [#1530](https://github.com/apache/arrow-go/pull/1530) +* fix(parquet/internal/encoding): clear buffered RLE booleans after each flush by @hcrosse in [#1539](https://github.com/apache/arrow-go/pull/1539) +* fix(parquet/file): count V2 page num_nulls from the definition levels by @hcrosse in [#1543](https://github.com/apache/arrow-go/pull/1543) +* fix(parquet/internal/encoding): clear buffered RLE booleans on Reset by @hcrosse in [#1545](https://github.com/apache/arrow-go/pull/1545) +* fix(parquet/internal/encoding): clear encoder state on Reset by @hcrosse in [#1546](https://github.com/apache/arrow-go/pull/1546) +* fix(parquet/file): return an error for dictionary encoded boolean columns by @hcrosse in [#1547](https://github.com/apache/arrow-go/pull/1547) + +#### Performance + +* perf(arrow/array): compare optional validity bitmaps in bulk by @fallintoplace in [#1327](https://github.com/apache/arrow-go/pull/1327) +* perf(arrow/array): compare approximate list values by valid runs by @fallintoplace in [#1330](https://github.com/apache/arrow-go/pull/1330) +* perf(arrow/array): compare fixed-size binary values in bulk by @fallintoplace in [#1347](https://github.com/apache/arrow-go/pull/1347) +* perf(arrow/array): copy view buffers directly by @fallintoplace in [#1324](https://github.com/apache/arrow-go/pull/1324) +* perf(arrow/array): concatenate list offsets directly by @fallintoplace in [#1323](https://github.com/apache/arrow-go/pull/1323) +* perf(arrow/array): write bulk binary offsets and payloads directly by @fallintoplace in [#1413](https://github.com/apache/arrow-go/pull/1413) +* perf(arrow/array): reuse unchanged dictionaries across builder arrays by @hcrosse in [#1444](https://github.com/apache/arrow-go/pull/1444) +* perf(arrow/array): reuse dictionary mappings and handle empty chunks in Concatenate by @fallintoplace in [#1403](https://github.com/apache/arrow-go/pull/1403) +* perf(compute): vectorize mixed 64-bit filters by @fallintoplace in [#1311](https://github.com/apache/arrow-go/pull/1311) +* perf(compute): vectorize mixed 16-bit filters by @fallintoplace in [#1349](https://github.com/apache/arrow-go/pull/1349) +* perf(compute): vectorize mixed 8-bit filters by @fallintoplace in [#1350](https://github.com/apache/arrow-go/pull/1350) +* perf(compute): add AVX2 take-index compaction by @fallintoplace in [#1348](https://github.com/apache/arrow-go/pull/1348) +* perf(compute): reuse amd64 SIMD comparisons for numeric-to-bool casts by @fallintoplace in [#1351](https://github.com/apache/arrow-go/pull/1351) +* perf(arrow/compute): read flat take values directly by @fallintoplace in [#1360](https://github.com/apache/arrow-go/pull/1360) +* perf(arrow/compute): avoid getter wrappers in chunked binary take by @fallintoplace in [#1359](https://github.com/apache/arrow-go/pull/1359) +* perf(compute): radix-sort null-free integer indices by @fallintoplace in [#1391](https://github.com/apache/arrow-go/pull/1391) +* perf(arrow/compute): unify changing dictionaries incrementally by @fallintoplace in [#1395](https://github.com/apache/arrow-go/pull/1395) +* perf(compute): use typed memo insertion for binary hashes by @fallintoplace in [#1315](https://github.com/apache/arrow-go/pull/1315) +* perf(bitutil): pack boolean bitmap writes by byte by @fallintoplace in [#1402](https://github.com/apache/arrow-go/pull/1402) +* perf(bitutil): accelerate misaligned bitmap equality by @fallintoplace in [#1414](https://github.com/apache/arrow-go/pull/1414) +* perf(bitutil): bulk-transfer unaligned bitmap ranges by @fallintoplace in [#1416](https://github.com/apache/arrow-go/pull/1416) +* perf(bitutil): process unaligned bitmap operations by word by @fallintoplace in [#1437](https://github.com/apache/arrow-go/pull/1437) +* perf(arrow/csv): stream full-file CSV reads into builders by @fallintoplace in [#1393](https://github.com/apache/arrow-go/pull/1393) +* perf: zstd implement Pooling + DecodeAll with no new reader. by @samueleresca in [#1331](https://github.com/apache/arrow-go/pull/1331) +* perf(arrow/ipc): reuse zstd encoders across writers by @vdpoora in [#1429](https://github.com/apache/arrow-go/pull/1429) +* perf(parquet/metadata): pool bloom filter hash scratch buffers by @zeroshade in [#1378](https://github.com/apache/arrow-go/pull/1378) +* perf(parquet): use typed memo insertion for fixed-width dictionaries by @fallintoplace in [#1312](https://github.com/apache/arrow-go/pull/1312) +* perf(parquet): encode dictionary indices in RLE batches by @fallintoplace in [#1328](https://github.com/apache/arrow-go/pull/1328) +* perf(parquet): reuse DELTA_BYTE_ARRAY discard storage by @fallintoplace in [#1321](https://github.com/apache/arrow-go/pull/1321) +* perf(parquet): reuse DELTA decoder scratch by @fallintoplace in [#1322](https://github.com/apache/arrow-go/pull/1322) +* perf(parquet): speed up delta byte array prefix matching by @fallintoplace in [#1388](https://github.com/apache/arrow-go/pull/1388) +* perf(parquet): fast-path constant delta bitpacked blocks by @fallintoplace in [#1396](https://github.com/apache/arrow-go/pull/1396) +* perf(parquet): batch DELTA_LENGTH byte-array decoding and discard by @fallintoplace in [#1415](https://github.com/apache/arrow-go/pull/1415) +* perf(parquet): decode narrow packed levels directly by @fallintoplace in [#1404](https://github.com/apache/arrow-go/pull/1404) +* perf(parquet): expand nullable boolean bitmaps by byte by @fallintoplace in [#1405](https://github.com/apache/arrow-go/pull/1405) +* perf(parquet): decode RLE booleans directly into bool slices by @fallintoplace in [#1417](https://github.com/apache/arrow-go/pull/1417) +* perf(parquet): pack narrow RLE literal levels eight at a time by @fallintoplace in [#1419](https://github.com/apache/arrow-go/pull/1419) +* perf(parquet/file): use dense decoding for batches without nulls by @hcrosse in [#1447](https://github.com/apache/arrow-go/pull/1447) +* perf(parquet/pqarrow): build struct child readers serially when Parallel is false by @elsteveogrande in [#1434](https://github.com/apache/arrow-go/pull/1434) +* perf(parquet/internal/encoding): preallocate dictionaries bounded by page data by @zeroshade in [#1473](https://github.com/apache/arrow-go/pull/1473) + +### New Contributors +* @matanper made their first contribution in [#1290](https://github.com/apache/arrow-go/pull/1290) +* @singhpratech made their first contribution in [#1303](https://github.com/apache/arrow-go/pull/1303) +* @jakthom made their first contribution in [#1308](https://github.com/apache/arrow-go/pull/1308) +* @CaptainAni187 made their first contribution in [#1353](https://github.com/apache/arrow-go/pull/1353) +* @samueleresca made their first contribution in [#1331](https://github.com/apache/arrow-go/pull/1331) +* @hpaliwal8 made their first contribution in [#1365](https://github.com/apache/arrow-go/pull/1365) +* @sankalpsthakur made their first contribution in [#1363](https://github.com/apache/arrow-go/pull/1363) +* @Asthenia0412 made their first contribution in [#1376](https://github.com/apache/arrow-go/pull/1376) +* @skyon0522-ai made their first contribution in [#1385](https://github.com/apache/arrow-go/pull/1385) +* @nirajan1111 made their first contribution in [#1420](https://github.com/apache/arrow-go/pull/1420) +* @autoCAT6 made their first contribution in [#1412](https://github.com/apache/arrow-go/pull/1412) +* @vdpoora made their first contribution in [#1429](https://github.com/apache/arrow-go/pull/1429) +* @elsteveogrande made their first contribution in [#1435](https://github.com/apache/arrow-go/pull/1435) +* @EdwardShen125 made their first contribution in [#1439](https://github.com/apache/arrow-go/pull/1439) +* @AynSiddiqui made their first contribution in [#1477](https://github.com/apache/arrow-go/pull/1477) +* @peterolkhov made their first contribution in [#1499](https://github.com/apache/arrow-go/pull/1499) +* @prtkgaur made their first contribution in [#1289](https://github.com/apache/arrow-go/pull/1289) + +**Full Changelog**: https://github.com/apache/arrow-go/compare/v18.8.0...v18.9.0