[Parquet] ALP encoder/decoder support - #9372

Merged

alamb merged 80 commits into

apache:mainfrom

Sep 4, 2026

Commits on Feb 6, 2026

Add alp to the thrift definitions
sdf-jkl
committed
54bc343
Add alp to the encoding enum
sdf-jkl
committed
622841b

Commits on Feb 12, 2026

Add alp header and page layout structs
sdf-jkl
committed
bf754b2

Commits on Feb 15, 2026

Add Interleaved Vectors support
sdf-jkl
committed
8337fcd
Merge branch 'main' into alp
sdf-jkl
authored
5349a9e

Commits on Feb 18, 2026

Add bit unpacking, reverse FOR and exceptions patching
sdf-jkl
committed
148b1d5
Decode page works
sdf-jkl
committed
5f0222e

Commits on Feb 19, 2026

Add more checks and tests
sdf-jkl
committed
027172f

Commits on Feb 20, 2026

Make decode page lazy + some tests
sdf-jkl
committed
8ee1b7e
Find vectors using offsets at decode time, not parse time
sdf-jkl
committed
18f885e
get decode fast path
sdf-jkl
committed
a9fa0c3
Simplify decode_page_values
sdf-jkl
committed
7754ebc

Commits on Mar 12, 2026

remove `version` from page header
sdf-jkl
committed
6b7e923
change `log_vetctor_size` range consts
sdf-jkl
committed
a13f244
redo the integration test using a test file from the c++ pr
sdf-jkl
committed
6e86d6f
cache parsed vectors
sdf-jkl
committed
ac56cc2
use `Bytes` for `packed_values`
sdf-jkl
committed
07ca6f0
clippy too many args
sdf-jkl
committed
def5c09
cargo fmt
sdf-jkl
committed
5e5c8d2

Commits on Mar 16, 2026

Merge branch 'main' of https://github.com/apache/arrow-rs into alp
sdf-jkl
committed
6060ca2
fix decode to two-step multiplication
sdf-jkl
committed
5ed70ae
make vector byte count more strict
sdf-jkl
committed
e328d7a
add stricter offset validation
sdf-jkl
committed
1807d32
fmt
sdf-jkl
committed
11c7905

Commits on May 8, 2026

Merge tag '58.2.0' into alp
alamb
committed
e9827e5
Fix up compilation
alamb
committed
cacacf4
Refactor: Pull common header structures into common module
alamb
committed
578481c

Commits on May 9, 2026

Merge pull request #3 from alamb/alamb/move_common_structures
sdf-jkl
authored
131851a
fmt
sdf-jkl
committed
f10279c
small fix
sdf-jkl
committed
fbf4ad2
Align structure names with spec
alamb
committed
f65ff9a
Merge pull request #5 from alamb/alamb/alp_names
sdf-jkl
authored
c912f4e
fmt
sdf-jkl
committed
26c382c

Commits on May 18, 2026

use arrow-csv
sdf-jkl
committed
82cdba3
add page layout ASCII art
sdf-jkl
committed
d15f0ca

Commits on Jun 2, 2026

add `AlpHeader` serde methods for `vector_size` and `num_elements`
sdf-jkl
committed
a6b5f05

Commits on Jun 7, 2026

Commits on Jun 8, 2026

even lazier decode
sdf-jkl
committed
93eb925
use `BitReader::get_batch` to support autovectorization with a scratch buffer
sdf-jkl
committed
ad84bde
update parquet-testing dep
sdf-jkl
committed
23fac76

Commits on Jul 14, 2026

avoid division
sdf-jkl
committed
b0199e0
v1 header doesn't know num_value upfront
sdf-jkl
committed
532b025
add encoder support
sdf-jkl
committed
3b7d362
Merge branch 'main' of https://github.com/apache/arrow-rs into alp
sdf-jkl
committed
3306fd2

Commits on Jul 16, 2026

skip loading each vector during skip
sdf-jkl
committed
dc193a6

Commits on Jul 17, 2026

add streaming ALP encoder + throughput bench

sdf-jkl
and
claude
committed
5b0d8ad

Commits on Jul 19, 2026

use copysign for fast_round
sdf-jkl
committed
0dd0324
branchless exceptions encode
sdf-jkl
committed
e5d9bf9
comment trim
sdf-jkl
committed
b58a7e5

Commits on Jul 31, 2026

drop the sign in fast rounding
sdf-jkl
committed
526b8af
drop the bench
sdf-jkl
committed
2196346
Merge branch 'main' of https://github.com/apache/arrow-rs into alp
sdf-jkl
committed
799d58c
fmt
sdf-jkl
committed
e23b476
comments clean up pass
sdf-jkl
committed
2c25e55

Commits on Aug 3, 2026

replace the per row test check with assert_eq
sdf-jkl
committed
b400009
add docs to explain `AlpFloat` and `AlpExact` connection
sdf-jkl
committed
6e99313
update `fast_round` docs
sdf-jkl
committed
6cf1712

Commits on Aug 4, 2026

drop the writer cpp parity test
sdf-jkl
committed
3c73757
pin the remainders behavior with debug_assert_eq!
sdf-jkl
committed
199a50a
add more tests for llvm-cov
sdf-jkl
committed
5a10e68

Commits on Aug 5, 2026

fix worst case `estimated_data_encoded_size` formula + add regression test
sdf-jkl
committed
bf7b13f

Commits on Aug 7, 2026

Update to latest test file
alamb
committed
9bebf39
Merge remote-tracking branch 'apache/main' into alp
alamb
committed
bb681e4
Update parquet_testing fixture test to use new file
alamb
committed
0521e86
limit f32 literal to max precision
alamb
committed
68010c9
fmt
alamb
committed
03e10a9
chore: reorganize parquet-testing test
alamb
committed
b4bffad
Merge remote-tracking branch 'apache/main' into alp
alamb
committed
34c9af8
fix merge
alamb
committed
564cd0e

Commits on Aug 11, 2026

Wire ALP to parquet-rewrite
sdf-jkl
committed
5e9ad1c

Commits on Aug 15, 2026

Merge branch 'main' into alp
alamb
authored
f9794b4

Commits on Aug 25, 2026

Merge branch 'main' of https://github.com/apache/arrow-rs into alp
sdf-jkl
committed
2d8002b
clippy
sdf-jkl
committed
baea4e9

Commits on Sep 3, 2026

Merge remote-tracking branch 'apache/main' into alp
alamb
committed
984968d
Merge branch 'main' into alp
alamb
authored
e27676e
Ignore test_alp_extended under Miri (zstd calls native C functions)
alamb
committed
ea6d049