Warning
Project-specific code and documentation were generated by large language models (LLMs) and have not been authored, read, or reviewed by a human.
A Parquet reader built on Vortex layouts and expressions, contributed by the SpiralDB team. Parquet pages remain compressed until an operation requires decoding; dictionary-encoded values can be filtered without expanding every row.
Filters and projections run in the Vortex scan. Row-group and page statistics prune reads, sparse row selections skip decoding, and row groups are scheduled in parallel. Nested fields decode through arrow-rs.
The DataFusion integration exposes the reader as a Vortex data source. The
DuckDB adapter provides read_parquet_vortex('path.parquet') as an embedded table
function or loadable extension. The benchmark harness compares readers using TPC-H,
TPC-DS, ClickBench, and focused workloads.
Run the scan example with your own Parquet file:
cargo run --locked --example scan_file -- data/events.parquetSee the reader reference for architecture and pushdown details, and development instructions for build and validation commands. Licensed under Apache-2.0; third-party notices cover vendored material.