Skip to content

About

Read Apache Parquet through Vortex layouts and expressions.

Resources

Stars

1 star

Watchers

0 watching

Forks

Repository files navigation

vortex-parquet

Warning

Project-specific code and documentation were generated by large language models (LLMs) and have not been authored, read, or reviewed by a human.

A Parquet reader built on Vortex layouts and expressions, contributed by the SpiralDB team. Parquet pages remain compressed until an operation requires decoding; dictionary-encoded values can be filtered without expanding every row.

Filters and projections run in the Vortex scan. Row-group and page statistics prune reads, sparse row selections skip decoding, and row groups are scheduled in parallel. Nested fields decode through arrow-rs.

The DataFusion integration exposes the reader as a Vortex data source. The DuckDB adapter provides read_parquet_vortex('path.parquet') as an embedded table function or loadable extension. The benchmark harness compares readers using TPC-H, TPC-DS, ClickBench, and focused workloads.

Run the scan example with your own Parquet file:

cargo run --locked --example scan_file -- data/events.parquet

See the reader reference for architecture and pushdown details, and development instructions for build and validation commands. Licensed under Apache-2.0; third-party notices cover vendored material.

About

Read Apache Parquet through Vortex layouts and expressions.

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages