You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
"end to end" test for reading parquet using custom PageIndexProvider #11029
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
#10842 introduces the PageIndexProvider trait so that applications can plug in custom page index implementations (for example, sparse or lazily-loaded indexes) into ParquetMetaData.
However, no test currently drives reading with a custom PageIndexProvider through the actual read path.
Several places in the code use as_any().downcast_ref::<PageIndex>() and take a different path for custom providers (for example ParquetMetaData::memory_size, PartialEq for ParquetMetaData, and ParquetMetaDataWriter), so adding specific reader coverage would be good.
The custom_page_index.rs example implements OnDemandPageIndexProvider, but it only prints index availability, and examples are compiled — not executed — in CI.
The existing reader tests exercise the built-in PageIndex implementation only.
Describe the solution you'd like
Add an integration test that:
Writes a parquet file with page indexes (multiple row groups, multiple columns, multiple pages per column).
Activity
parquet: Add end-to-end test of custom `PageIndexProvider` (apache#11132
parquet: Add end-to-end test of custom `PageIndexProvider` (apache#11132