Skip to content

parquet: PageIndex cannot be constructed outside the crate #10824

Closed

Description

@alamb

Which part is this question about

The new PageIndex struct introduced in #10719.

Describe the problem / reproducer

PageIndex::new is pub(crate), and PageIndex has no other constructor or mutators.

However, since external code cannot create (or modify) a PageIndex, the only thing a user outside the parquet crate can do with that method is remove or move an existing index.

Before #10719, external code could build ParquetMetaData values containing page indexes, e.g.:

ParquetMetaData::new(file_metadata, row_groups)
    .into_builder()
    .set_column_index(Some(vec![vec![column_index_builder.build().unwrap()]]))
    .set_offset_index(Some(vec![vec![offset_index_builder.build()]]))
    .build()

ColumnIndexBuilder and OffsetIndexBuilder are still public, so the individual ColumnIndexMetaData / OffsetIndexMetaData values can still be built — they just cannot be assembled into a PageIndex / attached to a ParquetMetaData anymore.

Expected behavior

External code can construct a ParquetMetaData with synthetic page indexes (DataFusion does this in tests that exercise page-level statistics handling, e.g. datafusion/datasource-parquet/src/statistics_order_tests.rs).

Actual behavior

There is no public way to construct or modify a PageIndex, so such code no longer compiles against main.

Additional context

Found while testing DataFusion against arrow-rs main ahead of the arrow 60 release (#9880, apache/datafusion#24366).

Activity

  1. alamb commented on Aug 24, 2026

    @alamb
    ContributorAuthor

    @etseidl has a PR with a builder that is probably a better idea:

  2. added a commit that references this issue on Sep 8, 2026
    7438071
  3. added 2 commits that reference this issue on Sep 18, 2026
    2adca4f
    c13ed77
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions