Skip to content

[Follow-up #23696] Prebuild parquet row-filter candidates once per file #24563

Description

@zhuqi-lucas

Follow-up to #23696, per @adriangb's decomposition proposal (PR 2).

RowFilterGenerator::build() currently redoes split_conjunction + FilterCandidateBuilder::build + reassign_expr_columns for every row group. On any multi-row-group scan with a pushdown predicate, that per-RG tree walk and column resolution is wasted work.

Move it to once per file: prebuild the row-filter candidate list when the file is opened, then instantiate a RowFilter per row group cheaply from the prebuilt list.

Proposed shape (extracted from #23696, decoupled from fully_matched):

  • add PrebuiltRowFilterCandidate / prebuild_row_filter_candidates (once per file: tree walk + candidate construction) and row_filter_from_prebuilt (per RG: cheap bind against the fresh array readers);
  • delete RowFilterGenerator;
  • reimplement the existing public build_row_filter on top of the two new functions (its signature must stay — parquet_nested_filter_pushdown / parquet_struct_filter_pushdown benches depend on it).

Parity to preserve: same conjunct order (sort_unstable_by_key(required_bytes) when reorder_predicates), same metric wiring (every predicate shares pushdown_rows_pruned, only the last predicate counts pushdown_rows_matched). One intended behavioural delta: reassign_expr_columns errors surface once at open time instead of being swallowed per-build by log::debug!.

Needs nothing from fully_matched.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions