Skip to content

perf: benchmark and document MIN/MAX reader filtering 3/n - #7683

Open
pingzh wants to merge 2 commits into
apache:mainfrom
pingzh:dev/codex/minmax-benchmark-docs
Open

pingzh wants to merge 2 commits into
apache:mainfrom
pingzh:dev/codex/minmax-benchmark-docs

Conversation

@pingzh

@pingzh pingzh commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Part of #5774. Follows the merged reader support (#7598) and Spark integration (#7657).

TL;DR performance

Measured on 8.39M INT rows, uncompressed Parquet, local[1], in both mode orders. Times below are filtering off → on.

  • Scan savings: favorable ordering and constant values reduced scan rows 8.39M → 1.05M and requested data ~32 → ~4 MiB (about 87.5% less), pruning 8 of 9 row groups per file.
  • Favorable elapsed time: descending MAX, one task, improved 74 → 48 ms and 87 → 44 ms across the two runs (about 1.5–2.0× faster).
  • Overhead cases: unfavorable ordered layouts with four tasks were 19–35% slower. Random MAX with four tasks took 63 → 179 ms and 67 → 125 ms, with high variance despite pruning.

Timings are sensitive to warmup and outliers; scan savings do not guarantee faster queries. The feature remains disabled by default. Full means, standard deviations, and counters are below.

Rationale for this change

MIN/MAX reader filtering can skip later Parquet row groups after a task observes a useful bound. The benefit depends on data order and task boundaries, while footer inspection and bound updates add overhead. This adds an off/on benchmark and documents eligibility, fallback limits, and metrics before considering broader enablement.

What changes are included in this PR?

  • CometMinMaxBenchmark: separate MIN and MAX queries over ascending, descending, deterministic random, and constant INT data, with one or four scan tasks by default. It checks Spark result parity, the direct scan/partial aggregate plan, actual task count, and per-task filter attachment before timing.
  • Scan and attachment metrics from untimed executions, plus elapsed query time in both configurable mode orders. Uncompressed Parquet v1 pages, disabled dictionaries, and disabled page-index/decoder filtering provide a controlled row-group comparison, including constant values.
  • User-guide configuration, supported plans, footer limits, fallback behavior, and metric interpretation; contributor instructions for reproducing measurements.

The existing configuration remains disabled by default. Runtime behavior and supported scope are unchanged. The two commits separate the benchmark from documentation.

How are these changes tested?

  • Native release build passed, with -Ctarget-cpu=native -Clink-arg=-fuse-ld=bfd and the standard release profile/default features.
  • JVM reactor test-compile, including formatting and style checks, passed. Packaged binary/source jar-content checks passed (the final check was rerun with JDK tools on PATH).
  • Both complete benchmark runs passed: 16 query/layout/task-count combinations × two modes × two mode orders. Results matched Spark, enabled filters attached once per scan task, and all pruning metrics agreed across mode orders.
  • git diff --check passed.

Local release measurements

Environment: AMD EPYC-Genoa, 32 logical CPUs, about 98 GiB RAM, Ubuntu 24.04.4/Linux 6.8; Spark 4.1.3, OpenJDK 21.0.13.0.3, Rust 1.99.1. Spark used local[1] and an 8 GiB benchmark JVM heap. Production code is from d187e6823; the benchmark is in this PR's d2f9fdfc6 commit. Each fixture contained 8,388,608 rows and nine row groups per file. Four scan tasks ran serially under local[1].

Equivalent standard invocations after the same release build:

COMET_BENCHMARK_MASTER='local[1]' make BENCH_HEAP=8g benchmark-org.apache.spark.sql.benchmark.CometMinMaxBenchmark --   8388608 1,4 ascending,descending,random,constant 5 off,on
COMET_BENCHMARK_MASTER='local[1]' make BENCH_HEAP=8g benchmark-org.apache.spark.sql.benchmark.CometMinMaxBenchmark --   8388608 1,4 ascending,descending,random,constant 5 on,off

Favorable ordering and constant values reduced emitted scan rows and requested data bytes by about 87.5%. The unfavorable ordered cases pruned no groups and expose filtering overhead. Random layouts pruned some groups but did not consistently improve elapsed time. These short local runs are sensitive to warmup and long-tail outliers; the first ascending-MIN timing even changes direction when case order reverses. This is evidence of scan-work reduction and overhead cases, not a general speedup claim or a basis to enable the feature by default.

All elapsed times and scan-work counters

Times are mean ± standard deviation in milliseconds, always shown as off → on, regardless of execution order. Every off execution emitted 8,388,608 rows and pruned zero groups. The byte column is bytes_scanned in bytes, excluding metadata. All on attachment counts equaled the scan-task count; skipped counts were zero. Statistics, page-index, and decoder pruning counters were zero.

Tasks Layout Aggregate On scan rows Groups pruned Data bytes off → on Mean ± SD, off-first Mean ± SD, on-first
1 ascending MIN 1,048,287 8 33,566,729 → 4,194,680 176 ± 21 → 101 ± 22 77 ± 10 → 135 ± 96
1 ascending MAX 8,388,608 0 33,566,729 → 33,566,729 105 ± 4 → 98 ± 5 81 ± 3 → 96 ± 7
1 descending MIN 8,388,608 0 33,566,728 → 33,566,728 73 ± 4 → 84 ± 6 69 ± 3 → 68 ± 6
1 descending MAX 1,048,287 8 33,566,728 → 4,194,682 74 ± 5 → 48 ± 5 87 ± 12 → 44 ± 3
1 random MIN 3,144,861 6 33,570,132 → 12,585,317 63 ± 4 → 67 ± 10 62 ± 4 → 54 ± 6
1 random MAX 5,241,435 4 33,570,132 → 20,975,530 62 ± 2 → 62 ± 1 65 ± 3 → 63 ± 3
1 constant MIN 1,048,287 8 33,566,756 → 4,194,685 53 ± 9 → 102 ± 152 60 ± 4 → 42 ± 8
1 constant MAX 1,048,287 8 33,566,756 → 4,194,685 56 ± 4 → 59 ± 14 64 ± 9 → 48 ± 1
4 ascending MIN 1,048,196 32 33,567,471 → 4,194,401 78 ± 3 → 73 ± 15 76 ± 1 → 66 ± 3
4 ascending MAX 8,388,608 0 33,567,471 → 33,567,471 70 ± 3 → 87 ± 18 79 ± 5 → 107 ± 21
4 descending MIN 8,388,608 0 33,567,469 → 33,567,469 73 ± 2 → 88 ± 15 79 ± 2 → 94 ± 5
4 descending MAX 1,048,196 32 33,567,469 → 4,194,400 59 ± 2 → 69 ± 17 74 ± 4 → 68 ± 15
4 random MIN 3,144,588 24 33,571,051 → 12,584,536 61 ± 2 → 72 ± 17 61 ± 3 → 64 ± 2
4 random MAX 3,406,637 23 33,571,051 → 13,633,247 63 ± 4 → 179 ± 196 67 ± 2 → 125 ± 46
4 constant MIN 1,048,196 32 33,567,496 → 4,194,404 75 ± 3 → 64 ± 14 95 ± 9 → 50 ± 14
4 constant MAX 1,048,196 32 33,567,496 → 4,194,404 68 ± 2 → 58 ± 1 81 ± 11 → 68 ± 13

Reader scan_io_data_bytes equaled bytes_scanned. scan_io_metadata_bytes was 524,288 bytes per task in each mode (524,288 for one task and 2,097,152 for four); attachment does not eliminate footer inspection. Timings include planning and result collection, while fixture generation, Spark parity checks, plan validation, and metric reporting are outside the timed cases.

@github-actions github-actions Bot added enhancement New feature or request performance labels Nov 19, 2026
@pingzh
pingzh marked this pull request as ready for review November 19, 2026 05:49

@sunchao sunchao left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 2d6b1842f1485857aa118e2f8b8b11eeab7f440f with five independent passes covering correctness/Spark compatibility, performance and measurement validity, design/reuse, test maintainability/CI integration, and documentation/release compatibility. No substantive issues found in the benchmark or the documented behavior. The off/on cases use matched fixtures and settings, validate the direct scan/partial aggregate and per-task attachment, and separate untimed metrics from query timings. The eligibility, fallback limits, and metric descriptions match Comet and its pinned DataFusion implementation. This PR changes no production code or configuration defaults.

Please format docs/source/user-guide/latest/metrics.md and docs/source/user-guide/latest/tuning/operators.md. Both upstream Preflight and fork CI fail prettier --check "**/*.md" on those two files at this head. Preflight stops the workflow before downstream build/validation jobs can run. Please get the formatting fix through hosted CI before merging.

Validation was source inspection and exact-head hosted CI/log review. I did not run local tests or benchmarks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants