Skip to content

[record] Parquet prefetch window prototype: method and results - #83

Closed
adriangb wants to merge 9 commits into
claude/datafusion-rowgroup-buffering-ec9abcfrom
adrian/prefetch-window-prototype-record
Closed

adriangb wants to merge 9 commits into
claude/datafusion-rowgroup-buffering-ec9abcfrom
adrian/prefetch-window-prototype-record

Conversation

@adriangb

Copy link
Copy Markdown
Member

Record only. Not for merge. This PR is closed on purpose. It keeps the prototype code, method and results in one place.

Base: the apache#24086 head (pool-counted read_ahead_bytes), built against the pydantic/arrow-rs batch-granular scan_plan branch (claude/push-decoder-batch-granular-scan-plan-60.0.0).

What the prototype does

datasource-parquet/src/prefetch.rs adds a per-file fetch window, enabled with DF_PREFETCH=1.

  • The window is built from the decoder's scan_plan(), in decode order. That order can move backward through the file (for example ORDER BY ... DESC reverses row groups). Chunks must not overlap; if they do, the reader falls back to the plain reader.
  • Ranges are split or coalesced into chunks of DF_PREFETCH_CHUNK bytes (default 4 MiB).
  • Each chunk is Pending, InFlight, Ready(Bytes) or Freed. A read cursor moves forward only and frees every chunk it passes.
  • Required reads (the chunk the decoder asks for now) claim with MemoryReservation::grow, which never fails.
  • Optional reads (chunks ahead of the cursor) are fetched by up to DF_PREFETCH_CONCURRENCY workers (default 4). They claim with try_grow, within DF_PREFETCH_BUDGET bytes (default 32 MiB) counted after the cursor chunk.
  • A Claim guard returns a cancelled fetch's chunk to Pending. A JoinSet aborts the workers when the reader drops. Moving the cursor wakes the workers.
  • push_decoder.rs: required reservations are a plain grow.
  • Stats (DF_PREFETCH_STATS=1): hit_mib miss_mib required_mib wait_ms denied.

Earlier designs in this branch's history, kept for reference, and why they were removed:

commit design result
7cd1f3d, c540892 background prefetch plus look-ahead into the next files look-ahead added memory, no speed
40160e3 fallible prefetch plus pool reclaim reclaim never reduced spill; evicted bytes were fetched again
446700d skip prefetch when the pool is nearly full 5-12x slower sorts: gated workers fetched chunks the decoder had already read directly (3.6-5.8 GiB of 6.3 GiB missed) and the stale bytes stayed pinned
875d273 promote missed ranges to required fetches worked, but patched over the previous problem
33f38f4 onward per-file window with a forward-only cursor current design; no gates, reclaim or priority

Method

  • Data: synthetic. 64 Parquet files of about 100 MB each, an id column and a attributes string column of about 20 KB per row. Two layouts: 16384-row row groups (d16384) and 2048-row row groups (d2048).
  • Storage: GCS bucket, read through object_store from an n2-highmem-32 VM in the same region. Each run is pinned to 8 cores (taskset -c 0-7).
  • Harness: a small Rust binary that registers the files, runs one query, and prints wall time, peak RSS (jemalloc), peak pool use and spill bytes. Pool: FairSpillPool wrapped in TrackConsumersPool. Batch size 4096.
  • Queries:
    • sort: SELECT id, concat(substr(attributes, 1, 300), chr(120)) AS a FROM t ORDER BY id DESC, with a 512 MiB or 768 MiB pool (forces spill).
    • scan: attributes LIKE ... filter, 3 GiB pool.
  • Lanes:
    • main: 64 partitions, no read-ahead, no prefetch.
    • best: 8 partitions, read_ahead_bytes 1 MiB, DF_PREFETCH=1, chunk 4 MiB, budget 32 MiB, concurrency 4.
  • Two reps of every cell. The sweep script is below.
sweep script
B=gs://<bucket>
R="taskset -c 0-7 ./target/release/ra2 run"
export BATCH=4096 DF_PREFETCH_STATS=1
S="SELECT id, concat(substr(attributes, 1, 300), chr(120)) AS a FROM t ORDER BY id DESC"
BEST="PARTS=8 RA=1048576 DF_PREFETCH=1"
for rep in 1 2; do for d in d16384 d2048; do
  for pool in 536870912 805306368; do
    one "$S" POOL=$pool PARTS=64
    for cb in "8388608 16777216" "8388608 33554432" "4194304 16777216" "4194304 25165824" "4194304 33554432"; do
      set -- $cb; one "$S" POOL=$pool $BEST DF_PREFETCH_CHUNK=$1 DF_PREFETCH_BUDGET=$2
    done
  done
  one like_filter POOL=3221225472 PARTS=64
  for cb in "8388608 33554432" "4194304 25165824" "4194304 33554432"; do
    set -- $cb; one like_filter POOL=3221225472 $BEST DF_PREFETCH_CHUNK=$1 DF_PREFETCH_BUDGET=$2
  done
done; done

Results: main vs best (2 reps)

layout query pool main wall s / peak RSS MiB best wall s / peak RSS MiB
d16384 sort 512 MiB fails at 2.9-3.0 s / 7781-7807 4.78-4.81 / 1080-1097
d16384 sort 768 MiB 4.83-5.07 / 7593-7761 4.98-4.99 / 1150-1168
d2048 sort 512 MiB fails at 0.6 s / 1146-1246 4.72-4.82 / 587-605
d2048 sort 768 MiB fails at 3.4 s / 1214-1222 4.96-5.0 / 691
d16384 scan 3 GiB 5.01-5.11 / 7780-7805 4.63-4.70 / 1108-1135
d2048 scan 3 GiB 4.58 / 1195 4.53 / 569

main fails 4 of 8 sort cells with Resources exhausted. best passes all of them, spills the same 325 MiB, and holds RSS at 0.6-1.2 GiB. Prefetch hit rate is 100% (miss_mib=0) in every best run.

Sizing

chunk budget effect
8 MiB 32 MiB fails the d16384 512 MiB sort at 0.23-0.25 s
8 MiB 16 MiB passes, about 6.0-6.3 s
4 MiB 16 MiB passes, about 5.9-6.0 s on d16384
4 MiB 24 MiB passes, about 4.8-5.0 s
4 MiB 32 MiB passes, about 4.7-5.0 s (default)
earlier runs 8 MiB about 7.5-8 s

The 8 MiB / 32 MiB failure: prefetch held about 290 MB and read-ahead about 200 MB, which left 2.2 MB for the sort's first 2.5 MB reservation. The chunk under the cursor is also counted twice (by prefetch and by the decoder), so a larger chunk costs more than its size suggests.

Findings for upstream

  • Optional prefetch must be counted in the MemoryPool and must be fallible (try_grow). Required reads must not be refused (grow).
  • Do not gate prefetch on "pool nearly full". Free everything behind a forward-only cursor instead.
  • Size the budget up front. First-come try_grow lets prefetch take the pool before other operators reserve. A per-scan share derived from MemoryPool::memory_limit() works (compare DuckDB's per-thread cap of limit / 8).
  • Allow plans in decode order that move backward, but reject overlapping chunks.
  • Look-ahead across files added memory without speed here. Keep the window within the file; cross-file scheduling belongs in the morsel scheduler.

Related: apache#26682, apache#24086, apache#23492, apache#26387, apache#25650, apache#21422; apache/arrow-rs#11609, apache#11238, apache#11613, apache#11240.

@adriangb

Copy link
Copy Markdown
Member Author

Closed on purpose: record only.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant