Skip to content

perf(query): prune and order immutable Parquet files by event-time window #273

Description

@vishr

PR #272 review follow-up for views.go: #272 (comment)

Eliminate per-statement glob/footer work for unrelated retained batches and bind selected files newest first under the existing publication pin. Span metadata has start bounds; logs/metrics have ingest bounds only. Obtain actual per-signal event-time bounds without pruning by ingest time, and preserve arbitrary SQL's complete approved snapshot.

Validation: compare pruned and full scans across all three signals, late/future/out-of-order timestamps, zero/negative instants, nanosecond and partial windows, reopen, compaction/retention, and canceled readers. Measure actual files/row groups read and newest-first top-N behavior on a multi-age dataset.

Implementation is authorized in PR #272; this issue tracks the fix and its validation.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions