Skip to content

I78: Per-insert full-page XXH3 re-stamp on the slot-packing path #138

Description

@Xof

Migrated from ISSUES.md, which was retired in favour of GitHub issues. Original id I78, priority P1, from the 2026-06-01 performance review (docs/reviews/perf-review-2026-06-01.md).

The text below is the triaged entry verbatim, including the corrections triage made to the original finding.

Status: OPTIMIZATION DEFERRED 2026-06-01, pending a bulk-insert bench on dedicated hardware.

Important

Later investigation corrected the caveat in the "Direction of fix" below: the eager stamp is NOT needed for the evict-mid-transaction case — the spillway round-trip verifies its own slot_checksum, never the internal page checksum (only a main-file cold load checks that, page_cache.rs:872). The eager stamp's real job is "valid internal checksum before the main-file write — commit flush OR spill-then-drain"; that drain path is precisely what makes the deferral non-trivial. The misleading transaction.rs comment has since been fixed.


Where: src/transaction.rs insert_into_data_page (cursor path ~:1856-1869, fresh-page path ~:1892); cost in src/page.rs compute_checksum

Problem: page::stamp_checksum(buf) runs after every DataPage::insert, hashing all 8184 page-body bytes regardless of how few changed. A 1000-small-value transaction packing ≈ 39 values/page re-hashes its data pages once per value instead of once per page — O(values × 8 KB) where O(pages × 8 KB) would do.

Direction of fix: defer stamping — stamp a data page lazily at flush time and on the eviction path (a "needs-stamp" sub-flag, or stamp on cursor retirement + an eviction hook). Caveat (Don't-Break #8): a stamped checksum must still be guaranteed before any eviction-to-spillway/disk and before flush; eager stamping exists for exactly the evict-mid-transaction case (documented at transaction.rs:1839-1842). Validate on allocate-1000pertx / update-1000pertx at small value sizes.

Metadata

Metadata

Assignees

No one assigned

    Labels

    type:performanceHot paths, allocation, measurement

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions