feat(encoding): add range and delta block codecs - #8041
Conversation
|
Important This PR touches the Lance format specification. Substantive changes to the format specification — the If this is a meaningful format change:
|
There was a problem hiding this comment.
🟡 Gate recommendation: maintainer decision required.
The codec direction is technically sound and addresses the documented uncompressed-offset gap with a bounded, checked zero-or-one-payload tree.
This adds CompressiveEncoding tags 14 and 15, so it is a substantive format-specification change. Repository policy requires a separate PMC vote, but no completed vote is linked. Please link the vote covering these Range/Delta semantics, or document that an existing umbrella vote covers this stacked change.
Please mark this PR with the breaking-change label.
Stack 3 of the generic block compression series. Depends on #8040.
This PR appends
RangeandDeltaasCompressiveEncodingtags 14 and 15 and implements their bounded u32/u64 codec trees. Range is metadata-only; Delta inherits its child payload arity. Both readers use checked cardinality and arithmetic, and malformed trees fail during shared validation.No production page writer selects these codecs yet. Lance 2.0–2.2 bytes, selector order, and SparseLayout wire remain unchanged.
Validation:
cargo fmt --allcargo check -p lance-encoding --no-default-features --testscargo check -p lance-encoding --tests --benchescargo test -p lance-encoding compression::blockcargo test -p lance-encoding compression::testscargo test -p lance-encoding encodings::physicaluv run mkdocs buildcargo clippy --all --tests --benches -- -D warningsStack navigation