Skip to content

feat(encoding): add range and delta block codecs - #8041

Closed
Xuanwo wants to merge 1 commit into
xuanwo/generic-block-stack-2-block-contractfrom
xuanwo/generic-block-stack-3-range-delta
Closed

feat(encoding): add range and delta block codecs#8041
Xuanwo wants to merge 1 commit into
xuanwo/generic-block-stack-2-block-contractfrom
xuanwo/generic-block-stack-3-range-delta

Conversation

@Xuanwo

@Xuanwo Xuanwo commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Stack 3 of the generic block compression series. Depends on #8040.

This PR appends Range and Delta as CompressiveEncoding tags 14 and 15 and implements their bounded u32/u64 codec trees. Range is metadata-only; Delta inherits its child payload arity. Both readers use checked cardinality and arithmetic, and malformed trees fail during shared validation.

No production page writer selects these codecs yet. Lance 2.0–2.2 bytes, selector order, and SparseLayout wire remain unchanged.

Validation:

  • cargo fmt --all
  • cargo check -p lance-encoding --no-default-features --tests
  • cargo check -p lance-encoding --tests --benches
  • cargo test -p lance-encoding compression::block
  • cargo test -p lance-encoding compression::tests
  • cargo test -p lance-encoding encodings::physical
  • uv run mkdocs build
  • cargo clippy --all --tests --benches -- -D warnings

Stack navigation

@github-actions github-actions Bot added A-encoding Encoding, IO, file reader/writer A-format On-disk format: protos and format spec docs labels Jul 28, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Important

This PR touches the Lance format specification.

Substantive changes to the format specification — the .proto definitions
and the spec docs under docs/src/format/ — require a PMC vote before merge.
Minor edits such as typo fixes, wording, or formatting are excluded; use your
judgment.

If this is a meaningful format change:

  • Start a vote following the Lance community voting process.
    Format specification modifications need 3 binding +1 votes (excluding the
    proposer), held on GitHub Discussions, with a minimum voting period of 1 week.
  • Once the vote passes, link the completed vote in this PR. It should not be
    merged until the vote is linked.

@lance-gatekeeper lance-gatekeeper Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Gate recommendation: maintainer decision required.

The codec direction is technically sound and addresses the documented uncompressed-offset gap with a bounded, checked zero-or-one-payload tree.

This adds CompressiveEncoding tags 14 and 15, so it is a substantive format-specification change. Repository policy requires a separate PMC vote, but no completed vote is linked. Please link the vote covering these Range/Delta semantics, or document that an existing umbrella vote covers this stacked change.

Please mark this PR with the breaking-change label.

@Xuanwo

Xuanwo commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator Author

Superseded by the rebuilt Range and Delta layer in #8325. The new stack is rooted at #8324 and ends at #8333; closing this obsolete layer to avoid duplicate review.

@Xuanwo Xuanwo closed this Aug 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-encoding Encoding, IO, file reader/writer A-format On-disk format: protos and format spec docs enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant