Skip to content

feat(encoding): adopt generic offsets in Lance 2.3 - #8046

Closed
Xuanwo wants to merge 1 commit into
xuanwo/generic-block-stack-7-offset-writerfrom
xuanwo/generic-block-stack-8-v23-offset-adopter
Closed

feat(encoding): adopt generic offsets in Lance 2.3#8046
Xuanwo wants to merge 1 commit into
xuanwo/generic-block-stack-7-offset-writerfrom
xuanwo/generic-block-stack-8-v23-offset-adopter

Conversation

@Xuanwo

@Xuanwo Xuanwo commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

Stack 8 of the Generic Block Sequence Compression series.

Depends on #8045.

This is the first production adopter. Ordinary Lance 2.3 variable-width mini-block pages may select the generic offset container when its complete serialized wire size is strictly smaller. Lance 2.0 through 2.2 retain legacy interleaved Flat offsets byte-for-byte; explicit outer general compression retains legacy offsets; SparseLayout remains on its existing wire.

Page-level coverage verifies Range, Delta, Delta(Range), legacy fallback, unchanged chunk-local take request shape, and the SparseLayout exclusion. The format reference documents the two unambiguous mini-block containers and the bounded block-sequence contract.

Validation:

  • cargo check -p lance-encoding --no-default-features --tests
  • cargo check -p lance-encoding --tests --benches
  • cargo test -p lance-encoding test_v2_3_
  • cargo test -p lance-encoding test_variable_offset_codec_is_version_gated
  • cargo clippy --all --tests --benches -- -D warnings
  • cargo fmt --all
  • make -C docs build

The docs build succeeded with existing missing-external-doc-repository and link warnings; its generated tracked-file side effects were restored before commit.

Stack navigation

@github-actions

Copy link
Copy Markdown
Contributor

Important

This PR touches the Lance format specification.

Substantive changes to the format specification — the .proto definitions
and the spec docs under docs/src/format/ — require a PMC vote before merge.
Minor edits such as typo fixes, wording, or formatting are excluded; use your
judgment.

If this is a meaningful format change:

  • Start a vote following the Lance community voting process.
    Format specification modifications need 3 binding +1 votes (excluding the
    proposer), held on GitHub Discussions, with a minimum voting period of 1 week.
  • Once the vote passes, link the completed vote in this PR. It should not be
    merged until the vote is linked.

@github-actions github-actions Bot added A-encoding Encoding, IO, file reader/writer A-format On-disk format: protos and format spec docs enhancement New feature or request labels Jul 28, 2026
@Xuanwo

Xuanwo commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator Author

Superseded by the rebuilt v2.3 generic-offset adopter in #8331. The new stack is rooted at #8324 and ends at #8333; closing this obsolete layer to avoid duplicate review.

@Xuanwo Xuanwo closed this Aug 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-encoding Encoding, IO, file reader/writer A-format On-disk format: protos and format spec docs enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant