chore(deps): bump lance 4.0.0 → 6.0.0, lancedb 0.27.2 → 0.29.0#423
Merged
Conversation
Per task scope: bump Lance family (4.0 → 6.0) and lancedb (0.27 → 0.29).
Note: target was lance 6.0.1, but lancedb 0.29.0 pins lance =6.0.0, so the
viable bump is 6.0.0 across the family (no 6.0.1 with current lancedb).
Crates touched (Cargo.toml pin updates):
- crates/lance-graph: lance/lance-linalg/lance-namespace/lance-arrow/lance-index = =6.0.0; lancedb = =0.29.0
- crates/lance-graph-catalog: lance-namespace = =6.0.0
- crates/lance-graph-ontology: lance = =6.0.0 (optional)
- crates/lance-graph-callcenter: lance = =6.0.0 (optional)
- crates/lance-graph-benches: lance = =6.0.0
- crates/holograph: lance = =6.0.0 (excluded crate, kept in sync)
- crates/surreal_container: lance = =6.0.0, lancedb = =0.29.0 (optional)
Transitive bumps required by lance 6.0:
- arrow 57 → 58 (lance 6.0 ships arrow 58; consumer types must match)
- datafusion 52 → 53 (lance 6.0 needs DataFusion 53)
- Applied to: lance-graph, lance-graph-catalog, lance-graph-ontology,
lance-graph-callcenter, lance-graph-benches, lance-graph-archetype,
lance-graph-python (excluded), holograph (excluded)
Source code: no changes required — boundary types stayed compatible after
arrow 57→58 + datafusion 52→53 alignment.
cargo check --workspace --all-features: clean, except 3 PRE-EXISTING errors
in crates/cognitive-shader-driver (QUALIA_DIMS missing import + ShaderDispatch
missing fields alpha_saturation_override / merge_override). These errors
exist on origin/main and are unrelated to this bump — verified by checking
out main and reproducing the same errors.
Notes / surprises:
- lancedb 0.29.0 pins lance =6.0.0 (strict equality, not ^6); lance 6.0.1
cannot coexist with lancedb 0.29.0. Bump achievable today is 6.0.0.
- lance 6.0 requires protoc at build time (lance-encoding build script
pulls prost-build). Solved via system protobuf-compiler install for CI;
crates with `protoc` feature could bundle protobuf-src instead.
- No `unimplemented!()` stubs needed.
- No surface renames in lance-namespace / lance-linalg between 4.0 and 6.0
(still published as separate crates at 6.0.0).
https://claude.ai/code/session_01LiUiGeUDLje8KMnxB4FfA3
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThis PR upgrades workspace Cargo manifests to the Lance 6.0 ecosystem: Arrow crates move from 57→58, DataFusion from 51/52→53, Lance workspace crates from =4.0.0→=6.0.0, and lancedb/deltalake pins updated; changes are applied across the primary crate and dependent crate manifests while preserving optional/default-features and feature flags. ChangesLance 6.0 Ecosystem Upgrade
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1db091f1f5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| chrono = "0.4" | ||
| async-trait = "0.1" | ||
| datafusion = { version = "52", default-features = false, features = [ | ||
| datafusion = { version = "53", default-features = false, features = [ |
There was a problem hiding this comment.
With the default delta feature enabled, this crate still pulls deltalake = 0.31 with its datafusion feature, and the regenerated lock shows deltalake-core still depends on datafusion 52.5.0 / Arrow 57. DeltaTableReader creates a deltalake::delta_datafusion::DeltaTableProvider and passes it to the crate's SessionContext; after this bump that context is DataFusion 53, so the provider implements the wrong TableProvider trait and default builds using Delta support will fail to type-check. Either bump deltalake to a DataFusion-53-compatible release or disable/remove the Delta reader until it can be aligned.
Useful? React with 👍 / 👎.
| arrow-array = "57" | ||
| arrow-schema = "57" | ||
| arrow-ipc = "57" | ||
| arrow = { version = "58", features = ["pyarrow"] } |
There was a problem hiding this comment.
The Python extension now constructs Arrow 58 RecordBatch/Schema values, but its datafusion dependency remains pinned to 51 while graph.rs feeds those batches into MemTable::try_new and passes a SessionContext into lance-graph APIs that now use DataFusion 53. In an opt-in maturin develop/Python build this leaves two incompatible DataFusion/Arrow type universes in the same crate, so the extension will fail to compile; bump this dependency alongside the Arrow/Lance bump.
Useful? React with 👍 / 👎.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/lance-graph-python/Cargo.toml`:
- Around line 13-17: The Cargo.toml dependency for datafusion is out of sync
with the pinned Arrow 58 and the workspace lockfile; update the datafusion entry
in crates/lance-graph-python/Cargo.toml to a DataFusion series that targets
Arrow 58 (e.g., set datafusion = { version = "53", default-features = false } or
the exact locked version like "53.1.0") so it matches the Arrow 58 pin and the
workspace Cargo.lock; this ensures code in src/graph.rs that uses MemTable,
DefaultTableSource, SessionContext and Arrow RecordBatch/Schema compiles against
a consistent DataFusion/Arrow pair.
In `@crates/surreal_container/Cargo.toml`:
- Line 44: The inline comment next to the lancedb dependency is stale (it says
"bump to 0.28" while the dependency is set to version 0.29.0); update the
comment next to the lancedb = { version = "=0.29.0", optional = true,
default-features = false } line to reflect the actual version (e.g., change
"bump to 0.28" to "bump to 0.29" or remove the BLOCKED note if no longer
applicable) so the BLOCKED annotation matches the declared lancedb version.
- Around line 43-44: The workspace test fails because the crate relies on a path
dependency named `ndarray` (referenced by the `ndarray = { path =
"../../../ndarray", ... }` entry) that isn’t present, so before running cargo
test for `crates/surreal_container` either ensure the `../../../ndarray`
directory is available (e.g., checkout or init the submodule/fork) or change the
dependency to the published crate by replacing the path override with a
versioned dependency in the manifest; locate the `ndarray` dependency entry
referenced by `crates/surreal_container/Cargo.toml` (or the workspace override)
and either populate the path or switch to `ndarray = "<appropriate_version>"` so
the workspace can resolve the dependency.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 913cc7b5-ed65-4e93-a524-bde07ea5c5a6
⛔ Files ignored due to path filters (1)
Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (9)
crates/holograph/Cargo.tomlcrates/lance-graph-archetype/Cargo.tomlcrates/lance-graph-benches/Cargo.tomlcrates/lance-graph-callcenter/Cargo.tomlcrates/lance-graph-catalog/Cargo.tomlcrates/lance-graph-ontology/Cargo.tomlcrates/lance-graph-python/Cargo.tomlcrates/lance-graph/Cargo.tomlcrates/surreal_container/Cargo.toml
Comment on lines
+43
to
+44
| lance = "=6.0.0" # BLOCKED(A): bump to Lance 6 once version confirmed | ||
| lancedb = { version = "=0.29.0", optional = true, default-features = false } # BLOCKED(B): bump to 0.28 |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Verify surreal_container builds and tests with Lance 6.0 dependencies
# Expected: Build succeeds (or fails with known BLOCKED surrealdb dependency issue)
cargo test --manifest-path crates/surreal_container/Cargo.tomlRepository: AdaWorldAPI/lance-graph
Length of output: 640
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== Find ndarray references in Cargo manifests =="
rg -n --hidden --glob='**/Cargo.toml' 'ndarray' .
echo
echo "== Show workspace Cargo.toml dependency entry for ndarray if present =="
if [ -f Cargo.toml ]; then
rg -n --hidden 'ndarray' Cargo.toml || true
echo
echo "---- Root Cargo.toml (around ndarray) ----"
awk '
BEGIN{found=0}
/ndarray/ {found=1}
{print NR ":" $0}
found && NR>0 { if(found && NR>0){} }
' Cargo.toml | head -n 120 || true
fi
echo
echo "== Show manifest paths for crates/lance-graph dependency bgz-tensor if present =="
if [ -f crates/lance-graph/Cargo.toml ]; then
rg -n --hidden 'bgz-tensor' crates/lance-graph/Cargo.toml
echo
echo "---- crates/lance-graph/Cargo.toml (around bgz-tensor) ----"
awk '{
if($0 ~ /bgz-tensor/){start=NR-20; if(start<1) start=1}
if(NR>=start && NR<=start+80) print NR ":" $0
}' crates/lance-graph/Cargo.toml || true
fiRepository: AdaWorldAPI/lance-graph
Length of output: 13587
cargo test for crates/surreal_container fails to load due to missing ndarray path dep
cargo test --manifest-path crates/surreal_container/Cargo.tomlexits 101 because the workspace can’t readndarray(failed to read .../ndarray/Cargo.toml: No such file or directory).- The workspace uses a path dependency to the AdaWorldAPI fork (
ndarray = { path = "../../../ndarray", ... }), so the../../../ndarraydirectory must be present (e.g., submodule/fork checkout or dependency wiring) before the Lance 6.0 / LanceDB 0.29 bump can be tested.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@crates/surreal_container/Cargo.toml` around lines 43 - 44, The workspace test
fails because the crate relies on a path dependency named `ndarray` (referenced
by the `ndarray = { path = "../../../ndarray", ... }` entry) that isn’t present,
so before running cargo test for `crates/surreal_container` either ensure the
`../../../ndarray` directory is available (e.g., checkout or init the
submodule/fork) or change the dependency to the published crate by replacing the
path override with a versioned dependency in the manifest; locate the `ndarray`
dependency entry referenced by `crates/surreal_container/Cargo.toml` (or the
workspace override) and either populate the path or switch to `ndarray =
"<appropriate_version>"` so the workspace can resolve the dependency.
- deltalake 0.31 -> 0.32 (codex P1): the previous pin pulled datafusion 52 + arrow 57 transitively, conflicting with the workspace's DF53 + arrow 58 after the lance 6.0 bump. DeltaTableProvider implements the wrong TableProvider trait in default builds with delta enabled. - lance-graph-python: datafusion 51 -> 53, pyo3 0.26 -> 0.28 (codex P2 + cargo resolve): arrow-pyarrow 58.0 transitively requires pyo3 0.28; only one crate can link "python" so the 0.26 + 0.28 coexistence is unresolvable. DF51 also incompatible with the SessionContext/Arrow 58 types now flowing through graph.rs into MemTable::try_new. - surreal_container: drop stale BLOCKED(A)/(B) comments (CodeRabbit): pins are =6.0.0 / =0.29.0, no longer blocked. Local cargo check --workspace --all-features passes.
Real CI failure root cause: `deltalake 0.32` removed
`delta_datafusion::DeltaTableProvider::try_new`; the new API is
`TableProviderBuilder::new().with_snapshot(...).with_log_store(...).build().await`
(async, fallible). `crates/lance-graph/src/table_readers.rs:119`
still calls the old API and fails to compile under default features.
Per codex P1's explicit fallback option ("disable/remove the Delta
reader until it can be aligned"), drop `delta` from
`crates/lance-graph/Cargo.toml :: default = [...]`. The `delta`
feature remains opt-in via `--features delta` for any caller that
wants to patch up the broken state themselves; `DeltaTableReader`
itself is already `#[cfg(feature = "delta")]` so non-default builds
don't even compile the broken file. Refactor to TableProviderBuilder
is queued as a follow-up — `TODO(lance-bump-delta)` left at the
feature definition with the new API surface noted.
Local `cargo build --manifest-path crates/lance-graph/Cargo.toml`
(matches CI command) finishes clean in 5m16s.
|
Actionable comments posted: 0 |
3 tasks
AdaWorldAPI
pushed a commit
that referenced
this pull request
May 28, 2026
….1 block #423 landed the lance 4→6, lancedb 0.27→0.29, datafusion 52→53, arrow 57→58 coordinated bump (RESOLVED A/B/D). The stale BLOCKED comments in the root Cargo.toml predate that landing. This commit: - Replaces the BLOCKED-(A)/(B)/(D) comment block with a dated RESOLVED record pointing to #423 + the live crate-level pins. - Records the user-authorised follow-on patch 6.0.0 → 6.0.1 as CURRENTLY BLOCKED: lancedb 0.29.0 (the latest 0.29.x on crates.io) transitively pins `lance = "=6.0.0"`, so cargo cannot satisfy `lance = "=6.0.1"` + the lancedb transitive simultaneously. Resolution path: wait for lancedb 0.29.1+ that bumps the lance pin, or drop the strict-= in lance-graph. - Keeps BLOCKED(C) (surrealdb kv-lance git source) since it's a separate fork-access concern still pending human input. No code touched. No version-string pin changed in any crate Cargo.toml — they all stay at the #423 state (=6.0.0 / =0.29.0 / 53 / 58). https://claude.ai/code/session_01FMooFcE7hgRWWvknNr2N4i
Merged
3 tasks
AdaWorldAPI
added a commit
that referenced
this pull request
May 28, 2026
…sis-LXmug chore(deps): bump lance 4.0.0 → 6.0.0, lancedb 0.27.2 → 0.29.0
AdaWorldAPI
pushed a commit
that referenced
this pull request
May 28, 2026
….1 block #423 landed the lance 4→6, lancedb 0.27→0.29, datafusion 52→53, arrow 57→58 coordinated bump (RESOLVED A/B/D). The stale BLOCKED comments in the root Cargo.toml predate that landing. This commit: - Replaces the BLOCKED-(A)/(B)/(D) comment block with a dated RESOLVED record pointing to #423 + the live crate-level pins. - Records the user-authorised follow-on patch 6.0.0 → 6.0.1 as CURRENTLY BLOCKED: lancedb 0.29.0 (the latest 0.29.x on crates.io) transitively pins `lance = "=6.0.0"`, so cargo cannot satisfy `lance = "=6.0.1"` + the lancedb transitive simultaneously. Resolution path: wait for lancedb 0.29.1+ that bumps the lance pin, or drop the strict-= in lance-graph. - Keeps BLOCKED(C) (surrealdb kv-lance git source) since it's a separate fork-access concern still pending human input. No code touched. No version-string pin changed in any crate Cargo.toml — they all stay at the #423 state (=6.0.0 / =0.29.0 / 53 / 58). https://claude.ai/code/session_01FMooFcE7hgRWWvknNr2N4i
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
lancefamily (lance,lance-arrow,lance-index,lance-linalg,lance-namespace) from=4.0.0to=6.0.0across 6 crates.lancedbfrom=0.27.2to=0.29.0(strict-pinslance = "=6.0.0", which forced the patch-level target).arrow 57 → 58,datafusion 52 → 53across 8 Cargo.toml files. Lance 6.0's public API takes arrow 58 types — keeping arrow 57 caused trait-incoherence at everyDataset::write/InsertBuilder/LanceTableProvidercall site.TODO(lance-bump):comments left next to each cascaded arrow/datafusion bump for traceability.Why now
Unblocks evaluating OQ-MBX-8 (
persisted_rowvs Lance native versioning) against the actual Lance 6.0 surface. Schema-evolution / versioning primitives that may answer the kvs/lance question live in 6.0, not 4.0.Build prereq
Lance 6.0 needs system
protocat build time (lance-encodingpullsprost-build). Either installprotocon the CI runners or enable theprotocfeature on lance which bundlesprotobuf-src. Worth confirming CI before merging.Next bump
Lance
v7.0.0is final on GitHub (May 27) but not yet on crates.io. Once it lands AND a matchinglancedbfinal ships (currently only0.30.0-beta.1), do another bump. Watching cadence; could be days.Pre-existing breakage (out of scope)
cognitive-shader-driverhas 3 errors (missingQUALIA_DIMSimport + missingShaderDispatchfieldsalpha_saturation_override/merge_override). Verified reproducible onorigin/mainpre-bump — not caused by this PR.Test plan
protocavailable on CI runnerOut of scope
surrealdb-corelance/lancedb bump — sequenced after this PR's CI goes green.https://claude.ai/code/session_01LiUiGeUDLje8KMnxB4FfA3
Generated by Claude Code
Summary by CodeRabbit