Skip to content

Give the bindings one local URL and one default database - #640

Merged
wudidapaopao merged 2 commits into
chdb-io:mainfrom
ShawnChen-Sirius:fix/durable-cross-binding-and-engine-hint
Sep 11, 2026
Merged

wudidapaopao merged 2 commits into
chdb-io:mainfrom
ShawnChen-Sirius:fix/durable-cross-binding-and-engine-hint

Conversation

@ShawnChen-Sirius

@ShawnChen-Sirius ShawnChen-Sirius commented Sep 10, 2026 •

Copy link
Copy Markdown
Contributor

Two things the same durable object needed different code for depending on which language opened it, and neither difference was anything the contract asked for.

One URL. The local backend answered only to local:, chdb-node only to file:, chdb-go to either — so no single namespace URL could be handed to all three. It now takes both here (chdb-node takes both as of chdb-io/chdb-node#94). A file: URL is percent-encoded and a local: path is not; that is the only difference between them.

One default database. A cold object was created in mem here and default in Node and Go. The head is authoritative once an object exists, so this never broke interop — but the same application ported between bindings addressed a differently-named table. default wins because two of three bindings already had it and it is what an unqualified statement means in chDB anyway. The contract now states it, so there is one answer rather than three implementations.

The refusal says which engine is loaded. It named the ABI it needed and nothing else, so its two causes read identically: an engine too old to have the ABI, and a new engine that the wrapper is not loading because something else owns chdb/. And the command it printed only became true with chdb-core 26.7.3 — the first release carrying the ABI that is published to PyPI. It names chdb-core rather than chdb because pip leaves a satisfied dependency alone, so upgrading chdb reports success and moves no engine.

chdb.durable needs chdb-core 26.7.2-rc.2 or newer, which added backup_database / restore_database / classify_query.
The engine loaded here reports 26.7.0.

    pip install -U chdb-core

Verified

Cross-binding recovery, contract §7.5 — the item no repository had implemented. Every writer produces a 100-row checkpointed base plus a 50-row WAL tail that is flushed and not checkpointed, so a reader has to restore the base and replay the WAL. 3×3, on the local backend and again on real AWS S3, every cell 150 rows / sum(id) 11175 / base 100 + wal 50, each reader taking the database name from head.json:

writer \ reader Python Node Go
Python ✓ ✓ ✓
Node ✓ ✓ ✓
Go ✓ ✓ ✓

After this change all three take the same URL string and the same database name.

Also: tests/test_durable.py 50 passed against chdb-core 26.7.2-rc.2, and the whole suite again against real AWS S3 (DAO_TEST_URL=s3://…, 49 passed in 7 minutes). The refusal message was checked from where a user actually is — chdb on PyPI's newest chdb-core, 26.7.0 — and it names 26.7.0.

Two tests changed rather than being added to

test_control_statements_refused and test_reopen_honors_persisted_db both relied on the default being mem. With the default moved, the first one's USE default would have been a no-op instead of a refusal, and the second would have had nothing to disagree with. Both now name mem explicitly, which is what they were really testing.

Follow-ups, once 26.7.3 is on PyPI

  • add chdb-core>=26.7.3 to the durable extra, so pip install "chdb[durable]" resolves an ABI-carrying engine deterministically
  • drop the CHDB_CORE_ABI_WHEEL pin from the durable conformance step in the four wheel workflows; it exists only because no chdb-core on PyPI exported the ABI

🤖 Generated with Claude Code

Note

Standardize durable bindings on default database and add file: URL support to make_backend

  • Adds the shared DEFAULT_DATABASE constant (value default) in protocol.py and switches DurableObject, Namespace, and contract docs from the old mem default to it. Persisted head data remains authoritative on reopen.
  • Updates make_backend in backends/init.py to route file: URLs into the local filesystem backend, percent-decoding the path before resolution; local: and bare paths stay literal.
  • Improves ABI mismatch errors in engine.py to report the loaded engine version and, for ManagedConnection, the missing required attributes.
  • Adds installation docs and bumps the documented minimum chdb-core to 26.7.2-rc.2.
  • Behavioral Change: cold-created durable objects without an explicit db now land in default instead of mem; existing tests that relied on the implicit mem default are updated to pass it explicitly.

Macroscope summarized 0dad0fc.

ShawnChen-Sirius and others added 2 commits September 9, 2026 14:48
Cross-binding recovery testing turned up two places where the same durable
object needed different code depending on which language opened it, and
neither difference was anything the contract asked for.

The local backend answered only to `local:`, while chdb-node answered only to
`file:` and chdb-go took either. A namespace URL is the sort of thing that
gets shared between a Python job and a Node service, so it now takes both
here too (chdb-node takes both as of its own change). A `file:` URL is
percent-encoded and a `local:` path is not, which is the only difference
between them.

A cold object was created in a database called `mem` here and `default` in
Node and Go. The head is authoritative once an object exists, so this never
broke interop -- but the same application ported between bindings addressed a
differently-named table, which is worse than a name nobody likes. `default`
wins because two of three bindings already had it and it is what an
unqualified statement means in chDB anyway. The contract now states it, so
there is one answer rather than three implementations.

The engine-compatibility refusal also stopped telling people to run
`pip install -U chdb-core`. Not every chdb-core release goes to PyPI -- the
wheels are large and the quota only moves one way -- so the newest one there
can be older than the release that has the ABI, and following that hint left
you exactly where you started. It now reports the engine version actually
loaded, which separates "no engine with the ABI" from "the wrapper is not
loading the engine you installed", and gives the install form that works: the
release wheel and chdb in one pip command, so chdb's own chdb/__init__.py
lands on top of the engine's.

Two tests changed rather than being added to. `test_control_statements_refused`
and `test_reopen_honors_persisted_db` both relied on the default being `mem`,
and with the default moved to `default` the first one's `USE default` would
have been a no-op and the second would have had nothing to disagree with. Both
now name `mem` explicitly, which is what they were really testing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The refusal named the ABI it needed and nothing else, so the two ways it fires
read identically: an engine too old to have the ABI, and a new engine that the
wrapper is not loading because something else owns `chdb/`. It now reports the
version actually loaded, which separates them in one line.

`pip install -U chdb-core` is the fix, and naming chdb-core rather than chdb is
the point of it. pip's default upgrade strategy leaves a dependency alone once
it is satisfied, and an installed 26.7.0 satisfies chdb's floor, so upgrading
chdb reports success and moves no engine at all.

That command only became true with 26.7.3: it is the first release carrying the
ABI that is published to PyPI, and before it the newest chdb-core there was
26.7.0. The durable docs gained an installation section saying the same thing,
including why a fresh install and an existing one need different commands.

Checked from where a user actually is: chdb on PyPI's chdb-core 26.7.0 refuses
with this message naming 26.7.0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ShawnChen-Sirius

Copy link
Copy Markdown
Contributor Author

@wudidapaopao please review this PR

@wudidapaopao wudidapaopao left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wudidapaopao
wudidapaopao merged commit 7abcc13 into chdb-io:main Sep 11, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants