Repository navigation
feat(l2): pluggable Connection — chdb/connection for @clickhouse/client - #52
Merged
wudidapaopao merged 8 commits intoJun 24, 2026
Merged
Conversation
Add the pluggable Connection surface (`chdb/connection`) so a user can
plug in-process chDB into `@clickhouse/client`:
import { createClient } from '@clickhouse/client'
import { createChdbConnection } from 'chdb/connection'
const client = createClient({
connection: createChdbConnection({ path: ':memory:' }),
})
ChdbConnection implements `@clickhouse/client-common`'s published
`Connection<Stream.Readable>` interface EXACTLY — same method
signatures, same result shapes, same error semantics. chdb-node
does NOT define its own Connection / ConnQueryResult / ConnInsertParams
/ … types; they are re-exported verbatim from `@clickhouse/client-common`
(now a devDependency). Bumping that package forces a deliberate
decision about which version of the contract we conform to.
Implementation:
src/connection/connection.ts re-export the real types from
@clickhouse/client-common + the
chdb-only ChdbConnectionOptions
src/connection/chdb-connection.ts ChdbConnection class that wraps
Layer 1's Session and bridges:
- query/exec materialize via
Session.queryAsync (eager-buffered;
errors at `await`, not mid-stream)
- insert reconstructs the
`${params.query}\n${body}` inline
INSERT … FORMAT X\n<body> shape
chDB's parser accepts
- ping(params) executes SELECT 1
- query_id synthesized as UUIDv4
- response_headers={}, http_status_code=200
(synthetic for the no-wire embedded path)
src/connection/index.ts public exports
package.json ./connection subpath; @clickhouse/client-common
devDependency
`:memory:` path uses a process-shared, reference-counted temp dir so N
ChdbConnections coexist and see each other's data (matching
clickhouse-js's `chdb://memory` shared-state semantics; the
multi-connection model from PR chdb-io#51 lets N native connections share the
same bound path).
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
Add the .chdb capability extension on ChdbConnection — a namespaced
surface for chDB-specific operations that do NOT belong on the cross-backend
Connection contract:
conn.chdb.session.path — bound on-disk path (or temp dir)
conn.chdb.session.isTemp — whether the path is process-managed
conn.chdb.queryAsync(sql, opts) — native single-shot ChdbResult; bytes()
/ text() / json() / toArrow() views
conn.chdb.queryStream(sql, opts) — true streaming chunked output
conn.chdb.rawInsert(params) — raw passthrough Buffer insert
conn.chdb.streamInsert(params) — backpressured streaming insert
The exposed set is intentionally LIMITED to what chdb-node supports
today; aspirational items (Python() table function, UDF registration)
are not present because chdb-node does not implement them yet.
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
Add the chdb-owned cross-suite parity machinery: all chdb-side
@clickhouse/client integration logic lives in chdb-node's repo. The
upstream @clickhouse/client repo carries zero chdb-specific code and
zero chdb CI dependency — the only upstream change is the
createClient({ connection }) injection point.
New tree:
tests/clickhouse-js/
skip_list.json AUTHORITATIVE blacklist of upstream tests
that chdb does not support today (with reasons).
Includes a syncedAgainst block recording which
@clickhouse/client ref this list is calibrated
against, and an underInvestigation list of
files where the chdb failure looks fixable in
ChdbConnection rather than a capability gap.
inject_setup.ts Vitest setup that wraps every createClient call
with connection: createChdbConnection({...}).
runner.mjs Clones @clickhouse/client at the configured ref,
links this chdb-node checkout into it, injects
the setup, and runs the integration suite with
skip_list.json applied as --exclude patterns.
README.md Sync policy: which @clickhouse/client ref the
runner targets at each stage of the upstream PR
(open on personal fork → merged to main → latest
release). Documents how to add/remove a skip
entry when a new release lands.
Sync policy: chdb-node re-runs parity and updates skip_list.json only
when @clickhouse/client cuts a new release. During the rolling period
the runner stays pointed at the ref in skip_list.json#syncedAgainst.ref;
no per-commit chdb participation in upstream CI.
Initial skip_list categories (23 files):
- HTTP transport (12)
- Auth/RBAC (5)
- Server runtime / cluster (2)
- HTTP session / abort semantics (2)
- HTTP exception/compression headers (2)
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
test/v3/connection/contract.test.ts pins the cross-backend invariants of the Connection<Readable> interface that ChdbConnection must satisfy: identity, capability flag, query/exec/command/insert return shape, ping result, close idempotence, and the presence of the .chdb extension. These are the assertions any pluggable Connection implementation should be able to pass. Server-vs-chdb byte-parity is OUT of scope here — it's covered by tests/clickhouse-js/runner.mjs running the upstream integration suite. vitest.config.ts gains a server.deps.external rule for /\\/index\\.js$/. The CJS entrypoint owns the process-wide session and pending-op registries; if vitest transforms the import copy, the entrypoint evaluates TWICE and the global afterEach safety net drains a different instance than the one ChdbConnection creates sessions on — sessions leak across files. Externalizing routes every importer through Node's require cache. Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering>
Two-job CI for the pluggable Connection surface:
- contract: runs test/v3/connection/contract.test.ts (the in-repo
cross-backend invariants) on every PR/push. Fast,
self-contained, no network.
- parity: invokes tests/clickhouse-js/runner.mjs — clones
@clickhouse/client at the ref configured in
tests/clickhouse-js/skip_list.json#syncedAgainst,
links this chdb-node checkout into it, injects
ChdbConnection into createClient, and runs the upstream
integration suite with skip_list.json applied as
--exclude patterns.
A new parity failure is either a real new chdb capability gap (add to
skip_list.json) or a ChdbConnection bug (fix in src/connection/). The
sync policy in skip_list.json's README makes the upstream-ref bump
explicit: chdb-node only re-runs the parity matrix when
@clickhouse/client cuts a new release.
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
Rewrite docs/design/pluggable-connection.md to reflect the final
design:
- Upstream-clean asymmetry: @clickhouse/client gains ONE option
(createClient({ connection })); everything else lives here.
- ChdbConnection RE-EXPORTS the Connection contract from
@clickhouse/client-common (not a chdb-local interface).
- tests/clickhouse-js/skip_list.json is the chdb-owned blacklist
of upstream tests that hit real chdb capability gaps; sync policy
is encoded in syncedAgainst.ref.
- Sync cadence: re-run parity only when @clickhouse/client cuts a
new release — between releases the skip list is frozen.
- Initial skip categories: HTTP transport, auth/RBAC/TLS, server
runtime / cluster, HTTP session_id, abort-signal divergence.
README.md adds a 'Pluggable Connection' section pointing at the
design doc.
Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
3 tasks done
Contributor
Author
|
@chibugai review it |
There was a problem hiding this comment.
Pull request overview
Introduces a new chdb/connection public entrypoint that implements @clickhouse/client-common’s Connection<Stream.Readable> so chdb-node can plug into @clickhouse/client via createClient({ connection }), plus adds an upstream parity runner + CI workflow to validate compatibility.
Changes:
- Added
src/connection/*withChdbConnection,.chdbextension surface, and type re-exports from@clickhouse/client-common. - Added a clickhouse-js parity runner (
tests/clickhouse-js/*) with a skip manifest, plus a CI workflow to run contract + parity suites. - Updated docs and package exports to publish the new
chdb/connectionsubpath.
Reviewed changes
Copilot reviewed 14 out of 15 changed files in this pull request and generated 13 comments.
Show a summary per file
| File | Description |
|---|---|
vitest.config.ts |
Externalizes index.js in vitest to avoid double-evaluation/global registry splits. |
src/connection/index.ts |
Public exports for the new chdb/connection surface. |
src/connection/connection.ts |
Re-exports @clickhouse/client-common Connection/result types; defines ChdbConnectionOptions. |
src/connection/extension.ts |
Defines .chdb extension namespace and binds it to a Layer 1 session. |
src/connection/chdb-connection.ts |
Implements ChdbConnection and createChdbConnection. |
test/v3/connection/contract.test.ts |
Adds an in-repo contract suite defining expected cross-backend invariants. |
tests/clickhouse-js/runner.mjs |
Adds a runner that clones/builds clickhouse-js and runs its integration suite with skips. |
tests/clickhouse-js/skip_list.json |
Authoritative skip manifest for upstream suite parity runs. |
tests/clickhouse-js/README.md |
Documents the parity runner workflow and skip policy. |
tests/clickhouse-js/inject_setup.ts |
Describes injected vitest setup for wrapping createClient with ChdbConnection. |
README.md |
Documents the new chdb/connection feature and links design docs. |
docs/design/pluggable-connection.md |
Design document for pluggable Connection + parity policy. |
.github/workflows/connection.yml |
Adds CI jobs for contract and parity suites. |
package.json |
Adds ./connection export and introduces @clickhouse/client-common. |
package-lock.json |
Lockfile updates for new dependencies/versioning. |
Comments suppressed due to low confidence (1)
package.json:66
src/connection/chdb-connection.tsimportsparseErrorfrom@clickhouse/client-commonat runtime, but the PR adds@clickhouse/client-commononly as a devDependency. Consumers installingchdbin production may omit devDependencies, causingrequire('@clickhouse/client-common')to fail at runtime. This needs to be a regular dependency (and lockfile updated accordingly).
"devDependencies": {
"@clickhouse/client-common": "^1.22.0",
"@types/node": "^20.14.0",
"apache-arrow": "^21.1.0",
"chai": "^4.5.0",
"mocha": "^10.7.3",
"node-gyp": "^9.3.1",
"typescript": "^5.5.0",
"vitest": "^2.0.0"
},
"dependencies": {
"node-addon-api": "^6.1.0",
"node-gyp-build": "^4.6.0"
},
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
chibugai
reviewed
Jun 23, 2026
…, runner setup wiring, SQL escape, contract test alignment Single follow-up commit addressing all 13 line-level review comments on PR chdb-io#52, leaving the original 6 commits unchanged. Fixes are grouped by severity below. ### Critical (would block ship / silently break in prod) - **`@clickhouse/client-common`: devDependency → dependency.** `src/connection/chdb-connection.ts` imports `parseError` at runtime (not just types). With it as a devDependency, `npm install chdb --omit=dev` (the default for production installs) does not pull it in, and `require("@clickhouse/client-common")` throws at first use. Verified via local tarball install + triggering the error-reshape path; `parseError` now resolves cleanly and emits the expected `ClickHouseError { code: '46', type: 'UNKNOWN_FUNCTION' }` shape. - **runner.mjs now actually loads the injection.** The previous version wrote `vitest.chdb.setup.cjs` but never wired it into vitest's `setupFiles`, so the upstream integration suite was running against the default HTTP backend (papered over by an uncommitted local edit to upstream's `vitest.node.setup.ts`). The runner now patches upstream's `vitest.node.setup.ts` in-place — appending a sentinel- delimited snippet that wraps `globalThis.environmentSpecificCreateClient` with `createChdbConnection` — and restores the original content in a `try/finally` block (same pattern as `applyPerTestSkips`). A diagnostic line confirms the wrapping in the logs: [chdb-runner] createClient wrapped with ChdbConnection With the injection now actually firing, 11 additional chdb-vs-server divergences surfaced (Tuple/Map/Nullable parameter-binding, HTTP-style mid-stream error delivery, error-type parsing, Parquet decoder version, nested-JSON input). Documented in `skip_list.json` with concrete reasons. - **SQL string escape uses `''` (doubling), not `\'`.** ClickHouse SQL string literals are escaped by doubling single quotes; backslash escapes either silently truncate the value or produce malformed SQL depending on engine settings. Affects any `clickhouse_settings` value containing `'`. ### contract.test.ts rewrite The previous contract suite asserted the **Client-layer** shape (`queryId`, `summary` on `query()`, `{ table, format, values }` on `insert`, `ping → boolean`). The real `Connection<Stream.Readable>` contract (re-exported from `@clickhouse/client-common`) uses the **Connection-layer** shape: snake_case `query_id` / `response_headers` / `http_status_code`, no summary on `ConnQueryResult`, `ConnInsertParams` takes `{ query, values }`, and `ConnPingResult = { success: true } | { success: false, error: Error }`. All 13 cases now pin the real upstream shape and pass (verified by actually running `npx vitest run test/v3/connection/contract.test.ts`). - Promoted `mapError` from private `#mapErrorLike` to a public method on `ChdbConnection`, mirroring `NodeBaseConnection.mapError` from clickhouse-js. ### Documentation + lint - `README.md`: stale `test_profile.json` reference replaced with the actual `tests/clickhouse-js/skip_list.json`. - `docs/design/pluggable-connection.md` and `tests/clickhouse-js/README.md`: removed references to a non-existent `underInvestigation` array in `skip_list.json`; replaced with the actual triage rule (every entry has a `reason`; on upstream release, new failures get classified as either `ChdbConnection` bugs or capability gaps). - Removed `tests/clickhouse-js/inject_setup.ts` — it was never loaded by vitest (see C12 above); the injection snippet now lives inline in `runner.mjs`. - Dropped unused `cpSync` import from `runner.mjs`. - `clone()` in `runner.mjs`: `git pull origin <ref>` → `git fetch && git checkout <ref>` so the sync-policy "track latest released tag" path works (a tag isn't a branch and can't be pulled). ### Final verification (clean env, fresh cache) rm -rf .chdb-runner npm run build:ts npx vitest run test/v3/connection/contract.test.ts → 13/13 ✓ node tests/clickhouse-js/runner.mjs → Test Files 27 passed | 2 skipped Tests 180 passed | 41 skipped | 0 failed Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering>
peter-leonov-ch
pushed a commit
to ClickHouse/clickhouse-js
that referenced
this pull request
Jun 23, 2026
…) injection point (#879) ## What A single, additive hook on the Node client: `createClient({ connection })` lets a caller plug an externally-built `Connection<Stream.Readable>` into `createClient` instead of letting the package build its default HTTP connection. ```ts import { createClient } from '@clickhouse/client'; import { createChdbConnection } from 'chdb/connection'; const client = createClient({ connection: createChdbConnection({ path: ':memory:' }), }); ``` `client.query` / `insert` / `command` / `exec` / `ping` / `close` then route through the injected `Connection`'s existing contract. When `connection` is omitted (the default), behavior is byte-identical to today. Tracking issue: #865. Companion PR (the chdb-node side): **[chdb-io/chdb-node#52](chdb-io/chdb-node#52. ## Shape 3 lines of actual code: `make_connection` factory override when `connection` is provided, spread of `NodeConfigImpl` otherwise. No HTTP/HTTPS code touched. No new public types added. The new field is `@experimental` so we can iterate on shape without semver pressure. See `docs/design/pluggable-connection.md` for the full rationale and the asymmetric upstream-clean design (zero chdb code, zero chdb tests, zero chdb CI in this repo — all chdb-side logic lives in chdb-node). ## Why this shape Two design pressures, in tension: 1. **chDB's reason for existing is in-process zero-copy I/O.** Hiding chdb behind a loopback HTTP server (or any wire-format boundary) erases that. 2. **`@clickhouse/client`'s public surface is intentionally slim.** Bolting a second client family onto it — one mirroring the entire public API for chdb — is a maintenance trap. The compromise: - One public client API stays `createClient` from `@clickhouse/client`. - The `Connection<Stream>` interface behind it is unchanged — only injectable. - The backend implementation lives in the backend's repo. chdb-node ships `ChdbConnection` from `chdb/connection`; all chdb-side data, skip list, CI matrix that runs **this** suite against ChdbConnection stays in chdb-node's repo. ## Verification (chdb-node side) The companion chdb-node PR carries `tests/clickhouse-js/runner.mjs`, which clones this repo's integration suite and runs all 232 tests against an in-process ChdbConnection. **Currently 202 / 232 pass.** The 30 skips are documented in chdb-node's `skip_list.json` with reasons — every one is either an HTTP-only behavior (no socket in-process) or a chdb engine gap (default timezone, etc.). This is the first real-world consumer of the injection point; if shape needs to change based on what we learn here, the `@experimental` tag is the room to do that without semver pressure. ## Backwards compatibility Default code path (`connection` omitted) is unchanged — same `NodeConfigImpl`, same `make_connection`, byte-identical behavior. No existing user of `createClient` sees a difference. ## Test plan - [x] Existing test suite green (no behavior change when `connection` is omitted) - [x] TypeScript check clean - [x] End-to-end verification via chdb-node companion PR: 202 / 232 integration tests pass against `ChdbConnection` Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Happy <yesreply@happy.engineering>
peter-leonov-ch
added a commit
to ClickHouse/clickhouse-js
that referenced
this pull request
Jun 23, 2026
…w-up to #879) (#880) ## What this does A small follow-up to #879. It dials the pluggable `connection` option back from a *public plugin surface* to a **deliberately narrow, internal `@experimental` hook** whose only intended consumer today is the chDB integration. Concretely: - **Removes the public re-exports** of the `Connection<Stream>` contract (`Connection`, `ConnBaseQueryParams`, `ConnQueryResult`, `ConnInsertParams`, `ConnPingResult`, `ClickHouseSummary`, …) from the `@clickhouse/client` (Node) entrypoint. - **Deletes `docs/design/pluggable-connection.md`**, which described a fuller pluggable-backend system we are intentionally not committing to yet. - **Trims the `CHANGELOG` and `connection?` JSDoc** to state plainly that the field exists to unblock chDB and may change. - **Moves the rationale into the `connection?` JSDoc itself**, so it lives right where anyone touching the field will read it. The runtime behavior is unchanged: the `make_connection` factory override still works exactly as merged in #879. This is purely a framing/surface-area change. ## Why Two pressures, gently in tension: 1. We genuinely want to **unblock chDB** ([chdb-io/chdb-node#52](chdb-io/chdb-node#52)) with an in-process backend, today. 2. We do **not** want to turn the Node client into the common package replacement — a public pluggable-backend framework implies a stable third-party contract and invites a second client family mirroring the entire public API, which is a real long-term maintenance burden. Keeping the `Connection` types out of the public entrypoint is what *enforces* this distinction. A backend has to deep-import or structurally match the shape — and that small friction is by design: it signals "unsupported, may change" rather than promising an API we are not ready to own. Nothing internal depends on the re-exports (internal code imports these types from `./common`), and `npm run typecheck` passes. ## Next steps (not in this PR) 1. **Verify it works and is actually needed in production** with the real chDB backend before hardening anything. 2. If validated, **build a specialized client** that drops the HTTP connection-pooling burden entirely and leans into the in-process transport — leaving the higher-level query API as the surface we invest in and improve, rather than retrofitting it onto the HTTP-shaped client. Feedback very welcome — this is intentionally the smallest step that unblocks chDB while keeping our options open. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
…khouse-js main post-#879 Two coupled changes on top of chdb-io#52, both purely docs/metadata (0 runtime code changed): ### Reframe: "pluggable Connection" → "chDB experimental integration" Upstream's clickhouse-js#880 (open) makes it explicit that the `createClient({ connection })` hook merged in #879 is a **deliberately narrow, internal experiment to unblock the chDB integration — NOT the start of a public pluggable-backend / plugin system**. The Connection contract types are deliberately NOT re-exported from `@clickhouse/client`'s public entry; the deep-import friction is by-design and signals "may change." This commit aligns chdb-node's framing with that stance: - README: section heading "Pluggable Connection" → "chDB ↔ `@clickhouse/client` integration (experimental)". Added a status callout referencing clickhouse-js#879 (the hook) and #880 (the framing). - docs/design/pluggable-connection.md: same heading change; added an upstream-status callout in the doc preface; renamed the "Design principle: chdb-owned, upstream-clean" section to the more neutral "Ownership split" (the technical content underneath is unchanged — same diagram, same boundaries, same chdb-side ownership of skip_list / runner / CI; just no longer marketed as a "pluggable backend system"). - File name `docs/design/pluggable-connection.md` is kept (renaming would break links and PR-review continuity for anyone scrolling the chdb-node chdb-io#52 conversation); the heading inside the file is the one users actually read. Code under `src/connection/*` is NOT changed: - ChdbConnection still imports `Connection` (and friends) from `@clickhouse/client-common` via re-export, which IS the "deep-import friction" upstream wants preserved per #880. No reason to move to a different import path today. ### Sync ref update: PR #879 merged → main PR #879 was merged to `ClickHouse/clickhouse-js` main as commit `c2e28b9015405b...` on 2026-06-23. Per our own sync policy table ("PR merged to upstream → ClickHouse/clickhouse-js main"), update: - `tests/clickhouse-js/skip_list.json#syncedAgainst.ref`: `ShawnChen-Sirius/clickhouse-js feat/pluggable-connection` → `ClickHouse/clickhouse-js main` - `syncedAgainst.lastSyncedSha`: `be9045d` → `c2e28b9` - Markdown sync-policy tables (README, design doc, tests README, runner.mjs header docstring) bolded to show the "merged to upstream → main" row as the current state. ### Verification (clean env) Removed `.chdb-runner/` cache so the new ref takes effect: rm -rf .chdb-runner npm run build:ts → clean npx vitest run test/v3/connection/contract.test.ts → 13/13 node tests/clickhouse-js/runner.mjs → Test Files 27 passed / 2 skipped Tests 180 passed / 41 skipped / 0 failed Runner now clones `ClickHouse/clickhouse-js main` at commit `c2e28b9` (verified via `git -C .chdb-runner/clickhouse-js log`); no fork checkout needed any more. Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering>
wudidapaopao
approved these changes
Jun 24, 2026
This was referenced Jun 24, 2026
This was referenced Jun 24, 2026
This was referenced Jul 2, 2026
dfrankland
added a commit
to dfrankland/chdb-node
that referenced
this pull request
Jul 2, 2026
formatParamValue rejected a top-level null/undefined with ChdbBindError. The reference client (@clickhouse/client-common formatQueryParams) instead emits the TSV null marker \N at the top level (and the NULL keyword when nested), which the server binds as SQL NULL for Nullable columns. chdb-node is a byte-compat @clickhouse/client facade (chdb-io#49, chdb-io#52), so throwing was a divergence, not an engine limitation: the native chdb_query_with_params path already accepts \N as NULL (verified for Nullable(String)/Int64). Emit \N for a top-level null/undefined; nested nulls already render as the NULL keyword via serializeValue. A null bound to a non-Nullable placeholder is now rejected by the engine (a CHDB_* query error) rather than a client-side ChdbBindError, matching the HTTP client + server. Removes the two NULL-binding entries from the clickhouse-js parity skip list; both now pass against @clickhouse/client 1.23.0. Fixes chdb-io#67 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ShawnChen-Sirius
added a commit
that referenced
this pull request
Jul 7, 2026
…w integrations Additive minor release on top of 3.1.0 (2026-06-24). No breaking changes; existing 3.1.0 APIs continue to work unchanged. libchdb bits are unchanged (chdb-core v26.5.1-rc.1 packaged as @chdb/lib-* 26.5.2 — no republish needed, the subpackage-matrix job will idempotent-skip). ## What's new ### Layer 3 fluent query builder SDK (#55, #59) First-class typed fluent query API — Kysely-/Drizzle-style: const db = connect({ session }) const rows = await db .from('users') .where(r => r.age.gt(18)) .select('id', 'name') .execute() - `src/layer3/{builder,execute,compile,dialect,introspection,connect,types,codegen}/` - `chdb-gen-types` CLI — generates row types from a schema (Drizzle / Prisma / raw SQL sources) - `.stream()` terminal with server-side parameter binding - Arrow C Data Interface support: register JS columnar data as `arrowstream()` tables - 12 new test files under `test/v3/layer3/` ### Cross-language agent-tool contract + 2 framework adapters (#61, #66) - `chdb/ai-sdk` — Vercel AI SDK tools (`generateText`/`streamText`) - `chdb/mastra` — Mastra tools + `ChDBVector` (HNSW RAG store) + `ChDBStore` - Both are thin wrappers over `ChDBTool.call()` from `integrations/agents/`, which implements the cross-language contract vendored from `chdb-io/chdb` (`chdb/agents/CONTRACT.md`, chdb-io/chdb#597). - 7 canonical tools: `run_select_query` / `list_databases` / `list_tables` / `describe_table` / `get_sample_data` / `list_functions` / `attach_file` - 5 pillars enforced: engine-level readonly=2 (P1), value-vs-identifier separation (P2), truncation flag (P3), error envelope for model self-correction (P4), resource/source controls (P5) - Behavior verified against the shared `conformance/cases.jsonl` fixture ### `chdb/hypequery` DatabaseAdapter (#63, #65) Lets `@hypequery/clickhouse` query builder run on embedded chDB instead of a remote ClickHouse server. Uses hypequery's own `substituteParameters` (imported, not copied) so rendered SQL stays byte-identical to hypequery's built-in HTTP adapter. ### `chdb/connection` stabilization for @clickhouse/client 1.23.0 - `peerDependency: "@clickhouse/client: >=1.23.0"` (optional) — old client triggers a clear npm warning instead of a runtime TS error - Parity runner switched to track `ClickHouse/clickhouse-js` release tag `client-1.23.0` (per the sync-policy table; #52 shipped the initial hook) - Runner auto-detects upstream Vitest layout (pre-/post-clickhouse-js#931) - Runner re-synced when clickhouse-js reorganized into a monorepo (#62) ### AI-discovery docs (#60) - Top-level `AGENTS.md` + `llms-full.txt` - Per-subpath `AGENTS.md` (`src/layer3/`, `src/connection/`, etc.) - All packaged in `files` so agents crawling the installed tarball find them ## Fixes / infrastructure - Bind `null` / `undefined` query params as SQL `\N` (contributor @dfrankland, #68) — matches `@clickhouse/client` behavior - `close()`-mid-flight engine abort + ResultSet teardown race (#56) - `ChDBVector` HNSW granularity / filter binding / dimension guard (#61) - Release workflow: publish → `unverified` dist-tag → clean-install verify matrix → promote to `latest` (#64). A broken release now never reaches `latest` users. - CI: cache deps + harden npm registry fetch against transient resets ## User-facing installation matrix ``` npm install chdb # base + Layer 3 SDK npm install chdb @clickhouse/client # + chdb/connection npm install chdb @hypequery/clickhouse # + chdb/hypequery npm install chdb ai zod # + chdb/ai-sdk npm install chdb @mastra/core zod # + chdb/mastra ``` Native libchdb: chdb-core v26.5.1-rc.1 (packaged as @chdb/lib-* 26.5.2), unchanged from 3.1.0. Generated with [Claude Code](https://claude.ai/code) via [Happy](https://happy.engineering) Co-Authored-By: Claude <noreply@anthropic.com> Co-Authored-By: Happy <yesreply@happy.engineering>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Ships the
chdb/connectionpublic surface — aChdbConnectionclass that implements@clickhouse/client-common'sConnection<Stream.Readable>interface and plugs into@clickhouse/client'screateClient({ connection })injection point (companion PR: ClickHouse/clickhouse-js#879, tracking issue ClickHouse/clickhouse-js#865).After that, all
client.*methods work the same as against a remote ClickHouse server. Switching backends is one extra import + one line.Stacked on #51
This PR's base is
feat/layer1-multi-conn(#51 — Layer 1 multi-connection model). ChdbConnection's:memory:refcount semantics require N parallel connections to one bound path, which is what #51 introduces. Please merge #51 first. After merge, this PR's base will auto-update tomain.Shape (6 commits)
Connection<Stream.Readable>verbatim from@clickhouse/client-common(we don't define our own lookalike), and the class implementing it on top of Layer 1Session. IncludeswithClickHouseErrorreshape (chdb errors come out asClickHouseErrorwith.code/.type/.message);HTTP_ONLY_SETTINGSfilter so HTTP-side settings likewait_end_of_querydon't blow up against the embedded engine;default_formatrerouted to chdb's queryAsync format opt;query_paramsrouted throughqueryBindAsync; empty-body insert/exec short-circuit.conn.chdb.queryAsync/queryStream/rawInsert/session.pathfor chdb-native escape hatches (zero-copy bytes, Arrow, raw passthrough insert) that don't fit the cross-backend contract.tests/clickhouse-js/runner.mjsclones@clickhouse/clientat the ref configured inskip_list.json#syncedAgainst.ref, links this checkout in, injects ChdbConnection viaglobalThis.environmentSpecificCreateClient, and runs the full integration suite with the skip list applied. Result today: 202 passing / 30 skipped / 0 failing.test/v3/connection/contract.test.tspins the cross-backend invariants (15 tests).vitest.config.tsgainsserver.deps.external: [/\/index\.js$/]so the CJS entrypoint isn't double-evaluated.contracton every PR, slowerparitythat runs the upstream suite viarunner.mjs.docs/design/pluggable-connection.mdcovers the upstream-clean asymmetry, the.chdbnamespace rationale, sync policy (re-run parity only when@clickhouse/clientcuts a new release).Test plan
@clickhouse/clientintegration suite viarunner.mjs: 202 / 232 pass, 30 skipped with documented reasons intests/clickhouse-js/skip_list.jsonnpm run buildclean (TypeScript enforces conformance to upstreamConnection<Stream.Readable>viaimport typere-export)node tests/clickhouse-js/runner.mjslocallySkip categories (30 skips)
JSONEachRowWithProgress,system.settingsiteration): 2 filesEach skip has a
reasoninskip_list.json.Sync policy
chdb-node tests against the latest released version of
@clickhouse/client. Only when@clickhouse/clientcuts a new release does chdb-node re-run its parity job and update its blacklist for any newly-failing tests.Generated with Claude Code
via Happy