[ShawnChen-Sirius]

What

Two related Layer 1 changes, split out from the longer-running feat/layer2-clickhouse-js-compat work so they can land independently:

1. feat(l1): multi-connection model — same-path Sessions run in parallel

Replace the single-active-connection registry with one that allows N independent connections to the same process-wide EmbeddedServer path, so same-path Sessions execute queries in parallel instead of serializing on one shared ChdbClient's per-instance mutex.

• native (lib/chdb_node.cpp): g_active single slot → registry of N connections per bound path; a different data directory is still rejected; the last connection out unbinds the server. Guard mutex added.
• index.js: each Session owns its own native connection (also fixes a latent same-path clobber). Non-parameterized async queries run in parallel across connections. Parameterized queries are serialized process-wide (one globalParamChain), because the bundled libchdb does not isolate parameter set/reset per connection under true parallelism — concurrent param queries on different connections otherwise clobber (456 Substitution not set) and can fatally abort the engine (236). The GIL hides this in thread-based bindings (chdb-python); libuv exposes it.
• test: test/v3/multi-connection.test.ts covers same-path coexistence, data sharing, different-path rejection, param correctness under concurrency, and real parallelism (~3.7× on 4 cores).
• docs: layer1-native-binding.md §9.6.1 documents the model and root cause.

This matches chdb-python's model (one ChdbClient per Connection). The prior binding refcount-collapsed every same-path Session onto one ChdbClient, whose client_mutex serialized all queries — which is why node did not parallelize while chdb-python did (verified: the same libchdb runs concurrent chdb_query_n in parallel from independent connections).

2. test(l1): stabilize async-stress timeouts on slow CI runners

Bumps the async-stress test's wall-clock budget so a slow CI runner finishes the stress matrix instead of getting killed mid-flight. No change to what the suite exercises — it still hits the same parallelism/cancellation surfaces; only the per-test deadline grows.

Scope

• 5 files, +381/-88
• No public-API change; behavior change is strictly additive parallelism plus a process-wide serialization rule for the parameterized path (whose prior per-connection isolation was a latent correctness bug under libuv concurrency, not an intended guarantee).

Why two commits in one PR

The async-stress timeout fix was a direct follow-up after the multi-connection model exposed a slow-path regression on resource-constrained CI runners. They share the same surface (concurrent async ops on a shared engine) and benefit from being reviewed together.

Follow-up

A separate PR will replace the current feat/layer2-clickhouse-js-compat byte-compat façade with a pluggable Connection design (one public client API in @clickhouse/client with chdb-node's ChdbConnection injected via a connection option). The design discussion is in clickhouse-js#865 and the parallel Python proposal is in clickhouse-connect#809. That PR will sit on top of this one.

Generated with Claude Code
via Happy

Co-Authored-By: Claude noreply@anthropic.com
Co-Authored-By: Happy yesreply@happy.engineering

[ShawnChen-Sirius]

@chibugai review this

[Copilot]

Pull request overview

Enables Layer 1 same-path session parallelism by moving from a single “active connection” model to a multi-connection registry (one EmbeddedServer path per process, N connections per same path), while stabilizing async-stress tests on slower CI runners.

Changes:

• Native: replace single-slot active connection with a guarded registry supporting multiple connections to the same bound path.
• JS: make each Session own its own native connection; serialize parameterized async queries process-wide via globalParamChain.
• Tests/docs: add multi-connection coverage, increase stress-test timeouts, and document the new model.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
lib/chdb_node.cpp	Implements multi-connection registry + cleanup changes.
index.js	Updates Session ownership model and parameterized async serialization strategy.
test/v3/multi-connection.test.ts	Adds same-path coexistence/parallelism and param-concurrency tests.
test/v3/async-stress.test.ts	Increases timeouts for long-running stress/race cases.
docs/design/layer1-native-binding.md	Documents the new multi-connection model and param-query serialization rationale.

Comments suppressed due to low confidence (1)

index.js:251

• runExclusiveParam() always calls startNative() once it reaches the head of the chain, even if the caller’s AbortSignal already fired or the timeout already elapsed while waiting behind earlier parameterized queries. With a process-wide chain this can execute queries (including DDL/DML) after the caller already observed CHDB_ABORT/CHDB_TIMEOUT, which is surprising and can cause unintended side effects. Add an abort/timeout pre-check at execution time to skip starting the native call once the request is already cancelled/expired.

function runExclusiveParam(chain, startNative, opts) {
  const nativeP = chain.tail.then(() => startNative());
  chain.tail = nativeP.then(() => {}, () => {});
  return withAbortTimeout(nativeP, opts);
}

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[chibugai]

Reviewed at a51768f9 — OCR deep pass over the two native/JS source files plus a manual read of the registry concurrency change. Advisory only.

Native registry (lib/chdb_node.cpp) — the g_active single-slot → Registry + std::set<chdb_connection*> migration looks correct:

• Every g_reg access is under g_reg_mu (all four registry fns), and the query path never touches the registry — consistent with the thread-safety note.
• None of the four functions call each other while holding the lock, so no self-deadlock.
• release_session_conn's conns.find() guard makes double-release idempotent (no double chdb_close_conn), and boundKey is cleared only when the last connection leaves so a different data dir can re-bind. The boundKey == "" ⇔ only the default is up invariant holds, which is what makes the "close default, fall through to bind path" branch in acquire_session_conn safe.

Worth fixing (concurring, not re-flagging inline): I independently landed on the same issue as the existing review note on the lib/chdb_node.cpp line-43 comment — it still claims per-connection param isolation ("the JS per-connection serialization only has to guard a single connection"), but index.js deliberately moved to a process-wide globalParamChain precisely because the bundled libchdb clobbers param state across connections (code 456 → fatal 236). Reconciling that comment matters here: a future reader trusting it could revert the global chain to per-connection and reintroduce the exact race this PR fixes. The ChdbConnectionError-not-exported test nit looks valid too.

No net-new blocking issues from my pass.

[ShawnChen-Sirius]

Addressing the suppressed (low-confidence) review note about runExclusiveParam dispatching the native call even after the caller already observed CHDB_ABORT/CHDB_TIMEOUT — this was a real concern with the process-wide chain (a queued DDL/DML could execute after the JS promise rejected), and is now fixed in 6371b6d.

runExclusiveParam pre-checks the signal/deadline at the moment the chain head fires:

function runExclusiveParam(chain, startNative, opts) {
  const signal = opts && opts.signal;
  const timeout = opts && opts.timeout;
  const deadline = timeout ? Date.now() + timeout : 0;
  const guardedStart = () => {
    if (signal && signal.aborted) {
      return Promise.reject(new ChdbAbortError(
        "Query aborted before execution (queued behind earlier parameterized queries)"));
    }
    if (deadline && Date.now() >= deadline) {
      return Promise.reject(new ChdbTimeoutError(
        `Query timed out (${timeout}ms) before execution (queued behind earlier parameterized queries)`));
    }
    return startNative();
  };
  const nativeP = chain.tail.then(guardedStart);
  chain.tail = nativeP.then(() => {}, () => {});
  return withAbortTimeout(nativeP, opts);
}

Notes:

• The chain still advances on the guarded promise, so the next queued query is not stalled.
• Non-parameterized queries are unaffected (they never enter this chain).
• A regression test (multi-connection: parameter chain cancellation pre-check) builds head-of-chain congestion, aborts the queued queries, asserts they all reject with code: CHDB_ABORT, and verifies the chain advances normally afterwards.