Skip to content

fix(ci): stop close()-mid-flight engine abort (#53) and absorb ResultSet teardown (#54) - #56

Merged
wudidapaopao merged 1 commit into
chdb-io:mainfrom
ShawnChen-Sirius:fix/ci-issues-53-54
Jun 24, 2026
Merged

wudidapaopao merged 1 commit into
chdb-io:mainfrom
ShawnChen-Sirius:fix/ci-issues-53-54

Conversation

@ShawnChen-Sirius

Copy link
Copy Markdown
Contributor

Summary

Two workflows were red on main (commit 08ffdb2, the #52 merge). Both failures come from a lifecycle teardown racing an operation that the layer above assumes has already settled — fixed here in one change each.

Closes #53, closes #54.

#53 — test red: async-stress 120s hang on macos-14/Node-20

The closing a session while its async query is in flight (100x) case hung the full 120 s on macos-14/20.x, then the abandoned in-flight op poisoned 48 later v3 tests with Code: 236 … server is shutting down (ABORTED).

Root cause: a plain queryAsync (no signal/timeout) takes withAbortTimeout's fast path, which skipped trackNative. So pendingNativeOps was empty and Session.close() ran CloseConnection immediately, mid-flight. Releasing a connection while its query is still executing on the libuv worker aborts the in-process engine and, on some platforms, leaves the worker blocked inside chdb_query so its promise never settles → the caller's await hangs to the test timeout. (The test's own header comment predicted exactly this: "plain queryAsync is not tracked for drain".)

Fix: track every native async op (query and insert), not just the abort/timeout ones, so close() defers teardown until in-flight ops drain. This structurally removes the mid-flight-release race rather than papering over the timing.

#54 — connection red: unhandled ResultSet has been closed in parity

The parity job exited non-zero on 1 unhandled error even though all 27 test files passed:

⎯⎯ Uncaught Exception ⎯⎯
Error: ResultSet has been closed
 ❯ ResultSet.close packages/client-node/dist/result_set.js:244:30

Root cause: clickhouse-js's ResultSet.close() tears down via this._stream.destroy(new Error("ResultSet has been closed")). For a never-consumed result (the "format is not stream-able" test — stream() throws, then result.close() in its finally), nothing has attached an 'error' listener to the Readable we hand back, so Node escalates that synthetic teardown error to an uncaughtException. It surfaces asynchronously and vitest blames the next test (can pause response stream).

Fix: attach a benign 'error' listener to the materialized buffer Readable. That buffer has no genuine async error source (read() only ever pushes), so the only 'error' it can ever receive is exactly that external teardown signal — absorbing it makes close() idempotent as the upstream contract expects. Real consumers (Stream.pipeline in ResultSet.stream) attach their own error handler, so genuine errors still propagate. Also gitignores the parity runner's .chdb-runner checkout.

Verification

  • v3 suite: 174 passed / 4 skipped; the lifecycle-race case is now ~2 s (was a 120 s hang).
  • clickhouse-js parity runner (node tests/clickhouse-js/runner.mjs, clickhouse-js@main injected with ChdbConnection): exits 0 with 0 unhandled errors — Test Files 27 passed | 2 skipped, Tests 180 passed | 41 skipped.

Note: #53's hang is macos-14/Node-20 timing-sensitive and could not be reproduced on the dev box's newer Node; the fix is structural (no longer releases a connection with an in-flight op) and does not depend on timing. Final confirmation is the CI leg itself.

🤖 Generated with Claude Code

… ResultSet teardown (chdb-io#54)

Two failures were red on main; both stem from a lifecycle teardown racing an
op that the layer above assumes has already settled.

chdb-io#53 — test/async-stress 'close session during in-flight
query' hung 120s on macos-14/Node-20, then poisoned 48 later v3 tests with
code 236 (engine ABORTED). A plain queryAsync (no signal/timeout) took
withAbortTimeout's fast path, which skipped trackNative, so pendingNativeOps
was empty and Session.close() ran CloseConnection immediately — mid-flight.
Releasing a connection while its query is still executing on the libuv
worker aborts the in-process engine and, on some platforms, leaves the
worker blocked inside chdb_query so its promise never settles and the
caller's await hangs to the test timeout. Fix: track EVERY native async op
(query and insert), not just the abort/timeout ones, so close() defers
teardown until in-flight ops drain.

chdb-io#54 — the connection/parity job exited non-zero on an
unhandled 'Error: ResultSet has been closed' even though all 27 files
passed. clickhouse-js's ResultSet.close() tears down via
this._stream.destroy(new Error('ResultSet has been closed')); for a
never-consumed result (a non-streamable-format query whose stream() threw,
closed in a finally) nothing has attached an 'error' listener to the
Readable we return, so Node escalates that synthetic teardown error to an
uncaughtException, surfacing async and blamed on the next test. Fix: attach
a benign 'error' listener to the materialized buffer Readable — it has no
genuine async error source, so the only 'error' it can receive is exactly
that external teardown signal; real consumers (Stream.pipeline in
ResultSet.stream) attach their own handler, so genuine errors still
propagate. Also gitignore the parity runner's .chdb-runner checkout.

Verification: full v3 suite 174 passed / 4 skipped (lifecycle race ~2s);
clickhouse-js parity runner exits 0 with 0 unhandled errors
(180 passed | 41 skipped).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@wudidapaopao
wudidapaopao merged commit 5fc8d38 into chdb-io:main Jun 24, 2026
17 checks passed
ShawnChen-Sirius added a commit that referenced this pull request Jul 7, 2026
…w integrations

Additive minor release on top of 3.1.0 (2026-06-24). No breaking changes;
existing 3.1.0 APIs continue to work unchanged. libchdb bits are unchanged
(chdb-core v26.5.1-rc.1 packaged as @chdb/lib-* 26.5.2 — no republish
needed, the subpackage-matrix job will idempotent-skip).

## What's new

### Layer 3 fluent query builder SDK (#55, #59)

First-class typed fluent query API — Kysely-/Drizzle-style:

    const db = connect({ session })
    const rows = await db
      .from('users')
      .where(r => r.age.gt(18))
      .select('id', 'name')
      .execute()

- `src/layer3/{builder,execute,compile,dialect,introspection,connect,types,codegen}/`
- `chdb-gen-types` CLI — generates row types from a schema (Drizzle /
  Prisma / raw SQL sources)
- `.stream()` terminal with server-side parameter binding
- Arrow C Data Interface support: register JS columnar data as
  `arrowstream()` tables
- 12 new test files under `test/v3/layer3/`

### Cross-language agent-tool contract + 2 framework adapters (#61, #66)

- `chdb/ai-sdk` — Vercel AI SDK tools (`generateText`/`streamText`)
- `chdb/mastra` — Mastra tools + `ChDBVector` (HNSW RAG store) + `ChDBStore`
- Both are thin wrappers over `ChDBTool.call()` from `integrations/agents/`,
  which implements the cross-language contract vendored from `chdb-io/chdb`
  (`chdb/agents/CONTRACT.md`, chdb-io/chdb#597).
- 7 canonical tools: `run_select_query` / `list_databases` / `list_tables` /
  `describe_table` / `get_sample_data` / `list_functions` / `attach_file`
- 5 pillars enforced: engine-level readonly=2 (P1), value-vs-identifier
  separation (P2), truncation flag (P3), error envelope for model
  self-correction (P4), resource/source controls (P5)
- Behavior verified against the shared `conformance/cases.jsonl` fixture

### `chdb/hypequery` DatabaseAdapter (#63, #65)

Lets `@hypequery/clickhouse` query builder run on embedded chDB instead
of a remote ClickHouse server. Uses hypequery's own `substituteParameters`
(imported, not copied) so rendered SQL stays byte-identical to hypequery's
built-in HTTP adapter.

### `chdb/connection` stabilization for @clickhouse/client 1.23.0

- `peerDependency: "@clickhouse/client: >=1.23.0"` (optional) — old client
  triggers a clear npm warning instead of a runtime TS error
- Parity runner switched to track `ClickHouse/clickhouse-js` release tag
  `client-1.23.0` (per the sync-policy table; #52 shipped the initial hook)
- Runner auto-detects upstream Vitest layout (pre-/post-clickhouse-js#931)
- Runner re-synced when clickhouse-js reorganized into a monorepo (#62)

### AI-discovery docs (#60)

- Top-level `AGENTS.md` + `llms-full.txt`
- Per-subpath `AGENTS.md` (`src/layer3/`, `src/connection/`, etc.)
- All packaged in `files` so agents crawling the installed tarball find them

## Fixes / infrastructure

- Bind `null` / `undefined` query params as SQL `\N` (contributor
  @dfrankland, #68) — matches `@clickhouse/client` behavior
- `close()`-mid-flight engine abort + ResultSet teardown race (#56)
- `ChDBVector` HNSW granularity / filter binding / dimension guard (#61)
- Release workflow: publish → `unverified` dist-tag → clean-install verify
  matrix → promote to `latest` (#64). A broken release now never reaches
  `latest` users.
- CI: cache deps + harden npm registry fetch against transient resets

## User-facing installation matrix

```
npm install chdb                                    # base + Layer 3 SDK
npm install chdb @clickhouse/client                 # + chdb/connection
npm install chdb @hypequery/clickhouse              # + chdb/hypequery
npm install chdb ai zod                             # + chdb/ai-sdk
npm install chdb @mastra/core zod                   # + chdb/mastra
```

Native libchdb: chdb-core v26.5.1-rc.1 (packaged as @chdb/lib-* 26.5.2),
unchanged from 3.1.0.

Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants