[ShawnChen-Sirius]

Layer 3 — fluent, immutable query builder for chdb-node

This PR adds Layer 3, the Kysely-shaped fluent query builder, on top of the pluggable Connection introduced in #52 (now merged into main). Code: src/layer3/, tests: test/v3/layer3/. Spec: chdb_node_layer3_lld_en.md.

What's included

• SQL compiler with server-side value binding — the compiler emits {pN:Type} placeholders plus raw values and calls Layer 1 queryBindAsync; Layer 1 owns value serialization (src/serialize.ts). No value ever enters the SQL string, identifiers are backtick-quoted. ChdbCompileError is the only net-new (build-time) error. Layer 1 null semantics are left exactly as they are on main.
• Builders — SELECT (Kysely-shape verbs), INSERT, UPDATE (ALTER … UPDATE), DELETE (ALTER … DELETE), and INSERT…SELECT; terminals .compile / .execute / .executeTakeFirst[OrThrow]; json/arrow/raw format views; root factory (selectFrom / database / session).
• ClickHouse dialect sugar — final / sample / prewhere / settings / format / limitBy; chTable table-function helpers (args bound); chFn function helper incl. parametric name(params)(args).
• connect() federation — maps all 14 URL schemes to table functions with every arg bound; Connection.table()/.describe()/.databases()/.tables()/.snapshot(). Metadata wired for ClickHouse, Postgres, and MySQL; clickhouseSettings forwarded to chDB, remote-only options accepted-and-ignored (no throw).
• Typed row inference — Database<DB> flows through to .execute(); chdb-gen-types CLI with runtime introspection plus static Drizzle/Prisma schema conversion.
• Arrow C Data Interface input — registerArrowTable(name, columns) / chTable.arrowstream(name) register JS columnar buffers zero-copy via 4 new N-API methods in lib/chdb_node.cpp.
• db.introspection — Kysely-instance-shaped getTables/getSchemas/getMetadata over system.databases/tables/columns.

Testing

• 190 Layer 3 tests; full v3 suite green (364 pass / 4 skip locally), build:ts clean, native addon rebuilt.
• Golden-SQL, injection, and adversarial suites for the builder; live federation tests against real local Postgres + MySQL (skip when servers are unreachable).

🤖 Generated with Claude Code

[ShawnChen-Sirius]

@chibugai, review this

[Copilot]

Pull request overview

This pull request introduces Layer 3, a fluent, immutable query builder on top of the existing Layer 1 runtime, along with a chdb-gen-types CLI for typed schema generation and an Arrow C Data Interface input path for zero-copy columnar ingestion.

Changes:

• Adds Layer 3 builder surface (SELECT/INSERT/UPDATE/DELETE), compiler, execution terminals, and local-engine introspection.
• Adds federation via connect() with URL-scheme → table-function mapping and metadata discovery helpers.
• Adds schema codegen (runtime introspection + static Prisma/Drizzle parsing) and an Arrow table registration API backed by new N-API exports.

Reviewed changes

Copilot reviewed 45 out of 47 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
test/v3/layer3/types.test.ts	Type-level assertions for CH type literal → TS type mapping and row inference.
test/v3/layer3/static-codegen.test.ts	Tests static schema conversion (Prisma/Drizzle) and CLI output behavior.
test/v3/layer3/introspection.test.ts	Exercises db.introspection against the embedded engine.
test/v3/layer3/functions.test.ts	Tests function/table-function helpers compilation and one execution path.
test/v3/layer3/fixtures/prisma-schema.prisma	Prisma fixture for static conversion tests.
test/v3/layer3/fixtures/drizzle-schema.ts	Drizzle-shaped TS fixture for static conversion tests.
test/v3/layer3/execute.test.ts	End-to-end execution tests for core builder behavior and types.
test/v3/layer3/dialect.test.ts	Tests ClickHouse dialect sugar compilation + observable execution behavior.
test/v3/layer3/connect.test.ts	Tests URL-scheme mapping, injection resistance, metadata SQL, and execution.
test/v3/layer3/compile.test.ts	Golden-SQL compilation tests to enforce value-binding and quoting.
test/v3/layer3/codegen.test.ts	Tests runtime introspection, deterministic emission, and CLI behavior.
test/v3/layer3/arrow-input.test.ts	Tests Arrow input registration, refresh/close semantics, and validation.
src/layer3/types/infer.ts	Defines schema → row inference and RowOf helper for builder typing.
src/layer3/types/ch-types.ts	Implements CH type literal → TS type mapping via template literal types.
src/layer3/runtime.ts	Lazy Layer 1 runtime accessor + minimal runtime interface for Layer 3.
src/layer3/introspection.ts	Kysely-shaped introspector implementation over system.* tables.
src/layer3/index.ts	Layer 3 public exports surface.
src/layer3/execute/terminal.ts	Execution boundary forwarding compiled queries to Layer 1 bind APIs.
src/layer3/execute/format.ts	Output-view planning (json/arrow/raw) and JSONEachRow parsing.
src/layer3/execute/arrow-input.ts	JS API for registering Arrow columnar buffers as arrowstream() sources.
src/layer3/database.ts	Root factories (selectFrom, database, session, connect) and context binding.
src/layer3/connect/url-scheme.ts	URL-scheme → table-function mapping and validation for federation sources.
src/layer3/connect/snapshot.ts	Builds INSERT … SELECT snapshot node for one-shot materialization.
src/layer3/connect/connect.ts	Connection wrapper: table expression, describe, catalog listing, snapshot.
src/layer3/compiler/param.ts	Placeholder naming + JS value → CH type inference for server-side binding.
src/layer3/compiler/nodes.ts	AST node definitions for the builder/compiler boundary.
src/layer3/compiler/identifier.ts	Identifier quoting + function name validation.
src/layer3/compiler/compile.ts	SQL compiler producing { sql, parameters } and settings/format rendering.
src/layer3/codegen/introspect.ts	Runtime DESCRIBE-based introspection over tables/files/expressions/URLs.
src/layer3/codegen/from-prisma.ts	Static Prisma schema parsing → IntrospectedDatabase conversion.
src/layer3/codegen/from-drizzle.ts	Static Drizzle schema parsing (TypeScript AST) → IntrospectedDatabase conversion.
src/layer3/codegen/emit.ts	Deterministic TS emitter for schema interfaces (ClickHouse type literals).
src/layer3/codegen/cli.ts	chdb gen-types CLI implementation and argument planning.
src/layer3/builder/update.ts	UPDATE builder compiling to ClickHouse mutations.
src/layer3/builder/table-functions.ts	chTable helpers for common table functions with bound arguments.
src/layer3/builder/select.ts	Immutable SELECT builder + dialect sugar and execution terminals.
src/layer3/builder/insert.ts	INSERT builder supporting row inserts and INSERT … SELECT.
src/layer3/builder/functions.ts	chFn helper functions including parametric aggregates.
src/layer3/builder/expression.ts	sql tag and expression helpers (ref, val, fn) for AST construction.
src/layer3/builder/delete.ts	DELETE builder compiling to ClickHouse mutations.
src/errors.ts	Adds ChdbCompileError for Layer 3 build/contract failures.
package.json	Adds chdb-gen-types binary entrypoint.
lib/chdb_node.cpp	Adds Arrow C Data Interface registration/unregistration N-API bindings.
index.mjs	Exposes ChdbCompileError as a named ESM export.
index.js	Exposes low-level Arrow registration/unregistration and re-exports Layer 3.
index.d.ts	Re-exports generated Layer 3 TypeScript declarations from dist/layer3.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.