Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ chtypes' SDKs are thin, honest bindings over one frozen C ABI (`include/chtypes.

- **Build and test locally.** `scripts/fetch.sh <clickhouse-line>` installs an artifact for this machine into the per-user cache; each binding's README says how to run its suite against it. Without one, every test that needs an artifact skips by name and the rest still runs — `scripts/check-suite.sh --no-artifacts <lang>` and `scripts/check-standalone.sh --no-artifacts` are exactly what CI runs first; it then runs the same suites with two published lines. The artifact-backed proof beyond that lives in the core repository.
- **Every binding follows the same contract** (`docs/reference/`). A change to what a call means belongs in the spec and in all four bindings, not one.
- **Goldens are served, not tracked.** The golden set is published in the rolling release as `sdk-goldens.json` and installed beside the artifacts by `scripts/fetch.sh`; there is no cases file in this repository to edit. Open an issue here for a missing case; it is routed to whoever owns the generator.
- **Goldens are served, not tracked.** The golden set is published in the rolling release as `sdk-goldens.json` and installed beside the artifacts by `scripts/fetch.sh`; there is no cases file in this repository to edit. Open an issue here for a missing case.
- **Report security issues privately**: see `SECURITY.md`.

By contributing you agree your work is licensed under the Apache License 2.0 (`LICENSE`).
Expand All @@ -17,6 +17,8 @@ Each binding has its own linter, configured to be strict but green on the curren

It runs two independent checks, because one is not enough: `misspell -locale US`, and a grep backstop for a family of words misspell's matcher has been proven not to fire on reliably even when they are in its own dictionary. The word list lives in the script and nowhere else, so this page cannot drift from it. Run `scripts/lint-prose.sh --selftest` for the proof: it builds a probe file, shows misspell missing most of it, and shows the backstop catching all of it.

Markdown formatting and structure (`dprint check`, `.markdownlint.json`) run in CI's `prose` job but do not block merges yet, for the same reason: this tree's markdown was written hard-wrapped and most of it hasn't been through a `dprint fmt` pass. Dependency vulnerability scanning (`govulncheck`, `pip-audit`, `pnpm audit`, `cargo audit`) runs in CI's `security` job and also does not block — a fresh advisory against a pinned dependency with no fix available yet should not stall every unrelated PR. Both jobs still report their findings on every run.
Markdown formatting and structure (`dprint check`, `.markdownlint.json`) run in CI's `prose` job and **do block merges** — `prose` is a required status check on `main`. It was report-only while the tree was still hard-wrapped; `dprint fmt` has since run over every markdown file, so the reason for the exemption is gone. `pnpm dlx dprint@0.57.4 fmt` fixes the formatting half automatically. Dependency vulnerability scanning (`govulncheck`, `pip-audit`, `pnpm audit`, `cargo audit`) runs in CI's `security` job and also does not block — a fresh advisory against a pinned dependency with no fix available yet should not stall every unrelated PR. Both jobs still report their findings on every run.

This repository is public, and two more jobs block on that being true. `scripts/lint-public.sh` rejects any pointer to the private core repository **by name**; `scripts/lint-paths.sh` rejects any citation of a repository path that is not here. Both are in the `public` job, and the fix for either is the same: say what a thing **is** — "the C ABI contract", with `include/chtypes.h` as its public authority — rather than where it lives. Both take `--selftest`, which proves the rules fire before you trust a clean run.

Run any of these locally with the same command CI uses; each job's step name in `.github/workflows/ci.yml` names the exact invocation.
9 changes: 5 additions & 4 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,9 +75,10 @@ chtypes derives that report and it is not optional: every accepted row carries a

### Reference

| | |
| --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ |
| [`reference/go.md`](reference/go.md) · [`reference/python.md`](reference/python.md) · [`reference/ts.md`](reference/ts.md) · [`reference/rust.md`](reference/rust.md) | every public symbol, the C entry point under it, and what it returns or raises |
| [`reference/bindings.md`](reference/bindings.md) | the normative shape every binding implements — read this when porting, or when two bindings seem to disagree |
| | |
| --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
| [`reference/go.md`](reference/go.md) · [`reference/python.md`](reference/python.md) · [`reference/ts.md`](reference/ts.md) · [`reference/rust.md`](reference/rust.md) | every public symbol, the C entry point under it, and what it returns or raises |
| [`reference/bindings.md`](reference/bindings.md) | the normative shape every binding implements — read this when porting, or when two bindings seem to disagree |
| [`reference/artifact.md`](reference/artifact.md) | the artifact and registry contract: file names, `manifest.json`, platform keys, and how a loader verifies one |

The two normative pages are the deep layer. You should not need either to use chtypes; they are where a disagreement is settled, and where the answer is when a per-language reference page says "see the ABI contract".
2 changes: 2 additions & 0 deletions docs/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,8 @@ npm install @wavehouse/chtypes

ESM only. Native calls go through `ffi-rs`, prebuilt for darwin arm64/x64 and linux arm64/x64 (gnu and musl), so there is no build step.

⚠️ That is the FFI loader's matrix, not the artifact's. chtypes artifacts are published for **darwin-arm64, linux-amd64 and linux-arm64 only** ([support.md](support.md)). On an Intel Mac or a musl distribution the package installs and `ffi-rs` resolves, and then `chtypes fetch <line>` has nothing to give you.

</details>

<details><summary><b>Rust</b></summary>
Expand Down
2 changes: 1 addition & 1 deletion docs/limitations.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ The rules genuinely differ: `256` into a `UInt8` column stores `0`, while `x = 2

## Filters are shadow-only for now

No read-side security may be enforced on the filter surface until the WHERE-truth rig gates green. Until then it is for shadow and replay: run it beside your existing enforcement and compare, do not replace.
No read-side security may be enforced on the filter surface until a release explicitly lifts this limitation — the CHANGELOG will say so, and until it does, assume it has not. Until then the surface is for shadow and replay: run it beside your existing enforcement and compare, do not replace.

The parse-once block twin does not change that — it is a performance shape, not a maturity signal, and sits under the same gate.

Expand Down
15 changes: 8 additions & 7 deletions docs/reference/artifact.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,13 +80,14 @@ The reference algorithm (`chtypes.NewRegistry` in `go/chtypes/multiversion.go`):
1. `ReadDir(registry)`. For each entry that is a directory:
2. Read `manifest.json`. **If it is missing or unparseable, skip the directory silently** — a registry may legitimately contain scratch directories, and a `.DS_Store` is not a version.
3. `dlopen(dir/<manifest.library>, RTLD_NOW | RTLD_LOCAL)`. A failure here IS an error and MUST abort with the path and the `dlerror()` text: a directory that has a manifest and does not load is broken, not absent.
4. Resolve the `chs_*` symbols by name. Four are **mandatory** — `chs_clickhouse_version`, `chs_init`, `chs_schema_compile`, `chs_rows`. If any is missing, `dlclose` and reject the library: it is not a chtypes artifact.
5. Resolve `chs_abi_revision`. **Absent** -> the artifact predates the probe; record revision `0` and continue with the rules below — absence is ignorance, not incompatibility. **Present** -> call it. If it returns a value that is neither `0` nor the revision the binding was written against (`CHS_ABI_REVISION`), **reject the library**, naming both numbers: the artifact has positively stated that the binding's declarations do not describe it, and calling through them is undefined. 6. Every other symbol is **optional**. A missing one means "this artifact predates the feature" and MUST degrade to `unsupported` at call time, never to a load failure. The reference returns a private `-3` from its C shims for a missing `chs_schema_engine` / `chs_schema_ttl` and turns it into `CodeUnsupported` with `"this artifact predates engine support (rebuild it)"`; a missing `chs_row` returns `NULL`, reported as `"this artifact predates chs_row (rebuild it)"`.
6. Column introspection is **all-or-nothing**: `chs_schema_column_count`, `_name`, `_type`, `_default_expr`, `_default_kind`, `_default_is_literal` shipped together. If any is missing, treat the whole group as absent and leave the schema's column list empty rather than partially populated.
7. Ask the library its own version: `chs_clickhouse_version()`. **The library names itself; nothing is inferred from the path.** Derive the minor line from that string.
8. `chs_init(timezone, unsafe_families, out_err)`, once per library, with the contents of that version's own `unsafe_families.txt`. Each library keeps its own DateLUT and its own refuse-list. `out_err` MAY be `NULL`; when it is not and the call fails, it carries ClickHouse's own message (free with `chs_free`) — the reachable failure is an unknown timezone.
9. Index the library under **both** its exact version and its minor line.
10. If zero libraries loaded, that is an error naming the directory — an empty registry is a configuration mistake, not an empty result.
4. Resolve the `chs_*` symbols by name. Four are **mandatory everywhere** — `chs_clickhouse_version`, `chs_init`, `chs_schema_compile`, `chs_rows`. If any is missing, `dlclose` and reject the library: it is not a chtypes artifact. **The four bindings do not agree beyond that**, and this is a known divergence rather than a rule. The Go reference named above also requires `chs_free`, `chs_validate_type` and `chs_schema_free` — seven in all — because its C shims call those three without NULL checks, so admitting a library that lacks one would trade a clean load error for a SIGSEGV on first use. Python, TypeScript and Rust treat all three as optional and degrade to `unsupported`. No artifact this repository builds is affected: every one exports all seven. Which number is the contract is open — see issue #13.
5. Resolve `chs_abi_revision`. **Absent** -> the artifact predates the probe; record revision `0` and continue with the rules below — absence is ignorance, not incompatibility. **Present** -> call it. If it returns a value that is neither `0` nor the revision the binding was written against (`CHS_ABI_REVISION`), **reject the library**, naming both numbers: the artifact has positively stated that the binding's declarations do not describe it, and calling through them is undefined.
6. Every other symbol is **optional**. A missing one means "this artifact predates the feature" and MUST degrade to `unsupported` at call time, never to a load failure. The reference returns a private `-3` from its C shims for a missing `chs_schema_engine` / `chs_schema_ttl` and turns it into `CodeUnsupported` with `"this artifact predates engine support (rebuild it)"`; a missing `chs_row` returns `NULL`, reported as `"this artifact predates chs_row (rebuild it)"`.
7. Column introspection is **all-or-nothing**: `chs_schema_column_count`, `_name`, `_type`, `_default_expr`, `_default_kind`, `_default_is_literal` shipped together. If any is missing, treat the whole group as absent and leave the schema's column list empty rather than partially populated.
8. Ask the library its own version: `chs_clickhouse_version()`. **The library names itself; nothing is inferred from the path.** Derive the minor line from that string.
9. `chs_init(timezone, unsafe_families, out_err)`, once per library, with the contents of that version's own `unsafe_families.txt`. Each library keeps its own DateLUT and its own refuse-list. `out_err` MAY be `NULL`; when it is not and the call fails, it carries ClickHouse's own message (free with `chs_free`) — the reachable failure is an unknown timezone.
10. Index the library under **both** its exact version and its minor line.
11. If zero libraries loaded, that is an error naming the directory — an empty registry is a configuration mistake, not an empty result.

`RTLD_LOCAL` is not a detail: it is what keeps each library's ClickHouse symbols private, so two builds that both define `DB::DataTypeFactory` never collide. A loader that uses `RTLD_GLOBAL` will appear to work and answer with the wrong version's semantics.

Expand Down
Loading