diff --git a/docs/superpowers/status/clickhouse-117740/.gitignore b/docs/superpowers/status/clickhouse-117740/.gitignore new file mode 100644 index 00000000..b530f7ba --- /dev/null +++ b/docs/superpowers/status/clickhouse-117740/.gitignore @@ -0,0 +1,8 @@ +# Each script writes its configs and logs here. The .out files are the evidence. +*.log +p_*.yml +d_*.yml +taxi_*.yml +win_late.yml +win_events.jsonl +sqlflow.diff diff --git a/docs/superpowers/status/clickhouse-117740/README.md b/docs/superpowers/status/clickhouse-117740/README.md new file mode 100644 index 00000000..9db4cb41 --- /dev/null +++ b/docs/superpowers/status/clickhouse-117740/README.md @@ -0,0 +1,82 @@ +# The harness behind the ClickHouse page's claims + +Read [STATUS.md](STATUS.md) first. It holds the findings and what each one +measured. This file says how to run them again. + +Everything here runs against `turbolytics/sql-flow:v2026.10.08` by default, +on the dev stack's `dev_default` network with the containers `kafka1` and +`clickhouse`. Set `IMG` to run another image. + +These scripts are scratch, not a supported harness. They are kept because they +are the configuration that reproduces each claim, which is the bar the page +has to meet. Folding them into something the repo checks is Pending item 3 in +STATUS.md. + +| File | What it proves | +| --- | --- | +| `probes.sh`, `probes.out` | Every single-message finding: NULL, Enum, Decimal, timestamps, unsupported types, a missing column, the retry contract and the flush interval. | +| `probe.sh` | One message through the sink into a table you declare. `probes.sh` calls it. | +| `dedup.sh`, `dedup.out` | A replayed block against `ReplacingMergeTree` and `MergeTree`, self-hosted. | +| `win.sh`, `win.yml`, `win.out` | The tumbling-window replay: one past bucket, read in ten batches. | +| `perf.sh`, `perf.out` | One throughput run against self-hosted, with a fresh consumer group. | +| `perfcloud.sh`, `perfcloud.out` | The same against Cloud. It reads `~/.sqlflow-clickhouse`, or the file `CLICKHOUSE_CREDS` names. | +| `cloudverify.out` | The Verify query on Cloud after a `v2026.10.08` run. | +| `taxi.yml`, `taxi.sql`, `publish.py` | The page's NYC taxi example, extracted verbatim from the MDX. | +| `bluesky.yml`, `bluesky.sql` | The page's WebSocket example, extracted verbatim. | +| `sqlflow.orig.mdx`, `sqlflow.mdx` | The page at the PR's head `fe238499`, and the draft with every finding applied. | + +## Setup + +Start the dev stack, then create the topics and the databases: + +```bash +make start-backing-services +for t in ch-probe nyc-taxi-trips; do + docker exec kafka1 kafka-topics --bootstrap-server localhost:9092 \ + --create --if-not-exists --topic $t --partitions 1 --replication-factor 1 +done +printf '{"x":1}\n' | docker exec -i kafka1 kafka-console-producer \ + --bootstrap-server localhost:9092 --topic ch-probe +docker exec clickhouse clickhouse-client -q 'CREATE DATABASE IF NOT EXISTS probe' +docker exec -i clickhouse clickhouse-client --multiquery < taxi.sql +``` + +`publish.py` reads `trips_0.gz` from its working directory and needs +`confluent-kafka`: + +```bash +curl -O https://datasets-documentation.s3.eu-west-3.amazonaws.com/nyc-taxi/trips_0.gz +python publish.py +``` + +## Running them + +```bash +./probes.sh > probes.out +./dedup.sh > dedup.out +./win.sh > win.out +./perf.sh s20k_a 20000 'clickhouse://default@clickhouse:8123/nyc_taxi' +``` + +`probe.sh [dsn]` runs one case: + +```bash +./probe.sh enum_nozero "e Enum8('a' = 1, 'b' = 2)" \ + "SELECT CAST(NULL AS VARCHAR) AS e FROM batch" +``` + +**`perf.sh` takes a fresh consumer group every run, on purpose.** An offset +reset fails silently when another consumer still holds the group, and a stuck +container then starves every later run while reporting zero. + +**`win.sh` runs without `--max-msgs`.** The bucket closes on +`idle_close_seconds`, and `--max-msgs` stops the process before that fires. +The script runs the pipeline for 40 seconds, then stops it. + +## Before a long run + +The Docker VM filling up is what killed Kafka in the middle of this work. + +```bash +docker run --rm alpine df -h / +``` diff --git a/docs/superpowers/status/clickhouse-117740/STATUS.md b/docs/superpowers/status/clickhouse-117740/STATUS.md new file mode 100644 index 00000000..772d1ddf --- /dev/null +++ b/docs/superpowers/status/clickhouse-117740/STATUS.md @@ -0,0 +1,415 @@ +# ClickHouse docs PR #117740: re-verification status + +Resumed and re-run on 2026-10-09 against `v2026.10.08`, and on Cloud on +2026-10-10. Paused first on +2026-09-18 against `v2026.09.17.1`. Everything below was produced by running +it, on the versions named. + +**The PR:** [ClickHouse/ClickHouse#117740](https://github.com/ClickHouse/ClickHouse/pull/117740), +the SQLFlow community integration page. Draft. Head `fe238499` on +`turbolytics/ClickHouse:docs-sqlflow-integration`. One file, 511 lines, plus a +navigation entry. Three review threads are open, at lines 372, 443 and 444. + +**The goal:** pin the newest release, re-verify every claim on it, fix the +reviewer's three open majors, and take the page out of draft. + +**The standard the user set, 2026-09-18:** every claim needs test evidence and +a configuration to reproduce it, with versions. The page does not meet that +bar today and neither does the repo. See Pending, item 3. + +**The page** is [sqlflow.mdx](sqlflow.mdx) beside this file. It applies every +finding below, and it is on the fork as commit `d4e94780`, the PR's head. + +## Versions + +| Component | Measured on | Page currently says | +| --- | --- | --- | +| SQLFlow | `turbolytics/sql-flow:v2026.10.08` | v1.1.0 | +| ClickHouse self-hosted | 26.8.1.2041, timezone UTC | 26.8 | +| ClickHouse Cloud | 26.6.1.2326, us-east-2 | 26.2 | +| Kafka | `confluentinc/cp-kafka:7.3.2` | not stated | +| Docker Desktop | 29.8.0 | not stated | + +Cloud was re-run on `v2026.10.08` on 2026-10-10. The deduplication half of +finding 3 is still the 2026-09-18 Cloud 26.4.1 measurement: it is server +behavior and does not depend on the SQLFlow release. + +## Findings + +Eight wrong claims. Three are the reviewer's open majors. Five came out of +re-running. Every one reproduces on `v2026.10.08`; the output below is from +that release. `probes.sh` runs every single-message case in one go, and +[probes.out](probes.out) is its output. + +### 1. The string row overstates what the matrix proves (reviewer, major) + +Page line 372 presents `VARCHAR`, `UUID`, `JSON` against nine destinations as +a cross-product. The matrix groups them only because they share the `utf8` +Arrow carrier, and it overrides the payload per destination. The matrix is +unchanged at `v2026.10.08` apart from the product name. + +From `docs/coverage/integrations/sink.clickhouse.yml`, the `utf8:` key: + +| Destination | Value the test actually writes | +| --- | --- | +| `String`, `LowCardinality(String)` | `L'Ε’il πŸ‘ "quoted" back\slash\ttab` | +| `DateTime64(3)` | `2026-09-01 12:00:00.123` | +| `DateTime` | `2026-09-01 12:00:00` | +| `Enum8('' = 0, 'a' = 1, 'b' = 2)` | `a` | +| `FixedString(36)`, `UUID` | `0b7e2c9a-4d31-4f6e-9a02-7c1de3f10b45` | +| `IPv4` | `192.168.1.1` | +| `Decimal(10, 2)` | `1234.56` | + +So a UUID payload does not reparse as `DateTime` or `Decimal`. + +**Fix, in the draft:** the row is `VARCHAR`, with `UUID` and `JSON` named as +types that reach the sink as text. A note under the table gives the text form +each destination needs. + +### 2. The NULL rule breaks on Enum (reviewer, major) + +A `NULL` into a non-`Nullable` `Enum8` with no zero member writes a row that +cannot be read back. + +``` +Enum8('a' = 1, 'b' = 2), non-Nullable, receives NULL + sqlflow exit 0 (the write succeeds) + SELECT count() 1 (the row is there) + SELECT e Code: 691. DB::Exception: Unexpected value 0 in enum. + (UNKNOWN_ELEMENT_OF_ENUM) (version 26.8.1.2041) + SELECT CAST(e AS Int8) 0 + +Enum8('' = 0, 'a' = 1, 'b' = 2) receives NULL -> empty string, reads fine +Nullable(Enum8('a' = 1, 'b' = 2)) receives NULL -> NULL +``` + +**Fix, in the draft:** the table row and the limits bullet carve out `Enum`. +An enum that may receive a `NULL` needs a `0` member or `Nullable`. + +### 3. ReplacingMergeTree does not remove replay duplicates on read (reviewer, major) + +Both halves are now measured. `dedup.sh` runs the self-hosted half, and +[dedup.out](dedup.out) is its output. + +Self-hosted 26.8.1, non-replicated, one replay sent as the identical block +twice: + +| Table | `count()` | `count() FINAL` | After | +| --- | --- | --- | --- | +| `ReplacingMergeTree`, a newer version of each key | 4 | 2 | `OPTIMIZE ... FINAL`: 2, the newer versions | +| `ReplacingMergeTree`, the identical block twice | 4 | 2 | | +| `MergeTree`, the identical block twice | 4 | | | +| `MergeTree SETTINGS non_replicated_deduplication_window = 100`, the identical block twice | 2 | | one part | + +Cloud 26.4.1, measured 2026-09-18 on `v2026.09.17.1`: a `ReplacingMergeTree` +read 4 rows, and 2 with `FINAL`. A plain `MergeTree` read 2 for an identical +block sent twice, because every Cloud table is replicated and +`replicated_deduplication_window` drops the second block at insert. + +So `ReplacingMergeTree` collapses duplicates on a background merge, and reads +see both until then. Insert-time deduplication drops only a byte-identical +block. A replay re-cuts its batches from the committed offset, and a batch the +flush interval cut short does not come back with the same boundaries, so a +replayed block is not guaranteed identical. + +**Fix, in the draft:** the at-least-once bullet states eventual deduplication +on merge, and points at `FINAL` for a read that must be exact now. It names +insert-time deduplication as a partial measure. + +### 4. The retry bullet is wrong + +The page says a value the driver cannot encode "is retried the full ladder +before failing, and reports as `system.sink.unreachable`". That changed in +sql-flow [#269](https://github.com/turbolytics/sql-flow/pull/269). + +ISO 8601 string into a `DateTime` column: + +``` +exit=10 wall=0s +Error: [user.sink.encode_failed] clickhouse sink: encode row: clickhouse [AppendRow]: +ts parsing time "2026-09-01T12:00:00Z" as "2006-01-02 15:04:05": cannot parse +"T12:00:00Z" as " " +``` + +`DOUBLE` into `Decimal(10, 2)`: + +``` +exit=10 wall=0s +Error: [user.sink.encode_failed] clickhouse sink: encode row: clickhouse [AppendRow]: +amount clickhouse [AppendRow]: converting float64 to Decimal(10, 2) is unsupported +``` + +The limits bullet and the timestamp troubleshooting entry both repeat the old +claim. The draft fixes both. + +### 5. The flush_interval_seconds floor does not exist + +The page says 30 "is also the smallest value the config schema accepts, so a +lower setting fails validation at startup", and steers readers to lower +`batch_size`. Both halves are false. + +``` +flush_interval_seconds: 5 + sqlflow validate -> /conf/p_flush5.yml: valid + sqlflow run -> exit=0, the row landed +``` + +**Decided by the code since the pause.** The 2026-09-18 note asked whether +the schema should gain the floor that a comment on `flushIntervalFor` +claimed. That comment no longer claims one. It says absent, zero and +negative all mean the default of 30. The schema carries no `minimum`. So the +page is wrong and the engine is not. This is a doc fix only. + +### 6. The tumbling-window bullet describes removed behavior + +The page says a window is evaluated against wall clock, that a replay running +behind real time publishes a bucket in parts, "five rows of 100", and that +window output must never be deduplicated. + +The window now closes on an event-time watermark that the engine asserts. The +config shape changed with it. A source names its `event_time`, the handler +cuts the bucket as `time_bucket(INTERVAL '', event_time)`, and +`validate` refuses `late_rows` and `poll_interval_seconds`. The 2026-09-18 +`win.yml` would no longer validate. It has been rewritten. + +`win.sh` replays 1,000 events in one past one-minute bucket, read in ten +batches of 100, with `idle_close_seconds: 15`. [win.out](win.out): + +``` + β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€bucket─┬────n─┐ +1. β”‚ 2026-09-01 12:00:00 β”‚ 1000 β”‚ + β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”˜ +rows published: 1 +``` + +One row, complete. A related check: `allowed_lateness_seconds` with a +ClickHouse window sink validates with a warning. A late row republishes its +bucket as a whole value, so the table must replace the row by key: + +``` +warning: [user.config.invalid] tables.sql[0] window: allowed_lateness_seconds is +60, so a late row republishes its bucket as a whole value. The clickhouse sink +must replace the row for (bucket, key) rather than add to it, which is its SQL's +or its table engine's to guarantee +``` + +### 7. The Docker Desktop throughput row no longer holds + +The page's third throughput row says Docker Desktop's port forwarding caps +the Kafka fetch at 10 to 15 MB/s, about 24,000 rows a second. On Docker +Desktop 29.8.0 a host binary reaching Kafka and ClickHouse through port +forwarding ran 57,322 and 55,313 rows a second. That matches the in-network +figure. The cap belonged to Docker Desktop 20.10. + +The host binary was built from `main` at `8ba6709`, not from the release tag. +The release ships as an image. + +**Fix, in the draft:** the row and its paragraph are gone. + +### 8. Version pins are stale + +`v1.1.0` appears in the prerequisites and in both `docker run` lines. The +draft pins `v2026.10.08` and links the tag. + +## What reproduces unchanged + +All on `v2026.10.08`. + +**The taxi example, extracted verbatim from the page and run end to end.** +The published configs under `dev/config/examples/published/` match the +harness copies byte for byte. + +``` +publish.py 1,000,660 rows published in 13s +sqlflow 1,000,000 rows, exit 0, 0 errors, wall 18s +``` + +The Verify query returns exactly the counts the page prints: + +``` +β”Œβ”€payment_type─┬──trips─┬─avg_total─┐ +β”‚ CSH β”‚ 617237 β”‚ 17.82 β”‚ +β”‚ CRE β”‚ 377897 β”‚ 13.91 β”‚ +β”‚ NOC β”‚ 3573 β”‚ 13.89 β”‚ +β”‚ DIS β”‚ 1293 β”‚ 12.09 β”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ +``` + +Zero U+FFFD in `pickup_ntaname` or `dropoff_ntaname`. `pickup_datetime` spans +2015-07-01 00:00:16 to 2015-09-30 23:59:58. The page says publishing "takes a +couple of minutes". It took 13 seconds, and 28 on 2026-09-18. + +**The Bluesky WebSocket example**, self-hosted, 2,000 posts: + +``` +exit=0 wall=40s errors=0 +count() 2000 +β”Œβ”€lang─┬─posts─┐ +β”‚ en β”‚ 1070 β”‚ +β”‚ ja β”‚ 236 β”‚ +β”‚ es β”‚ 63 β”‚ +β”‚ de β”‚ 53 β”‚ +β”‚ pt β”‚ 50 β”‚ +β””β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”˜ +``` + +**NULL handling, the four cases the page distinguishes,** plus an omitted +column: + +``` +DateTime DEFAULT now() <- NULL -> 1970-01-01 00:00:00 +String DEFAULT 'unset' <- NULL -> (empty string) +Int32 DEFAULT 7 <- NULL -> 0 +Nullable(String) <- NULL -> NULL +String DEFAULT 'omitted-default', column omitted -> omitted-default +``` + +**Timestamp strings.** The sink parses them in UTC +(`temporalFromString`, `internal/sinks/clickhouse.go`). + +``` +'2026-09-01 12:00:00' -> 2026-09-01 12:00:00 (DateTime('UTC') and plain DateTime alike) +'2026-09-01 12:00:00 +09:00' -> 2026-09-01 03:00:00 +``` + +**Decimal via string, and the STRUCT workaround.** + +``` +CAST(12.34 AS VARCHAR) -> Decimal(10, 2) -> 12.34 +to_json({'k': 1}) -> String -> {"k":1} +STRUCT into a String column, exit=10: + clickhouse sink: column "s": [user.sink.type_unsupported] + unsupported arrow type struct +``` + +**A missing column.** The page quotes +`column "" is not present in the table `. The real message has no +quotes around the name, and carries a code and a prefix. The draft quotes it +as it reads: + +``` +exit=1 +[system.sink.write_failed] clickhouse sink: prepare batch: failed to init block +for HTTP batch: failed to determine columns for HTTP insert: column extra is +not present in the table colmiss +``` + +## Throughput, re-measured + +Self-hosted, SQLFlow and Kafka on the same Docker network, one million +17-column rows, a fresh consumer group per run, every run landing 1,000,000 +rows with zero errors. [perf.out](perf.out): + +| `batch_size` | `v2026.10.08` run A | run B | `v2026.09.17.1`, 2026-09-18 | +| --- | --- | --- | --- | +| 5,000 | 33,908 | 34,019 | 36,640 and 36,811 | +| 20,000 | 57,877 | 55,908 | 62,733 and 68,103 | +| 50,000 | 58,253 | 56,949 | 68,911 and 68,829 | +| 100,000 | 56,153 | 57,694 | 69,986 and 63,197 | + +The shape still holds, so the page's `batch_size` guidance stands. The draft +carries the `v2026.10.08` figures. + +**ClickHouse Cloud,** us-east-2, over the public internet from the same +machine, `batch_size` 20,000. [perfcloud.out](perfcloud.out): + +| Image | Rows/second | +| --- | --- | +| `v2026.10.08` | 13,654, 13,346, 12,885 and 12,760 | +| `v2026.09.17.1`, the same hour | 12,495 and 12,926 | +| `v2026.09.17.1`, 2026-09-18 | 28,215 and 25,436 | + +Both releases ran at the same rate over the same link, so the drop from +September is the network, not the engine. The Cloud path cannot show the +self-hosted regression below, because the link bounds it first. + +**Cloud undercounted once, and lost nothing.** The first run read 980,000 +rows on a `count()` taken right after exit, exactly one batch short. +Cloud's own logs show every row written: `system.asynchronous_insert_log` +holds 156 flushes, all `Ok`, for 3,000,000 rows across the three runs, and +`system.query_log` puts 1,000,000 in the first. Cloud enables `async_insert` +and `wait_for_async_insert` by default, so the acknowledgement the engine +commits offsets on follows the flush. The short read came from a replica that +had not yet fetched the newest part. `perfcloud.sh` now counts with +`select_sequential_consistency=1`. + +**The Verify query on Cloud** returns the same counts as self-hosted. +[cloudverify.out](cloudverify.out): + +``` +β”Œβ”€payment_type─┬──trips─┬─avg_total─┐ +β”‚ CSH β”‚ 617237 β”‚ 17.82 β”‚ +β”‚ CRE β”‚ 377897 β”‚ 13.91 β”‚ +β”‚ NOC β”‚ 3573 β”‚ 13.89 β”‚ +β”‚ DIS β”‚ 1293 β”‚ 12.09 β”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ +fffd: 0 +``` + +**The self-hosted drop, found.** `v2026.10.08` runs about 11 percent slower +than `v2026.09.17.1`: 58,457 against 65,928 rows a second, three runs each, +alternating, no overlap. Two commits account for it, about 6 percent each. +[throughput-bisect.out](throughput-bisect.out) holds every run. + +- **#345 lowered the Kafka fetch defaults** from 100 MiB and 10 MiB to 4 MiB + and 1 MiB. That is a deliberate memory bound: the commit measured a log + pipeline's peak fall from 985 MiB to 193 MiB. The taxi topic has one + partition, so the 1 MiB partition bound caps each fetch at about 3,000 + rows. Setting the old bounds in the `fetch` block returns `v2026.10.08` to + 62,873 and 61,639. Accepted as the trade it was meant to be. +- **#338 bumped jsonparser from 1.1.2 to 1.6.1.** Applied alone to the last + fast commit, it costs as much as the whole bump. The other bumps, franz-go, + Arrow, ADBC and compress, measure flat. The cost arrived in jsonparser + 1.3.0: 1.2.0 ran 67,433 and 67,500, and 1.3.0 ran 62,989 and 63,015. Not + investigated further. + +The user ruled on 2026-10-10 that the drop is acceptable while memory stays +flat. A production instance's RSS held at about 92 MB from 2026-10-03 to +2026-10-10, with the Go heap flat. + +## Pending + +1. **Cloud on `v2026.10.08`.** Done on 2026-10-10. See Throughput. +2. **The throughput drop.** Found and accepted. See Throughput. +3. **Design the evidence layer the user asked for.** Every claim needs a + config and a test behind it. Two existing patterns to extend: + - `dev/config/examples/published/` holds the source-of-truth config for + every config in third-party docs. + - `docs/coverage/integrations/sink.clickhouse.yml` declares each type cell + and an integration test proves it. + + The page's behavioral claims, NULL, Enum, retry, window, flush interval + and deduplication, have neither. `probes.sh`, `win.sh` and `dedup.sh` are + the configurations. They are not yet tests. +4. **Push the draft.** Done on 2026-10-10 with the user's go: fork commit + `d4e94780`, a reply on each of the three open threads, and one summary + comment. The threads are left for the reviewer to resolve. +5. **The user decides** whether to mark it ready for review. + +## Notes for whoever resumes + +**Benchmark with a fresh consumer group per run.** Not an offset reset. A +stuck container holds the group, and every later run consumes nothing and +reports zero. That poisoned two rounds of numbers on 2026-09-17. `perf.sh` +does it the right way. + +**Check the Docker VM's free disk first.** A full VM killed Kafka mid-run on +2026-09-17. On 2026-10-09 it was at 94 percent before the run. +`docker builder prune -af` freed 22 GB. + +```bash +docker run --rm alpine df -h / +``` + +**Kafka can fail to start after a VM restart** with +`KeeperErrorCode = NodeExists` on broker registration: the old session still +holds the broker's ZooKeeper node. Start `kafka1` again once ZooKeeper has +expired it. + +**Cloud credentials** live in `~/.sqlflow-clickhouse`, one line, +`clickhouse://user:password@host`. That scheme is plain HTTP, which Cloud +refuses. `perfcloud.sh` derives the `clickhouses://...:8443` DSN from it. + +**Environment as left.** The `nyc-taxi-trips` topic holds its 1,000,660 +messages. `nyc_taxi.trips_small` holds the last run's million rows on +self-hosted. The `probe` database holds every probe table. The dev stack is up. diff --git a/docs/superpowers/status/clickhouse-117740/bluesky.sql b/docs/superpowers/status/clickhouse-117740/bluesky.sql new file mode 100644 index 00000000..190ab96a --- /dev/null +++ b/docs/superpowers/status/clickhouse-117740/bluesky.sql @@ -0,0 +1,6 @@ +CREATE TABLE bluesky_posts ( + ts DateTime, + did String, + langs Array(String), + text String +) ENGINE = MergeTree ORDER BY ts; diff --git a/docs/superpowers/status/clickhouse-117740/bluesky.yml b/docs/superpowers/status/clickhouse-117740/bluesky.yml new file mode 100644 index 00000000..7f312f16 --- /dev/null +++ b/docs/superpowers/status/clickhouse-117740/bluesky.yml @@ -0,0 +1,20 @@ +pipeline: + batch_size: 200 + source: + type: websocket + websocket: + uri: 'wss://jetstream2.us-east.bsky.network/subscribe?wantedCollections=app.bsky.feed.post' + handler: + type: 'handlers.InferredMemBatch' + sql: | + SELECT + to_timestamp(time_us / 1000000) AS ts, + did, + COALESCE(commit.record.langs, []) AS langs, + COALESCE(commit.record.text, '') AS text + FROM batch + sink: + type: clickhouse + clickhouse: + dsn: {{ SQLFLOW_CLICKHOUSE_DSN }} + table: bluesky_posts diff --git a/docs/superpowers/status/clickhouse-117740/cloudverify.out b/docs/superpowers/status/clickhouse-117740/cloudverify.out new file mode 100644 index 00000000..95e07359 --- /dev/null +++ b/docs/superpowers/status/clickhouse-117740/cloudverify.out @@ -0,0 +1,8 @@ +# Verify query on Cloud 26.6.1.2326 + β”Œβ”€payment_type─┬──trips─┬─avg_total─┐ +1. β”‚ CSH β”‚ 617237 β”‚ 17.82 β”‚ +2. β”‚ CRE β”‚ 377897 β”‚ 13.91 β”‚ +3. β”‚ NOC β”‚ 3573 β”‚ 13.89 β”‚ +4. β”‚ DIS β”‚ 1293 β”‚ 12.09 β”‚ + β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ +fffd: 0 diff --git a/docs/superpowers/status/clickhouse-117740/dedup.out b/docs/superpowers/status/clickhouse-117740/dedup.out new file mode 100644 index 00000000..5d5e8a22 --- /dev/null +++ b/docs/superpowers/status/clickhouse-117740/dedup.out @@ -0,0 +1,35 @@ +image: turbolytics/sql-flow:v2026.10.08 +26.8.1.2041 +Created topic ch-dedup. + +=== ReplacingMergeTree, two distinct blocks with the same keys (a newer version of each row) + run into rmt_distinct (v = v): exit=0 + run into rmt_distinct (v = 'second'): exit=0 + count() 4 + count() FINAL 2 + active parts 2 + rows [(1,'first'),(1,'second'),(2,'first'),(2,'second')] + after OPTIMIZE ... FINAL: count() 2, rows [(1,'second'),(2,'second')] + +=== ReplacingMergeTree, the identical block twice (a replay) + run into rmt_replay (v = v): exit=0 + run into rmt_replay (v = v): exit=0 + count() 4 + count() FINAL 2 + active parts 2 + rows [(1,'first'),(1,'first'),(2,'first'),(2,'first')] + +=== MergeTree, the identical block twice (a replay) + run into mt_replay (v = v): exit=0 + run into mt_replay (v = v): exit=0 + count() 4 + active parts 2 + rows [(1,'first'),(1,'first'),(2,'first'),(2,'first')] + +=== MergeTree SETTINGS non_replicated_deduplication_window = 100, the identical block twice + run into mt_dedup_replay (v = v): exit=0 + run into mt_dedup_replay (v = v): exit=0 + count() 2 + active parts 1 + rows [(1,'first'),(2,'first')] + insert_deduplicate (session default): 1 diff --git a/docs/superpowers/status/clickhouse-117740/dedup.sh b/docs/superpowers/status/clickhouse-117740/dedup.sh new file mode 100755 index 00000000..15f270fb --- /dev/null +++ b/docs/superpowers/status/clickhouse-117740/dedup.sh @@ -0,0 +1,84 @@ +#!/bin/bash +# dedup.sh: what a replayed batch does to each table engine, on self-hosted, +# non-replicated ClickHouse. A replay is the same two messages consumed twice +# by two consumer groups, so the sink sends the same block twice. IMG selects +# the image; DSN the server (its database must hold the tables). Each run +# writes a config of its own: a file rewritten within a second was once read +# stale through the Docker Desktop bind mount, and named a truncated table. +W="$(cd "$(dirname "$0")" && pwd)" +IMG="${IMG:-turbolytics/sql-flow:v2026.10.08}" +DSN="${DSN:-clickhouse://default@clickhouse:8123/probe}" +ch() { docker exec clickhouse clickhouse-client -q "$1" 2>&1; } +kt() { docker exec kafka1 kafka-topics --bootstrap-server localhost:9092 "$@" 2>&1 | tail -1; } + +# run
: consume the two messages once, into
. +run() { + local table="$1" vexpr="$2" g="dedup-$1-$RANDOM" + n=$((n + 1)); f="d_${table}_$n" + cat > "$W/$f.yml" < "$W/$f.log" 2>&1 + echo " run into $table (v = $vexpr): exit=$?" +} +report() { + local t="$1" + echo " count() $(ch "SELECT count() FROM probe.$t")" + [ "$2" = final ] && echo " count() FINAL $(ch "SELECT count() FROM probe.$t FINAL")" + echo " active parts $(ch "SELECT count() FROM system.parts WHERE database = 'probe' AND table = '$t' AND active")" + echo " rows $(ch "SELECT groupArray((id, v)) FROM (SELECT id, v FROM probe.$t ORDER BY id, v)")" +} + +echo "image: $IMG" +ch "SELECT version()" +kt --delete --if-exists --topic ch-dedup; sleep 2 +kt --create --topic ch-dedup --partitions 1 --replication-factor 1 +printf '{"id":1,"v":"first"}\n{"id":2,"v":"first"}\n' | docker exec -i kafka1 kafka-console-producer --bootstrap-server localhost:9092 --topic ch-dedup + +for t in rmt_distinct rmt_replay mt_replay mt_dedup_replay; do ch "DROP TABLE IF EXISTS probe.$t"; done +ch "CREATE TABLE probe.rmt_distinct (id Int32, v String) ENGINE = ReplacingMergeTree ORDER BY id" +ch "CREATE TABLE probe.rmt_replay (id Int32, v String) ENGINE = ReplacingMergeTree ORDER BY id" +ch "CREATE TABLE probe.mt_replay (id Int32, v String) ENGINE = MergeTree ORDER BY id" +ch "CREATE TABLE probe.mt_dedup_replay (id Int32, v String) ENGINE = MergeTree ORDER BY id SETTINGS non_replicated_deduplication_window = 100" + +echo +echo "=== ReplacingMergeTree, two distinct blocks with the same keys (a newer version of each row)" +run rmt_distinct "v"; run rmt_distinct "'second'" +report rmt_distinct final +ch "OPTIMIZE TABLE probe.rmt_distinct FINAL" +echo " after OPTIMIZE ... FINAL: count() $(ch 'SELECT count() FROM probe.rmt_distinct'), rows $(ch "SELECT groupArray((id, v)) FROM (SELECT id, v FROM probe.rmt_distinct ORDER BY id)")" + +echo +echo "=== ReplacingMergeTree, the identical block twice (a replay)" +run rmt_replay "v"; run rmt_replay "v" +report rmt_replay final + +echo +echo "=== MergeTree, the identical block twice (a replay)" +run mt_replay "v"; run mt_replay "v" +report mt_replay + +echo +echo "=== MergeTree SETTINGS non_replicated_deduplication_window = 100, the identical block twice" +run mt_dedup_replay "v"; run mt_dedup_replay "v" +report mt_dedup_replay +echo " insert_deduplicate (session default): $(ch "SELECT value FROM system.settings WHERE name = 'insert_deduplicate'")" diff --git a/docs/superpowers/status/clickhouse-117740/perf.out b/docs/superpowers/status/clickhouse-117740/perf.out new file mode 100644 index 00000000..fd03ee8f --- /dev/null +++ b/docs/superpowers/status/clickhouse-117740/perf.out @@ -0,0 +1,17 @@ +# v2026.10.08 unless a comment names another image; perf.sh
` {#trouble-column} + +The full error starts with `[system.sink.write_failed] clickhouse sink: prepare batch:`. The handler SQL produced a column the target table does not have. SQLFlow inserts by column name, so either alias the column in the SQL to match the table, drop it from the `SELECT`, or add it to the table. + +### `clickhouse sink: column "": unsupported arrow type ` {#trouble-type} + +The column's Arrow type has no ClickHouse mapping, such as a struct or a map value. Cast or flatten in the handler SQL; see [Type mapping](#type-mapping). + +### `converting float64 to Decimal(10, 2) is unsupported` {#trouble-decimal} + +A `DOUBLE` was sent to a `Decimal` column. Cast to a string instead: `CAST(amount AS VARCHAR) AS amount`. The driver parses the decimal text exactly, with no floating-point rounding. + +### `parsing time "…" as "2006-01-02 15:04:05"` {#trouble-timestamp} + +A `VARCHAR` timestamp reached a `DateTime` column in a shape the driver does not accept. It takes `'2006-01-02 15:04:05'` and the same with a trailing offset, and nothing else. ISO 8601 with a `T` separator or a `Z` suffix fails. Cast in the handler SQL with `CAST(ts AS TIMESTAMP)`, or reformat the string. See [Type mapping](#type-mapping). + +The error carries the code `user.sink.encode_failed`, and the process exits with code 10. It is not retried, because the value would fail every attempt. + +### The pipeline runs but rows never appear {#trouble-no-rows} + +Check the batching. A batch is executed and inserted when `batch_size` messages have arrived, or when `flush_interval_seconds` elapses with a partial batch waiting. Whichever comes first wins. `flush_interval_seconds` defaults to `30`, so a partial batch on a low-volume topic waits up to 30 seconds. For faster inserts on a quiet topic, lower `flush_interval_seconds`, which takes any whole number of seconds, or lower `batch_size`. With `--max-msgs`, the final partial batch is inserted on exit. + +## Further reading {#further-reading} + +- SQLFlow on GitHub: configuration reference, all sources and sinks, error policies, windowing. +- SQLFlow documentation: tutorials, including a longer ClickHouse walkthrough. diff --git a/docs/superpowers/status/clickhouse-117740/sqlflow.orig.mdx b/docs/superpowers/status/clickhouse-117740/sqlflow.orig.mdx new file mode 100644 index 00000000..aacf3716 --- /dev/null +++ b/docs/superpowers/status/clickhouse-117740/sqlflow.orig.mdx @@ -0,0 +1,511 @@ +--- +sidebarTitle: 'sqlflow' +keywords: ['clickhouse', 'sqlflow', 'duckdb', 'kafka', 'streaming', 'stream processing', 'sink', 'data ingestion', 'integrate', 'websocket'] +slug: /integrations/sqlflow +description: 'Stream Kafka, WebSocket and webhook data into ClickHouse, transformed with DuckDB SQL, using sqlflow' +title: 'sqlflow and ClickHouse' +doc_type: 'guide' +integration: + - support_level: 'community' + - category: 'data_ingestion' + - website: 'https://turbolytics.io' +--- + +import ConnectionDetails from '/snippets/_gather_your_details_http.mdx'; +import { CommunityMaintainedBadge } from "/snippets/components/CommunityMaintainedBadge/CommunityMaintainedBadge.jsx"; + + + +sqlflow is an open-source (MIT) stream processing engine. It consumes a stream (Kafka, a WebSocket, or webhooks), runs DuckDB SQL over each micro-batch, and inserts the result into a ClickHouse table. A pipeline is a single YAML file, and sqlflow ships as one binary and one Docker image, so there is no cluster to run. + +This page covers the ClickHouse sink: connecting to ClickHouse Cloud and self-hosted ClickHouse, which types map, an end-to-end example on the NYC taxi sample dataset, and the limits that were measured rather than assumed. + +## Prerequisites {#prerequisites} + +- sqlflow **v1.1.0** or later. +- ClickHouse Cloud, or self-hosted ClickHouse. Everything on this page was run against Cloud 26.2 and self-hosted 26.8; no minimum below those has been established. +- A Kafka broker the pipeline can reach, for the Kafka source used in the example. The WebSocket and webhook sources need no broker. +- Python 3 with the `confluent-kafka` package (`pip install confluent-kafka`), only for the script in the example that publishes the sample data to Kafka. sqlflow itself needs no Python. +- Docker is the simplest install. `turbolytics/sql-flow` is published multi-arch for `linux/amd64` and `linux/arm64`. sqlflow can also be built from source; it needs Go, a C toolchain, and `libduckdb` present at runtime. + +## Setup {#setup} + +sqlflow connects with a single DSN. The scheme selects the protocol and whether TLS is used; the port defaults from the scheme if omitted. + +| Scheme | Protocol | TLS | Default port | Use for | +| ------------------------------- | -------- | --- | ------------ | ------- | +| `clickhouses://`, `https://` | HTTP | yes | 8443 | ClickHouse Cloud, or any TLS-terminated HTTP endpoint | +| `clickhouse://`, `http://` | HTTP | no | 8123 | Self-hosted ClickHouse | +| `natives://` | native | yes | 9440 | ClickHouse Cloud over the native protocol | +| `tcp://`, `native://` | native | no | 9000 | Self-hosted, native protocol | + +The target table must already exist. sqlflow does not create tables and does not evolve schemas: it names the columns in its `INSERT` from the column names its SQL produces, so **the columns your handler SQL emits must match the table's column names**. Use aliases in the SQL to rename. + +### Connect to ClickHouse Cloud {#connect-to-clickhouse-cloud} + + + +With those details, the DSN is: + +``` +clickhouses://default:@.clickhouse.cloud:8443/ +``` + +Keep it out of the YAML by passing it as an environment variable; the config's `dsn` field is a template: + +```yaml +sink: + type: clickhouse + clickhouse: + dsn: {{ SQLFLOW_CLICKHOUSE_DSN }} + table: trips_small +``` + +```bash +export SQLFLOW_CLICKHOUSE_DSN='clickhouses://default:@.clickhouse.cloud:8443/nyc_taxi' +``` + + +Use `clickhouses://` (with the `s`) for Cloud. `clickhouse://` sends plain HTTP to port 8443, and the connection is reset before any request is made. See [Troubleshooting](#troubleshooting). + + +### Connect to self-hosted ClickHouse {#connect-to-self-hosted-clickhouse} + +Point the DSN at the HTTP interface, port 8123 by default: + +```bash +export SQLFLOW_CLICKHOUSE_DSN='clickhouse://default:@clickhouse.internal:8123/nyc_taxi' +``` + +If you have put TLS in front of the HTTP interface, use `clickhouses://` exactly as for Cloud. For the native protocol use `tcp://` (9000) or `natives://` (9440). + +When sqlflow runs in Docker, `localhost` inside the container is the container itself, not your machine. Put sqlflow and ClickHouse on the same Docker network and use the ClickHouse container's name as the host. + +## Authentication {#authentication} + +Username and password are carried in the DSN and, with `clickhouses://` or `natives://`, are sent over TLS. The server certificate is verified against the system trust store; there is no option to skip verification. + +- **Special characters in the password.** The DSN is parsed as a URL, so `/`, `#` and `?` must be percent-encoded (`%2F`, `%23`, `%3F`); left raw they produce a confusing `invalid port ":p" after host`. `@` and `:` are handled correctly either way. +- **Cloud IP access lists.** If your service restricts source IPs, the address sqlflow connects from has to be on the list. +- **Not supported:** mutual TLS / client certificates, and custom CA bundles. Use username and password. + +## Example: streaming the NYC taxi dataset from Kafka {#example-nyc-taxi} + +This walks the [NYC taxi sample dataset](/get-started/sample-datasets/nyc-taxi) through Kafka into ClickHouse, transformed by sqlflow on the way. It reproduces on Cloud or self-hosted; only the DSN changes. + +### 1. Create the target table {#example-create-table} + +The same table the sample-dataset page defines: + +```sql +CREATE DATABASE IF NOT EXISTS nyc_taxi; + +CREATE TABLE IF NOT EXISTS nyc_taxi.trips_small ( + trip_id UInt32, + pickup_datetime DateTime, + dropoff_datetime DateTime, + pickup_longitude Nullable(Float64), + pickup_latitude Nullable(Float64), + dropoff_longitude Nullable(Float64), + dropoff_latitude Nullable(Float64), + passenger_count UInt8, + trip_distance Float32, + fare_amount Float32, + extra Float32, + tip_amount Float32, + tolls_amount Float32, + total_amount Float32, + payment_type Enum('CSH' = 1, 'CRE' = 2, 'NOC' = 3, 'DIS' = 4, 'UNK' = 5), + pickup_ntaname LowCardinality(String), + dropoff_ntaname LowCardinality(String) +) +ENGINE = MergeTree +PRIMARY KEY (pickup_datetime, dropoff_datetime); +``` + +### 2. Publish the data to Kafka {#example-publish} + +The dataset is published as gzipped TSV. This step converts one file (about a million rows) to newline-delimited JSON and publishes it to a topic named `nyc-taxi-trips`. + +Install the publisher's dependency and download the file: + +```bash +pip install confluent-kafka +curl -O https://datasets-documentation.s3.eu-west-3.amazonaws.com/nyc-taxi/trips_0.gz +``` + +Create the topic. A stock Apache Kafka broker creates it on first publish, but managed and production clusters commonly set `auto.create.topics.enable=false`, where an uncreated topic fails the example before any row reaches ClickHouse: + +```bash +kafka-topics.sh --bootstrap-server localhost:9092 \ + --create --topic nyc-taxi-trips --partitions 1 --replication-factor 1 +``` + +Confluent's images name that command `kafka-topics`, without the `.sh`. + +Save the publisher as `publish.py`. Change `bootstrap.servers` if the broker is not on `localhost:9092`: + +```python +import gzip, json +from confluent_kafka import Producer + +COLUMNS = ["trip_id", "pickup_datetime", "dropoff_datetime", "pickup_longitude", + "pickup_latitude", "dropoff_longitude", "dropoff_latitude", "passenger_count", + "trip_distance", "fare_amount", "extra", "tip_amount", "tolls_amount", + "total_amount", "payment_type", "pickup_ntaname", "dropoff_ntaname"] +INTS = {"trip_id", "passenger_count"} +FLOATS = {"pickup_longitude", "pickup_latitude", "dropoff_longitude", "dropoff_latitude", + "trip_distance", "fare_amount", "extra", "tip_amount", "tolls_amount", "total_amount"} + +producer = Producer({"bootstrap.servers": "localhost:9092"}) + +def send(topic, value): + # The producer's queue holds 100,000 messages by default, and this file + # has a million rows; poll() drains it as it fills. + while True: + try: + producer.produce(topic, value) + return + except BufferError: + producer.poll(0.5) + +with gzip.open("trips_0.gz", "rb") as f: + header = f.readline().decode().rstrip("\n").split("\t") + index = {name: i for i, name in enumerate(header)} + for raw in f: + # Split on the raw bytes and decode only the columns being published. + # The file carries columns this example does not use that are not + # valid UTF-8, and decoding the whole line would have to corrupt them. + parts = raw.rstrip(b"\n").split(b"\t") + row = {} + for col in COLUMNS: + value = parts[index[col]].decode() + if value == "": + row[col] = None + elif col in INTS: + row[col] = int(float(value)) + elif col in FLOATS: + row[col] = float(value) + else: + row[col] = value + send("nyc-taxi-trips", json.dumps(row)) +producer.flush() +``` + +Run it. Publishing the million rows takes a couple of minutes: + +```bash +python publish.py +``` + +### 3. Write the pipeline {#example-pipeline} + +Save this as `taxi.yml`. The handler SQL runs in DuckDB over each batch of JSON messages; the casts produce Arrow types that map onto the table's ClickHouse types (see [Type mapping](#type-mapping)). + +```yaml +pipeline: + name: nyc-taxi-clickhouse + batch_size: 20000 + + source: + type: kafka + kafka: + brokers: [{{ SQLFLOW_KAFKA_BROKERS|default('localhost:9092') }}] + group_id: nyc-taxi + auto_offset_reset: earliest + topics: + - nyc-taxi-trips + + handler: + type: 'handlers.InferredMemBatch' + sql: | + SELECT + CAST(trip_id AS UINTEGER) AS trip_id, + CAST(pickup_datetime AS TIMESTAMP) AS pickup_datetime, + CAST(dropoff_datetime AS TIMESTAMP) AS dropoff_datetime, + pickup_longitude, + pickup_latitude, + dropoff_longitude, + dropoff_latitude, + CAST(passenger_count AS UTINYINT) AS passenger_count, + CAST(trip_distance AS FLOAT) AS trip_distance, + CAST(fare_amount AS FLOAT) AS fare_amount, + CAST(extra AS FLOAT) AS extra, + CAST(tip_amount AS FLOAT) AS tip_amount, + CAST(tolls_amount AS FLOAT) AS tolls_amount, + CAST(total_amount AS FLOAT) AS total_amount, + payment_type, + pickup_ntaname, + dropoff_ntaname + FROM batch + + sink: + type: clickhouse + clickhouse: + dsn: {{ SQLFLOW_CLICKHOUSE_DSN }} + table: trips_small +``` + +`batch_size` is the number of messages per DuckDB batch and per ClickHouse `INSERT`. See [Performance](#performance) for why 20,000. + +### 4. Run it {#example-run} + +```bash +docker run --rm \ + --network \ + -v "$(pwd)":/conf \ + -e SQLFLOW_CLICKHOUSE_DSN \ + -e SQLFLOW_KAFKA_BROKERS= \ + turbolytics/sql-flow:v1.1.0 \ + run /conf/taxi.yml --max-msgs=1000000 +``` + + +**The broker address must be one the container can reach.** A Kafka client is redirected to whatever the broker *advertises*, not the address you dial. A broker advertising `127.0.0.1:9092` for external clients sends the container to itself, and every fetch fails with `dial tcp 127.0.0.1:9092: connect: connection refused`, even though the same address works from your shell. Put the container on the broker's own network and use the address advertised there, or run the binary on the host instead: + +```bash +sqlflow run taxi.yml --max-msgs=1000000 +``` + + +`--max-msgs` stops after that many messages; leave it off to run continuously. Kafka offsets are committed only after each `INSERT` has been acknowledged by ClickHouse, and only up to the last message actually processed. + +### 5. Verify {#example-verify} + +```sql +SELECT payment_type, count() AS trips, round(avg(total_amount), 2) AS avg_total +FROM nyc_taxi.trips_small +GROUP BY payment_type +ORDER BY trips DESC; +``` + +``` +β”Œβ”€payment_type─┬──trips─┬─avg_total─┐ +β”‚ CSH β”‚ 617237 β”‚ 17.82 β”‚ +β”‚ CRE β”‚ 377897 β”‚ 13.91 β”‚ +β”‚ NOC β”‚ 3573 β”‚ 13.89 β”‚ +β”‚ DIS β”‚ 1293 β”‚ 12.09 β”‚ +β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ +``` + +The same query gives the same counts against Cloud and self-hosted, which is a reasonable check that nothing was dropped or coerced on either path. + +### A second example: WebSocket source, no Kafka {#example-websocket} + +The WebSocket source needs no broker, and this shows an `Array(String)` column being written and queried natively. It consumes the public Bluesky firehose. + +```sql +CREATE TABLE bluesky_posts ( + ts DateTime, + did String, + langs Array(String), + text String +) ENGINE = MergeTree ORDER BY ts; +``` + +```yaml +pipeline: + batch_size: 200 + source: + type: websocket + websocket: + uri: 'wss://jetstream2.us-east.bsky.network/subscribe?wantedCollections=app.bsky.feed.post' + handler: + type: 'handlers.InferredMemBatch' + sql: | + SELECT + to_timestamp(time_us / 1000000) AS ts, + did, + COALESCE(commit.record.langs, []) AS langs, + COALESCE(commit.record.text, '') AS text + FROM batch + sink: + type: clickhouse + clickhouse: + dsn: {{ SQLFLOW_CLICKHOUSE_DSN }} + table: bluesky_posts +``` + +Save it as `bluesky.yml` and run it. No `SQLFLOW_KAFKA_BROKERS` here: the source is the WebSocket, and the only variable the pipeline reads is the DSN. + +```bash +docker run --rm \ + -v "$(pwd)":/conf \ + -e SQLFLOW_CLICKHOUSE_DSN \ + turbolytics/sql-flow:v1.1.0 \ + run /conf/bluesky.yml --max-msgs=2000 +``` + +Add `--network` as in the Kafka example if ClickHouse runs in Docker on the same machine. With the binary instead of the image: + +```bash +sqlflow run bluesky.yml --max-msgs=2000 +``` + +The firehose delivers about 50 posts a second, so 2,000 messages take under a minute. Then: + +```sql +SELECT lang, count() AS posts +FROM (SELECT arrayJoin(langs) AS lang FROM bluesky_posts) +GROUP BY lang ORDER BY posts DESC LIMIT 5; +``` + +## Type mapping {#type-mapping} + +sqlflow hands each batch to ClickHouse as rows built from the Arrow table DuckDB produces, so what matters is the Arrow type each SQL column ends up with. Cast in the handler SQL to control it. + +Every row below is generated from the engine's type matrix and proven by an integration test that writes the value, reads it back through the server, and compares. The matrix and the test live in the [sqlflow repository](https://github.com/turbolytics/sql-flow/blob/main/docs/coverage/clickhouse-types.mdx). A pairing in the table is proven. A pairing missing from it is untested, not refused: the refused ones are listed under [Unsupported](#unsupported) and fail the batch with a code. Parameterised column types are listed as the instance the test wrote. + +| DuckDB SQL type (cast to) | ClickHouse column types that accept it | Notes | +| --- | --- | --- | +| `BOOLEAN` | `Bool`, `UInt8` | | +| `TINYINT` | `Int8` | | +| `SMALLINT` | `Int16` | | +| `INTEGER` | `Int32` | | +| `BIGINT` | `Int64` | | +| `UTINYINT` | `UInt8` | | +| `USMALLINT` | `UInt16` | | +| `UINTEGER` | `UInt32` | | +| `UBIGINT` | `UInt64` | | +| `REAL` | `Float32` | | +| `DOUBLE` | `Float64` | | +| `VARCHAR`, `UUID`, `JSON` | `String`, `LowCardinality(String)`, `DateTime64(3)`, `DateTime`, `Enum8('' = 0, 'a' = 1, 'b' = 2)`, `FixedString(36)`, `UUID`, `IPv4`, `Decimal(10, 2)` | | +| `BLOB`, `BIT`, `VARINT` | `String` | | +| `DATE` | `Date`, `Date32` | | +| `TIMESTAMP_S` | `DateTime` | | +| `TIMESTAMP_MS` | `DateTime64(3)` | | +| `TIMESTAMP` | `DateTime64(6)`, `DateTime` | DateTime keeps whole seconds; DateTime64(6) keeps all six digits | +| `TIMESTAMP_NS` | `DateTime64(9)` | | +| `TIMESTAMPTZ` | `DateTime64(6)`, `DateTime` | stored as the same UTC instant; the zone is dropped and the column's own zone governs rendering; DateTime keeps whole seconds | +| `BOOLEAN[]` | `Array(Bool)` | | +| `TINYINT[]` | `Array(Int8)` | | +| `SMALLINT[]` | `Array(Int16)` | | +| `INTEGER[]` | `Array(Int32)` | | +| `BIGINT[]` | `Array(Int64)` | | +| `UTINYINT[]` | `Array(UInt8)` | | +| `USMALLINT[]` | `Array(UInt16)` | | +| `UINTEGER[]` | `Array(UInt32)` | | +| `UBIGINT[]` | `Array(UInt64)` | | +| `REAL[]` | `Array(Float32)` | | +| `DOUBLE[]` | `Array(Float64)` | | +| `VARCHAR[]` | `Array(String)` | | +| `BLOB[]` | `Array(String)` | | +| `DATE[]` | `Array(Date32)` | | +| `TIMESTAMP_S[]` | `Array(DateTime)` | | +| `TIMESTAMP_MS[]` | `Array(DateTime64(3))` | | +| `TIMESTAMP[]` | `Array(DateTime64(6))` | | +| `TIMESTAMP_NS[]` | `Array(DateTime64(9))` | | +| `TIMESTAMPTZ[]` | `Array(DateTime64(6))` | stored as the same UTC instant; the zone is dropped and the column's own zone governs rendering | +| `BIGINT[][]` | `Array(Array(Int64))` | | +| `NULL` in any column | the column's own type | the column type's zero value; the DEFAULT expression is not evaluated because every produced column is named in the INSERT | +| `NULL` inside a `LIST` | the element's own type | the element type's zero value, because ClickHouse's Array(T) holds no nulls unless T is Nullable | + +Values are matched to columns **by name**, not by position. + + +**Prefer `CAST(ts AS TIMESTAMP)` over passing timestamps as strings.** When sqlflow sends a `VARCHAR` to a `DateTime` column, the Go driver parses the string, not the server. It accepts exactly two shapes: + +- `'2026-09-01 12:00:00'`, which it reads as UTC. The server's timezone and the column's timezone do not change the instant stored. Measured against a server running `Asia/Tokyo`: the row landed on `2026-09-01 12:00:00 UTC`, the same instant a `DateTime('UTC')` column received. +- `'2026-09-01 12:00:00 +09:00'`, whose offset is applied. That row stores `2026-09-01 03:00:00 UTC`. + +Anything else fails the batch, including ISO 8601 with a `T` separator or a `Z` suffix. `'2026-09-01T12:00:00Z'` is rejected with `parsing time "2026-09-01T12:00:00Z" as "2006-01-02 15:04:05"`. + +This differs from writing the same literal in an `INSERT` statement yourself, where ClickHouse parses the string in the target column's timezone, and a plain `DateTime` column uses the server timezone. Casting in the handler SQL sidesteps both sets of rules. + +If the field can be `NULL`, a `DateTime` column stores it as `1970-01-01 00:00:00`. Use `Nullable(DateTime)`, or `coalesce(CAST(ts AS TIMESTAMP), now()::TIMESTAMP)` in the handler SQL. + + + +`Decimal` columns accept the value as a **string**, not as a `DOUBLE`. `CAST(amount AS VARCHAR)` works; `CAST(amount AS DOUBLE)` fails the batch with `converting float64 to Decimal(10, 2) is unsupported`. + + +### Unsupported {#unsupported} + +These fail the batch with `user.sink.type_unsupported` rather than reaching the table. Cast or flatten them in the handler SQL. + +- `DECIMAL(p, s)`, `HUGEINT`, `UHUGEINT` +- `TIME`, `TIMETZ` +- `TIME_NS` +- `INTERVAL` +- `DECIMAL(p, s)[]`, `HUGEINT[]` +- `TIME[]` +- `STRUCT(...)[]` +- `INTEGER[3]` +- `STRUCT(...)` +- `MAP(K, V)` +- `UNION(...)` +- `ENUM` + +## Known limits {#known-limits} + +- **Unsupported types:** listed under [Unsupported](#unsupported). `DECIMAL`, `STRUCT`, `MAP`, `TIME` and `INTERVAL` are the ones a handler reaches for. Each fails the batch with `user.sink.type_unsupported` before any row is written. Flatten a `STRUCT` or `MAP` into scalar columns in the handler SQL, or serialise it with `to_json()` into a `String` column. Cast a `DECIMAL` to `VARCHAR`. `Nested` columns were not tested. +- **No table creation or schema evolution.** The table must exist, and a column the handler emits that the table lacks fails the batch (see [Troubleshooting](#troubleshooting)). Columns the handler omits take the table's `DEFAULT`. +- **A `NULL` written to a non-`Nullable` column is stored as the type's zero value**, not as the column's `DEFAULT` expression, and no error is raised. See [Type mapping](#type-mapping) for the workaround. +- **At-least-once delivery.** Kafka offsets are committed after the `INSERT` is acknowledged. A crash between the two replays the batch, so duplicates are possible. For a pipeline that writes one row per message, `ReplacingMergeTree` keyed on something that identifies the row removes them, as does an idempotent insert pattern. Do not apply that to an aggregate table fed by a tumbling window. See the next point. +- **Transient failures are retried.** A refused, reset, or timed-out connection to ClickHouse is retried with exponential backoff under a deadline, configured by the sink's `retry` block (`max_attempts`, `initial_backoff_ms`, `max_backoff_ms`, `deadline_seconds`). A rejected `INSERT` is not retried once the server has seen it. A value the driver itself cannot encode, such as the ISO 8601 timestamp above, is retried the full ladder before failing, and reports as `system.sink.unreachable`. The sink also dials ClickHouse once at startup, so an unreachable host fails before any message is consumed. +- **One connection, one `INSERT` per batch.** There is no parallel writer. Throughput is governed by `batch_size`; see below. +- **Streaming aggregation** (tumbling windows) is supported: a managed table is polled on an interval and its closed windows are published to their own sink. What counts as "closed" is the SQL you write, and it is evaluated against wall clock rather than an event-time watermark. + + That matters whenever the pipeline is running behind real time by more than the grace period, which a backfill, a replay, or plain consumer lag will all do. A window then closes while rows still belong to it, and it is published in parts, once per poll. Measured on v1.0.6: 500 events in a single one-minute bucket, consumed over ten seconds with a one-second poll and a 60-second grace, published as five rows of 100. + + The parts carry the same key and can be byte-identical to each other, so nothing downstream can tell them apart from a delivery duplicate. Deduplicating the aggregate table therefore drops real counts with no error: those five rows collapse to one row of 100, against a true count of 500. Insert the window output into a plain `MergeTree` and do not deduplicate it. Keep the pipeline caught up if the counts must be exact, and prefer the row-per-message path for a stream you intend to replay. + +## Performance {#performance} + +Measured with the example above on sqlflow v1.1.0: one million 17-column rows, one sqlflow process, Kafka co-located with sqlflow, two runs per row. + +| Target | Rows/second | +| ------------------------------------------ | ----------- | +| Self-hosted ClickHouse, sqlflow and Kafka on the same Docker network | 67,000 and 70,000 | +| ClickHouse Cloud (us-east-2), sqlflow beside the broker, over the public internet | 29,800 and 29,200 | +| Self-hosted ClickHouse, sqlflow on the host reaching Kafka through Docker Desktop's port forwarding | 23,400 and 24,700 | + +Two things those rows say. The Cloud figure is the network between the laptop and the service, not sqlflow: the same pipeline against a self-hosted server on the same Docker network runs more than twice as fast. Run sqlflow near the service, same region and ideally same VPC, for numbers closer to the self-hosted figure. And Docker Desktop's host-to-container port forwarding is not a tax to shrug off: it caps the Kafka fetch at roughly 10 to 15 MB/s, which for rows this wide is about 24,000 a second whatever `batch_size` is set to. Benchmark from inside the network. + +`batch_size` is the lever on a nearby target, measured on the same Docker network: + +| `batch_size` | Rows/second | +| ------------ | ----------- | +| 5,000 | 36,900 | +| 20,000 | 67,000 | +| 50,000 | 67,300 | +| 100,000 | 71,200 | + +Below about 20,000 rows per batch, per-`INSERT` overhead dominates; above it there is little further to gain, and memory per batch grows. 20,000 is a good default for rows of this width; verify against your own. + +## Troubleshooting {#troubleshooting} + +### `connection reset by peer` immediately, with `...:8443?...` in the message {#trouble-reset} + +The DSN uses `clickhouse://` (plain HTTP) against a TLS port. The full error looks like `failed to query server hello: … Post "http://…:8443?…": … connection reset by peer`. Change the scheme to `clickhouses://`. + +### `[HTTP 401] code: 194 … Authentication failed: password is incorrect, or there is no user with such name` {#trouble-auth} + +The credentials in the DSN are wrong, or a special character in the password was not URL-encoded. On Cloud, the `default` user's password is shown once at service creation and can be reset from the console. If the password is right, check the service's IP access list. + +### `column "" is not present in the table
` {#trouble-column} + +The handler SQL produced a column the target table does not have. sqlflow inserts by column name, so either alias the column in the SQL to match the table, drop it from the `SELECT`, or add it to the table. + +### `clickhouse sink: column "": unsupported arrow type ` {#trouble-type} + +The column's Arrow type has no ClickHouse mapping, such as a struct or a map value. Cast or flatten in the handler SQL; see [Type mapping](#type-mapping). + +### `converting float64 to Decimal(10, 2) is unsupported` {#trouble-decimal} + +A `DOUBLE` was sent to a `Decimal` column. Cast to a string instead: `CAST(amount AS VARCHAR) AS amount`. The driver parses the decimal text exactly, with no floating-point rounding. + +### `parsing time "…" as "2006-01-02 15:04:05"` {#trouble-timestamp} + +A `VARCHAR` timestamp reached a `DateTime` column in a shape the driver does not accept. It takes `'2006-01-02 15:04:05'` and the same with a trailing offset, and nothing else. ISO 8601 with a `T` separator or a `Z` suffix fails. Cast in the handler SQL with `CAST(ts AS TIMESTAMP)`, or reformat the string. See [Type mapping](#type-mapping). + +This one is retried the full `retry` ladder before it fails, and the final error reads `system.sink.unreachable`, because the encode happens on the client before anything is sent. The message it wraps names the real cause. + +### The pipeline runs but rows never appear {#trouble-no-rows} + +Check the batching. A batch is executed and inserted when `batch_size` messages have arrived, or when `flush_interval_seconds` elapses with a partial batch waiting. Whichever comes first wins. `flush_interval_seconds` defaults to `30`, which is also the smallest value the config schema accepts, so a partial batch on a low-volume topic waits up to 30 seconds and a lower setting fails validation at startup. For faster inserts on a quiet topic, lower `batch_size` instead. With `--max-msgs`, the final partial batch is inserted on exit. + +## Further reading {#further-reading} + +- sqlflow on GitHub: configuration reference, all sources and sinks, error policies, windowing. +- sqlflow documentation: tutorials, including a longer ClickHouse walkthrough. diff --git a/docs/superpowers/status/clickhouse-117740/taxi.sql b/docs/superpowers/status/clickhouse-117740/taxi.sql new file mode 100644 index 00000000..6834a2a7 --- /dev/null +++ b/docs/superpowers/status/clickhouse-117740/taxi.sql @@ -0,0 +1,24 @@ +CREATE DATABASE IF NOT EXISTS nyc_taxi; + +CREATE TABLE IF NOT EXISTS nyc_taxi.trips_small ( + trip_id UInt32, + pickup_datetime DateTime, + dropoff_datetime DateTime, + pickup_longitude Nullable(Float64), + pickup_latitude Nullable(Float64), + dropoff_longitude Nullable(Float64), + dropoff_latitude Nullable(Float64), + passenger_count UInt8, + trip_distance Float32, + fare_amount Float32, + extra Float32, + tip_amount Float32, + tolls_amount Float32, + total_amount Float32, + payment_type Enum('CSH' = 1, 'CRE' = 2, 'NOC' = 3, 'DIS' = 4, 'UNK' = 5), + pickup_ntaname LowCardinality(String), + dropoff_ntaname LowCardinality(String) +) +ENGINE = MergeTree +PRIMARY KEY (pickup_datetime, dropoff_datetime); +``` diff --git a/docs/superpowers/status/clickhouse-117740/taxi.yml b/docs/superpowers/status/clickhouse-117740/taxi.yml new file mode 100644 index 00000000..fbeee6ac --- /dev/null +++ b/docs/superpowers/status/clickhouse-117740/taxi.yml @@ -0,0 +1,41 @@ +pipeline: + name: nyc-taxi-clickhouse + batch_size: 20000 + + source: + type: kafka + kafka: + brokers: [{{ SQLFLOW_KAFKA_BROKERS|default('localhost:9092') }}] + group_id: nyc-taxi + auto_offset_reset: earliest + topics: + - nyc-taxi-trips + + handler: + type: 'handlers.InferredMemBatch' + sql: | + SELECT + CAST(trip_id AS UINTEGER) AS trip_id, + CAST(pickup_datetime AS TIMESTAMP) AS pickup_datetime, + CAST(dropoff_datetime AS TIMESTAMP) AS dropoff_datetime, + pickup_longitude, + pickup_latitude, + dropoff_longitude, + dropoff_latitude, + CAST(passenger_count AS UTINYINT) AS passenger_count, + CAST(trip_distance AS FLOAT) AS trip_distance, + CAST(fare_amount AS FLOAT) AS fare_amount, + CAST(extra AS FLOAT) AS extra, + CAST(tip_amount AS FLOAT) AS tip_amount, + CAST(tolls_amount AS FLOAT) AS tolls_amount, + CAST(total_amount AS FLOAT) AS total_amount, + payment_type, + pickup_ntaname, + dropoff_ntaname + FROM batch + + sink: + type: clickhouse + clickhouse: + dsn: {{ SQLFLOW_CLICKHOUSE_DSN }} + table: trips_small diff --git a/docs/superpowers/status/clickhouse-117740/throughput-bisect.out b/docs/superpowers/status/clickhouse-117740/throughput-bisect.out new file mode 100644 index 00000000..7f70ed1b --- /dev/null +++ b/docs/superpowers/status/clickhouse-117740/throughput-bisect.out @@ -0,0 +1,37 @@ +# Throughput bisect, 2026-10-10. Host binaries, Go 1.26.0 for every build, taxi bs=20000, +# self-hosted ClickHouse 26.8.1 through Docker Desktop 29.8.0 port forwarding. Rows/second. + +## The two releases, alternating +v2026.09.17.1 run 1: 65782 +v2026.10.08 run 1: 58871 +v2026.09.17.1 run 2: 65762 +v2026.10.08 run 2: 57489 +v2026.09.17.1 run 3: 66238 +v2026.10.08 run 3: 59010 + +## Main-branch commits between them (87, first parent) +3011131 serve: ask twice against the image and prove the cache answered (#339): 67072 65554 +c24e0ed build(deps): bump the go-modules group with 9 updates (#338): 62873 62556 +466184d ci: say what the registry said, and stop deciding a release with a gre: 62511 60692 63043 +99180b6 dev: publish-logs, and kafka fetch defaults that were never chosen (#3: 59358 58767 +c5bb1a4 turbostats: drop allow, a scope ceiling that enforced nothing (#355): 59597 59474 +b4b4bfd core: a windowing pipeline refuses a record whose event time it cannot: 59036 58517 + +## v2026.10.08 with the fetch limits set in config (max_bytes / max_partition_bytes) +100 MiB / 10 MiB, the defaults before #345: 62873 61639 +16 MiB / 4 MiB: 61558 60900 +4 MiB / 1 MiB, the defaults since #345: 58751 58116 + +## 3011131 (the last fast commit) with one module bump from #338 applied +franz: 67209 66538 +jsonparser: 62142 61849 +arrow: 67072 66585 +adbc: 66281 66063 +compress: 66109 65294 + +## 3011131 with each jsonparser release +v1.1.2 (as shipped at 3011131): 67072 65554 +v1.2.0: 67433 67500 +v1.3.0: 62989 63015 +v1.4.0: 61979 61620 +v1.6.1: 62142 61849 diff --git a/docs/superpowers/status/clickhouse-117740/win.out b/docs/superpowers/status/clickhouse-117740/win.out new file mode 100644 index 00000000..975522ba --- /dev/null +++ b/docs/superpowers/status/clickhouse-117740/win.out @@ -0,0 +1,19 @@ +image: turbolytics/sql-flow:v2026.10.08 +Created topic ch-win. +events published: 1000 +--- validate +skipped source.webhook.ack: this pipeline has no webhook source +supplied but never read: [SQLFLOW_DUCKDB_LIB] +/conf/win.yml: valid +--- run, no --max-msgs: the bucket closes on idle_close_seconds +2026-10-09T13:18:13.489Z INFO sqlflow.run.manager.watermark managers/watermark.go:182 starting watermark manager {"table": "agg"} +--- what landed + β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€bucket─┬────n─┐ +1. β”‚ 2026-09-01 12:00:00 β”‚ 1000 β”‚ + β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”˜ +rows published: 1 +--- validate: allowed_lateness_seconds with a ClickHouse window sink +skipped source.mqtt: this pipeline has no mqtt source +skipped source.webhook.ack: this pipeline has no webhook source +supplied but never read: [SQLFLOW_DUCKDB_LIB] +/conf/win_late.yml: valid diff --git a/docs/superpowers/status/clickhouse-117740/win.sh b/docs/superpowers/status/clickhouse-117740/win.sh new file mode 100755 index 00000000..a0244190 --- /dev/null +++ b/docs/superpowers/status/clickhouse-117740/win.sh @@ -0,0 +1,44 @@ +#!/bin/bash +# win.sh: the tumbling-window replay. 1,000 events in one past one-minute +# bucket, read in ten batches of 100, closed by idle_close_seconds. The page's +# claim is one row of 1,000. IMG selects the image. +W="$(cd "$(dirname "$0")" && pwd)" +IMG="${IMG:-turbolytics/sql-flow:v2026.10.08}" +ch() { docker exec clickhouse clickhouse-client -q "$1" 2>&1; } +kt() { docker exec kafka1 kafka-topics --bootstrap-server localhost:9092 "$@" 2>&1 | tail -1; } + +echo "image: $IMG" +ch "DROP TABLE IF EXISTS probe.win" +ch "CREATE TABLE probe.win (bucket DateTime, n Int32) ENGINE = MergeTree ORDER BY bucket" +kt --delete --if-exists --topic ch-win; sleep 2 +kt --create --topic ch-win --partitions 1 --replication-factor 1 +for i in $(seq 0 999); do + printf '{"ts":"2026-09-01T12:00:%02dZ","id":%d}\n' $((i % 60)) $i +done > "$W/win_events.jsonl" +head -3 "$W/win_events.jsonl" > "$W/win_events.sample.jsonl" +docker exec -i kafka1 kafka-console-producer --bootstrap-server localhost:9092 --topic ch-win < "$W/win_events.jsonl" +echo "events published: $(wc -l < "$W/win_events.jsonl" | tr -d ' ')" + +echo "--- validate" +docker run --rm -v "$W":/conf -e SQLFLOW_CLICKHOUSE_DSN=clickhouse://default@clickhouse:8123/probe -e SQLFLOW_GROUP_ID=x \ + "$IMG" validate /conf/win.yml 2>&1 | tail -3 + +echo "--- run, no --max-msgs: the bucket closes on idle_close_seconds" +g="win-$RANDOM" +docker rm -f win_rig >/dev/null 2>&1 +docker run -d --name win_rig --network dev_default -v "$W":/conf \ + -e SQLFLOW_CLICKHOUSE_DSN=clickhouse://default@clickhouse:8123/probe -e SQLFLOW_GROUP_ID="$g" \ + "$IMG" run /conf/win.yml >/dev/null +sleep "${WIN_WAIT:-40}" +docker stop -t 20 win_rig >/dev/null +docker logs win_rig > "$W/win.log" 2>&1 +docker rm win_rig >/dev/null +/usr/bin/grep -i 'window\|watermark\|closed\|published' "$W/win.log" | /usr/bin/grep -v '"level":"debug"' | tail -${WIN_TAIL:-6} | cut -c1-400 +echo "--- what landed" +ch "SELECT bucket, n FROM probe.win ORDER BY bucket FORMAT PrettyCompact" +echo "rows published: $(ch 'SELECT count() FROM probe.win')" + +echo "--- validate: allowed_lateness_seconds with a ClickHouse window sink" +sed -e 's/idle_close_seconds: 15/idle_close_seconds: 15\n allowed_lateness_seconds: 60/' "$W/win.yml" > "$W/win_late.yml" +docker run --rm -v "$W":/conf -e SQLFLOW_CLICKHOUSE_DSN=clickhouse://default@clickhouse:8123/probe -e SQLFLOW_GROUP_ID=x \ + "$IMG" validate /conf/win_late.yml 2>&1 | tail -4 diff --git a/docs/superpowers/status/clickhouse-117740/win.yml b/docs/superpowers/status/clickhouse-117740/win.yml new file mode 100644 index 00000000..9184ff9f --- /dev/null +++ b/docs/superpowers/status/clickhouse-117740/win.yml @@ -0,0 +1,40 @@ +tables: + sql: + - name: agg + sql: | + CREATE TABLE agg (bucket TIMESTAMPTZ, n INT); + window: + time_column: bucket + size_seconds: 60 + grace_seconds: 0 + idle_close_seconds: 15 + emit_sql: | + SELECT bucket, sum(n)::INT AS n FROM closed GROUP BY ALL + sink: + type: clickhouse + clickhouse: + dsn: {{ SQLFLOW_CLICKHOUSE_DSN }} + table: win + +pipeline: + batch_size: 100 + flush_interval_seconds: 5 + source: + type: kafka + kafka: + brokers: [kafka1:19092] + group_id: {{ SQLFLOW_GROUP_ID }} + auto_offset_reset: earliest + topics: + - ch-win + event_time: + path: ts + format: rfc3339 + handler: + type: 'handlers.InferredMemBatch' + sql: | + INSERT INTO agg BY NAME + SELECT time_bucket(INTERVAL '1 minute', event_time) AS bucket, count(*)::INT AS n + FROM batch GROUP BY bucket + sink: + type: noop diff --git a/docs/superpowers/status/clickhouse-117740/win_events.sample.jsonl b/docs/superpowers/status/clickhouse-117740/win_events.sample.jsonl new file mode 100644 index 00000000..3e6d413b --- /dev/null +++ b/docs/superpowers/status/clickhouse-117740/win_events.sample.jsonl @@ -0,0 +1,3 @@ +{"ts":"2026-09-01T12:00:00Z","id":0} +{"ts":"2026-09-01T12:00:01Z","id":1} +{"ts":"2026-09-01T12:00:02Z","id":2}