Skip to content

Add compressed TCP INSERT support - #28

Merged
joe-clickhouse merged 2 commits into
mainfrom
joe/tcp-compressed-insert
Nov 18, 2026
Merged

joe-clickhouse merged 2 commits into
mainfrom
joe/tcp-compressed-insert

Conversation

@joe-clickhouse

@joe-clickhouse joe-clickhouse commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

Adds compressed INSERT to ch-tcp-proto. Follows #27.

  • On a compressed connection, insert blocks and the end marker go out as LZ4 frames of up to 1 MiB.
  • The schema reply and other server Data during an INSERT use the compressed reader from Add compressed TCP SELECT support #27.
  • CompressedInsertUnsupported is deprecated and no longer returned.

Behavior change: a server Exception before InsertSchema now leaves the connection Ready instead of Failed. This also applies to uncompressed INSERT. An Exception after the schema is still terminal. The server can fail partway through a block and lose its place in the stream.


Note

Medium Risk
Changes core TCP INSERT protocol behavior and connection reuse rules on exceptions; well-covered by fixtures and tests but drivers must adopt new flush/reuse semantics.

Overview
ch-tcp-proto now supports compressed client-fed INSERT when Connection::with_compression(true) is enabled. Outbound upload blocks and the upload terminator are written as LZ4 frames (up to 1 MiB plaintext per frame, one Native block boundary per flush). Insert schema server Data uses the same compressed receive path as SELECT. CompressedInsertUnsupported is deprecated and no longer returned.

INSERT Exception handling changed: errors before InsertSchema can leave the connection Ready if the initial request was fully taken and flushed (same rules as SELECT wire-settings / terminal codes). Exceptions after the schema or during upload remain terminal, including after finish_insert.

Docs and roadmap entries are updated to reflect compressed INSERT as shipped. Captured revision-54460 fixtures, a generator script, replay tests, and extensive unit tests cover multi-frame uploads, rejection, and before-schema recovery.

Reviewed by Cursor Bugbot for commit 5c82906. Bugbot is set up for automated code reviews on this repo. Configure here.

Send insert blocks and the end marker as LZ4 frames on a compressed
connection, and read the schema reply through the compressed receiver.
A server Exception before InsertSchema now leaves the connection Ready.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@joe-clickhouse
joe-clickhouse requested a review from a team as a code owner November 17, 2026 23:47

@security-ai-gate-reviewer security-ai-gate-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Auto-approved by the AI review gate on bce8342. Details in the approver's comment on this pull request.

@security-ai-gate-reviewer

security-ai-gate-reviewer Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

AI review gate — auto-approved

5c82906aef79 is approved by security-ai-gate-reviewer[bot], an automated identity, not a human. The approval stands on the required AI review: Cursor Bugbot concluded green on this commit and the AI review gate is SUCCESS (its own comment has the verdict), and the change touches no path that requires a human reviewer. It is bound to this commit: a new push dismisses it and the approver re-evaluates.

History (latest 5 heads, newest first)

  • 5c82906aef79 approved — green: Cursor Bugbot
  • bce83423d768 approved — green: Cursor Bugbot

@polyglotAI-bot

Copy link
Copy Markdown
Collaborator

This PR adds compressed TCP INSERT with bounded LZ4 frames and permits connection reuse after eligible errors before the upload schema. No material correctness, safety, or contract issues found; approve. The added tests cover fragmented schema reads, multi-frame uploads, malformed input, compression-setting snapshots, and exception recovery. This was a static review: builds and tests were not run, and the pinned ClickHouse server source was unavailable for independent verification.

Final verdict

✅ Approve

Automated review by Polyglot AI (gpt-6-astra) of bce8342. Requested by @joe-clickhouse. Replies on the inline threads are read on the next review. A maintainer can ask for a new review by mentioning @polyglotAI-bot or requesting a review from it.

@polyglotAI-bot polyglotAI-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review of bce8342 posted: see the summary comment and the inline threads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@security-ai-gate-reviewer security-ai-gate-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Auto-approved by the AI review gate on 5c82906. Details in the approver's comment on this pull request.

@joe-clickhouse
joe-clickhouse merged commit 9b7d97e into main Nov 18, 2026
6 checks passed
@joe-clickhouse
joe-clickhouse deleted the joe/tcp-compressed-insert branch November 18, 2026 23:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants