Skip to content

labs/vector-route-dual-write: Vector route to ClickStack next to Loki - #162

Merged
litkhai merged 1 commit into
mainfrom
vector-route-dual-write
Nov 13, 2026
Merged

litkhai merged 1 commit into
mainfrom
vector-route-dual-write

Conversation

@litkhai

@litkhai litkhai commented Nov 13, 2026

Copy link
Copy Markdown
Owner

New lab, run end to end on local k3d on 2026-11-13 (Vector 0.61.0, clickstack-all-in-one 2.44.0 / ClickHouse 26.8.36.3, Loki 3.7.9, Kong 3.9.4).

  • One Vector DaemonSet keeps shipping every pod's logs to Loki; a route transform sends two Kong gateways and one API deployment into logs.kong_access / logs.app_logs. Counts match Loki exactly (300/300/600); unmatched pods reach Loki only.
  • ClickHouse scaled to zero for ~2 min under three ClickHouse sink buffers: block (default) stalls the whole source, Loki included, then delivers everything; drop_newest keeps Loki whole and drops exactly the counted rows; a 256 MiB disk buffer keeps both.
  • One negative Kong latency in a UInt32 column failed a 1,000-row insert with HTTP 400 and Vector dropped the batch (not retriable); latency columns are Int32 now.
  • ClickStack sources on both tables, three screenshots.
  • lab.yaml has no publishing keys (card left to the owner).

Not covered: GCP Private Service Connect, Cloud regions, a dedicated insert-only user, log rotation during long outages.

Closes #161

🤖 Generated with Claude Code

…, sink buffer modes under a ClickHouse outage

Closes #161

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@litkhai
litkhai merged commit ab432bb into main Nov 13, 2026
5 checks passed
@litkhai
litkhai deleted the vector-route-dual-write branch November 13, 2026 10:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

labs/vector-route-dual-write: Vector route to ClickStack next to an existing Loki pipeline

1 participant