Skip to content

[DBMON-6892] Fan out ClickHouse cluster queries over the resolved cluster name - #24920

Open
sangeetashivaji wants to merge 3 commits into
masterfrom
sangeeta/clickhouse-cluster-fanout-fix
Open

[DBMON-6892] Fan out ClickHouse cluster queries over the resolved cluster name#24920
sangeetashivaji wants to merge 3 commits into
masterfrom
sangeeta/clickhouse-cluster-fanout-fix

Conversation

@sangeetashivaji

Copy link
Copy Markdown
Contributor

What does this PR do?

Fixes the clusterAllReplicas fan-out in single endpoint mode to use the cluster the instance actually belongs to, instead of a hardcoded 'default'.

  • Adds ClickhouseCheck.fanout_cluster_name: the resolved cluster name, falling back to 'default' only when the deployment is not known to be self-hosted (Cloud always uses that name).
  • get_queries() and get_system_table() fan out over that cluster, and read the local system.<table> when no cluster can be resolved.
  • cluster_aware_query(base, cluster) now takes the cluster, and cluster names are quoted and escaped through quote_string() since they arrive as server-supplied data.

Split out of #24889, which now carries only the database_instance metadata additions and is stacked on top of this branch.

Motivation

Only ClickHouse Cloud names its cluster default. Against a self-hosted deployment a hardcoded 'default' either raises UNKNOWN_CLUSTER, or — when the stock localhost-only default cluster exists — silently returns the local node alone while the result is presented as a whole-cluster view.

Keeping this in its own PR keeps the changelog entries separate: this one is a fixed entry, #24889 is an added entry.

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Add the qa/skip-qa label if the PR does not need to be tested during QA. If the PR needs to be tested during QA, you should add qa/required label.
  • If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

…ault

Only ClickHouse Cloud names its cluster 'default'. Against a self-hosted
deployment a hardcoded 'default' either raises UNKNOWN_CLUSTER or, when the
stock localhost-only 'default' cluster exists, silently reads the local node
alone while presenting the result as a whole-cluster view. Fan out over the
cluster the instance actually belongs to, and read the local system table when
no cluster can be resolved.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Aug 19, 2026

Copy link
Copy Markdown

evalya-impact-summary

evalya impact analysis
Impact analysis: 0 selected, 0 skipped (of 0 test tasks)
Publish tasks:   1 (always emitted)
Diff (5 files):
  clickhouse/changelog.d/24920.fixed
  clickhouse/datadog_checks/clickhouse/clickhouse.py
  clickhouse/datadog_checks/clickhouse/utils.py
  clickhouse/tests/test_table_metrics.py
  clickhouse/tests/test_unit.py

Debug a specific task: evalya plan impact --path <path> --task <task>

Learn more about CI impact filtering

@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented Aug 19, 2026

Copy link
Copy Markdown

Tests  Code Coverage

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 93.92% (+5.32%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: d17393a | Docs | View more details | Give us feedback!

@sangeetashivaji
sangeetashivaji marked this pull request as ready for review August 19, 2026 18:18
@sangeetashivaji
sangeetashivaji requested review from a team as code owners August 19, 2026 18:18
@sangeetashivaji sangeetashivaji changed the title Fan out ClickHouse cluster queries over the resolved cluster name [DBMON-6892] Fan out ClickHouse cluster queries over the resolved cluster name Aug 19, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 423e93124a

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

instead and callers fall back to the local system table.
"""
if self.cluster_name:
return self.cluster_name

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Require an unambiguous cluster before fanning out

When a node participates in multiple configured clusters and has no {cluster} macro, cluster_name comes from CLUSTER_NAME_QUERY, which deliberately uses ORDER BY cluster LIMIT 1 (utils.py:62-70); that makes the tag stable but does not establish that the selected topology matches the load-balanced endpoint. Returning that value here now routes the affected single-endpoint metrics and DBM queries through an arbitrary cluster, potentially omitting nodes or querying an unrelated topology. Use this value for fan-out only when resolution is unambiguous, or provide an explicit fan-out cluster setting.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does make sense but its a feature functionality that needs to be built to support multi cluster setup for single endpoint mode in self-hosted which we do not want to do.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all system tables in cloud are under default cluster, self hosted single_endpoint_mode is the only impacted path here

@dd-octo-sts

dd-octo-sts Bot commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Validation Report

All 21 validations passed.

Show details
Validation Description Status
agent-reqs Verify check versions match the Agent requirements file
ci Validate CI configuration and code coverage settings
codeowners Validate every integration has a CODEOWNERS entry
config Validate default configuration files against spec.yaml
dep Verify dependency pins are consistent and Agent-compatible
http Validate integrations use the HTTP wrapper correctly
imports Validate check imports do not use deprecated modules
integration-style Validate check code style conventions
jmx-metrics Validate JMX metrics definition files and config
labeler Validate PR labeler config matches integration directories
legacy-signature Validate no integration uses the legacy Agent check signature
license-headers Validate Python files have proper license headers
licenses Validate third-party license attribution list
metadata Validate metadata.csv metric definitions
models Validate configuration data models match spec.yaml
openmetrics Validate OpenMetrics integrations disable the metric limit
package Validate Python package metadata and naming
qa-label Validate the pull request declares whether it needs QA for the next Agent release
readmes Validate README files have required sections
saved-views Validate saved view JSON file structure and fields
version Validate version consistency between package and changelog

View full run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant