[DBMON-6892] Report ClickHouse cluster topology in the database_instance payload

[sangeetashivaji]

What does this PR do?

Adds the hosting type, the connection mode and the cluster node inventory (the connected node plus the node list) to the ClickHouse database_instance metadata payload. Keys are omitted rather than reported empty, so a failed probe never claims a cluster has no nodes.

The clusterAllReplicas fan-out fix that used to live here is split out into #24920, keeping the two changelog entries separate (fixed there, added here). This PR is stacked on that branch and should be merged after it.

1. cloud with single_endpoint_mode

[
    {
        "host": "bpzyyp5jlh.us-east-2.aws.clickhouse.cloud",
        "port": 8443,
        "database_instance": "bpzyyp5jlh.us-east-2.aws.clickhouse.cloud:8443:default",
        "database_hostname": "bpzyyp5jlh.us-east-2.aws.clickhouse.cloud",
        "agent_version": "7.77.2",
        "ddagenthostname": "dbm-orders-clickhouse-cloud-sangeeta.shivajirao",
        "dbms": "clickhouse",
        "kind": "database_instance",
        "collection_interval": 300,
        "dbms_version": "26.2.1.558",
        "integration_version": "7.2.0",
        "tags": [
            "env:local",
            "deployment:cloud",
            "user:sangeeta",
            "server:bpzyyp5jlh.us-east-2.aws.clickhouse.cloud",
            "port:8443",
            "database_hostname:bpzyyp5jlh.us-east-2.aws.clickhouse.cloud",
            "database_instance:bpzyyp5jlh.us-east-2.aws.clickhouse.cloud:8443:default",
            "clickhouse_cluster:default",
            "hosting_type:clickhouse-cloud"
        ],
        "timestamp": 1787147945024.457,
        "metadata": {
            "dbm": true,
            "connection_host": "bpzyyp5jlh.us-east-2.aws.clickhouse.cloud",
            "hosting_type": "clickhouse-cloud",
            "single_endpoint_mode": true,
            "cluster_name": "default",
            "connect_node": "c-mintaws-bz-27-server-rbdgg8n-0",
            "nodes": [
                "c-mintaws-bz-27-server-rbdgg8n-0",
                "c-mintaws-bz-27-server-t2d8t8j-0"
            ]
        }
    }
]

2. self-hosted using single-endpoint-mode with a cluster name that is not default

The cluster name is not always default

[
    {
        "host": "clickhouse-lb",
        "port": 8123,
        "database_instance": "clickhouse-lb:8123:default",
        "database_hostname": "clickhouse-lb",
        "agent_version": "7.77.2",
        "ddagenthostname": "dbm-orders-local-clickhouse-se-sangeeta.shivajirao",
        "dbms": "clickhouse",
        "kind": "database_instance",
        "collection_interval": 300,
        "dbms_version": "24.8.14.39",
        "integration_version": "7.2.0",
        "tags": [
            "env:local",
            "deployment:self-hosted",
            "endpoint:single",
            "user:sangeeta",
            "server:clickhouse-lb",
            "port:8123",
            "database_hostname:clickhouse-lb",
            "database_instance:clickhouse-lb:8123:default",
            "clickhouse_cluster:dbm_single_endpoint",
            "hosting_type:self-hosted"
        ],
        "timestamp": 1787153884466.4001,
        "metadata": {
            "dbm": true,
            "connection_host": "clickhouse-lb",
            "hosting_type": "self-hosted",
            "single_endpoint_mode": true,
            "cluster_name": "dbm_single_endpoint",
            "connect_node": "clickhouse-se-01",
            "nodes": [
                "clickhouse-se-01",
                "clickhouse-se-02",
                "clickhouse-se-03"
            ]
        }
    }
]

3. self-hosted with single_endpoint_mode false

[
    {
        "host": "clickhouse-03",
        "port": 8123,
        "database_instance": "clickhouse-03:8123:default",
        "database_hostname": "clickhouse-03",
        "agent_version": "7.77.2",
        "ddagenthostname": "dbm-orders-local-clickhouse-sangeeta.shivajirao",
        "dbms": "clickhouse",
        "kind": "database_instance",
        "collection_interval": 300,
        "dbms_version": "24.8.14.39",
        "integration_version": "7.2.0",
        "tags": [
            "env:local",
            "node:clickhouse-03",
            "deployment:self-hosted",
            "server:clickhouse-03",
            "port:8123",
            "database_hostname:clickhouse-03",
            "database_instance:clickhouse-03:8123:default",
            "clickhouse_cluster:default",
            "hosting_type:self-hosted"
        ],
        "timestamp": 1787152211758.7336,
        "metadata": {
            "dbm": true,
            "connection_host": "clickhouse-03",
            "hosting_type": "self-hosted",
            "single_endpoint_mode": false,
            "cluster_name": "default",
            "connect_node": "clickhouse-03",
            "nodes": [
                "clickhouse-03"
            ]
        }
    }
]

Motivation

The backend needs to know whether the hostName() it sees behind a single endpoint is one node or a whole cluster, and which nodes make up that cluster.

Testing

Tested on staging.

Review checklist (to be filled by reviewers)

• Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
• Add the qa/skip-qa label if the PR does not need to be tested during QA. If the PR needs to be tested during QA, you should add qa/required label.
• If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

[cit-pr-commenter-54b7da]

evalya-impact-summary

evalya impact analysis

Impact analysis: 0 selected, 0 skipped (of 0 test tasks)
Publish tasks:   0 (always emitted)
Diff (8 files):
  clickhouse/changelog.d/24889.added
  clickhouse/changelog.d/24920.fixed
  clickhouse/datadog_checks/clickhouse/clickhouse.py
  clickhouse/datadog_checks/clickhouse/utils.py
  clickhouse/tests/metrics.py
  clickhouse/tests/test_clickhouse.py
  clickhouse/tests/test_table_metrics.py
  clickhouse/tests/test_unit.py

Debug a specific task: evalya plan impact --path <path> --task <task>

Learn more about CI impact filtering

[datadog-datadog-prod-us1]

 

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 98.92%
• Overall Coverage: 94.11% (+5.51%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: f0a4cf5 | Docs | View more details | Give us feedback!}

[dd-octo-sts]

Validation Report

All 21 validations passed.

Show details

Validation	Description	Status
agent-reqs	Verify check versions match the Agent requirements file	✅
ci	Validate CI configuration and code coverage settings	✅
codeowners	Validate every integration has a CODEOWNERS entry	✅
config	Validate default configuration files against spec.yaml	✅
dep	Verify dependency pins are consistent and Agent-compatible	✅
http	Validate integrations use the HTTP wrapper correctly	✅
imports	Validate check imports do not use deprecated modules	✅
integration-style	Validate check code style conventions	✅
jmx-metrics	Validate JMX metrics definition files and config	✅
labeler	Validate PR labeler config matches integration directories	✅
legacy-signature	Validate no integration uses the legacy Agent check signature	✅
license-headers	Validate Python files have proper license headers	✅
licenses	Validate third-party license attribution list	✅
metadata	Validate metadata.csv metric definitions	✅
models	Validate configuration data models match spec.yaml	✅
openmetrics	Validate OpenMetrics integrations disable the metric limit	✅
package	Validate Python package metadata and naming	✅
qa-label	Validate the pull request declares whether it needs QA for the next Agent release	✅
readmes	Validate README files have required sections	✅
saved-views	Validate saved view JSON file structure and fields	✅
version	Validate version consistency between package and changelog	✅

View full run