Skip to content

Filter offline DataNode URLs and add Java client seed fallback - #18920

Closed
JackieTien97 wants to merge 1 commit into
apache:masterfrom
JackieTien97:codex/available-urls-seed-fallback
Closed

JackieTien97 wants to merge 1 commit into
apache:masterfrom
JackieTien97:codex/available-urls-seed-fallback

Conversation

@JackieTien97

@JackieTien97 JackieTien97 commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Superseded by #18921, which uses the branch in apache/iotdb. The implementation and validation are unchanged.

Description

Related to #18633.

SHOW AVAILABLE URLS currently includes Unknown and Stopped DataNodes. Java SDK auto-discovery puts these addresses into the normal query rotation, so a persistently unreachable node can repeatedly add a connection timeout to otherwise successful queries. This change filters offline nodes on the server and preserves the client's initial addresses as recovery seeds.

Rolling restart scenario

A client can discover only nodes 2 and 3 while node 1 is stopped. Node 1 may recover before the next successful discovery refresh, while subsequent connection attempts to the cached nodes fail during maintenance. Filtering the server response alone leaves both business reconnection and background discovery dependent on the same stale list, even though node 1 is reachable again.

The client now tries the discovered endpoints first, then falls back to the initial nodeUrls. If node 1 was configured as a seed, it remains a recovery candidate after being omitted from a discovery snapshot. A successful business reconnection through a seed requests a prompt background refresh so recovered nodes can re-enter normal routing.

Implementation

  • Exclude Unknown and Stopped from SHOW AVAILABLE URLS, continue excluding Removing, and retain Running and ReadOnly nodes.
  • Preserve independent copies of the initial addresses in Session/SessionPool and the node supplier, including single-host configuration. Discovery updates and changes to the default endpoint do not replace the recovery seeds.
  • Apply discovered-first, seed-second recovery to both SessionConnection.reconnect() and NodesSupplier.run(). Deduplicate addresses within each attempt round and retain the existing bounded business reconnection rounds.
  • Let background discovery try other discovered nodes and seeds in the same run after its current connection fails. Coalesce refresh requests from sessions sharing a supplier.
  • Publish successful empty discovery results while retaining seeds. Select query endpoints from one snapshot so a concurrent refresh to an empty list cannot invalidate selection. Seeds are not merged into the normal query rotation.

Recovery still requires a reachable configured seed when all discovered endpoints fail. The seed fallback requires the updated Java SDK. This change does not add endpoint backoff for nodes that the server reports as Running but that are unreachable from a particular client.

Validation

Validated on JDK 17:

  • 239 targeted unit tests passed: 237 session/client tests and 2 ShowAvailableUrlsTaskTest tests. Coverage includes seed recovery, discovered-node priority, empty snapshots, duplicate addresses, bounded retries, caller mutation, refresh coalescing, and concurrent query selection during refresh.
  • 2 SessionSeedNodeFallbackIT tests passed on a real 1 ConfigNode / 3 DataNode cluster:
    • testQueriesDuringRollingRestart: Session and SessionPool queries return the expected data before and after each DataNode is stopped and restarted; discovery excludes stopped nodes and restores recovered nodes.
    • testBusinessAndBackgroundRecoveryThroughOmittedSeed: freeze periodic discovery, recover an omitted seed, make the two cached endpoints unavailable (Unknown and Stopped), and verify business reconnection with a metadata query and independent background discovery recovery through the seed.
  • Full-reactor mvn test-compile -DskipTests and mvn test-compile -DskipTests -P with-zh-locale passed.
  • Spotless and Git whitespace checks passed.

Validation notes: RAT remained enabled; a local excludes file omitted five pre-existing untracked SVG files outside this PR. The rolling-restart test tolerates IoTDBConnectionException only when closing stale sessions during cleanup. Query and discovery failures are not ignored; production close behavior is unchanged.


This PR has:

  • been self-reviewed.
  • added comments explaining recovery ordering and snapshot handling.
  • added unit tests covering the new paths.
  • added integration tests.
  • been tested in a test IoTDB cluster.
Key changed/added classes

ShowAvailableUrlsTask, INodeSupplier, NodesSupplier, SessionConnection, Session, SessionPool, ShowAvailableUrlsTaskTest, SessionSeedNodeFallbackTest, and SessionSeedNodeFallbackIT.

@JackieTien97
JackieTien97 deleted the codex/available-urls-seed-fallback branch October 20, 2026 01:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant