Skip to content

Refactor shared Pipe, consensus and subscription helpers - #18837

Open
Caideyipi wants to merge 1 commit into
apache:masterfrom
Caideyipi:refactor/align-shared-pipe-consensus-subscription
Open

Caideyipi wants to merge 1 commit into
apache:masterfrom
Caideyipi:refactor/align-shared-pipe-consensus-subscription

Conversation

@Caideyipi

@Caideyipi Caideyipi commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Refactor shared Pipe, IoTConsensus, and subscription code into focused interfaces and helpers while preserving existing behavior.

  • Introduce PipeBatchMetricsSettable and extract batch detachment while keeping synchronization and atomic ownership handoff intact.
  • Extract tree-pattern splitting, modification-map construction, schema-snapshot batch flushing, and ConfigNode snapshot file-list helpers; use consistent source naming and database/table matching.
  • Extract consensus retry, missing-group response, and snapshot-cleanup helpers without changing retry decisions, status codes, diagnostic messages, or locking.
  • Route subscription tablet projection through TableViewTabletProjectorProvider, reusing the existing schema preparation logic and the batch's immutable topic configuration. Keep projector lifetime scoped to each batch and preserve TAG filtering, column pruning, and WAL replay behavior.

Existing constructors and metric setters remain available. Regression coverage includes missing consensus groups, schema availability, projector lifetime, topic changes between capture and emission, and TAG filters without tree capture.

Validation

  • 248 focused unit tests passed: 0 failures, 0 errors, and 0 skipped tests.
  • Full-reactor mvn clean package -DskipTests passed for both the default locale and -P with-zh-locale. Each build used a fresh source tree and a separate empty Maven repository, with Maven and Develocity build caches disabled.
  • Spotless with with-integration-tests, Checkstyle, and git diff --check passed.

This PR has:

  • been self-reviewed, including synchronization, atomic batch ownership handoff, projector lifetime, and compatibility with the current master branch.
  • added regression coverage for missing consensus groups, schema availability, cached projectors, immutable batch topic configuration, and TAG filtering.

@Caideyipi
Caideyipi force-pushed the refactor/align-shared-pipe-consensus-subscription branch from 7a22815 to 5cb1d22 Compare October 26, 2026 10:40
@Caideyipi

Copy link
Copy Markdown
Collaborator Author

Self-review passed.

I reviewed the complete diff and relevant surrounding code. No actionable issues were found, so no fixes were needed.

Verification:

  • git diff --check — passed.
  • mvn -pl iotdb-core/consensus -am test -Dtest=IoTConsensusRPCServiceProcessorTest -Dsurefire.failIfNoSpecifiedTests=false — passed (2 tests, 0 failures, 0 errors).
  • mvn -pl iotdb-core/datanode -am test -Dtest=TableViewTabletProjectorProviderTest -Dsurefire.failIfNoSpecifiedTests=false — could not complete because the JVM terminated with an insufficient native memory error before producing test results.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant