Skip to content

Reuse DataRegion insert lock for IoTConsensus shutdown fencing - #18893

Merged
jt2594838 merged 1 commit into
apache:masterfrom
Caideyipi:fix/reuse-insert-lock-iot-consensus-shutdown
Oct 15, 2026
Merged

jt2594838 merged 1 commit into
apache:masterfrom
Caideyipi:fix/reuse-insert-lock-iot-consensus-shutdown

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Description

Follow up to #18841. IoTConsensus shutdown fencing currently acquires an additional region read/write lock around every apply. Reuse DataRegion.insertLock to check write admission and permanently close admission before storage and WAL shutdown. Memory backpressure and state-machine retry waits stay outside the insert lock.

Write successful insertions' WAL separators before releasing the insert lock, preserving their subscription metadata. Table deletions continue to release the lock during sealed-file I/O. Shutdown waits on a condition from the same lock until admitted deletions finish, including separator emission on success; failed deletions also unblock shutdown. Remove the state-machine apply wrapper.

Validation

  • Targeted tests: IoTConsensusShutdownTest, DataRegionTest, DataRegionStateMachineTest, WALNodeTest, and testCompactionWithDeletionsDuringCompactions in both InnerSeqCompactionWithFastPerformerTest and InnerSeqCompactionWithReadChunkPerformerTest (74 tests, 0 failures/errors, 1 existing skip).
  • Shutdown tests cover writers queued for the insert lock, admitted writes and pending WAL separators, memory backpressure, all storage write paths, unlocked table deletion I/O and failure, and other consensus protocols.
  • Full reactor test compilation in English and Chinese: mvn test-compile -DskipTests and mvn test-compile -P with-zh-locale -DskipTests.
  • mvn spotless:apply -pl iotdb-core/datanode.

This PR has:

  • been self-reviewed for concurrent writes and shutdown.
  • added or updated unit tests for the changed write and shutdown paths.
  • added comments explaining the lock and deletion-draining behavior.

@jt2594838
jt2594838 merged commit d2b97bc into apache:master Oct 15, 2026
39 checks passed
@jt2594838
jt2594838 deleted the fix/reuse-insert-lock-iot-consensus-shutdown branch October 15, 2026 10:41
jt2594838 pushed a commit that referenced this pull request Oct 16, 2026
#18893) (#18896)

* Fence IoTConsensus applies before storage shutdown (#18841)

* Fix queued IoTConsensus writes during shutdown

* Test queued IoTConsensus writes across storage shutdown

* Reproduce WAL checkpoint loss from writes after shutdown

* Fence IoTConsensus applies before storage shutdown

(cherry picked from commit 532d6c1)

* Reuse DataRegion insert lock for IoTConsensus shutdown fencing (#18893)

(cherry picked from commit d2b97bc)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants