Repository navigation
Fix deletion WAL logging and flush completion ordering - #18872
Merged
jt2594838 merged 2 commits intoOct 15, 2026
Merged
Conversation
Caideyipi
reviewed
Oct 14, 2026
Caideyipi
left a comment
Collaborator
There was a problem hiding this comment.
Reviewed commit 0b7733ded585a39441988358d8ac29e40b9a7946.
No blocking findings from this review. I checked the deletion WAL ID selection for both tree and table nodes, the deletion/flush/checkpoint ordering, and recovery of multiple unflushed memtables in the same TsFile:
- Holding
flushQueryLockthrough WAL enqueue prevents the selected memtable from being released before the deletion is queued. - Writing queued modifications and releasing the memtable under the same write lock closes the late-deletion registration window. The deque monitor is still acquired before the write lock, consistent with
syncReleaseFlushedMemTable. - All pending memtable IDs for a shared TsFile now route to one recovery performer, with its lifecycle calls deduplicated.
Validation on Windows with JDK 17:
mvn test -o -pl iotdb-core/datanode -am -Dtest=TsFileProcessorTest,WALRecoverManagerTest,UnsealedTsFileRecoverPerformerTest -Dsurefire.failIfNoSpecifiedTests=false -Ddevelocity.off=true
git diff --check 0c0116afe22532f32312a3cb30352a818fba4e38...HEAD
All 36 tests passed, with no failures, errors, or skipped tests, including the new regression cases. Spotless and Checkstyle also passed during the build.
Process-crash and disk-I/O fault injection were not part of this review.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
A TsFileProcessor is created before table schema registration. If registration fails before the first insert reaches memtable creation, the processor can remain in the working map with a null working memtable. A subsequent tree/table deletion dereferences that null memtable while logging to WAL.
Separately, deletion registration can race with file-close flush completion: after flush scans its queued modifications but before it releases the memtable, a new deletion can be registered and reported as handled in memory. That deletion is never written to the modification file.
Changes
MEMTABLE_NOT_EXISTsentinel when the working memtable is null. Retain the WAL entry for IoTConsensus synchronization.asyncClose()path that checks the actual persisted modification file.Production flush closes the processor and removes it from the working map under the DataRegion write lock. The change preserves the current one-memtable-per-TsFile lifecycle and makes no WAL recovery mapping changes. WAL/RPC formats and configuration remain unchanged.
Validation
mvn spotless:apply -o -pl iotdb-core/datanode mvn test -o -pl iotdb-core/datanode -am -Dtest=TsFileProcessorTest,WALRecoverManagerTest,UnsealedTsFileRecoverPerformerTest -Dsurefire.failIfNoSpecifiedTests=false git diff --checkAll 35 tests passed, with no failures, errors, or skipped tests. The flush-completion test controls the deque monitor and confirms the flush thread is blocked on that exact monitor before registering deletion, then reads the modification file from disk.
Cluster tests, process-crash fault injection, disk-I/O fault injection, and performance benchmarks were not run.
This PR has:
Key changed classes
TsFileProcessorTsFileProcessorTest