Skip to content

Fix Load task cleanup and input handling - #18772

Closed
Caideyipi wants to merge 1 commit into
apache:masterfrom
Caideyipi:codex/fix-load-p2-issues
Closed

Caideyipi wants to merge 1 commit into
apache:masterfrom
Caideyipi:codex/fix-load-p2-issues

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Summary

  • avoid preallocating the complete sliced Load RPC body from the peer-provided original size
  • defer rollback cleanup until in-flight Load work finishes and reject new work after rollback is requested
  • skip the active-load failure directory when it is nested below a listening directory

Tests

  • mvn spotless:apply -pl iotdb-core/datanode
  • mvn -pl iotdb-core/datanode "-Dtest=LoadTsFilePieceNodeAssemblerTest,ActiveLoadDirScannerTest,LoadTsFileManagerTest" test (13 tests passed)

@Caideyipi

Copy link
Copy Markdown
Collaborator Author

Closing in favor of #18936, which preserves all three improvements on dev/1.3. #18698 replaced the global Load manager and EXECUTE/ROLLBACK lifecycle on master with per-DataRegion consensus commands and consensus-aware staged-file retention, so the cleanup/rollback implementation in this PR no longer fits the master Load path. The buffer-allocation and failure-directory scanning improvements remain independently applicable to master, but the legacy cleanup changes should not be restored there just to resolve these conflicts.

@Caideyipi Caideyipi closed this Oct 20, 2026
jt2594838 pushed a commit that referenced this pull request Oct 20, 2026
Port the improvements from PR #18772 to the legacy Load implementation. Adapt the active loader tests and cover rollback while nested Load work remains active.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant