Skip to content

[To dev/1.3] Fix Load task cleanup and input handling - #18936

Merged
jt2594838 merged 1 commit into
apache:dev/1.3from
Caideyipi:codex/fix-load-p2-issues-dev-1.3
Oct 20, 2026
Merged

jt2594838 merged 1 commit into
apache:dev/1.3from
Caideyipi:codex/fix-load-p2-issues-dev-1.3

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Description

On dev/1.3, rollback can close a Load task's writers while its RPC work is still running. Track active work, including nested slice assembly and writing, so rollback rejects further work while cleanup is pending and releases resources after the last active call finishes.

This also preserves the other improvements from #18772:

  • Grow the sliced Load RPC buffer as bytes arrive instead of preallocating the peer-declared original body size.
  • Exclude the active-load failure directory from scanning when it is nested beneath a listening directory.

The cleanup changes target the legacy global Load manager on dev/1.3. #18698 replaced that lifecycle on master with per-DataRegion consensus commands and consensus-aware staging retention, so #18772 is being replaced by this PR for the legacy implementation. Adapt the scanner regression test to the dev/1.3 loader API and cover rollback while nested work remains active.

Validation

  • mvn -o -pl iotdb-core/datanode spotless:apply

  • mvn -o -pl iotdb-core/datanode -am "-Dtest=LoadTsFilePieceNodeAssemblerTest,ActiveLoadDirScannerTest,LoadTsFileManagerTest" "-Dsurefire.failIfNoSpecifiedTests=false" test — 13 tests passed, no failures or errors.

  • mvn -o test-compile -DskipTests — full reactor passed (44 modules).

  • mvn -o test-compile -P with-zh-locale -DskipTests — full reactor passed (44 modules).

  • Self-reviewed concurrent task admission, rollback and cleanup.

  • Added or adapted regression tests for all three changes.

Port the improvements from PR apache#18772 to the legacy Load implementation. Adapt the active loader tests and cover rollback while nested Load work remains active.
@jt2594838
jt2594838 merged commit fd0f76b into apache:dev/1.3 Oct 20, 2026
15 of 16 checks passed
@jt2594838
jt2594838 deleted the codex/fix-load-p2-issues-dev-1.3 branch October 20, 2026 10:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants