Repository navigation
Fix table schema map registration for table-model TsFiles - #19100
Merged
jt2594838 merged 7 commits intoNov 5, 2026
Merged
Conversation
Caideyipi
approved these changes
Nov 3, 2026
Caideyipi
left a comment
Caideyipi
left a comment
Collaborator
There was a problem hiding this comment.
Reviewed commit f868dc1. I found no actionable issues in the diff. The changes preserve table-schema entries at normal file close, WAL recovery, and compaction boundaries. I did not run the newly added tests locally; the related unit-test and dual-table integration CI jobs are still pending.
Keep the TableSchema map entry for every table that has chunk data in a TsFile, even when the schema has no FIELD or TIME column. Restore missing entries during flush and recovery, and retain them during compaction so table-model files do not degrade into tree-model files. Add unit tests and a two-cluster integration test covering compaction and LOAD.
luoluoyuyu
force-pushed
the
fix/table-schema-registration
branch
from
November 3, 2026 04:33
f868dc1 to
2c6e96f
Compare
jt2594838
reviewed
Nov 3, 2026
…LOAD Write a single isTableModel file property when a table-model TsFile is closed by flush, compaction, recovery and the pipe sink, so a file whose TableSchema map is empty is no longer mistaken for a tree-model file. The property is constant time and independent of the number of tables. LOAD now completes a schema entry that has no column other than time, and an entry that is missing, from the table in IoTDB, and only fails with a dedicated message when the table exists neither in the file nor in IoTDB. The storage path keeps its previous behavior: it neither synthesizes entries for dropped tables nor scans chunks during recovery. Add unit tests for the file property, the model detection and the schema completion, and an integration test covering time-only entries, missing entries and table creation.
CompactionTableSchema.merge behaves exactly as before: it merges id columns and never keeps the TIME column. Keeping the table-name entry of a table that wrote data into the target file is enough for a compacted file to stay identifiable, so the merge decision does not need to change. Restore the corresponding unit tests.
Compaction drops a TableSchema again when it has no FIELD or TIME column, as it did before this change. The target file stays identifiable through the isTableModel file property, so the writer only records that property, read before the unused entries are removed.
The compaction writer learns the model from the database it compacts, so the isTableModel property is written whenever the target file belongs to a table-model database, even when its TableSchema map keeps no entry. Assert the property in the compaction and LOAD integration test.
…ealed The marker is one entry of the in-memory file property map, serialized by endFile(). Spell that out at the helper and at the flush and pipe call sites so the marker is never moved to a per-record, per-tablet or per-chunk-group position.
jt2594838
approved these changes
Nov 4, 2026
UnsealedTsFileRecoverPerformer derived the data model from the database name via PathUtils.isTableModelDatabase, and the table-schema registration helper repeated the same name-based check. That duplicates knowledge the owning DataRegion already holds through its isTableModel field and misclassifies a table-model region whose database name happens to start with "root.", which leaves the recovered TsFile with the chunks of a table but no TableSchema entry for them. Pass DataRegion.isTableModel into the recover performer, gate the schema registration on it, and keep databaseName only as the DataNodeTableCache lookup key. Update the recovery tests to pass the model explicitly and turn the registration-helper test into a check that the database name is never inspected to guess the model.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Table-model TsFiles are identified by a non-empty TableSchema map. A table could have chunks in a file while its schema map entry was missing or removed, causing the file to be treated as a tree-model file and loaded with zero rows.
This change only fixes map membership and preserves the existing TableSchema content:
LoadTsFileAnalyzeris intentionally untouched.Tests
spotless:checkandgit diff --checkpassed.