Repository navigation
Conversation
Collaborator
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as outdated.
This comment was marked as outdated.
Enjection
force-pushed
the
draft/schemeshard-tpathref
branch
from
July 13, 2026 22:36
8dd0f03 to
8fbb967
Compare
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as outdated.
This comment was marked as outdated.
Enjection
force-pushed
the
draft/schemeshard-tpathref
branch
from
July 14, 2026 10:28
8fbb967 to
16802af
Compare
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as outdated.
This comment was marked as outdated.
Enjection
force-pushed
the
draft/schemeshard-tpathref
branch
from
July 14, 2026 22:14
16802af to
ec51957
Compare
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as outdated.
This comment was marked as outdated.
Enjection
force-pushed
the
draft/schemeshard-tpathref
branch
from
July 14, 2026 23:47
ec51957 to
58013c6
Compare
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as outdated.
This comment was marked as outdated.
Enjection
force-pushed
the
draft/schemeshard-tpathref
branch
from
July 15, 2026 05:17
58013c6 to
70e06c7
Compare
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as duplicate.
This comment was marked as duplicate.
Enjection
force-pushed
the
draft/schemeshard-tpathref
branch
from
July 15, 2026 07:42
70e06c7 to
2d63886
Compare
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as outdated.
This comment was marked as outdated.
Enjection
force-pushed
the
draft/schemeshard-tpathref
branch
from
July 15, 2026 11:13
2d63886 to
22d1d51
Compare
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment was marked as outdated.
This comment was marked as outdated.
Enjection
force-pushed
the
draft/schemeshard-tpathref
branch
from
July 15, 2026 14:21
22d1d51 to
6753296
Compare
- Set() now debug-asserts the acquired path was grabbed (GrabPath/GrabNewPath) in this tx, so a create op that forgets to snapshot before acquiring a ref crashes in debug instead of drifting. Backstops the create-op grab fixes. - TMemoryChanges::IsPathTracked scans the (small, per-tx) Paths stack; the stack is subclassed to expose its container, no shadow index or release-build cost. - Document that erase() records no undo (execute-phase only). Untracked-in-armed guard intentionally omitted: legitimate untracked-in-armed flows exist (ResolveDomainInfo, extsubdomain dual-tracking), so a blanket assert would false-fire. Validated: ut_base + ut_ttl 298/298 GOOD (relwithdebinfo, asserts active).
…tract - Add TRefLabel: a DbRefCount log label that only binds char arrays (string literals). A raw const char* (e.g. TString::c_str()) has no viable ctor and fails to compile, so a stored label can never dangle. Thread it through TPathRef, AcquirePathDbRef/ReleasePathDbRef, TSelfRefMap, and AcquireSelfDbRef; convert to const char* only at the Increment/DecrementPathDbRefCount boundary. Direct Increment/Decrement callers keep the raw TStringBuf API by design. - Document the tracked/untracked-by-phase contract on TSelfRefMap and note the extsubdomain untracked-in-armed SubDomains flow. Validated: ut_base + ut_ttl 298/298 GOOD.
The LOG_ERROR after the Y_VERIFY_S was unreachable: an orphaned source is skipped earlier when TolerateOrphanedPaths is on, so reaching that arm means the control is off and the assert always fires first. Remove the dead log; fail-fast and ICB-gating semantics are unchanged.
States the contract (intentionally drop the count; a snapshot owns it) instead of a metaphor, and can't be confused with Reset(). Mechanical rename plus a shorter class comment; no behavior change.
…heir alters Both held a THolder<...Config> AlterConfig, making them non-copyable, so their armed-propose alters had to use UpdateUntracked (no rollback) while every sibling alter uses tracked Update. Add a copy ctor that deep-copies AlterConfig (delete copy-assign; the undo path pointer-swaps, never assigns), then switch alter_kesus and alter_fs to tracked Update. Removes the untracked-in-armed exception outside SubDomains. Validated: ut_base + ut_filestore + ut_ttl 306/306 GOOD.
Semantics-free rename so the type names state their contract: - TPathRef -> TPathDbRef (owns a DbRefCount unit, not a TPath) - TSelfRefMap/I... -> TDbRefMap/IDbRefMap - SelfRefMaps -> DbRefMaps - AcquireSelfDbRef / ReleaseSelfDbRef / SelfDbRefs -> Acquire/ReleaseOwnDbRef / OwnDbRefs - RecordSelfRefUndo / SelfRefUndos / SelfRefUndoClone / SelfRefUndoRestoreSlot -> RecordDbRefUndo / DbRefUndos / DbRefUndoClone / DbRefUndoRestoreSlot - DebugCheckSelfRefIntegrity -> DebugCheckDbRefIntegrity - NSelfRefDetail -> NDbRefDetail Filenames and the TargetPathRef/SourcePathRef member names kept to limit churn. No behavior change; ut_base + ut_ttl + ut_filestore 306/306 GOOD.
Follow-up to the type rename:
- schemeshard_self_ref_map.h -> schemeshard_db_ref_map.h (TDbRefMap)
- schemeshard_path_ref.{h,cpp} -> schemeshard_path_db_ref.{h,cpp} (TPathDbRef)
- ut_base/ut_self_ref_map.cpp -> ut_base/ut_db_ref_map.cpp
Include sites and ya.make updated. No behavior change; ut_base + ut_ttl 298/298 GOOD.
CreateTx returns a TTxState with its target ref already acquired; the copy-assign in split_merge released it and substituted a hand-built local's empty handles, so an in-flight split held no DbRefCount ref on its target path. Introduced by the TPathRef conversion - pre-PR CreateTx bumped DbRefCount directly, so the same copy was harmless. TPathDbRef is now move-only, making the class of bug a compile error; it was the only copy site in the tree. Tests: ut_split_merge, ut_split_merge_reboots 212/212.
ResolveDomainInfo and GetPathVersion each did contains() followed by a value-returning accessor. TDbRefMap::FindPtr is already const and returns const V*, so both collapse to a single lookup: ResolveDomainInfo no longer needs the const_cast, and GetPathVersion no longer copies a std::shared_ptr (atomic refcount) per describe of a schema-preset column table. Also note in TTableInfo::DeepCopy that the copy stays O(partitions) via Stats.PartitionStats and VerifyConsistency, despite sharing Partitioning. Tests: ut_base, ut_olap 286/286.
erase records no undo, so an armed-propose erase cannot roll back. That contract was documented but unenforced: Set and Update have IsArmed tripwires, erase had none because it takes no TMemoryChanges. Arm now publishes the active TMemoryChanges on the shard, so erase can assert against it via a free function - db_ref_map.h holds only a forward declaration of TSchemeShard. The destructor withdraws only its own registration, since several TMemoryChanges coexist and only the propose one arms. All 31 current erase sites are plan-step or later, so this locks in behaviour rather than changing it. Tests: ut_base, ut_olap, ut_index, ut_move, ut_cdc_stream, ut_sequence, ut_replication, ut_external_table, ut_resource_pool, ut_backup_collection, ut_consistent_copy_tables, ut_extsubdomain - 645 GOOD, guard did not fire (6 TAsyncIndexTests reboots timeouts are pre-existing, same on main).
Arm() set a flag cleared only when TMemoryChanges died - a member of the propose tx - so the window ran past Complete(). Any erase() legitimately running later in that tx would have tripped the new guard. Disarm() via Y_DEFER closes the window on every exit, including the early returns for quota, rewrite and split failures. AbortOperationPropose runs from inside ProcessOperationParts, so rollback is still armed; the Set and Update tripwires, NeedsUpdateSnapshot and RecordDbRefUndo all fire during part-propose and are unaffected. The destructor keeps Disarm as a backstop. Tests: same 12 suites, 645 GOOD, guard did not fire (6 TAsyncIndexTests reboots timeouts are pre-existing, same on main).
Enjection
force-pushed
the
draft/schemeshard-tpathref
branch
from
August 20, 2026 08:17
26f764e to
6b1b4ba
Compare
This comment was marked as duplicate.
This comment was marked as duplicate.
This comment has been minimized.
This comment has been minimized.
dorooleg
previously approved these changes
Aug 20, 2026
Contributor
|
🔴 Unable to merge your PR into the base branch. Please rebase or merge it with the base branch. |
Merge upstream main bd9080b without rewriting the PR history. Adapt new generated-column, metrics, and topic code to read-only ownership-map lookups.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Contributor
|
Main concerns:
|
This was referenced Sep 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
YDBBUGS-452 · YDBBUGS-585
Changelog entry
Internal SchemeShard path-reference ownership and cleanup refactoring.
Description for reviewers
Automate path database-reference ownership for transactions, publications, and type-info map entries; retain restart checks and cleanup fixes. Shard reference accounting remains unchanged.
Keep cheap map-membership undo, explicit field undo, and staged table
AlterData. Remove automatic whole-object snapshots and the associated partitioning CoW prototype:Update()no longer copies a table's partitioning or per-shard statistics. Existing operations that prohibit proposal rollback keep that contract; this is not an MVCC redesign.Deferred snapshot prototype: Enjection/ydb#45 (draft).
Pre-merge verification (
fe39a0b): 53 related SchemeShard suites, 1,870/1,870 tests passed after reruns; three initial timeout cases each passed three serial repeats and both affected complete suites passed with lower test concurrency. Includes the 200,000-shard snapshot-freeUpdate()regression and reference-count, undo-ordering, and stable-alias tests.Merged fresh upstream
main(bd9080b) without rebasing; resolved the generated-column helper conflict and adapted new metrics/topic lookups to the ownership API. Post-merge verification (96311a4): 15 affected SchemeShard suites, 1,033/1,033 tests passed withhya make -T --build=relwithdebinfo -ttt --test-threads=4; no failures, timeouts, filters, or increased timeout limits.