[Filestore] [4/N] Persist shard creation progress across throttled create and resize retries by Maximus5 · Pull Request #7253 · ydb-platform/nbs

Maximus5 · 2026-09-21T15:10:31Z

Notes

Main issue: [Filestore] Send requests to SchemeShard in batches for CreateFileStore, ResizeFileStore, and DestroyFileStore #6310
Design doc: [Filestore] [1/N] Document the filestore resize flow in legacy and throttled modes #6376

Implement persistent shard creation state integration for throttled filestore create/resize flows.

The PR extends UnsafeChangeTabletState so the main tablet can store and return CAS-protected shard creation state, preserves this state across UpdateConfig, and trims tablet logs so bitmap payloads are not dumped at INFO.

Create and resize actors now read the persisted state before shard creation, skip shards already recorded as physically created, persist newly created shard bits before shard configuration, merge and resend state after CAS conflicts, fall back to the legacy throttled flow for old tablets that do not return ShardCreationState, and fail retryably if state updates fail.

The PR also adds service-level coverage for create/resize retry recovery, skip-all retry configuration, state update errors, CAS conflict resend, old-tablet fallback, and an e2e resize retry/restart scenario.

Stack

github-actions · 2026-09-21T16:29:28Z

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🟢 linux-x86_64-relwithdebinfo target: cloud/tasks/,cloud/storage/ (test time: 271s): all tests PASSED for commit ff09111.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1580	1580	0	0	0	0	0

🟢 linux-x86_64-relwithdebinfo target: cloud/disk_manager/ (test time: 324s): all tests PASSED for commit ff09111.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1746	1746	0	0	0	0	0

🟢 linux-x86_64-relwithdebinfo target: cloud/blockstore/ (test time: 1412s): all tests PASSED for commit ff09111.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
7574	7573	0	0	0	1	0

🟢 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 4451s): all tests PASSED for commit ff09111.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
5047	5046	0	0	0	1	0

github-actions · 2026-09-24T15:19:00Z

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🟢 linux-x86_64-relwithdebinfo target: cloud/tasks/,cloud/storage/,cloud/fastshard/ (test time: 281s): all tests PASSED for commit 704d3fd.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1671	1671	0	0	0	0	0

🟢 linux-x86_64-relwithdebinfo target: cloud/disk_manager/ (test time: 320s): all tests PASSED for commit 704d3fd.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1746	1746	0	0	0	0	0

🟢 linux-x86_64-relwithdebinfo target: cloud/blockstore/ (test time: 1630s): all tests PASSED for commit 704d3fd.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
7436	7435	0	0	0	1	0

🟢 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 4835s): all tests PASSED for commit 704d3fd.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
4822	4820	0	0	0	2	0

github-actions · 2026-09-30T16:46:41Z

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🟢 linux-x86_64-relwithdebinfo target: cloud/tasks/,cloud/storage/,cloud/fastshard/ (test time: 274s): all tests PASSED for commit b81a239.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1718	1718	0	0	0	0	0

🟢 linux-x86_64-relwithdebinfo target: cloud/disk_manager/ (test time: 315s): all tests PASSED for commit b81a239.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1755	1755	0	0	0	0	0

🟢 linux-x86_64-relwithdebinfo target: cloud/blockstore/ (test time: 1487s): all tests PASSED for commit b81a239.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
7493	7492	0	0	0	1	0

🔴 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 5257s): some tests FAILED for commit b81a239.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
4912	4910	0	1	0	1	0

🟢 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 33s): all tests PASSED for commit b81a239.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
2	2	0	0	0	0	0

github-actions · 2026-10-01T08:23:56Z

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🟢 linux-x86_64-relwithdebinfo target: cloud/tasks/,cloud/storage/,cloud/fastshard/ (test time: 276s): all tests PASSED for commit fc5c35d.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1718	1718	0	0	0	0	0

🟢 linux-x86_64-relwithdebinfo target: cloud/disk_manager/ (test time: 329s): all tests PASSED for commit fc5c35d.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1755	1755	0	0	0	0	0

🔴 linux-x86_64-relwithdebinfo target: cloud/blockstore/ (test time: 1283s): some tests FAILED for commit fc5c35d.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
7482	7479	0	2	0	1	0

🟢 linux-x86_64-relwithdebinfo target: cloud/blockstore/ (test time: 1112s): all tests PASSED for commit fc5c35d.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
7494	7493	0	0	0	1	0

🟢 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 5307s): all tests PASSED for commit fc5c35d.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
4912	4911	0	0	0	1	0

github-actions · 2026-10-21T18:36:53Z

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🟢 linux-x86_64-relwithdebinfo target: cloud/disk_manager/ (test time: 304s): all tests PASSED for commit 13edc78.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1849	1849	0	0	0	0	0

🟢 linux-x86_64-relwithdebinfo target: cloud/blockstore/ (test time: 1498s): all tests PASSED for commit 13edc78.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
7689	7688	0	0	0	1	0

🟢 linux-x86_64-relwithdebinfo target: cloud/tasks/,cloud/storage/,cloud/fastshard/ (test time: 270s): all tests PASSED for commit 13edc78.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1806	1806	0	0	0	0	0

🔴 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 5460s) Test run completed, no test results found for commit 13edc78.
Please check build logs.

🟢 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 5087s): all tests PASSED for commit 13edc78.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
5069	5068	0	0	0	1	0

SvartMetal · 2026-10-23T12:23:23Z


+namespace {
+
+ui64 GetChunkDataSize(const NCloud::NProto::TCompressedBitmapData& bitmap)


Can we put it to some reusable place?

SvartMetal · 2026-10-23T12:29:31Z

    // Compress ShardId and ShardNodeName when we write them to NodeRefs table.
    optional bool CompressNodeRef = 3;
+
+    // Update shard creation state during large operations.


Let's explain what is meant by "large"

Maximus5 added this pull request to stack #7257 September 21, 2026 16:12

Maximus5 added the recheck Add this label to relaunch checks, it will be automatically removed label Sep 21, 2026

github-actions Bot removed the recheck Add this label to relaunch checks, it will be automatically removed label Sep 21, 2026

Maximus5 force-pushed the users/maximus5/filestore-resize-persist-state-fs-state branch from ff09111 to 704d3fd Compare September 24, 2026 15:17

Maximus5 added the recheck Add this label to relaunch checks, it will be automatically removed label Sep 24, 2026

github-actions Bot removed the recheck Add this label to relaunch checks, it will be automatically removed label Sep 24, 2026

Maximus5 force-pushed the users/maximus5/filestore-resize-persist-state-fs-state branch from 704d3fd to b81a239 Compare September 30, 2026 16:44

Maximus5 added the recheck Add this label to relaunch checks, it will be automatically removed label Sep 30, 2026

github-actions Bot removed the recheck Add this label to relaunch checks, it will be automatically removed label Sep 30, 2026

Maximus5 force-pushed the users/maximus5/filestore-resize-persist-state-fs-state branch from b81a239 to fc5c35d Compare October 1, 2026 08:22

Maximus5 added the recheck Add this label to relaunch checks, it will be automatically removed label Oct 1, 2026

github-actions Bot removed the recheck Add this label to relaunch checks, it will be automatically removed label Oct 1, 2026

[Filestore] Persist resize state in the main FS tablet

13edc78

Maximus5 force-pushed the users/maximus5/filestore-resize-persist-state-fs-state branch from fc5c35d to 13edc78 Compare October 21, 2026 18:35

Maximus5 added the recheck Add this label to relaunch checks, it will be automatically removed label Oct 21, 2026

github-actions Bot removed the recheck Add this label to relaunch checks, it will be automatically removed label Oct 21, 2026

SvartMetal reviewed Oct 23, 2026

View reviewed changes

Navigation Menu

Repository navigation

[Filestore] [4/N] Persist shard creation progress across throttled create and resize retries - #7253

[Filestore] [4/N] Persist shard creation progress across throttled create and resize retries#7253
Maximus5 wants to merge 1 commit into
users/maximus5/filestore-resize-persist-state-companionfrom
users/maximus5/filestore-resize-persist-state-fs-state

Maximus5 commented Sep 21, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 21, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 24, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 30, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 1, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 21, 2026 •

edited

Loading

Uh oh!

SvartMetal Oct 23, 2026

Uh oh!

SvartMetal Oct 23, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants


		namespace {

		ui64 GetChunkDataSize(const NCloud::NProto::TCompressedBitmapData& bitmap)

Conversation

Maximus5 commented Sep 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Notes

Stack

Uh oh!

github-actions Bot commented Sep 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Sep 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Sep 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SvartMetal Oct 23, 2026

Choose a reason for hiding this comment

Uh oh!

SvartMetal Oct 23, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Maximus5 commented Sep 21, 2026 •

edited

Loading

github-actions Bot commented Sep 21, 2026 •

edited

Loading

github-actions Bot commented Sep 24, 2026 •

edited

Loading

github-actions Bot commented Sep 30, 2026 •

edited

Loading

github-actions Bot commented Oct 1, 2026 •

edited

Loading

github-actions Bot commented Oct 21, 2026 •

edited

Loading