Skip to content

[Filestore] Eliminate allocation of fresh blocks and move blob allocation to TFlushActor - #7467

Open
antonmyagkov wants to merge 12 commits into
mainfrom
users/myagkov/eliminate-copy-of-fresh-blocks
Open

antonmyagkov wants to merge 12 commits into
mainfrom
users/myagkov/eliminate-copy-of-fresh-blocks

Conversation

@antonmyagkov

@antonmyagkov antonmyagkov commented Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator

issue: #7468

  1. Keep block index in BlockBuffer and shared pointer to BlockBuffer instead of allocating fresh blocks

struct TBlockDataHolder { size_t BufferBlockIndex; IBlockBufferPtr BlockBuffer; }

  1. Move blob allocation to TFlushActor

Also I removed allocator for fresh blocks, so it will not report statistics. However fresh blocks can be additionally allocated for unaligned tail and after tablet restart.

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

  • ⚫ tasks + storage + fastshard (job skipped)
  • ⚫ blockstore (job skipped)
  • ⚫ disk + manager (job skipped)
  • ⚫ filestore (job skipped)

@antonmyagkov
antonmyagkov force-pushed the users/myagkov/eliminate-copy-of-fresh-blocks branch from acd1644 to 84cc23f Compare October 7, 2026 11:39
@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

@antonmyagkov
antonmyagkov force-pushed the users/myagkov/eliminate-copy-of-fresh-blocks branch from 84cc23f to d7313b3 Compare October 7, 2026 11:47
@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🟢 linux-x86_64-relwithdebinfo target: cloud/tasks/,cloud/storage/,cloud/fastshard/ (test time: 279s): all tests PASSED for commit d7313b3.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
1733 1733 0 0 0 0 0

🟢 linux-x86_64-relwithdebinfo target: cloud/disk_manager/ (test time: 364s): all tests PASSED for commit d7313b3.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
1798 1798 0 0 0 0 0

🟢 linux-x86_64-relwithdebinfo target: cloud/blockstore/ (test time: 1528s): all tests PASSED for commit d7313b3.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
7611 7610 0 0 0 1 0

@antonmyagkov
antonmyagkov force-pushed the users/myagkov/eliminate-copy-of-fresh-blocks branch from d7313b3 to 7fefcd2 Compare October 7, 2026 12:36
@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

  • ⚫ tasks + storage + fastshard (job skipped)
  • ⚫ blockstore (job skipped)
  • ⚫ disk + manager (job skipped)
  • ⚫ filestore (job skipped)

@antonmyagkov
antonmyagkov force-pushed the users/myagkov/eliminate-copy-of-fresh-blocks branch from 7fefcd2 to 702213d Compare October 7, 2026 12:46
@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🟢 linux-x86_64-relwithdebinfo target: cloud/tasks/,cloud/storage/,cloud/fastshard/ (test time: 279s): all tests PASSED for commit 702213d.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
1733 1733 0 0 0 0 0

🟢 linux-x86_64-relwithdebinfo target: cloud/disk_manager/ (test time: 332s): all tests PASSED for commit 702213d.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
1798 1798 0 0 0 0 0

🟢 linux-x86_64-relwithdebinfo target: cloud/blockstore/ (test time: 1491s): all tests PASSED for commit 702213d.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
7611 7610 0 0 0 1 0

@antonmyagkov
antonmyagkov marked this pull request as ready for review October 7, 2026 14:00
@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🟢 linux-x86_64-relwithdebinfo target: cloud/tasks/,cloud/storage/,cloud/fastshard/ (test time: 273s): all tests PASSED for commit b0136be.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
1733 1733 0 0 0 0 0

🟢 linux-x86_64-relwithdebinfo target: cloud/disk_manager/ (test time: 319s): all tests PASSED for commit b0136be.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
1798 1798 0 0 0 0 0

🔴 linux-x86_64-relwithdebinfo target: cloud/blockstore/ (test time: 1535s): some tests FAILED for commit b0136be.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
7586 7584 0 1 0 1 0

🟢 linux-x86_64-relwithdebinfo target: cloud/blockstore/ (test time: 965s): all tests PASSED for commit b0136be.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
7611 7610 0 0 0 1 0

🔴 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 5460s) Test run completed, no test results found for commit b0136be.
Please check build logs.

🟢 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 5144s): all tests PASSED for commit b0136be.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
4915 4914 0 0 0 1 0

@github-actions

github-actions Bot commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🟢 linux-x86_64-relwithdebinfo target: cloud/tasks/,cloud/storage/,cloud/fastshard/ (test time: 266s): all tests PASSED for commit e31a409.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
1733 1733 0 0 0 0 0

🟢 linux-x86_64-relwithdebinfo target: cloud/disk_manager/ (test time: 314s): all tests PASSED for commit e31a409.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
1798 1798 0 0 0 0 0

🟢 linux-x86_64-relwithdebinfo target: cloud/blockstore/ (test time: 1651s): all tests PASSED for commit e31a409.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
7611 7610 0 0 0 1 0

@github-actions

github-actions Bot commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

@antonmyagkov antonmyagkov changed the title [Filestore] Eliminate copy of fresh blocks [Filestore] Move copy of fresh blocks to TFlushActor Oct 9, 2026
@github-actions

github-actions Bot commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

@antonmyagkov antonmyagkov added filestore Add this label to run only cloud/filestore build and tests on PR large-tests Launch large tests for PR msan Launch builds with memory sanitizer along with regular build tsan Launch builds with thread sanitizer along with regular build ubsan Launch builds with undefined behaviour sanitizer along with regular build asan Launch builds with address sanitizer along with regular build labels Oct 9, 2026
@github-actions

github-actions Bot commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🔴 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 7934s): some tests FAILED for commit f392ba1.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
4968 4966 0 1 0 1 0

🔴 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 33s): some tests FAILED for commit f392ba1.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
2 1 0 1 0 0 0

🔴 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 19s): some tests FAILED for commit f392ba1.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
2 1 0 1 0 0 0

@github-actions

github-actions Bot commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-release-ubsan have completed.

Tip

Planned checks for linux-x86_64-release-ubsan.

🟢 linux-x86_64-release-ubsan target: cloud/filestore/ (test time: 4681s): all tests PASSED for commit f392ba1.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
4258 4258 0 0 0 0 0

@github-actions

github-actions Bot commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-release-tsan have completed.

Tip

Planned checks for linux-x86_64-release-tsan.

🔴 linux-x86_64-release-tsan target: cloud/filestore/ (test time: 5430s): some tests FAILED for commit f392ba1.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
4198 4197 0 1 0 0 0

@github-actions

github-actions Bot commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-release-asan have completed.

Tip

Planned checks for linux-x86_64-release-asan.

🟢 linux-x86_64-release-asan target: cloud/filestore/ (test time: 4975s): all tests PASSED for commit f392ba1.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
4208 4208 0 0 0 0 0

@github-actions

github-actions Bot commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-release-msan have completed.

Tip

Planned checks for linux-x86_64-release-msan.

🟢 linux-x86_64-release-msan target: cloud/filestore/ (test time: 4948s): all tests PASSED for commit f392ba1.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
4208 4208 0 0 0 0 0

Comment thread cloud/filestore/libs/storage/tablet/model/blob_builder.cpp Outdated
Comment thread cloud/filestore/libs/storage/tablet/model/block.h Outdated
Comment thread cloud/filestore/libs/storage/tablet/model/fresh_blocks.cpp
Comment thread cloud/filestore/libs/storage/tablet/model/fresh_blocks.cpp
Comment thread cloud/filestore/libs/storage/tablet/tablet_actor_flush.cpp
Comment thread cloud/filestore/libs/storage/tablet/tablet_actor_writedata.cpp Outdated
@antonmyagkov antonmyagkov removed tsan Launch builds with thread sanitizer along with regular build msan Launch builds with memory sanitizer along with regular build ubsan Launch builds with undefined behaviour sanitizer along with regular build labels Oct 16, 2026
@antonmyagkov
antonmyagkov force-pushed the users/myagkov/eliminate-copy-of-fresh-blocks branch from bce152e to 6038244 Compare October 16, 2026 07:53
@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🔴 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 7781s): some tests FAILED for commit 6038244.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
5060 5058 0 1 0 1 0

🔴 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 32s): some tests FAILED for commit 6038244.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
2 1 0 1 0 0 0

🔴 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 17s): some tests FAILED for commit 6038244.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
2 1 0 1 0 0 0

Comment thread cloud/filestore/libs/storage/tablet/model/fresh_blocks.cpp Outdated
Comment thread cloud/filestore/libs/storage/tablet/tablet_actor_writedata.cpp Outdated
Comment thread cloud/filestore/libs/storage/model/block_buffer.cpp
@antonmyagkov
antonmyagkov force-pushed the users/myagkov/eliminate-copy-of-fresh-blocks branch from 6038244 to 57c03d8 Compare October 23, 2026 09:33
@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🔴 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 7465s): some tests FAILED for commit 57c03d8.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
5122 5120 0 1 0 1 0

🔴 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 34s): some tests FAILED for commit 57c03d8.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
2 1 0 1 0 0 0

🔴 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 17s): some tests FAILED for commit 57c03d8.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
2 1 0 1 0 0 0

@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🔴 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 7472s): some tests FAILED for commit 1bde286.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
5122 5120 0 1 0 1 0

🔴 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 36s): some tests FAILED for commit 1bde286.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
2 1 0 1 0 0 0

🔴 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 17s): some tests FAILED for commit 1bde286.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
2 1 0 1 0 0 0

@github-actions

github-actions Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🔴 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 7828s): some tests FAILED for commit a0a90fd.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
5122 5120 0 1 0 1 0

🔴 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 36s): some tests FAILED for commit a0a90fd.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
2 1 0 1 0 0 0

🔴 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 18s): some tests FAILED for commit a0a90fd.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
2 1 0 1 0 0 0

@antonmyagkov antonmyagkov changed the title [Filestore] Move copy of fresh blocks to TFlushActor [Filestore] Eliminate copy of fresh blocks and move blob allocation to to TFlushActor Oct 24, 2026
@antonmyagkov antonmyagkov changed the title [Filestore] Eliminate copy of fresh blocks and move blob allocation to to TFlushActor [Filestore] Eliminate allocation of fresh blocks and move blob allocation to to TFlushActor Oct 24, 2026
@antonmyagkov antonmyagkov changed the title [Filestore] Eliminate allocation of fresh blocks and move blob allocation to to TFlushActor [Filestore] Eliminate allocation of fresh blocks and move blob allocation to TFlushActor Oct 24, 2026
@antonmyagkov
antonmyagkov force-pushed the users/myagkov/eliminate-copy-of-fresh-blocks branch from a0a90fd to 5294bb9 Compare October 26, 2026 16:30
@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🔴 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 7141s): some tests FAILED for commit 5294bb9.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
5122 5115 0 6 0 1 0

🟢 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 34s): all tests PASSED for commit 5294bb9.

TESTS PASSED ERRORS FAILED FAILED BUILD SKIPPED MUTED?
12 12 0 0 0 0 0

@antonmyagkov antonmyagkov added the rebase Add this label if you want to rebase your PR for test run label Oct 27, 2026
@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Note

This is an automated comment that will be appended during run.

Important

Workload for linux-x86_64-relwithdebinfo is not finished yet. This comment will be updated after all workloads complete.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

filestore Add this label to run only cloud/filestore build and tests on PR large-tests Launch large tests for PR rebase Add this label if you want to rebase your PR for test run

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants