profile_tool : support multiple input files by proller · Pull Request #7441 · ydb-platform/nbs

proller · 2026-10-05T23:33:43Z

Adds support for multiple profile logs, sorted by filename timestamp or mtime. Skips files outside the requested time range, with --all-files to bypass pruning and --ignore-errors to continue past unreadable logs. Includes multi-file masking and regression tests.

github-actions · 2026-10-05T23:34:48Z

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

✅ tasks + storage + fastshard
✅ blockstore
✅ disk + manager
❌ filestore (job failed before reporting completion)

🟢 linux-x86_64-relwithdebinfo target: cloud/tasks/,cloud/storage/,cloud/fastshard/ (test time: 273s): all tests PASSED for commit 1badea8.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1733	1733	0	0	0	0	0

🟢 linux-x86_64-relwithdebinfo target: cloud/disk_manager/ (test time: 323s): all tests PASSED for commit 1badea8.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1773	1773	0	0	0	0	0

🟢 linux-x86_64-relwithdebinfo target: cloud/blockstore/ (test time: 1614s): all tests PASSED for commit 1badea8.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
7611	7610	0	0	0	1	0

🔴 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 5274s): some tests FAILED for commit 1badea8.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
4905	4903	0	0	1	1	0

🔴 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 4967s): some tests FAILED for commit 1badea8.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
4905	4903	0	0	1	1	0

…le.files

github-actions · 2026-10-06T15:14:07Z

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🟢 linux-x86_64-relwithdebinfo target: cloud/tasks/,cloud/storage/,cloud/fastshard/ (test time: 277s): all tests PASSED for commit 248e11b.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1736	1736	0	0	0	0	0

🟢 linux-x86_64-relwithdebinfo target: cloud/disk_manager/ (test time: 328s): all tests PASSED for commit 248e11b.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1798	1798	0	0	0	0	0

🟢 linux-x86_64-relwithdebinfo target: cloud/blockstore/ (test time: 1573s): all tests PASSED for commit 248e11b.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
7627	7626	0	0	0	1	0

🔴 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 5408s) Test run completed, no test results found for commit 248e11b.
Please check build logs.

🟢 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 4910s): all tests PASSED for commit 248e11b.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
4924	4923	0	0	0	1	0

github-actions · 2026-10-06T21:57:50Z

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🟢 linux-x86_64-relwithdebinfo target: cloud/tasks/,cloud/storage/,cloud/fastshard/ (test time: 275s): all tests PASSED for commit 99f67a1.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1736	1736	0	0	0	0	0

🟢 linux-x86_64-relwithdebinfo target: cloud/disk_manager/ (test time: 314s): all tests PASSED for commit 99f67a1.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1798	1798	0	0	0	0	0

🟢 linux-x86_64-relwithdebinfo target: cloud/blockstore/ (test time: 1578s): all tests PASSED for commit 99f67a1.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
7627	7626	0	0	0	1	0

🟢 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 5200s): all tests PASSED for commit 99f67a1.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
4924	4923	0	0	0	1	0

…le.files

…le, and it has no tests

…bytes_access.cpp)

github-actions · 2026-10-07T16:03:58Z

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🟢 linux-x86_64-relwithdebinfo target: cloud/tasks/,cloud/storage/,cloud/fastshard/ (test time: 276s): all tests PASSED for commit 789eb2f.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1744	1744	0	0	0	0	0

🟢 linux-x86_64-relwithdebinfo target: cloud/disk_manager/ (test time: 323s): all tests PASSED for commit 789eb2f.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1821	1821	0	0	0	0	0

🟢 linux-x86_64-relwithdebinfo target: cloud/blockstore/ (test time: 1604s): all tests PASSED for commit 789eb2f.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
7629	7628	0	0	0	1	0

🟢 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 5032s): all tests PASSED for commit 789eb2f.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
4981	4980	0	0	0	1	0

github-actions · 2026-10-09T14:29:45Z

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🟢 linux-x86_64-relwithdebinfo target: cloud/tasks/,cloud/storage/,cloud/fastshard/ (test time: 271s): all tests PASSED for commit 54ddfdc.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1744	1744	0	0	0	0	0

🟢 linux-x86_64-relwithdebinfo target: cloud/disk_manager/ (test time: 336s): all tests PASSED for commit 54ddfdc.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1821	1821	0	0	0	0	0

🟢 linux-x86_64-relwithdebinfo target: cloud/blockstore/ (test time: 1826s): all tests PASSED for commit 54ddfdc.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
7629	7628	0	0	0	1	0

🟢 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 5402s): all tests PASSED for commit 54ddfdc.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
4981	4980	0	0	0	1	0

🟢 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 5144s): all tests PASSED for commit 54ddfdc.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
4981	4980	0	0	0	1	0

…le.files

github-actions · 2026-10-09T18:14:46Z

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🟢 linux-x86_64-relwithdebinfo target: cloud/disk_manager/ (test time: 303s): all tests PASSED for commit 18bd192.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1836	1836	0	0	0	0	0

🟢 linux-x86_64-relwithdebinfo target: cloud/tasks/,cloud/storage/,cloud/fastshard/ (test time: 281s): all tests PASSED for commit 18bd192.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1745	1745	0	0	0	0	0

🟢 linux-x86_64-relwithdebinfo target: cloud/blockstore/ (test time: 1813s): all tests PASSED for commit 18bd192.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
7643	7642	0	0	0	1	0

🔴 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 5460s) Test run completed, no test results found for commit 18bd192.
Please check build logs.

🟢 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 5168s): all tests PASSED for commit 18bd192.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
5005	5004	0	0	0	1	0

SvartMetal · 2026-10-16T17:18:07Z

+                     << Endl;
+                return 0;
+            }
+            return 1;


Why are we using C-style error codes here? NProto::TError would be more natural

All interfaces around uses int return type -
int IterateEventLog(..
int Execute()
And with converting type need to write strange return transforms:

int Execute() override { TEventProcessor processor(Filter); return HasError(ProcessProfileLogs(processor)) ? 1 : 0; }

….files

…le.files

github-actions · 2026-10-19T17:14:16Z

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🚫 tasks + storage + fastshard (cancelled or timed out before reporting completion)
🚫 blockstore (cancelled or timed out before reporting completion)
🚫 disk + manager (cancelled or timed out before reporting completion)
🚫 filestore (cancelled or timed out before reporting completion)

github-actions · 2026-10-19T17:35:25Z

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🟢 linux-x86_64-relwithdebinfo target: cloud/tasks/,cloud/storage/,cloud/fastshard/ (test time: 276s): all tests PASSED for commit 45911bd.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1754	1754	0	0	0	0	0

🟢 linux-x86_64-relwithdebinfo target: cloud/disk_manager/ (test time: 306s): all tests PASSED for commit 45911bd.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1849	1849	0	0	0	0	0

🟢 linux-x86_64-relwithdebinfo target: cloud/blockstore/ (test time: 1551s): all tests PASSED for commit 45911bd.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
7661	7660	0	0	0	1	0

🟢 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 5133s): all tests PASSED for commit 45911bd.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
5071	5070	0	0	0	1	0

…le.files

github-actions · 2026-10-20T16:34:50Z

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🚫 tasks + storage + fastshard (cancelled or timed out before reporting completion)
🚫 blockstore (cancelled or timed out before reporting completion)
🚫 disk + manager (cancelled or timed out before reporting completion)
🚫 filestore (cancelled or timed out before reporting completion)

github-actions · 2026-10-20T16:42:50Z

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🚫 tasks + storage + fastshard (cancelled or timed out before reporting completion)
🚫 blockstore (cancelled or timed out before reporting completion)
🚫 disk + manager (cancelled or timed out before reporting completion)
🚫 filestore (cancelled or timed out before reporting completion)

github-actions · 2026-10-20T16:54:38Z

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🚫 tasks + storage + fastshard (cancelled or timed out before reporting completion)
🚫 blockstore (cancelled or timed out before reporting completion)
🚫 disk + manager (cancelled or timed out before reporting completion)
🚫 filestore (cancelled or timed out before reporting completion)

github-actions · 2026-10-20T17:09:58Z

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

🟢 linux-x86_64-relwithdebinfo target: cloud/tasks/,cloud/storage/,cloud/fastshard/ (test time: 270s): all tests PASSED for commit 0c087e5.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1782	1782	0	0	0	0	0

🟢 linux-x86_64-relwithdebinfo target: cloud/disk_manager/ (test time: 307s): all tests PASSED for commit 0c087e5.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1849	1849	0	0	0	0	0

🟢 linux-x86_64-relwithdebinfo target: cloud/blockstore/ (test time: 1689s): all tests PASSED for commit 0c087e5.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
7670	7669	0	0	0	1	0

🔴 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 5449s) Test run completed, no test results found for commit 0c087e5.
Please check build logs.

🟢 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 4949s): all tests PASSED for commit 0c087e5.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
5067	5066	0	0	0	1	0

…le.files

github-actions · 2026-10-23T15:31:31Z

Note

This is an automated comment that will be appended during run.

Note

All workloads for linux-x86_64-relwithdebinfo have completed.

Tip

Planned checks for linux-x86_64-relwithdebinfo.

✅ tasks + storage + fastshard
✅ blockstore
✅ disk + manager
❌ filestore (job failed before reporting completion)

🟢 linux-x86_64-relwithdebinfo target: cloud/tasks/,cloud/storage/,cloud/fastshard/ (test time: 279s): all tests PASSED for commit 0c8112b.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1798	1798	0	0	0	0	0

🟢 linux-x86_64-relwithdebinfo target: cloud/disk_manager/ (test time: 321s): all tests PASSED for commit 0c8112b.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
1849	1849	0	0	0	0	0

🔴 linux-x86_64-relwithdebinfo target: cloud/blockstore/ (test time: 1437s): some tests FAILED for commit 0c8112b.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
7682	7679	0	2	0	1	0

🔴 linux-x86_64-relwithdebinfo target: cloud/blockstore/ (test time: 27s): some tests FAILED for commit 0c8112b.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
4	2	0	2	0	0	0

🔴 linux-x86_64-relwithdebinfo target: cloud/blockstore/ (test time: 14s): some tests FAILED for commit 0c8112b.

TESTS	PASSED	ERRORS	FAILED	FAILED BUILD	SKIPPED	MUTED^?
4	2	0	2	0	0	0

🔴 linux-x86_64-relwithdebinfo target: cloud/filestore/ (test time: 5461s) Test run completed, no test results found for commit 0c8112b.
Please check build logs.

profile.files

1badea8

proller added 2 commits October 6, 2026 17:11

fix

2840b3b

Merge remote-tracking branch 'upstream/main' into users/proller/profi…

248e11b

…le.files

better

99f67a1

proller added 8 commits October 7, 2026 16:25

Merge remote-tracking branch 'upstream/main' into users/proller/profi…

ced1a9e

…le.files

Simpler

e390fed

SelectProfileLogFiles is ~90 lines of which a large part is unreachab…

237f82b

…le, and it has no tests

--ignore-errors

d91d288

GetProfileLogEndTime returns TMaybe yet throws TFileError

a6f82c3

Since/Until promoted to members for no reason (dump_events.cpp, find_…

2dd4914

…bytes_access.cpp)

--all-files

c3205d1

progress

789eb2f

proller changed the title ~~profile_tool : support muliple files~~ profile_tool : support multiple files Oct 7, 2026

proller changed the title ~~profile_tool : support multiple files~~ profile_tool : support multiple input files Oct 7, 2026

SvartMetal self-requested a review October 8, 2026 21:02

SvartMetal reviewed Oct 8, 2026

View reviewed changes

Comment thread cloud/filestore/tools/analytics/profile_tool/main.cpp Outdated

Not required profile-log

54ddfdc

SvartMetal reviewed Oct 9, 2026

View reviewed changes

Comment thread cloud/filestore/tools/analytics/profile_tool/lib/time_range.cpp

SvartMetal reviewed Oct 9, 2026

View reviewed changes

Comment thread cloud/filestore/tools/analytics/profile_tool/lib/command_ut.cpp

proller added 3 commits October 9, 2026 19:07

fix

2fec1f1

otbivkas

efa861d

Merge remote-tracking branch 'upstream/main' into users/proller/profi…

18bd192

…le.files

SvartMetal requested changes Oct 16, 2026

View reviewed changes

At least one profile log is required

ffd4a19

proller added 10 commits October 19, 2026 18:28

masksensitivedata now honors --ignore-errors

3c6a923

Changed --profile-log to .AppendTo(&ProfileLogPaths)

d0b625c

Remove SetFreeArgDefaultTitle

12313ab

Moved iterator creation before the loop.

6151cc0

style

1dbba65

in -> profileLogFiles

fd49566

Clean

fa62a4f

Made pruning opt-in with --prune-by-time

26ddb30

Merge remote-tracking branch 'origin/main' into users/proller/profile…

127dbc2

….files

Merge remote-tracking branch 'upstream/main' into users/proller/profi…

cac53e7

…le.files

proller added 4 commits October 19, 2026 19:18

Clean

1319ee5

Moved ProcessProfileLog into the anonymous namespace

c3c93d1

Simplified suffix parsing with TryRSplit

cc69025

Replaced in-place compaction with a new result vector

45911bd

proller added 2 commits October 20, 2026 18:33

Read start/end

9887987

Merge remote-tracking branch 'upstream/main' into users/proller/profi…

fcceb45

…le.files

Fixed

9144af7

clean

0c087e5

proller added 2 commits October 23, 2026 16:56

Merge remote-tracking branch 'upstream/main' into users/proller/profi…

b702840

…le.files

Fix ordering when mixed dated/current files

0c8112b

Conversation

proller commented Oct 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Oct 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Oct 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SvartMetal Oct 16, 2026

Choose a reason for hiding this comment

Uh oh!

proller Oct 19, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

proller commented Oct 5, 2026 •

edited

Loading

github-actions Bot commented Oct 5, 2026 •

edited

Loading

github-actions Bot commented Oct 6, 2026 •

edited

Loading

github-actions Bot commented Oct 6, 2026 •

edited

Loading

github-actions Bot commented Oct 7, 2026 •

edited

Loading

github-actions Bot commented Oct 9, 2026 •

edited

Loading

github-actions Bot commented Oct 9, 2026 •

edited

Loading

github-actions Bot commented Oct 19, 2026 •

edited

Loading

github-actions Bot commented Oct 19, 2026 •

edited

Loading

github-actions Bot commented Oct 20, 2026 •

edited

Loading

github-actions Bot commented Oct 20, 2026 •

edited

Loading

github-actions Bot commented Oct 20, 2026 •

edited

Loading

github-actions Bot commented Oct 20, 2026 •

edited

Loading

github-actions Bot commented Oct 23, 2026 •

edited

Loading