Skip to content

OptEnableOlapSortedRead has been added to ydb tools dump - #59404

Open
dorooleg wants to merge 2 commits into
ydb-platform:mainfrom
dorooleg:read_optimization
Open

dorooleg wants to merge 2 commits into
ydb-platform:mainfrom
dorooleg:read_optimization

Conversation

@dorooleg

Copy link
Copy Markdown
Contributor

Changelog entry

...

Description for reviewers

Включаю OptEnableOlapSortedRead если она доступна на сервере. Позволяет с ограниченным объемом памяти зачитывать данные с сервера в сортированном виде

@dorooleg
dorooleg requested review from a team as code owners November 14, 2026 09:58
Copilot AI balanced review requested due to automatic review settings November 14, 2026 09:58

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The dump can fail during rolling upgrades because actual reads lack an unknown-pragma retry fallback, and the tests do not verify sorted-read requests during Dump() itself.

2 open issues
What changed in this PR

This PR updates ydb tools dump to use primary-key-ordered reads for column tables when the server supports OptEnableOlapSortedRead, reducing memory needed for sorting.

Changes:

  • Probe and cache pragma support, retaining the previous read path when unsupported.
  • Prefetch the next result part while serializing the current part.
  • Add sorted-read, resume-query, and backup/restore tests, plus changelog entries.
File Description
ydb/​services/​ydb/​backup_ut/​ydb_backup_ut.cpp Adds query-plan checks and a column-table dump/restore test.
ydb/​library/​backup/​ut/​ut.cpp Tests unknown-pragma error detection.
ydb/​library/​backup/​backup.h Declares the pragma constant and compatibility helper.
ydb/​library/​backup/​backup.cpp Adds capability probing, pragma injection, and result prefetching.
ydb/​core/​kqp/​ut/​olap/​reading/​sorted_read_ut.cpp Tests dump-shaped sorted reads and parameterized resume queries.
ydb/​apps/​ydb/​CHANGELOG.md Documents sorted reads and prefetching for column-table dumps.

Review effort: Balanced

Comment thread ydb/library/backup/backup.cpp Resolved
Comment thread ydb/library/backup/backup.cpp Resolved
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

⚪ 2026-11-15 22:49:02 UTC Pre-commit check linux-x86_64-relwithdebinfo for 5d111cc has started.
⚪ 2026-11-15 22:49:24 UTC Artifacts will be uploaded here
⚪ 2026-11-15 22:50:48 UTC ya make is running...
🟡 2026-11-15 23:26:12 UTC Some tests failed, follow the links below. Going to retry failed tests...

Details

Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
9034 8958 0 2 71 3

⚪ 2026-11-15 23:26:19 UTC ya make is running... (failed tests rerun, try 2)
🟢 2026-11-15 23:33:27 UTC Tests successful.

Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
51 (only retried tests) 51 0 0 0 0

🟢 2026-11-15 23:33:31 UTC Build successful.

@dorooleg
dorooleg requested a review from XJIE6 November 16, 2026 10:32
Comment thread ydb/library/backup/backup.cpp Outdated
}

const auto issues = result.GetIssues().ToOneLineString();
if (IsUnknownOlapSortedReadPragma(TStringBuf(issues.data(), issues.size()))) {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Проблемы с отступами

@github-actions

Copy link
Copy Markdown
Contributor

🔴 Unable to merge your PR into the base branch. Please rebase or merge it with the base branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants