Skip to content

Reject dumps of tables with generated columns - #59402

Merged
dahbka-lis merged 4 commits into
ydb-platform:mainfrom
dahbka-lis:fix-dump-generated-columns
Nov 18, 2026
Merged

dahbka-lis merged 4 commits into
ydb-platform:mainfrom
dahbka-lis:fix-dump-generated-columns

Conversation

@dahbka-lis

@dahbka-lis dahbka-lis commented Nov 14, 2026 •

Copy link
Copy Markdown
Member

Changelog entry

Reject dumps of tables with STORED or VIRTUAL generated columns.

Description for reviewers

Dump saves table schemas as Ydb::Table::CreateTableRequest, which does not support generated-column definitions. Return UNSUPPORTED for these dumps to avoid losing schema information.

  • Validate included tables before copying or exporting them.
  • Preserve regular DescribeTable, backup exclusions, and direct changefeed dumps.
  • Reuse table descriptions where possible to avoid duplicate requests.
  • Add tests for rejection and successful dumps of Serial and literal-default columns.

@dahbka-lis
dahbka-lis requested a review from a team as a code owner November 14, 2026 09:36
Copilot AI balanced review requested due to automatic review settings November 14, 2026 09:36
@github-actions

github-actions Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

⚪ 2026-11-14 09:39:14 UTC Pre-commit check linux-x86_64-relwithdebinfo for 05dc7c0 has started.
⚪ 2026-11-14 09:39:54 UTC Artifacts will be uploaded here
⚪ 2026-11-14 09:41:20 UTC ya make is running...
⚫ 2026-11-14 09:59:19 UTC Check cancelled

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Validation runs before exclusion filtering and during metadata-only lookups, causing excluded generated-column tables to abort backups and incorrectly rejecting direct changefeed dumps.

1 open issue
What changed in this PR

This PR adds coordinated CLI and server checks to reject table dumps containing STORED or VIRTUAL generated columns, while preserving ordinary DescribeTable behavior.

Changes:

  • Marks backup table descriptions using existing request metadata.
  • Returns UNSUPPORTED for dump requests involving generated columns.
  • Adds CLI rejection coverage across dump modes and a regular-description regression test.
File Description
ydb/​tests/​functional/​ydb_cli/​test_ydb_backup.py Tests generated-column dump rejection across column kinds and dump modes.
ydb/​library/​backup/​backup.cpp Tags table descriptions with the dump request type.
ydb/​core/​kqp/​ut/​scheme/​kqp_generated_columns_ut.cpp Checks that ordinary descriptions still support generated columns with WithSetVal.
ydb/​core/​grpc_services/​rpc_describe_table.cpp Rejects dump-tagged descriptions of tables with generated columns.

Review effort: Balanced

Comment thread ydb/library/backup/backup.cpp Resolved
@github-actions

github-actions Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

⚪ 2026-11-14 10:19:18 UTC Pre-commit check linux-x86_64-relwithdebinfo for 1131de3 has started.
⚪ 2026-11-14 10:19:57 UTC Artifacts will be uploaded here
⚪ 2026-11-14 10:21:22 UTC ya make is running...
🟡 2026-11-14 11:37:48 UTC Some tests failed, follow the links below. Going to retry failed tests...

Details

Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
54973 53163 0 5 1800 5

⚪ 2026-11-14 11:38:03 UTC ya make is running... (failed tests rerun, try 2)
🟢 2026-11-14 11:45:26 UTC Tests successful.

Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
73 (only retried tests) 72 0 0 0 1

🟢 2026-11-14 11:45:30 UTC Build successful.
🟡 2026-11-14 11:45:53 UTC ydbd size 2.9 GiB changed* by +16.2 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 7c1bcec merge: 1131de3 diff diff %
ydbd size 3 045 473 272 Bytes 3 062 488 696 Bytes +16.2 MiB +0.559%
ydbd stripped size 643 672 192 Bytes 645 514 816 Bytes +1.8 MiB +0.286%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@robot-vibe-db robot-vibe-db Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Review Summary

Verdict: ✅ No Critical or Major issues found

Critical and Major issues

No Critical or Major issues found.

Other findings

  • Minor | Medium: Redundant extra DescribeTable RPC per table on the dump critical path — ydb/library/backup/backup.cpp:1227
  • Minor | Medium: Generated-column dump gate covers only the row-table branch; column tables return earlier and are never checked — ydb/core/grpc_services/rpc_describe_table.cpp:180
  • Minor | Medium: New unit test asserts only success and column count; no assertion on set_val content and no dump-request-type case — ydb/core/kqp/ut/scheme/kqp_generated_columns_ut.cpp:1044
  • Minor | Medium: Missing negative control that Serial (sequence-default) and literal-default columns still dump with RequestType("dump") — ydb/tests/functional/ydb_cli/test_ydb_backup.py:406

This review was generated automatically. Critical and Major issues require attention; Minor and Nit findings are advisory.
If this comment was useful, please give it a 👍 — it helps us improve the review bot.

Comment thread ydb/library/backup/backup.cpp Resolved
@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

⚪ 2026-11-16 12:34:37 UTC Pre-commit check linux-x86_64-relwithdebinfo for 8915b13 has started.
⚪ 2026-11-16 12:35:32 UTC Artifacts will be uploaded here
⚪ 2026-11-16 12:36:59 UTC ya make is running...
🟡 2026-11-16 14:45:47 UTC Some tests failed, follow the links below. Going to retry failed tests...

Details

Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
55077 53277 0 4 1791 5

⚪ 2026-11-16 14:46:04 UTC ya make is running... (failed tests rerun, try 2)
🟢 2026-11-16 14:53:18 UTC Tests successful.

Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
74 (only retried tests) 73 0 0 0 1

🟢 2026-11-16 14:53:22 UTC Build successful.
🟡 2026-11-16 14:53:48 UTC ydbd size 2.9 GiB changed* by +18.3 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 7c1bcec merge: 8915b13 diff diff %
ydbd size 3 045 473 272 Bytes 3 064 613 920 Bytes +18.3 MiB +0.628%
ydbd stripped size 643 672 192 Bytes 645 822 080 Bytes +2.1 MiB +0.334%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@dahbka-lis
dahbka-lis enabled auto-merge (squash) November 16, 2026 13:04
Comment thread ydb/library/backup/backup.cpp Resolved
Comment thread ydb/library/backup/backup.cpp Resolved
Comment thread ydb/library/backup/backup.cpp Resolved
@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

⚪ 2026-11-18 08:50:47 UTC Pre-commit check linux-x86_64-relwithdebinfo for f61dd2e has started.
⚪ 2026-11-18 08:51:08 UTC Artifacts will be uploaded here
⚪ 2026-11-18 08:52:33 UTC ya make is running...
🟡 2026-11-18 10:10:58 UTC Some tests failed, follow the links below. Going to retry failed tests...

Details

Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
55312 53506 0 5 1795 6

⚪ 2026-11-18 10:11:15 UTC ya make is running... (failed tests rerun, try 2)
🟡 2026-11-18 10:12:37 UTC Some tests failed, follow the links below. Going to retry failed tests...

Details

Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
420 (only retried tests) 417 0 1 0 2

⚪ 2026-11-18 10:12:42 UTC ya make is running... (failed tests rerun, try 3)
🟢 2026-11-18 10:13:38 UTC Tests successful.

Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
402 (only retried tests) 401 0 0 0 1

🟢 2026-11-18 10:13:43 UTC Build successful.
🟡 2026-11-18 10:14:09 UTC ydbd size 2.9 GiB changed* by +27.0 MiB, which is >= 100.0 KiB vs main: Warning

ydbd size dash main: 7c1bcec merge: f61dd2e diff diff %
ydbd size 3 045 473 272 Bytes 3 073 833 120 Bytes +27.0 MiB +0.931%
ydbd stripped size 643 672 192 Bytes 646 947 456 Bytes +3.1 MiB +0.509%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@dahbka-lis
dahbka-lis requested a review from pnv1 November 18, 2026 09:29
@dahbka-lis
dahbka-lis merged commit dd7e24d into ydb-platform:main Nov 18, 2026
12 checks passed
@ydbot

ydbot commented Nov 18, 2026

Copy link
Copy Markdown
Collaborator

Backport

To backport this PR, click the button next to the target branch and then click "Run workflow" in the Run Actions UI.

Branch Run
stable-26-3-1, stable-26-4-1 ▶  Backport
stable-26-4-1 ▶  Backport

▶  Backport manual

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants