Repository navigation
Conversation
It is currently only used for source tasks and HashShuffle ignores it. One can override it with ydb.OverridePlanner, but it is quite fragile and user-unfriendly.
|
⚪ ⚪ Details
⚪
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
There was a problem hiding this comment.
AI Review Summary
Verdict: ❌ Critical: 0; Major: 1
Critical and Major issues
- Major | Medium:
MaxTasksPerStageis not propagated to the per-partition requests created byTKqpPartitionedExecuter(FillRequestFrom()), so BATCH UPDATE/DELETE executions run with the pragma cap silently disabled —ydb/core/kqp/executer_actor/kqp_partitioned_executer.cpp:56
Other findings
- Minor | Medium: stages with a forced task count (
PRAGMA ydb.OverridePlanner) take the FIXED branch and bypass the newMaxTasksPerStageclamp —ydb/core/kqp/executer_actor/kqp_tasks_graph.cpp:4442 - Minor | Medium: the clamp added to
CountComputeTasks()has no test coverage; existing pragma tests only exercise external-source stages —ydb/core/kqp/executer_actor/kqp_tasks_graph.cpp:4448
This review was generated automatically. Critical and Major issues require attention; Minor and Nit findings are advisory.
If this comment was useful, please give it a 👍 — it helps us improve the review bot.
| } else { | ||
| auto [newPartitionCount, _] = GetMaxTasksAggregation(stageInfo, inputTasks, nodesCount); | ||
| partitionsCount = std::max(newPartitionCount, partitionsCount); | ||
| if (MaxTasksPerStage != 0) { |
There was a problem hiding this comment.
Severity: Minor
Confidence: Medium
The clamp added here has no test coverage. The existing PRAGMA ydb.MaxTasksPerStage tests (kqp_federated_query_ut.cpp, kqp_s3_plan_ut.cpp, streaming_*) only exercise external-source stages, where the limit was already honored via the compiler before this change; nothing asserts that a hash-shuffle stage with the pragma set produces at most MaxTasksPerStage tasks in CountComputeTasks(). A test over a multi-shard table (e.g. a group-by/join query with PRAGMA ydb.MaxTasksPerStage = "2" verifying the produced task count via stats/plan) is needed to keep the changed behavior safe.
Finding outside the inline diffydb/core/kqp/executer_actor/kqp_partitioned_executer.cpp:56 Severity: Major
Suggested fix: add |
Finding outside the inline diffydb/core/kqp/executer_actor/kqp_tasks_graph.cpp:4442 Severity: Minor The new If the pragma is meant to be an absolute cap, apply it to the FIXED branch as well (or validate/intersect the two limits at compile time in the compiler); otherwise the precedence between the two pragmas is worth documenting. |
It is currently only used for source tasks and HashShuffle ignores it. One can override it with ydb.OverridePlanner, but it is quite fragile and user-unfriendly (they must guess tx and stage, which may wildly variate depending on query)
Changelog entry
...
Description for reviewers
...