Repository navigation
Benchmark parsed YQL requests at the KQP logical-root boundary - #59987
asmyasnikov wants to merge 7 commits into
Conversation
✅ Documentation buildRevision built successfully |
|
⚪
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
✅ Documentation buildRevision built successfully |
✅ Documentation buildRevision built successfully |
There was a problem hiding this comment.
🔵 Needs a closer look
The compiler-resume APIs reconstruct state outside normal frontend setup, requiring a KQP compiler maintainer to validate imported-KQL context parity beyond what static inspection and covered fixtures establish.
No open issues 🎉
What changed in this PR
Adds an offline benchmark comparing four query representations through the legacy KQP frontend, stopping at logical-root construction in the new rule-based optimizer. It measures server compilation and producer serialization costs while checking request and logical-output equivalence.
Changes:
- Adds text, grammar AST, translated YQL AST, and KQL expression-graph inputs, with timing, verification, and regression tests.
- Exposes compiler entry points and callbacks for replaying and capturing intermediate stages.
- Guards Bison’s GNU-only linker flag on Darwin.
| File | Description |
|---|---|
ydb/tests/tools/ya.make |
Registers the benchmark tool. |
ydb/tests/tools/parsed_query_bench/ya.make |
Defines the executable and fingerprint generation. |
ydb/tests/tools/parsed_query_bench/ut/ya.make |
Registers Python integration tests. |
ydb/tests/tools/parsed_query_bench/ut/test_legacy.py |
Tests equivalence, timing output, and invalid inputs. |
ydb/tests/tools/parsed_query_bench/syntax.h |
Declares syntax conversion interfaces. |
ydb/tests/tools/parsed_query_bench/syntax.cpp |
Implements grammar and YQL AST conversion. |
ydb/tests/tools/parsed_query_bench/README.md |
Documents stages, timing boundaries, and usage. |
ydb/tests/tools/parsed_query_bench/proto/ya.make |
Builds experimental protobuf schemas. |
ydb/tests/tools/parsed_query_bench/proto/parsed_query.proto |
Defines grammar and YQL AST payloads. |
ydb/tests/tools/parsed_query_bench/proto/kql_query_list.proto |
Defines serialized KQL expression graphs. |
ydb/tests/tools/parsed_query_bench/proto/execute_query.proto |
Defines the benchmark request envelope. |
ydb/tests/tools/parsed_query_bench/measurement.h |
Provides bounded decoding and timing helpers. |
ydb/tests/tools/parsed_query_bench/main.cpp |
Implements export, verification, and measurement commands. |
ydb/tests/tools/parsed_query_bench/kql_codec.h |
Declares KQL graph serialization interfaces. |
ydb/tests/tools/parsed_query_bench/kql_codec.cpp |
Encodes, restores, and validates KQL graphs. |
ydb/tests/tools/parsed_query_bench/grammar_fingerprint.py |
Generates a parser-schema fingerprint. |
ydb/tests/tools/parsed_query_bench/engine.h |
Defines benchmark engine and result interfaces. |
ydb/tests/tools/parsed_query_bench/engine.cpp |
Implements offline catalog binding and compiler replay. |
ydb/core/kqp/tracing/kqp_compile_trace.h |
Adds intermediate-stage capture callbacks. |
ydb/core/kqp/opt/rbo/kqp_rbo_transformer.cpp |
Captures logical roots and stops benchmark compilation. |
ydb/core/kqp/host/kqp_runner.cpp |
Exposes optimizer entry and KQL capture boundaries. |
ydb/core/kqp/host/kqp_host.h |
Declares the KQL inspection entry point. |
ydb/core/kqp/host/kqp_host.cpp |
Supports imported inputs and translation timing callbacks. |
ydb/core/kqp/host/kqp_host_impl.h |
Declares the runner’s new optimizer entry point. |
ydb/core/kqp/common/simple/query_ref.h |
Adds an optional translated-AST reference. |
contrib/tools/bison/ya.make |
Excludes the GNU linker flag on Darwin. |
contrib/tools/bison/.yandex_meta/__init__.py |
Preserves the Darwin guard during regeneration. |
Review effort: Balanced
Measure four ways to supply a read-only query to the legacy KQP SQL frontend on main with
EnableNewRBO. This is the main-branch counterpart to the NewParser benchmark.The text baseline enters the stock
PrepareGenericQuery(SQL). The grammar stage supplies the generatedTSQLv1ParserASTand its original tokens toSqlASTToYql. A translated YQL AST resumes beforeCompileExpr. The latekql_query_listinput restores the expression DAG produced byBuildKqlQuery, rebinds table metadata against the supplied catalog and enters the same RBO pipeline. The host entry points let the tool resume at these actual compiler boundaries and the callbacks expose intermediate results that the ordinary prepare API consumes internally.Each server sample starts immediately before decoding a complete in-memory
ExecuteQueryRequestand stops inTKqpNewRBOTransformer::DoTransform(), just beforeif (Roots.size() > 1). At this point the compiler has set up the logical roots and calledComputeParents(). A compilation-local observer records the timestamp on the calling thread and terminates the replay before statistics or optimizer rules.The benchmark warms the offline catalog, verifies the full request envelope and the logical outputs against SQL, rotates input formats and records wall and thread CPU time for every iteration. A separate producer run starts with rendered SQL and ends after serialization of the complete request. The experimental messages live under the test tool. Field 4 holds the public YQL text input. Field 5 holds
ParsedQuery, whoseoneof stageselects a protobuf representation of an intermediate compiler result. Timer endpointPublic corpus results
Cells give median wall milliseconds per request. A dash means the series has no verified measurement for that input and format.
On the 90-join query,
texttook 184.713 ms at p95. Mean CPU was 178.447 ms on the server and 0.004 ms on the producer. Its full request used 12,241 bytes.On the 90-join query,
kql_query_listtook 57.248 ms at p95. Mean CPU was 55.447 ms on the server and 141.425 ms on the producer. Its full request used 123,564 bytes.The table below compares the stock
PrepareGenericQueryentry points for the same public text request. Pairing requires matching ordered output names and types and rejects NewParser fallback. 6 of 10 public inputs qualified.These measurements used one Apple M3 Pro with 36 GiB RAM and macOS 26.6.2 arm64. The two executables ran sequentially. A cross-branch pair was excluded for the five-table join (7 versus 11 output columns) and for TPC-DS Q88 (different first output column). The main replay could not annotate DateTime2.GetYear for TPC-H Q8. Its forced YqlSelect translation rejected ONLY/SEMI for TPC-DS Q14.
Reproduction
The build also guards a GNU-only linker flag in the transitive Bison host tool on Darwin. Each public result uses the schema and query fixtures for the selected workload. Commands, pipeline and input formats
The public 96-join control uses the same SQL generator as the upstream 90-join test, with six more LEFT JOINs and six more tables. From the build checkout, run:
The complete text request has SHA-256
46c0af5998103d3da27a11ca528c6bf7436c1aecfccafc188de91eb8fb3ed9c2.