Repository navigation
fix(query_engine): surface plan execution errors - #1814
fengelniederhammer wants to merge 6 commits into
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
The plan-completion wait can hang indefinitely, bypassing timeouts and bounded cancellation, while the new query exception is incorrectly mapped to RuntimeError for Python callers.
2 open issues
What changed in this PR
This PR makes RhyDB’s query engine surface Arrow plan execution failures instead of silently returning empty or truncated results.
Changes:
- Checks the plan’s final status before completing output.
- Tags cast failures as query errors and translates them at the HTTP boundary.
- Adds error-classification tests and updates cast regressions to expect failures.
| File | Description |
|---|---|
src/rhydb/query_engine/scalar_expressions/cast.test.cpp |
Expects errors for invalid column casts. |
src/rhydb/query_engine/scalar_expressions/cast.cpp |
Routes casts through the error-tagging wrapper. |
src/rhydb/query_engine/query_plan.test.cpp |
Adds a regression test for execution failures. |
src/rhydb/query_engine/query_plan.cpp |
Checks final status and distinguishes query errors. |
src/rhydb/query_engine/query_execution_error.test.cpp |
Tests status tagging and exception messages. |
src/rhydb/query_engine/query_execution_error.h |
Declares query-error tags and exceptions. |
src/rhydb/query_engine/query_execution_error.cpp |
Implements query-error tagging and detection. |
src/rhydb/query_engine/exec_node/string_to_bool_expression.cpp |
Tags boolean parsing failures as query errors. |
src/rhydb/query_engine/exec_node/cast_expression.h |
Declares the tagged cast wrapper. |
src/rhydb/query_engine/exec_node/cast_expression.cpp |
Implements and registers the tagged cast kernel. |
src/rhydb/query_engine/exec_node/arrow_util.h |
Declares shared kernel-input conversion. |
src/rhydb/query_engine/exec_node/arrow_util.cpp |
Implements scalar-to-array input conversion. |
app/src/query_response.cpp |
Translates query execution exceptions to BadRequest. |
Review effort: Balanced
There was a problem hiding this comment.
🟡 Changes recommended
An insert source failure after a buffer flush can leave partial writes in the target table without updating its data version, requiring rollback or staging and a regression test.
1 open issue
2 resolved since last review
Review effort: Balanced
| ) | ||
| ); | ||
| } | ||
| ARROW_RETURN_NOT_OK(finished.status()); |
There was a problem hiding this comment.
AI reasoning:
Valid finding. Confirmed the chain:
TableInsertSink::writeBatch → NdjsonInsertStream::insertAll → TableInserter::insert, where at table_inserter.cpp:332-333 the input buffer is flushed via table->bulkInsert(...) once it reaches COLUMN_CHUNK_SIZE (65536 rows) — i.e. rows are appended to the live table before commit(). InsertCommand::execute throws out of executeAndWrite on an execution failure, so finish()/commit() (finalize()+validate()) and updateDataVersion() never run. A failed insert producing >65536 rows can therefore leave flushed chunks in the target table, un-finalized, under the old data version, while the caller sees an exception. The doc comment on TableInsertSink ("leaving the insert uncommitted") only holds for inserts smaller than one chunk.
Why it surfaces now: this is a pre-existing hole, but previously a column-value cast failure was silently swallowed (empty/truncated result, no throw), so updateDataVersion() still ran and there was no observable partial-write symptom. This PR makes such failures throw, which exposes the partial insert.
A proper fix is transactional staging + rollback of insert writes, which RhyDB doesn't have today (updateDataVersion() is just a timestamp bump, no snapshot). That's a larger change than this PR's scope ("surface plan execution errors"), so I'll track it as a separate issue rather than fold it in here, and fix the misleading doc comment.
|
This is a preview of the changelog of the next release. If this branch is not up-to-date with the current main branch, the changelog may not be accurate. Rebase your branch on the main branch to get the most accurate changelog. Note that this might contain changes that are on main, but not yet released. Changelog: 0.17.6 (2026-11-17)FeaturesBug Fixes |
eed6f64 to
19883f7
Compare


Summary
PR Checklist