Repository navigation
Simplify execution and add upstream SQLLogicTest coverage - #28
Merged
Merged
Conversation
jakthom
marked this pull request as draft
September 9, 2026 20:08
jakthom
marked this pull request as ready for review
September 10, 2026 00:03
Collaborator
Author
|
Weeee lfg |
This was referenced Sep 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Go execution duplicated statement ownership and cleanup, and the bridge had limited SQL coverage. This change consolidates execution, splits Rust by responsibility, and runs the pinned DataFusion SQLLogicTest corpus through the public Go Arrow API.
The reviewed corpus has four temporary exclusions. Two
FixedSizeList(0, Null)assertions are commented out until the Arrow Go dependency supports zero-length lists; the proposed fix is tracked in apache/arrow-go PR #1308. Case G's result and EXPLAIN ANALYZE assertions inaggregate_memory_spill.sltrun separately because four-partition aggregation can exhaust its 1 MiB pool or time out. Cases A–F and the ordinary single-partition Go spill regression remain required. The exclusions and restoration conditions are documented, and their checksums are updated.The Spark
cscandsecfixtures retain ordinary-input and special-value checks. Platform-dependent snapshots near trigonometric poles are replaced with known values away from the poles, checked with an absolute tolerance of1e-12. The suite's comparison rules are unchanged.make test.sqllogic.spillattempts 25 repetitions through Go and in each of two native Rust execution modes. It checks the result values and a nonzero spill count in the final aggregation. CI runs only these diagnostic steps withcontinue-on-error; logs and exit codes are retained underreports/current/spill/without changing required corpus outcomes. CI also replays failed required SQL files through native DataFusion and retains separate comparison logs.Validation on macOS arm64 with released Arrow Go v18.8.0 and no module replacement:
make lint make test make test.source make rust.test make sqllogic.check sqllogic.tools.test TOKIO_WORKER_THREADS=4 make test.sqllogicAll passed, along with workflow lint. The required corpus passed all 24,956 active assertions across 365 executable files, with all 511 files reported and no documentation gaps. The separate spill diagnostic returned failure as intended when it reproduced Go memory exhaustion, a native blocking-read timeout, and native async memory exhaustion; all three outcomes were retained. The underlying parallel-spill issue remains unresolved. Linux and Windows verification of these changes awaits CI.
Function evidence excludes expected-error, failed, empty-result, EXPLAIN-only, and Spark-only queries. Reviewed section mappings record discrepancies in the pinned documentation, including obsolete compression syntax. These example counts do not prove every overload, input, branch, or possible SQL program correct. Required corpus success and documentation witnesses remain separate metrics.