Repository navigation
fix(embedding): preserve exact decimals in NULL predicates and true joins - #29
Merged
aunjgr merged 1 commit intoOct 9, 2026
Conversation
aunjgr
marked this pull request as ready for review
October 9, 2026 03:24
This was referenced Oct 9, 2026
Merged
XuPeng-SH
added a commit
to matrixorigin/matrixone
that referenced
this pull request
Oct 10, 2026
) Opt-in embedded MO-reader queries now lower MO exact Decimal64/128/256 values and descriptors to Sirius GPU execution. One immutable capability-scoped profile owns admission, physical types/literals, bound scalar/aggregate overloads, input publication, result reconstruction and public numeric error classes. Terminal evidence records verified descriptors and a schema digest without logging headings or values. This implements approved PR C from #29690 and the versioned [design/validation map](https://github.com/matrixorigin/matrixone/blob/a4e34ef05de6a6d02ceb0fa81219cacd452ae6c8/docs/design/sirius-mo-exact-lowering.md). Physical width is preserved at narrow declared precision; MO-bound division metadata and 32-byte Decimal256 coefficients use the existing credit-before-copy and borrowed-result contracts. Embedded fetch defaults and DATE extraction use supported native wire forms. All prerequisites are merged and inherited: - Sirius [#29](matrixorigin/sirius#29): `908ffc75a58b3be496b2172795e416326416e7ec`, with merged importer #5 at `99c7ca3b6f8f3159239e119ed2982d42f98c4690`. - Kernel [#29775](#29775): `af3f7232a31879a2d6ed590785c8e9a560361c3f`. Prepared SELECT metadata comes from the executed generation; incomplete saved SELECTs clear session state before reuse. - Main [#29794](#29794) supplies the existing MORPC shutdown-test race correction. Main now also includes the histogram-export synchronization fix; C uses main's bounded two-send notification wait unchanged. Final release/native integration at executable C head `ad61b5b5d3`, on merged-kernel main `af3f7232a3`, passes: - Default, Sirius-only and combined Sirius/cuVS release builds. The clean merged SDK, all 73 fingerprints, compiler/header/proto identity and runtime package/provenance guards pass. - All 22 native preparations without reader/GPU-task admission. The complete public MySQL numeric fixture passes exact values, metadata/NULLs, arithmetic/aggregates/joins/sort, operation-owned errors, masked errors, healthy reuse and prepared division increments 0/4/10/30/4. Native MO supplies the independent oracle. - All 31 public terminal events show capability 31, `mo-exact-decimal-v1`, completed GPU tasks, no fallback, healthy cleanup and zero retained input/result credit. - Four C-owning packages plus frontend pass default normal/race checks; native-tag owning checks and the complete combined bridge suite pass. cuVS-before/Sirius/cuVS-after passes in one process. - All nine public prepared-metadata/saved-result cases pass normal mode and three race repetitions, including a real persisted-batch failure followed by successful save/replay on the same physical connection. - Native data/cancellation passes 68 race repetitions in one process (measured 0.44s; 30s budget). The exact MORPC shutdown test passes 100 race repetitions. Metrics focused/owning normal/race and 100 focused race repetitions pass. The delivered head is `b34111b4c9e94cda034d199a2c362072fe98d808`, normally merged with authoritative main `e5dd4724f782067e1c76c38c9d2f55cebceda4a9`. Review fixes are committed in `af4f342066`: typed DATE lowering preserves persisted MO zero dates; exact CASE retains the MO-bound descriptor around IfThen; ordinary embedded integer add/subtract/multiply declines before readers because the pinned consumer does not implement MO checked overflow. Flight encoding/admission, reader bytes, native ABI and the merged Sirius pin are unchanged. The installed and loaded host drivers now both report NVIDIA 615.78.08; final GPU checks run directly with the installed driver. The bounded local native fixture uses 256 MiB GPU and 2 GiB host capacity. An initial 512 MiB host fixture correctly rejected larger progress-window reservations; only the test fixture was corrected. Production windows, limits and assertions remain unchanged. Review-resolution validation at prior head `a4e34ef05d`: - Both incoming counterexamples have real GPU red/green proof. Zero DATE now returns 0/0/0 alongside year-0001/leap-day/year-9999/NULL controls, with SQL mode restored. Nullable division conditions with two required decimal arms execute with native-MO values and nullable DECIMAL(15,2) metadata. - The existing public fixture covers integer +/-/* overflow declines and healthy integral/decimal reuse. No new cluster, production hook, native library dependency, sleep or skip was added. - All 22 native preparations pass. Full public GPU normal mode passes (9.596s); measured race mode followed by three in-process repetitions passes (29.571s). Its 34 terminal events verify exact profile/capability31, completed GPU work, no fallback, healthy cleanup and zero retained input/result credits. - Fresh combined release/native provenance and full bridge/coexistence pass. Exporter/compiler/CN owning normal/race, focused new wire/domain regressions, config and full mandatory pre-push SCA pass with zero lint findings. Self-review has no unresolved implementation blocker. **Ready for review; merge gates remain required CI and reviewer approval.** Both incoming findings were reproduced on actual GPU/public MySQL and fixed with regressions. A separate admitted integer-overflow counterexample returned -2 instead of native MO 1690/22003; it now declines before execution. Fresh CI is pending for the delivered head, and the reviewer has been asked to reassess the fixes. Local passes do not establish required CI success. C remains opt-in. D begins after C merges and stays one complete public campaign PR for all-22 SF1/SF10, bounded comparison, lifecycle/resource/process evidence and performance gates. E/F retain their separate recovery-readiness design. References #28968 and #28966; neither issue is closed by C. Conflict resolution at `b34111b4c9` (2026-10-10): normally merged authoritative main `e5dd4724f7`. The single conflict was two equivalent asynchronous histogram-test fixes; main's stopped-timer notification barrier is retained unchanged. All 28 prior C paths were checked: only this test changed, while the DATE/CASE/integer review fixes and merged native pins remain intact. Validation at this merged head passes: affected owning packages normal/race, 100 focused metrics race repetitions, fresh native generations and combined release/package, all 22 actual preparations (1.116s), full public GPU normal/race (9.762s/11.999s), and full combined bridge/coexistence (1.108s). Both public runs' 34 terminal events verify completed GPU work, exact profile/capability31, healthy cleanup, no fallback and zero retained credits. Mandatory pre-push SCA passes with zero findings; self-review has no unresolved implementation blocker. New-head required CI and human reviewer approval remain the merge gates. --------- Co-authored-by: XuPeng-SH <xupeng3112@163.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
After #28, MO exact-decimal queries still hit two GPU consumer gaps:
IS NOT NULLcan send Decimal256's private STRUCT through an unsupported cuDF AST unary operation, and canonical TPC-H Q11/Q15/Q22 retainJOIN ON trueas unsupported DuckDB ANY_JOIN because exact preparation skips the ordinary optimizer.Materialize exact NULL predicates from the canonical top-level validity mask, using the existing task stream/resource and AST temporary-column owner. Lower literal TRUE joins to equal constant keys in the existing GPU join, retaining join kind, projection maps, mark binding, duplicate rows and empty-side outer NULLs. Private key columns remain outside public output. FALSE/NULL predicates retain rejection before readers or GPU work start. The numeric URI, capability mask and ABI-v1 layouts remain unchanged.
Validation at clean head
2a39339beb0c2312dc4b71fbdd472e822450637c, frozen Piximo, cuDF/RMM 26.08, RTX 3070:The joint MO result uses an explicitly marked development SDK. C still pins merged #28 and must advance only after this PR merges, then repeat merged-pin delivery checks. This is preparation/small-fixture evidence; SF1/SF10, lifecycle/resource baselines and performance acceptance remain PR D. No rollout gate is waived. Local validation used matching official NVIDIA 615.71.09 libraries in a temporary test launcher; host packages and shipped artifacts were unchanged. CI status is reported separately from these local passes.
Refs MO #28968, MO #28966. Contract and ownership map: docs/mo-exact-null-predicates.md.
Consumer integration is available as draft MO C #29757.