Repository navigation
Conversation
added 12 commits
November 21, 2026 23:35
Fma(a, b, c) computes a * b + c rounded once at the DOUBLE and FLOAT lanes, as Math.fma does, for row 346's remainder and round-to-odd. No Spark expression builds it and no multiply and add is contracted into it. Its validity is the three operands' AND, the reference evaluates Math.fma at each lane's own precision, and every NaN it answers is open in VarkaSameAnswer, since the instruction and Math.fma in Java pick different NaN bits. Generated-by: Claude Code (Claude Opus 5.5)
…N rule Generated-by: Claude Code (Claude Opus 5.5)
The floating grammar draws row 342's nodes behind a flag - the add, subtract and multiply, the unchecked and nulling division, negation and abs, the square root and the FMA - from two new seeds, so the committed floating sequences keep their shapes. VarkaIrFuzzSuite runs each sequence against the reference and holds each to every form it owns. Generated-by: Claude Code (Claude Opus 5.5)
…grammar draws the FMA Generated-by: Claude Code (Claude Opus 5.5)
…census Generated-by: Claude Code (Claude Opus 5.5)
…rators Generated-by: Claude Code (Claude Opus 5.5)
Generated-by: Claude Code (Claude Opus 5.5)
…' result Generated-by: Claude Code (Claude Opus 5.5)
Generated-by: Claude Code (Claude Sonnet 5.5)
Generated-by: Claude Code (Claude Sonnet 5.5)
Generated-by: Claude Code (Claude Opus 5.5)
…tion table Generated-by: Claude Code (Claude Opus 5.5)
Author
|
Option matrix run: https://github.com/MaxGekk/spark/actions/runs/45060442549 (jobs: 23 completed, 5 in_progress, 148 queued) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Row 342 is floating arithmetic. Its operators fuse already.
This pull request adds the one missing node, and fuzz and census parts for the family.
Fma(a, b, c)is a new IR node at theDOUBLEandFLOATlanes. It computesa * b + cwith one rounding, asMath.fmadoes.operator_census.jsonis regenerated.VARKA-342.mdrecords the decisions and runs. A calibration entry scores one prediction.Design details
Row 342's operators are
+,-,*, division, negation,abs, the square root and the fused multiply-add.VARKA-342.mdalso records what is left of the row.Fmais lowered as onefmacall on the Vector API's vector.No multiply and add is ever contracted into it. It exists for row 346's remainder and round-to-odd.
Its validity is the AND of the three operands' validity. The masked body stores the AND for a node that owns its word.
The reference evaluates
Math.fmaat each lane's own precision.Every NaN it answers is open in
VarkaSameAnswer, because the hardware instruction andMath.fmain Java pick different NaN bits.Every exhaustive switch over the IR learns the node.
The warm-up calls
fmain its double and float blocks.VarkaEmitterFmaSuite(new) holds the node to the reference at both lanes.The fuzz grammar draws add, subtract, multiply, the unchecked and nulling division, negation,
abs, the square root and the FMA behind a flag.The new seeds,
FLOAT_ARITH_FUZZ_SEEDandDOUBLE_ARITH_FUZZ_SEED, keep the committed floating sequences and their pinned bytes in their shapes.VarkaIrFuzzSuiteruns each sequence against the reference and makes sure that it drew every form it owns.VarkaOperatorCensusCatalogand the probe list the floating arithmetic's operators.operator_census.jsonholds their verdicts from the laptop's run.Its host
jdkstring changes to the build that the run used. The run did not measure the other cells of the file again.Built here, the node lets row 346 land without a change to the IR.
The calibration entry is
sql/varka/skills/calibration/VARKA-342-5.3-1.md: predicted under 20 s, measured 9.1 s, ratio 0.5.Why are the changes needed?
A separate multiply and add cannot round
a * b + conce, and the remainder and rounding of row 346 need that.The fuzz and census show that the arithmetic agrees with Spark on NaN payloads, signed zeros and subnormals, which the pinned sequences do not draw.
Does this PR introduce any user-facing change?
No. Nothing in the compiler builds
Fmayet. The fuzz and census additions are test code and records.How was this patch tested?
On the laptop, 22 November 2026, final head
a6b9053cf42: the branch merged withorigin/masteratb4cbee926d1, with no conflict.UseAVX=2, benchmarks.Full test record and what was not run
All runs used one sbt at a time and
SPARK_LOCAL_IP=127.0.0.1.The merge was conflict-free. The diff against
masterhas the same lines before and after it.The same gates, without
VarkaOperatorCensusSuite, also passed on3cf4cc0c3f1, before the calibration commit.Run on the final head
a6b9053cf42:dev/varka_regen_generated.sh --all. It left nothing to commit.python3 dev/varka_push_check.pypassed. Its suites were the R1 registry suites for a new IR node and the generated-file suites:VarkaCastMatrixSuite,VarkaCoverageSuite,VarkaEmitCostSuite,VarkaEmittedBytesSuite,VarkaEmitterBudgetSuite,VarkaFuzzTypesSuite,VarkaIrFuzzSuite,VarkaLaneTypeSuite,VarkaReproductionSuite,VarkaSmtNodesSuite,VarkaTypesSuite,VarkaVectorIRSharingSuiteandVarkaWithChildrenSuite. Under the defaults, 192 tests passed, 0 failed, 5 cancelled.dev/varka_matrix.sh --suites VarkaEmitterFmaSuite,VarkaIrFuzzSuitewith the armscse=false,denseValidityOnce=false,lanesOverride=4,lanesOverride=16,useAVX=2,validityByBitmap=falseandvalidityByWord=true: each arm and the defaults had 52 passed, 2 cancelled, 0 failed.dev/lint-java,dev/lint-scala, andcatalyst/testOnly *VarkaInternalErrorSuite *VarkaProofFilesSuite.build/sbt 'catalyst/testOnly *VarkaEmitterFmaSuite *VarkaRangeAnalysisSuite *VarkaShapeCacheSuite *VarkaSmallTreeSuite *VarkaEmitterContractSuite *VarkaSameAnswerContractSuite *VarkaWarmupReachSuite *VarkaEmitterFloatingArithSuite *VarkaEmittedBytesSuite *VarkaOperatorCensusSuite': 141 passed, 3 cancelled, 0 failed.build/sbt 'sql/testOnly *VarkaSameAnswerCoreContractSuite': 1 passed.dev/varka_errorprone.sh.No proof file changed, so
dev/varka_prove.shwas not run.Not run: a CI Build, the dispatched option matrix over every arm, the width audit under
-XX:UseAVX=2, and any benchmark. The census cells were measured on the laptop only.Review
/code-review highat3cf4cc0c3f1, after the merge, found no important defect.One of its three nits is fixed in
a6b9053cf42.Two are open test limits: the FMA suite runs 578 of 4913 operand triples, and its
FLOATrows hold no subnormal or finite extreme.The size is 965 counted lines, under the cap of 1000.
Review record and the nit list
Open nits from the review at
3cf4cc0c3f1:VARKA-342.mdsay "every ordered triple of seventeen operands". The suite's operand function gives about 578 rows of the 4913 triples, so the third operand is always the first or its neighbour.FLOATlane, the special values collapse when they are cast to float. So theFLOATmatrix has no subnormal, no finite extreme and no inexact product.Fixed:
VARKA-342.md5.3 scored a time prediction without a calibration entry. The entry is added.An earlier
/code-review highran on the branch merged withmasterat96f7ff5402b, before32baf6607e4. Its SHA was not recorded. It found no important defect.After it, the push check failed in
VarkaReproductionSuite: no round trip reachedFma. The review did not see this failure. The fix adds anFmatree to that suite.Nits from that review, left as they are:
Vector.fmaunconditionally and the budget prices it as one call. This is an under-price on a host with no hardware FMA. Row 346a owns this.Fma's cached hash has no tag, unlike its siblings.operator_census.jsonmoves its host block'sjdkstring to the laptop's build. The other hosts' sections were measured on other builds.fmacalls run on every host.FLOAT, which the compiler never builds.varka.fuzz.seedwhen it is set.FLOATanswer is compared sign-extended on both sides.Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 5.5)