Repository navigation
GH-50517: [C++] RunEndEncodedBuilder does not merge consecutive NaN values #51325
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -108,6 +108,19 @@ class EqualOptions { | |
| return res; | ||
| } | ||
|
|
||
| /// Whether to compare floating-point values by their physical representation. | ||
| bool physical_repr_equal() const { return physical_repr_equal_; } | ||
|
|
||
| /// Return a copy of this EqualOptions object with the "physical_repr_equal" | ||
| /// property set to the specified value. | ||
| /// If both "physical_repr_equal" and "nans_equal" are enabled, two values | ||
| /// are considered equal if either condition is satisfied. | ||
| EqualOptions physical_repr_equal(bool v) const { | ||
|
Comment on lines
+111
to
+118
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Could we also propagate this new option to sparse floating-point equality? For independently allocated float32 and float64 tensors containing identical NaN bits, Since both equality APIs accept |
||
| auto res = EqualOptions(*this); | ||
| res.physical_repr_equal_ = v; | ||
| return res; | ||
| } | ||
|
|
||
| /// Whether the \ref arrow::Schema property is used in the comparison. | ||
| /// | ||
| /// This option only affects the Equals methods | ||
|
|
@@ -160,6 +173,7 @@ class EqualOptions { | |
| std::optional<int32_t> ulp_distance_ = std::nullopt; | ||
| bool nans_equal_ = false; | ||
| bool signed_zeros_equal_ = true; | ||
| bool physical_repr_equal_ = false; | ||
| bool use_schema_ = true; | ||
| bool use_metadata_ = false; | ||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could this test append the same NaN in two separate
AppendScalarcalls? The consecutive NaNs here are quiet versus signaling, which should remain separate. The other NaN is isolated, and+0and-0are separated by5.As a narrow negative control, reverting only the builder's
Equals(scalar, options_)call toEquals(scalar), while keeping the rest of the current code and tests unchanged, still leaves this test passing. An independent repeated-NaN probe fails under that same reversion.Two calls to
AppendScalar(nan, 3)with identical NaN bits should produce run ends[6], not[3, 6]. Could we cover that case, two quiet NaNs with different payloads, and adjacent signed zeros, asserting both the run boundaries and exact decoded bits? This would protect the intended merge without allowing payload or sign loss.