Repository navigation
Commit aeaddb1
authored
Order -0.0 below 0.0 in data file metrics bounds (#4331)
* Order -0.0 below 0.0 in data file metrics bounds
StatsAggregator.update_min/update_max feed DataFile lower/upper bounds,
but min(val, current) and max(val, current) keep the first argument on a
tie, so whichever zero arrives last wins: a late -0.0 lowers an existing
+0.0 upper bound, and a late +0.0 raises an existing -0.0 lower bound.
The serialized bounds then order above a value present in the file.
Mirror the fix that #4324 applied to partition field summaries: restore
the stored bound when a zero arrives that would invert the -0.0 < 0.0
ordering. The argument order here (val first) flips which direction
needs the guard compared to manifest.py.
Verified failing-first: the new tests fail on main for both arrival
orders and both float types, and pass with the guard.
Signed-off-by: Ayush Kumar Saha <225407166+AP24110011695@users.noreply.github.com>
* Simplify the negative-zero guards per review
Flip the min/max argument order so a tie keeps the existing value (the
same shape as PartitionFieldStats.update in manifest.py), which lets the
guard only handle an incoming zero of the wrong sign and drops the
previous-value capture. Behavior verified unchanged: the regression
tests and all zero-arrival interleavings still produce the canonical
-0.0-below-0.0 bounds.
Signed-off-by: Ayush Kumar Saha <225407166+AP24110011695@users.noreply.github.com>
---------
Signed-off-by: Ayush Kumar Saha <225407166+AP24110011695@users.noreply.github.com>
Co-authored-by: Ayush Kumar Saha <225407166+AP24110011695@users.noreply.github.com>1 parent 0008d65 commit aeaddb1
2 files changed
Lines changed: 32 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2402 | 2402 | | |
2403 | 2403 | | |
2404 | 2404 | | |
2405 | | - | |
| 2405 | + | |
| 2406 | + | |
| 2407 | + | |
| 2408 | + | |
2406 | 2409 | | |
2407 | 2410 | | |
2408 | 2411 | | |
2409 | 2412 | | |
2410 | 2413 | | |
2411 | | - | |
| 2414 | + | |
| 2415 | + | |
| 2416 | + | |
| 2417 | + | |
2412 | 2418 | | |
2413 | 2419 | | |
2414 | 2420 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
44 | 44 | | |
45 | 45 | | |
46 | 46 | | |
| 47 | + | |
47 | 48 | | |
48 | 49 | | |
49 | 50 | | |
| |||
2954 | 2955 | | |
2955 | 2956 | | |
2956 | 2957 | | |
| 2958 | + | |
| 2959 | + | |
| 2960 | + | |
| 2961 | + | |
| 2962 | + | |
| 2963 | + | |
| 2964 | + | |
| 2965 | + | |
| 2966 | + | |
| 2967 | + | |
| 2968 | + | |
| 2969 | + | |
| 2970 | + | |
| 2971 | + | |
| 2972 | + | |
| 2973 | + | |
| 2974 | + | |
| 2975 | + | |
| 2976 | + | |
| 2977 | + | |
| 2978 | + | |
| 2979 | + | |
| 2980 | + | |
2957 | 2981 | | |
2958 | 2982 | | |
2959 | 2983 | | |
| |||
0 commit comments