Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions doc/sql-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -533,7 +533,8 @@ SELECT a FROM employees e(a, b, c); -- Parquet t
even for empty inputs.
- Unknown functions in grouped `SELECT` and `HAVING` raise `SQLSyntaxErrorException` with SQLState `42000` when
evaluated.
- `SUM` over `REAL` or `DOUBLE` inputs reports JDBC `DOUBLE` metadata and returns `Double`.
- `SUM` over `REAL` or `DOUBLE` inputs reports JDBC `DOUBLE` metadata and returns `Double`. A sum of finite values
that is not finite raises `SQLDataException` with SQLState `22003`.

### DISTINCT and row limits on grouped results

Expand Down Expand Up @@ -722,7 +723,7 @@ JDBC types for arithmetic are derived before rows are read, including through de
| Operands | Result |
| --- | --- |
| Whole numbers with `+`, `-`, `*`, `%`, unary sign | `BIGINT` (`Long`); overflow raises `22003` |
| `REAL` or `DOUBLE` | `DOUBLE` (`Double`), using double arithmetic |
| `REAL` or `DOUBLE` | `DOUBLE` (`Double`), using double arithmetic; finite operands whose result is not finite raise `22003` |
| Exact decimals | `DECIMAL` (`BigDecimal`); declared bounds reserve carry and product digits |
| Non-approximate division | `MathContext.DECIMAL64`, so `7 / 2` returns `3.5`. Precision and scale are unknown until materialization infers them from the values, keeping value scales such as `3.5`. |
| NULL operand | NULL |
Expand Down Expand Up @@ -764,7 +765,9 @@ DuckDB:
`-Infinity` < every finite value < `+Infinity` < `NaN`, where `NaN = NaN` and `-0.0 = 0.0`.

Comparisons with `NaN` and infinities are therefore TRUE or FALSE rather than UNKNOWN, and `ORDER BY`, `MIN`, and
`MAX` place `NaN` last. Arithmetic follows IEEE 754: for example, `Infinity * 0` and `NaN + 1` return `NaN`.
`MAX` place `NaN` last. Arithmetic whose operands are non-finite follows IEEE 754: for example, `Infinity * 0` and
`NaN + 1` return `NaN`. Arithmetic on finite operands that produces a non-finite result raises `SQLDataException`
with SQLState `22003`, like `BIGINT` overflow and floating-point `CAST` overflow.
`DISTINCT` and `GROUP BY` currently keep `-0.0` and `0.0` apart
([#477](https://github.com/Alipsa/JParq/issues/477)).

Expand Down
23 changes: 22 additions & 1 deletion src/main/java/se/alipsa/jparq/engine/NumericCasts.java
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@
import java.sql.SQLDataException;

/**
* Shares numeric conversion and range validation across CAST and JDBC getters.
* Shares numeric conversion and range validation across CAST, aggregate
* results, and JDBC getters.
*/
public final class NumericCasts {
/** Prevents utility class instantiation. */
Expand Down Expand Up @@ -68,6 +69,26 @@ public static Number convert(Object value, NumericCastTarget type, String target
}
}

/**
* Converts a decimal value to a DOUBLE result, rejecting overflow.
*
* @param value
* the decimal value
* @param overflowMessage
* the diagnostic message used when the value overflows the DOUBLE
* range
* @return the finite double value
* @throws IllegalArgumentException
* if the value overflows the DOUBLE range (22003)
*/
public static double toFiniteDouble(BigDecimal value, String overflowMessage) {
double result = value.doubleValue();
if (!Double.isFinite(result)) {
throw SqlExceptions.numericRangeFailure(overflowMessage);
}
return result;
}

/**
* Converts a finite numeric input without leaking unclassified parse errors.
*
Expand Down
11 changes: 11 additions & 0 deletions src/main/java/se/alipsa/jparq/engine/SqlExceptions.java
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,17 @@ public static SQLDataException invalidRowCount(String clause, Object value) {
"OFFSET".equals(clause) ? "2201X" : "2201W");
}

/**
* Creates an engine failure for numeric values outside the target range.
*
* @param message
* the diagnostic message
* @return an engine exception carrying SQLState 22003
*/
public static IllegalArgumentException numericRangeFailure(String message) {
return new IllegalArgumentException(message, new SQLDataException(message, "22003"));
}

/**
* Creates an engine failure carrying the JDBC syntax classification.
*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2793,19 +2793,13 @@ Object result() {
return null;
}
Class<?> type = observedType;
if (type == null) {
return sum.doubleValue();
}
if (type == Byte.class || type == Short.class || type == Integer.class || type == Long.class) {
return sum.longValue();
}
if (type == Float.class || type == Double.class) {
return sum.doubleValue();
}
if (type == BigDecimal.class) {
return sum;
}
return sum.doubleValue();
return NumericCasts.toFiniteDouble(sum, "SUM overflows the DOUBLE range");
}

@Override
Expand Down Expand Up @@ -3574,7 +3568,8 @@ private Object aliasValue(String name) {
* @return the numeric result, or null for NULL operands or a zero divisor
* @throws IllegalArgumentException
* if text conversion fails (22018), text exceeds the DOUBLE range
* (22003), or BIGINT arithmetic overflows (22003)
* (22003), BIGINT arithmetic overflows (22003), or double arithmetic
* on finite operands produces a non-finite result (22003)
*/
public static Object calculate(Object leftVal, Object rightVal, Operation op) {
return calculate(leftVal, rightVal, op, null);
Expand All @@ -3595,7 +3590,8 @@ public static Object calculate(Object leftVal, Object rightVal, Operation op) {
* @return the numeric result, or null for NULL operands or a zero divisor
* @throws IllegalArgumentException
* if text conversion fails (22018), text exceeds the DOUBLE range
* (22003), or BIGINT arithmetic overflows (22003)
* (22003), BIGINT arithmetic overflows (22003), or double arithmetic
* on finite operands produces a non-finite result (22003)
*/
public static Object calculate(Object leftVal, Object rightVal, Operation op, NumericType resultType) {
if (leftVal == null || rightVal == null) {
Expand All @@ -3607,13 +3603,20 @@ public static Object calculate(Object leftVal, Object rightVal, Operation op, Nu
|| rightVal instanceof Float) {
double left = arithmeticDouble(leftVal);
double right = arithmeticDouble(rightVal);
return switch (op) {
if (right == 0 && (op == Operation.DIV || op == Operation.MOD)) {
return null;
}
double result = switch (op) {
case ADD -> left + right;
case SUB -> left - right;
case MUL -> left * right;
case DIV -> right == 0 ? null : left / right;
case MOD -> right == 0 ? null : left % right;
case DIV -> left / right;
case MOD -> left % right;
};
if (Double.isFinite(left) && Double.isFinite(right) && !Double.isFinite(result)) {
throw SqlExceptions.numericRangeFailure("DOUBLE arithmetic overflow");
}
return result;
}
if (resultType != null
? resultType.jdbcType() == Types.BIGINT
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

import java.math.BigDecimal;
import java.math.BigInteger;
import se.alipsa.jparq.engine.NumericCasts;

final class SumComputationState {
private BigDecimal sum = BigDecimal.ZERO;
Expand Down Expand Up @@ -30,10 +31,7 @@ Object result() {
|| observedType == Long.class) {
return sum.longValue();
}
if (observedType == Float.class || observedType == Double.class) {
return sum.doubleValue();
}
return sum.doubleValue();
return NumericCasts.toFiniteDouble(sum, "SUM overflows the DOUBLE range");
}

private void trackType(Object value) {
Expand Down
174 changes: 174 additions & 0 deletions src/test/java/jparq/FloatingPointArithmeticOverflowTest.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,174 @@
package jparq;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertInstanceOf;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;

import java.math.BigDecimal;
import java.nio.file.Path;
import java.sql.DriverManager;
import java.sql.SQLDataException;
import java.util.List;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;

/**
* Verifies that floating-point arithmetic overflow reports SQLState 22003
* (#323).
*/
class FloatingPointArithmeticOverflowTest {
/** Expected value of an arithmetic or SUM expression. */
private record Expectation(String expression, Double expected) {
}

/**
* Rejects arithmetic on finite approximate operands that produces a non-finite
* result, consistently through materialization.
*
* @param predicatePushdown
* whether predicate pushdown is enabled
* @param projectionPushdown
* whether projection pushdown is enabled
* @throws Exception
* if fixture lookup or connection setup fails
*/
@ParameterizedTest
@CsvSource({
"true,true", "true,false", "false,true", "false,false"
})
void finiteOperandOverflowReportsSqlState(boolean predicatePushdown, boolean projectionPushdown) throws Exception {
String url = jdbcUrl() + "?predicatePushdown=" + predicatePushdown + "&projectionPushdown=" + projectionPushdown;
try (var connection = DriverManager.getConnection(url); var statement = connection.createStatement()) {
for (String expression : List.of("CAST(1e300 AS DOUBLE) * CAST(1e300 AS DOUBLE)",
"CAST(-1e300 AS DOUBLE) * CAST(1e300 AS DOUBLE)",
"CAST(1.7976931348623157e308 AS DOUBLE) + CAST(1.7976931348623157e308 AS DOUBLE)",
"CAST(-1.7976931348623157e308 AS DOUBLE) - CAST(1.7976931348623157e308 AS DOUBLE)",
"CAST(1e300 AS DOUBLE) / CAST(1e-300 AS DOUBLE)", "CAST(-1e300 AS DOUBLE) / CAST(1e-300 AS DOUBLE)",
"CAST(1e38 AS REAL) * CAST(1e300 AS DOUBLE)")) {
String body = "SELECT " + expression + " AS x FROM mtcars WHERE cyl = 6 LIMIT 1";
for (String sql : List.of(body, "SELECT x FROM (" + body + ") d", "WITH c AS (" + body + ") SELECT x FROM c",
"SELECT (" + body + ") AS x", "SELECT " + expression + " AS x",
"SELECT cyl FROM mtcars WHERE " + expression + " > 0 LIMIT 1",
"SELECT cyl FROM mtcars WHERE mpg > " + expression + " LIMIT 1")) {
SQLDataException failure = assertThrows(SQLDataException.class, () -> {
try (var result = statement.executeQuery(sql)) {
assertTrue(result.next(), sql);
result.getObject(1);
}
}, sql);
assertEquals("22003", failure.getSQLState(), sql);
}
}
}
}

/**
* Rejects sums of finite DOUBLE values that exceed the DOUBLE range in
* aggregate and window forms.
*
* @param predicatePushdown
* whether predicate pushdown is enabled
* @param projectionPushdown
* whether projection pushdown is enabled
* @throws Exception
* if fixture lookup or connection setup fails
*/
@ParameterizedTest
@CsvSource({
"true,true", "true,false", "false,true", "false,false"
})
void sumOverflowReportsSqlState(boolean predicatePushdown, boolean projectionPushdown) throws Exception {
String url = jdbcUrl() + "?predicatePushdown=" + predicatePushdown + "&projectionPushdown=" + projectionPushdown;
try (var connection = DriverManager.getConnection(url); var statement = connection.createStatement()) {
for (String sum : List.of("SUM(CAST(1e308 AS DOUBLE))", "SUM(CAST(-1e308 AS DOUBLE))")) {
for (String sql : List.of("SELECT " + sum + " AS x FROM mtcars",
"SELECT x FROM (SELECT " + sum + " AS x FROM mtcars) d",
"WITH c AS (SELECT " + sum + " AS x FROM mtcars) SELECT x FROM c",
"SELECT cyl, " + sum + " AS x FROM mtcars GROUP BY cyl",
"SELECT cyl FROM mtcars GROUP BY cyl HAVING " + sum + " > 0",
"SELECT cyl, " + sum + " AS x FROM mtcars GROUP BY cyl ORDER BY x",
"SELECT " + sum + " OVER () AS x FROM mtcars")) {
SQLDataException failure = assertThrows(SQLDataException.class, () -> {
try (var result = statement.executeQuery(sql)) {
while (result.next()) {
result.getObject(1);
}
}
}, sql);
assertEquals("22003", failure.getSQLState(), sql);
}
}
}
}

/**
* Keeps finite approximate results, NULL zero-divisor results, and in-range
* sums flowing through unchanged.
*
* @param predicatePushdown
* whether predicate pushdown is enabled
* @param projectionPushdown
* whether projection pushdown is enabled
* @throws Exception
* if fixture lookup or JDBC execution fails
*/
@ParameterizedTest
@CsvSource({
"true,true", "true,false", "false,true", "false,false"
})
void finiteResultsAndNullDivisorsArePreserved(boolean predicatePushdown, boolean projectionPushdown)
throws Exception {
String url = jdbcUrl() + "?predicatePushdown=" + predicatePushdown + "&projectionPushdown=" + projectionPushdown;
try (var connection = DriverManager.getConnection(url); var statement = connection.createStatement()) {
for (Expectation expectation : List.of(new Expectation("CAST(1e300 AS DOUBLE) + CAST(1e300 AS DOUBLE)", 2.0E300),
new Expectation("CAST(1.7976931348623157e308 AS DOUBLE) / CAST(1e300 AS DOUBLE)", 1.7976931348623157E8),
new Expectation("CAST(10.5 AS DOUBLE) % CAST(2.0 AS DOUBLE)", 0.5),
new Expectation("CAST(1 AS DOUBLE) / CAST(0 AS DOUBLE)", null),
new Expectation("CAST(1 AS DOUBLE) % CAST(0 AS DOUBLE)", null))) {
for (String sql : List.of("SELECT " + expectation.expression() + " AS x FROM mtcars WHERE cyl = 6 LIMIT 1",
"SELECT " + expectation.expression() + " AS x")) {
try (var result = statement.executeQuery(sql)) {
assertTrue(result.next(), sql);
Object value = result.getObject(1);
if (expectation.expected() == null) {
assertNull(value, sql);
} else {
assertEquals(expectation.expected(), assertInstanceOf(Double.class, value, sql), sql);
}
assertFalse(result.next(), sql);
}
}
}
double sum = BigDecimal.valueOf(1.0E300).multiply(BigDecimal.valueOf(32L)).doubleValue();
for (Expectation expectation : List.of(new Expectation("SUM(CAST(1e300 AS DOUBLE))", sum),
new Expectation("SUM(CAST(-1e300 AS DOUBLE))", -sum),
new Expectation("SUM(CAST(1e300 AS DOUBLE)) OVER ()", sum),
new Expectation("SUM(CAST(-1e300 AS DOUBLE)) OVER ()", -sum))) {
for (String sql : List.of("SELECT " + expectation.expression() + " AS x FROM mtcars",
"SELECT x FROM (SELECT " + expectation.expression() + " AS x FROM mtcars) d",
"WITH c AS (SELECT " + expectation.expression() + " AS x FROM mtcars) SELECT x FROM c")) {
try (var result = statement.executeQuery(sql)) {
while (result.next()) {
assertEquals(expectation.expected(), assertInstanceOf(Double.class, result.getObject(1), sql), sql);
}
}
}
}
}
}

/**
* Locates the committed Parquet fixture.
*
* @return the fixture directory's JDBC URL
* @throws Exception
* if fixture lookup fails
*/
private static String jdbcUrl() throws Exception {
return "jdbc:jparq:"
+ Path.of(FloatingPointArithmeticOverflowTest.class.getResource("/mtcars.parquet").toURI()).getParent();
}
}
Loading