Skip to content

Retain declared MIN/MAX argument type for empty input - #570

Merged
perNyfelt merged 3 commits into
mainfrom
fix/320-min-max-empty-metadata
Nov 17, 2026
Merged

perNyfelt merged 3 commits into
mainfrom
fix/320-min-max-empty-metadata

Conversation

@perNyfelt

@perNyfelt perNyfelt commented Nov 16, 2026 •

Copy link
Copy Markdown
Member

Problem

MIN/MAX result metadata was inferred from the returned value. Over empty input (or an all-NULL group) the column reported OTHER / java.lang.Object even though the argument has a declared type, and set operations then treated such a branch as an untyped NULL and accepted incompatible branch types. Decimal bounds also depended on the returned rows: MIN(CAST(mpg AS DECIMAL(10,2))) reported value-inferred decimal(4,2) with rows and OTHER without.

Reproduction (mtcars fixture: mpg DOUBLE, model VARCHAR)

Query Before After
SELECT MIN(mpg) FROM mtcars WHERE cyl = 99 OTHER / java.lang.Object DOUBLE / java.lang.Double
SELECT MAX(model) FROM mtcars WHERE cyl = 99 OTHER / java.lang.Object VARCHAR / java.lang.String
SELECT MIN(CAST(mpg AS DECIMAL(10,2))) FROM mtcars DECIMAL p=4 s=2 (value-inferred) DECIMAL p=10 s=2 (declared)
SELECT MIN(CAST(mpg AS DECIMAL(10,2))) FROM mtcars WHERE cyl = 99 OTHER / java.lang.Object DECIMAL p=10 s=2 (declared)
… UNION ALL SELECT CAST(1 AS DECIMAL(3,0)) (with rows / empty branch) p=5 s=2 / p=10 s=2 — depends on data p=10 s=2 in both cases
SELECT MIN(cyl / 3) FROM mtcars WHERE cyl = 99 OTHER / java.lang.Object DECIMAL p=0 s=0 (unbounded)
SELECT MIN(ROUND(mpg, 1)) FROM mtcars WHERE cyl = 99 OTHER / java.lang.Object DECIMAL p=0 s=0 (unbounded)
SELECT MIN(mpg) FROM mtcars WHERE cyl = 99 UNION ALL SELECT 'a' accepted (OTHER) rejected, SQLState 42000
SELECT MAX(model) FROM mtcars WHERE cyl = 99 UNION ALL SELECT 1 accepted (INTEGER) rejected, SQLState 42000
SELECT SUM(cyl) FROM mtcars WHERE cyl = 99 UNION ALL SELECT 'a' (control) rejected, 42000 unchanged

Non-empty observed values are unchanged: MIN(mpg) still returns 10.4 as Double, and unbounded decimal arguments keep their value-derived bounds when rows exist (MIN(cyl / 3) stays decimal(16,15)).

Root cause

Aggregate result types come from the accumulator's observed value class: an empty or all-NULL ExtremumAccumulator reports sqlTypeForClass(null) = OTHER, and the rows-based constructor infers decimal bounds only from materialized values. The declared-type fallback, JParqResultSetMetaData.declaredProjectionSchema, resolved window MIN/MAX but had no branch for top-level aggregate MIN/MAX, so nothing replaced the OTHER in the metadata. Set operations read the branch type through ResultSetMetaData / columnValueSchema, classified the branch as untyped NULL, and merged it with anything.

Fix (metadata side, no value-side changes)

  • JParqResultSetMetaData.declaredProjectionSchema: a top-level single-argument MIN/MAX now resolves its argument's declared schema via ArrayExpressionSchemas.resolve, mirroring the existing window (AnalyticExpression) branch. The existing AggregateResultSetMetaData.retainExpressionTypes mechanism then applies the declaration, which also fixes set-operation rejection because executeSetOperation reads branch types from the same metadata. Value-side inference in AggregateFunctions is intentionally untouched.
  • JParqResultSetMetaData.unboundedExtremumArgumentType (new): arguments whose declaration is an unbounded decimal (cyl / 3, ROUND(mpg, 1)) have no Avro value schema (NumericType.schema() is null for precision 0), so their NumericType is exposed directly.
  • AggregateResultSetMetaData.retainExpressionTypes: uses the unbounded-decimal numeric declaration when the value-schema channel cannot express it (fixing empty MIN(cyl / 3) → DECIMAL p=0 s=0, matching the plain empty projection), and applies a resolved decimal declaration's bounds unconditionally, so declared bounds no longer depend on whether rows came back. Unbounded declarations (precision 0) still never override value-derived bounds.

Tests

jparq.MinMaxEmptyInputTest (13 tests): empty MIN/MAX report the argument's declared type and class (DOUBLE/Double, VARCHAR/String, INTEGER/Integer); non-empty MIN/MAX keep observed values and types; an all-NULL group reports the argument's declared type; an empty DECIMAL(10,2) MIN retains the declared bounds and a set operation over it merges the declared decimal(10,2) bounds whether or not the branch returns rows; empty unbounded-decimal MIN/MAX (cyl / 3, ROUND(mpg, 1)) report DECIMAL p=0 s=0; both incompatible-branch UNION variants are rejected with 42000 and the existing message; a compatible numeric branch still merges.

Out of scope, tracked as #572: a scalar subquery argument with a nonnumeric result (e.g. MIN((SELECT MAX(model) FROM mtcars))) still reports OTHER over empty input; the same gap pre-exists for top-level scalar subquery projections over empty outer input.

Verified: checkstyle/PMD/spotless clean (no new violations vs main baseline), full mvn -o verify green.

Fixes #320.

An empty or all-NULL MIN/MAX reported OTHER/Object because the result
type was inferred from the observed value class and the declared-type
fallback had no branch for top-level aggregate MIN/MAX.
declaredProjectionSchema now resolves the single-argument MIN/MAX
argument like the existing window branch, and retainExpressionTypes
retains the declared decimal bounds when no numeric type is inferred, so
set operations consistently reject incompatible branches.

Fixes #320.
…ents

Address review round 1 on PR #570:

- retainExpressionTypes applies a resolved decimal declaration's bounds
  unconditionally, so declared bounds no longer depend on whether rows
  came back and set operations merge row-independent bounds.
- unboundedExtremumArgumentType exposes the NumericType of an unbounded
  decimal MIN/MAX argument, which the value-schema channel cannot
  express because NumericType.schema() is null for precision 0; empty
  MIN(cyl / 3) and MIN(ROUND(mpg, 1)) now report DECIMAL p=0 s=0,
  matching the plain empty projection, while value-derived bounds are
  preserved when rows exist.
- emptyDecimalMinimumRetainsDeclaredBounds no longer depends on VALUES
  literal typing (issue #559) and uses a real source column cast; add
  row-independence tests for decimal bounds and set-operation merges.

Scalar subquery arguments with nonnumeric results stay untyped over
empty input; tracked as #572.
@perNyfelt

Copy link
Copy Markdown
Member Author

All four addressed in the new commit; verified against your exact repro queries.

1. Decimal bounds no longer depend on rows. Dropped the precisions[i] == 0 guard — a resolved decimal declaration now always supplies its bounds. Measured before/after:

Query Before After
SELECT MIN(CAST(mpg AS DECIMAL(10,2))) FROM mtcars p=4 s=2 p=10 s=2
SELECT MIN(CAST(mpg AS DECIMAL(10,2))) FROM mtcars WHERE cyl = 99 p=10 s=2 p=10 s=2
… UNION ALL SELECT CAST(1 AS DECIMAL(3,0)) p=5 (rows) / p=10 (empty) p=10 s=2 in both cases

2. The VALUES-based test is replaced. emptyDecimalMinimumRetainsDeclaredBounds now uses SELECT MIN(CAST(mpg AS DECIMAL(10,2))) FROM mtcars WHERE cyl = 99 and expects 10/2, so it no longer encodes the #559 VALUES behavior and will not flip when #559 is fixed. Added decimalMinimumBoundsDoNotDependOnRows (10/2 with and without rows, value 10.40 with rows) and setOperationBoundsDoNotDependOnRows (the merged UNION bounds are asserted identical for the filled and empty MIN branch).

3. Unbounded decimal arguments report DECIMAL when empty. One wrinkle that shaped the implementation: NumericType.schema() returns null for precision 0 (no Avro bounds to carry), so the declaration channel structurally cannot express decimal(0,0). The fix routes that case through the NumericType itself: new JParqResultSetMetaData.unboundedExtremumArgumentType(column) returns the argument's numeric declaration when it is exactly an unbounded DECIMAL, and retainExpressionTypes uses it when the value-schema channel has nothing. Result:

Query Before After
SELECT MIN(cyl / 3) FROM mtcars WHERE cyl = 99 OTHER / p=0 s=0 DECIMAL / p=0 s=0
SELECT MIN(ROUND(mpg, 1)) FROM mtcars WHERE cyl = 99 OTHER / p=0 s=0 DECIMAL / p=0 s=0
SELECT MIN(cyl / 3) FROM mtcars DECIMAL p=16 s=15 DECIMAL p=16 s=15 (unchanged)

Empty MIN(cyl / 3) now matches the plain empty projection (SELECT cyl / 3 FROM mtcars WHERE 1 = 0 → DECIMAL p=0 s=0), and the value-derived bounds survive when rows exist, because the numeric branch only overwrites precision for DECIMAL with precision > 0.

4. Scalar subquery argument — tracked as #572. While investigating I found it's narrower and broader than it looks. Narrower: the numeric case already works on this branch — SELECT MIN((SELECT MAX(mpg) FROM mtcars)) FROM mtcars WHERE cyl = 99 reports DOUBLE when empty, via the numeric declaration channel; only nonnumeric results fail. Broader: the same gap exists without any aggregate — SELECT (SELECT MAX(model) FROM mtcars) FROM mtcars WHERE cyl = 99 (top-level scalar subquery projection over an empty outer input) also reports OTHER, so it is the nonnumeric result-declaration channel (SubqueryExecutor.projectionResultSchema / ProjectionDeclaration.result) that cannot establish a declaration for a shape that never runs. Filed #572 with both repros rather than expanding this PR.

Full mvn -o verify is green on the new commit.

@perNyfelt perNyfelt left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at head f7df136.

Existing feedback. No GitHub reviews, inline review comments, or review threads exist on this PR (REST pulls/570/reviews, pulls/570/comments and GraphQL reviewThreads are all empty; no Copilot/bot comments). The only feedback record is the issue comment from the prior local review round listing four points; each is addressed at this head, verified against the repro queries:

  1. Row-independent decimal bounds — verified: MIN(CAST(mpg AS DECIMAL(10,2))) reports p=10 s=2 with rows, without rows, and in the UNION merge in both row states (previously p=4 s=2 / p=5 s=2 with rows on main).
  2. VALUES-based test replaced — verified: emptyDecimalMinimumRetainsDeclaredBounds now uses a real source-column CAST.
  3. Unbounded decimal arguments — verified: empty MIN(cyl / 3) and MIN(ROUND(mpg, 1)) report DECIMAL p=0 s=0, matching the plain empty projection, while non-empty MIN(cyl / 3) keeps its value-derived p=16 s=15.
  4. Scalar subquery argument — tracked as #572 (open, with both repros).

Verification.

  • Full mvn verify (no skip flags, Java 21): BUILD SUCCESS, 4034 tests, 0 failures, checkstyle/PMD/spotless clean.
  • Ran a JDBC repro (18 queries) against builds of main and this branch. Every row in the PR's before/after table matches, including: empty MIN(mpg) → DOUBLE/Double, empty MAX(model) → VARCHAR/String, both incompatible-branch UNIONs rejected with 42000, the SUM control still rejected, MIN(mpg) UNION ALL SELECT 2.5 now DOUBLE (was DECIMAL(2,1) on main), and non-empty values/classes unchanged.
  • Traced the metadata flow (JParqResultSet.getMetaData → retainExpressionTypes, executeSetOperation family validation): the fix is metadata-side only, AggregateFunctions untouched, and retainedExpressionTypes's new decimal-bounds assignment can only fire for MIN/MAX columns because numericType returns null for function projections and the new declaredProjectionSchema branch is the only changed declaration channel.

Minor findings (non-blocking):

  1. PR body: the "Before" cell for standalone SELECT MIN(CAST(mpg AS DECIMAL(10,2))) FROM mtcars WHERE cyl = 99 says "DECIMAL p=10 s=2 (unchanged)", but on main that query measures OTHER p=0 s=0. The table's Before column appears to mix the pre-first-commit state (this row) with the pre-PR state (the rest). Consider correcting the cell or noting the baseline.
  2. setOperationBoundsDoNotDependOnRows asserts row-independence (equality) but not the exact expected bounds; pinning {DECIMAL, 10, 2} would anchor the declared-bounds behavior directly.
  3. Only MIN is exercised for unbounded decimal arguments over empty input; extremumArgument handles both, so a symmetric MAX case would be cheap to add.

No out-of-scope bugs found to file as issues; the remaining scalar-subquery gap is already #572.

@perNyfelt perNyfelt left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at head 55df396 (new commit: "Pin set operation decimal bounds and cover unbounded decimal MAX"). LGTM — approving (formal approval blocked because the reviewer is the PR author).

Previous feedback status (all resolved):

  • My review of f7df136, finding 1 (PR body "Before" baseline for standalone empty MIN(CAST(mpg AS DECIMAL(10,2)))): fixed — the body now shows OTHER / java.lang.Object as the pre-PR state for that row, matching the measurement on main.
  • Finding 2 (pin exact merged bounds): fixed — setOperationBoundsDoNotDependOnRows now asserts {DECIMAL, 10, 2} for both the empty and filled MIN branch instead of only asserting row-independence.
  • Finding 3 (MAX coverage for unbounded decimal arguments): fixed — new emptyUnboundedDecimalMaximumReportsDecimal mirrors the MIN test, covering empty MAX(cyl / 3) (DECIMAL p=0 s=0), empty MAX(ROUND(mpg, 1)), and the non-empty value case.
  • The four points from the prior local round (author's issue comment) remain addressed; 55df396 is a test-only delta, so the previously verified behavior is unchanged.
  • No review threads, Copilot, or bot comments on the PR.

Verification at 55df396:

  • Full mvn verify (no skip flags, Java 21): BUILD SUCCESS, 4035 tests, 0 failures (13 in MinMaxEmptyInputTest), checkstyle/PMD/spotless clean.
  • Delta inspected: test-only changes to src/test/java/jparq/MinMaxEmptyInputTest.java plus the corrected PR-body row; no main-code changes, so no re-verification of the metadata flow was needed.

No open findings and no out-of-scope bugs found; nothing new to file as issues.

@perNyfelt
perNyfelt merged commit 17cdb5a into main Nov 17, 2026
2 checks passed
@perNyfelt
perNyfelt deleted the fix/320-min-max-empty-metadata branch November 17, 2026 14:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MIN/MAX over empty input report OTHER metadata; set operations then accept incompatible branches

1 participant