Skip to content

Preserve NaN semantics for floating-point array_position pushdown - #341

Open
fallintoplace wants to merge 5 commits into
ClickHouse:mainfrom
fallintoplace:fix/array-position-not-found
Open

Preserve NaN semantics for floating-point array_position pushdown#341
fallintoplace wants to merge 5 commits into
ClickHouse:mainfrom
fallintoplace:fix/array-position-not-found

Conversation

@fallintoplace

@fallintoplace fallintoplace commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Overview

Fix array_position() pushdown for real[] and double precision[] columns.

ClickHouse indexOf() can miss a NaN when the array value and the search value were produced independently. PostgreSQL treats those values as a match for array_position(), so the existing pushdown could return NULL incorrectly.

Implementation

  • Keep the existing nullIf(indexOf(...), 0) path for non-floating-point arrays.
  • Use arrayFirstIndex() with an explicit predicate for floating-point arrays.
  • Match ordinary values, NULL, and any NaN payload.
  • Qualify captured columns with the generated relation alias so lambda parameters cannot shadow outer columns such as x.
  • Emit NaN, Infinity, and -Infinity constants as ClickHouse numeric float literals.
  • Preserve positive start handling with arraySlice() and restore the original PostgreSQL position.

The floating-point predicate is equivalent to:

nullIf(
    arrayFirstIndex(
        value -> (
            (isNull(value) AND isNull(r1.needle))
            OR (value = r1.needle)
            OR (isNaN(value) AND isNaN(r1.needle))
        ),
        r1.float_array
    ),
    0
)

Regression coverage

  • Float32 and Float64 arrays
  • Ordinary matches and missing values
  • Independently produced NaNs with different payloads
  • A captured column named x, matching the lambda parameter name
  • NaN, positive infinity, and negative infinity constants
  • Positive start positions and empty arrays
  • Nullable elements and NULL needles, including exact returned positions
  • Generated remote SQL and relation qualification

Related issue

ClickHouse/ClickHouse#113169 tracks the underlying floating-point NaN matching behavior in ClickHouse.

Follow-up fix

The follow-up commit also covers array_position() expressions used only in pushed-down ORDER BY pathkeys. These expressions are resolved after the base relation is emitted, so they now share the same pathkey resolver during alias preflight and SQL generation. This keeps captured columns qualified in cases such as ORDER BY array_position(float64_vals, x), id.

Added regression coverage for the ORDER BY path and qualified generated SQL.

@theory theory left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partial review, more later.

Comment thread src/deparse.c Outdated
Comment on lines +3527 to +3537
if ((node->consttype == FLOAT4OID || node->consttype == FLOAT8OID) &&
strcmp(extval, "NaN") == 0) {
appendStringInfoString(buf, "nan");
} else if (
(node->consttype == FLOAT4OID || node->consttype == FLOAT8OID) &&
(strcmp(extval, "Infinity") == 0 || strcmp(extval, "+Infinity") == 0)
) {
appendStringInfoString(buf, "inf");
} else if (
(node->consttype == FLOAT4OID || node->consttype == FLOAT8OID) &&
strcmp(extval, "-Infinity") == 0

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use case-insensitive comparisons

@theory theory added pushdown Improvements to query pushdown functions Improve function pushdown labels Aug 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

functions Improve function pushdown pushdown Improvements to query pushdown

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants