Preserve PostgreSQL substring offset semantics - #332
Open
fallintoplace wants to merge 1 commit into
Open
Conversation
serprex
reviewed
Aug 3, 2026
| } | ||
|
|
||
| start = (Expr*)lsecond(fn->args); | ||
| if (!IsA(start, Const)) { |
Member
There was a problem hiding this comment.
can we push down conditional logic to match postgres instead?
cc @JoshDreamland who recently did this sort of thing to get correct NULL handling with IN in #317
fallintoplace
force-pushed
the
fix/substring-pushdown-semantics
branch
from
August 3, 2026 16:43
d86b3f6 to
688c16d
Compare
fallintoplace
force-pushed
the
fix/substring-pushdown-semantics
branch
from
August 3, 2026 17:14
688c16d to
7e77665
Compare
ClickHouse interprets non-positive substring starts and negative lengths differently from PostgreSQL. Directly pushing down substring() and substr() can therefore change results or suppress PostgreSQL's negative-length error. Deparse text and bytea substring calls with PostgreSQL-compatible start and length adjustments, preserving NULL strictness and raising the PostgreSQL error for negative lengths. Keep safe constant bounds on the direct ClickHouse path. Add regression coverage for text and bytea offsets, negative lengths, NULL arguments, and generic plans.
fallintoplace
force-pushed
the
fix/substring-pushdown-semantics
branch
from
August 5, 2026 16:24
7e77665 to
09b2525
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
pg_clickhousetranslates PostgreSQL substring functions to ClickHouse for remote execution:substring(text, ...)/substr(text, ...)substringUTF8substring(bytea, ...)/substr(bytea, ...)substringThose mappings are only semantically safe for a subset of PostgreSQL bounds. This change retains pushdown for safe constant bounds and evaluates every other case locally in PostgreSQL.
PostgreSQL and ClickHouse differ
The difference affects both mappings, not just text values:
textsubstring('val1' FROM -1)val1textsubstring('val1' FROM 0 FOR 2)vtextsubstring('val1' FROM 2 FOR -1)negative substring length not allowedbyteasubstring('val1'::bytea FROM -1 FOR 3)vbyteasubstring('val1'::bytea FROM 0 FOR 2)vbyteasubstring('val1'::bytea FROM 2 FOR -1)negative substring length not allowedUnconditionally shipping these calls changes results and can suppress a PostgreSQL error.
Approach
Push down
substringandsubstronly when their bounds are known-safe constants at planning time:substring(value, start)/substr(value, start)startis a non-NULL constant>= 1startis NULL, non-constant, zero, or negativesubstring(value, start, length)/substr(value, start, length)start >= 1andlength >= 0, both non-NULL constantsThe guard is applied consistently to text and bytea overloads. Safe cases such as
substring(val, 1, 3)andsubstr(val, 2)continue to use ClickHouse. A parameterized offset is evaluated locally because its value is unavailable when the remote expression is planned.Tests
Regression coverage includes:
EXPLAINoutput confirming unsafe calls stay out of the remote SQLVerified with:
on PostgreSQL 19 against ClickHouse 26.3.17.56 and 23.3.22.3.