Push down JSONB existence predicates

[k-bx]

Summary

• Push PostgreSQL jsonb ? text predicates down for native ClickHouse JSON columns.
• Preserve PostgreSQL top-level object-key and array-string semantics.
• Support compatibility views that parse JSON documents stored in ClickHouse String columns, while leaving arbitrary JSON expressions local.
• Handle nullable String-backed documents without producing unsupported nullable nested array types in ClickHouse.

Validation

• make -j2
• Focused PostgreSQL 18 regression test for native JSON, nullable String-backed JSON, combined predicates, aggregate pushdown, and local fallback
• Tested on a real-world production canary system with PostgreSQL 18 and ClickHouse 26.3: EXPLAIN (FORMAT JSON, VERBOSE) confirmed that JSONHas, nullable-document handling, and count were all executed remotely; a month-range aggregate completed successfully, and a pushed single-row predicate matched a deliberately local PostgreSQL evaluation on the same replicated row

[theory]

Thank you, looks handy, but the tests are insufficient. Can you improve them?

[theory]

This test does not properly validate execution on the ClickHouse side. Please create the corresponding tables and data in ClickHouse and execute the queries with and without EXPLAIN (VERBOSE COSTS OFF to show both the CH SQL generated and its successful execution.

[k-bx]

Updated in 9fa88ff.

The regression now creates the native JSON and Nullable(String)-backed tables in ClickHouse, inserts representative object, array, scalar, and NULL data, and runs each predicate both with EXPLAIN (VERBOSE, COSTS OFF) and as an executing query. The expected output includes the full generated ClickHouse SQL and returned rows for native JSON, String-backed compatibility views, combined predicates, and the intentionally local fallback.

Validation:

• make -j2
• Full focused regression against PostgreSQL 18 / ClickHouse 26.3 on an isolated Kharkiv test database
• The same test update was applied to agent/v0.3.2-analytics-backports at 1869dcf

The PR branch was also rebased onto current main. GitHub Actions are currently awaiting maintainer approval.

[k-bx]

@theory I want to add that this has been used in (relatively-heavy) production instances at my org for quite some time, ever since PR was opened

[theory]

Almost there. I'm unable to push to your branch for some reason, so I pushed to the k-bx-agent/jsonb-exists-pushdown in our repo. Please have a look. In addition to the comments here in the PR, the results for ClickHouse 23.8-24.3 are unexpected. Here's the diff:

--- test/expected/jsonb_exists.out	2026-08-05 16:45:10
+++ test/expected/jsonb_exists_1.out	2026-08-05 16:52:46
@@ -84,7 +84,9 @@
  id 
 ----
   1
-(1 row)
+  2
+  3
+(3 rows)

Those results are quite wrong.

[theory]

Wow, okay, I think this makes sense now that I format it to make it more legible:

WHERE (
    (
        if(
            isNull(toJSONString(document)) OR isNull('key'),
            NULL,
            multiIf(
                JSONType(ifNull(toJSONString(document), 'null')) = 'Object',
                    JSONHas(ifNull(toJSONString(document), 'null'), 'key'),
                JSONType(ifNull(toJSONString(document), 'null')) = 'Array',
                arrayExists(
                    x -> JSONType(x) = 'String' AND JSONExtractString(x) = 'key', 
                    JSONExtractArrayRaw(ifNull(toJSONString(document), 'null'))
                ),
                0
            )
        )
    )
)

I get it for String, but is all that really necessary for a ClickHouse JSON column?

[theory]

Woof, okay, this makes sense:

SELECT id
FROM jsonb_exists.string_documents
WHERE (
    (
        if(
            isNull(document) OR isNull('key'),
            NULL,
            if(
                NOT isValidJSON(ifNull(document, 'null')),
                throwIf(1, 'invalid input syntax for type json'),
                multiIf(
                    JSONType(ifNull(document, 'null')) = 'Object',
                        JSONHas(ifNull(document, 'null'), 'key'),
                    JSONType(ifNull(document, 'null')) = 'Array',
                        arrayExists(
                            x -> JSONType(x) = 'String' AND JSONExtractString(x) = 'key',
                            JSONExtractArrayRaw(ifNull(document, 'null'))
                        ),
                    0
                )
            )
        )
    )
)

I don't suppose we can include any contextual information for the error? Maybe it could include the text it tires to pars into JSON.

[theory]

Okay, trying to understand this. I gather the point is to allow ? to work on a text column that Postgres implicitly uses jsonb_in to convert before evaluating. Is that correct?

I wonder if we'd be better off adding explicit pushdown for jsonb_in() (and perhaps other functions) as its own thing. That would mean adding case F_JSONB_IN: to lookup_builtin_func and creating the appropriate departed for it. Probably fine to do as a follow-up PR, but I suspect this won't be the last time we need to push down _in functions.