[raminqaf]

What is the purpose of the change

Flink compares VARIANT values by their binary encoding, but one value has many encodings. So functions and operators that compare array elements, map keys or collected values treat equal VARIANT values as different and return wrong results, without an error. #29411 rejects VARIANT in GROUP BY, joins and comparisons. This PR does the same for these functions and operators.

-- t(s STRING) has three rows: {"a":1,"b":2}, {"b":2,"a":1}, {"a":1,"c":3}
SELECT ARRAY_CONTAINS(ARRAY[PARSE_JSON(s)['a']], PARSE_JSON('1')) FROM t;

Query	Before	After
the ARRAY_CONTAINS above	false	An array element of type VARIANT cannot be used in ARRAY_CONTAINS, because the type has no equality. Cast the value to a comparable type first.

The same validator rejects VARIANT in GROUP BY and joins, so the message has the same form.

Brief change log

• ComparableTypeValidator checks the array elements and map keys of ARRAY_CONTAINS, ARRAY_DISTINCT, ARRAY_POSITION, ARRAY_REMOVE, ARRAY_UNION, ARRAY_EXCEPT, ARRAY_INTERSECT, MAP_CONTAINS_KEY and MAP_UNION. It also checks the map key of an item access m[k] and the argument of COLLECT. An index into an array and a field access on a VARIANT compare nothing and stay allowed. The query fails during planning.
• The validator names what it checks in sets: the functions, the operators that compare elements, and the comparison operators that need equality or order.
• LogicalTypeChecks#areComparable rejects every type that is not comparable, see LogicalTypeChecks#isComparableType. This also covers a structured type with a VARIANT attribute. GREATEST, LEAST, MAP_FROM_ENTRIES and Table API comparisons reject VARIANT through it.
• The VARIANT section of the data types page gives a few examples of where a query compares values, instead of a full list that would soon be out of date.
• KeyTypeValidator from [FLINK-40910][table] Reject VARIANT where a query needs equality or ordering #29411 is now ComparableTypeValidator, and isComparableKeyType is now isComparableType. The planner also uses key for upsert, unique and primary keys, and the validator checks every comparison, not only keys.

An earlier version declared the check in the input type strategy of each function. That needed a new argument strategy and a flag in three collection strategies, about 160 lines of main code, for a check that a future VARIANT equality may remove. The validator does it in a few lines. The trade-off is a function list kept apart from the function definitions. A new function that compares elements or keys has to be added to it.

Ten functions build their equality check again during code generation: ARRAY_CONTAINS, ARRAY_DISTINCT, ARRAY_POSITION, ARRAY_REMOVE, ARRAY_UNION, ARRAY_EXCEPT, ARRAY_INTERSECT, MAP_CONTAINS_KEY, MAP_UNION and MAP_FROM_ENTRIES. That check goes through LogicalTypeChecks#areComparable, not through the validator, which only runs when a query is planned. So a job restored from a compiled plan that calls one of them on VARIANT elements or keys now fails when it is executed, before the job is submitted. Such a plan returned wrong results before. A user SpecializedFunction that builds an = evaluator on VARIANT values now fails too, in new and in restored plans. Other restored plans are not affected. A restored plan with m[k] or COLLECT on VARIANT still runs and matches by bytes, because the validator never sees it. No test restores such a plan. ComparableInputTypeStrategyTest covers the check that makes it fail.

Not in this PR: MAP[...] and MAP_FROM_ARRAYS still build maps with VARIANT keys, and such a map can hold the same key twice. The data types page already says so. A follow-up issue will track them, together with MAP<VARIANT, ...> and MULTISET<VARIANT> as types.

Verifying this change

This change added tests and can be verified as follows:

• ComparableTypeValidatorTest rejects a call of each of the nine functions with VARIANT elements or keys, m[k] with a VARIANT key, and COLLECT on VARIANT values in an aggregate and in an OVER window. It accepts MAP_CONTAINS_KEY on a map with VARIANT values, an index into an ARRAY<VARIANT> and a field access on a VARIANT.
• ComparableInputTypeStrategyTest covers VARIANT in areComparable, which GREATEST, LEAST, MAP_FROM_ENTRIES and restored plans go through. LogicalTypeChecksTest covers VARIANT nested in other types, including a structured type with a VARIANT attribute.

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changed class annotated with @Public(Evolving): no. Calls of the functions and operators above with VARIANT elements or keys now fail, which needs a release note.
• The serializers: no
• The runtime per-record code paths (performance sensitive): no
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: yes. A restored compiled plan that calls one of the ten functions above on VARIANT elements or keys fails when it is executed.
• The S3 file system connector: no

Documentation

• Does this pull request introduce a new feature? no
• If yes, how is the feature documented? not applicable

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Generated-by: Claude Code 2.1.310 (Claude Opus 5.5)

[flinkbot]

CI report:

• 1c96af0 Azure: SUCCESS
• 25e6104 UNKNOWN

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[VasShabu]

Hi Ramin

First review so please let me know if my logic is flawed

Noticed that your comment key type of a MAP are the first child however if this changes in the future so that "key type of a MAP are the second child" some builtin functions may not give the intended output like Map_Contains_Key.

[twalthr]

LGTM, I just had one last doc simplification comment.