Track remaining native operator and type admission gaps after the PR #220 parity audit

[liuyongvs]

Track the remaining native admission and parity-test work at PR #220 commit 16ebb1e5a45cc5420abf90473b785d6c0b908a82 (rebased onto main fb6865a3, including #222). This is an inventory for prioritization, not a claim that every Flink operator or overload has been executed.

Executed coverage

Released Flink 2.2.1/JDK 17 was the reference. SQL expressions use bounded runtime inputs, independent host/native executions, result/schema comparison and routing evidence. The adapted upstream SQL arms and targeted probes produced 2,454 records: 1,720 native successes, 551 fallback successes, 167 both-error records and 16 scan/constant plans. These are execution records, including timezone variants and four valid-primary-key JOIN controls, not 2,454 unique functions. Both-error records are excluded from successful parity and are not proof of equivalent exception semantics.

Time-related cases include UTC, America/Los_Angeles and Asia/Shanghai. A separate portable SQL control suite and operator regressions passed 111 JUnit tests; five decimal probes overlap the records above. No confirmed new engine wrong-result or native-only execution failure remains in the reviewed results. This does not establish correctness of unexecuted contexts, input representations or profiles.

Runtime-confirmed gaps

The following issues include SQL/fixtures, observed results, fallback reasons and acceptance criteria. P2 basic-type coverage comes before P3 complex/floating-point expansion. Safe fallback is a support gap, not a wrong-result defect.

Priority	Work	Issue
P2	IF with TIMESTAMP_LTZ branches	#224
P2	DECIMAL ROUND/TRUNCATE with runtime scale	#225
P2	DECIMAL FLOOR/CEIL in verified scalar consumers	#226
P2	STRING TRY_CAST to BOOLEAN and temporal types	#228
P2	TINYINT/SMALLINT SUM/MIN/MAX and coverage-document discrepancy	#229
P2	DATE/TIME/BOOLEAN extrema and value aggregates	#230
P2	DISTINCT and two-phase aggregate/type coverage	#231
P2	LISTAGG and JSON string aggregates	#232
P2	ENCODE/DECODE with runtime charset	#233
P2/P3	JVM table/aggregate functions; nested scalar results	#237
P3	Remaining ARRAY/MAP functions, casts and aggregate results	#234
P3	Fixed-length BINARY generated-expression boundaries	#235
P3	Remaining floating math and compatibility gates	#236
P3	Test comparator: arrays nested in MAP/ROW	#238

The DECIMAL tickets explicitly distinguish successful scalar-consumer queries from released-host direct-collection scale/assertion problems. Do not infer intended decimal behavior from intuitive rounding alone or silently remove a compatibility gate.

Existing tickets to retain

• CI: reconcile function fallback assertions with admission in #220 #223: stale fallback assertions on PR Extend native Calc function coverage with released Flink semantics #220; a routing assertion can prevent the result comparison from running. Update the contract and execute value/schema/error checks.
• Native SQL/JSON: extend selectors beyond the verified definite-path grammar #91: remaining SQL/JSON non-Calc and MAP/MULTISET boundaries.
• Native window aggregation: support retracting input from upstream Top-N #99: remaining retracting window/aggregate shapes; much of the original scope is already shipped.
• Native retracting OFFSET: preserve upstream stateful mini-batch changelog order #102: retracting OFFSET with upstream stateful mini-batch changelog ordering.
• Native Top-N: support nullable and changing variable rank bounds #104: remaining nullable/changing rank-bound restrictions.
• Native window aggregation: support Flink split-distinct physical plans #166: window split-distinct physical plans.
• Implement MATCH_RECOGNIZE and pattern time functions #78: MATCH_RECOGNIZE and its pattern time functions.
• Track native SQL support gaps and parity-harness follow-ups from the September audit #110: Table API scalar AS wrapper admission and the older support tracker.
• Native GREATEST/LEAST: support runtime STRING, mixed exact numerics and TIMESTAMP/LTZ #209 and Native trim functions: support column-dependent trim sets #216: PR Extend native Calc function coverage with released Flink semantics #220 addresses the core extrema and dynamic-trim work; representation-dependent residual gates below must remain explicit. Do not interpret their older examples as all still failing on this branch.

Documented remaining operators: reproduction/triage required

These are explicit entries in docs/operators/unsupported.md at the audit commit. They are recorded here as unimplemented native paths, not new runtime reproductions from this audit. Establish a released-host-valid minimal plan and actual fallback before implementing each item.

• P2: GroupTableAggregate and GroupWindowTableAggregate (table aggregates; distinct from ordinary scalar-result UDAFs in Native JVM functions: track table, aggregate and nested scalar result gaps #237).
• P2: DeltaJoin plans under table.optimizer.delta-join.strategy; retain Flink's FORCE-mode validation behavior.
• P2: DropUpdateBefore; prove row-kind/changelog behavior across native boundaries.
• P3: Values; first distinguish an actual compute-path blocker from harmless constant-only plans.
• P3: LegacyTableSourceScan and LegacySink; evaluate need separately from modern connector acceleration.
• P3: PythonCalc, PythonCorrelate, PythonGroupAggregate, PythonOverAggregate and other Python operators; establish an explicit runtime boundary and lifecycle/state contract before support.

General table-function Correlate is covered by #237. Plain admitted UNNEST is already supported. A non-temporal Sort rejected by Flink in streaming is not a StreamFusion feature gap.

IncrementalGroupAggregate is a documented deliberate non-goal, not an implementation checkbox: the distinct-split five-node chain is excluded for the reasons recorded in docs/operators/unsupported.md. This tracker does not change that decision.

Documented expression/type boundaries needing dedicated reproduction

The following are conservative gates in docs/operators/calc-filter.md. They are not evidence of wrong results and were not all separately exercised by this runtime batch.

• P2: STRING extrema and dynamic trim with unknown/binary-backed source representations. Preserve the host's representation-dependent ordering/set behavior; related to Native GREATEST/LEAST: support runtime STRING, mixed exact numerics and TIMESTAMP/LTZ #209/Native trim functions: support column-dependent trim sets #216.
• P2: error-capable exact arithmetic, casts, rounding and similar expressions under AND/OR. Preserve row short-circuiting; direct arithmetic and selected CASE branches already have broader coverage. Do not remove these guards based only on non-error inputs.
• P2: intermediate JSON-derived STRING across GROUP BY, DISTINCT, JOIN, sort or other operator boundaries. Fused scalar consumers are supported; preserving final output bytes alone does not establish intermediate UTF-16 identity. The original wrong-result defect Preserve surrogate identity in intermediate JSON STRING expressions #81 is closed and is not being reopened here.
• P2/P3: SpecializedFunction and unsupported scalar UDF conversion signatures, including temporal and nested types. Resolve Flink call specialization and external/internal value contracts before admitting them; related to Native JVM functions: track table, aggregate and nested scalar result gaps #237 and the AS backlog in Track native SQL support gaps and parity-harness follow-ups from the September audit #110.
• P2/P3: review type/key/accumulator restrictions in each operator coverage page and reduce uncovered combinations to independent host-valid tests; scalar function admission does not imply GROUP BY/window/OVER/join-key support.

Completion requirements and scope limits

• Keep one entry per remaining contract, link dedicated issues as a scope becomes independently reproducible, and trim entries when shipped. Do not reopen already solved issues merely because this tracker references their history.
• Compare values, declared types, NULLs, failure/short-circuit behavior, precision/scale and temporal timezone behavior; stateful operators also require changelog/retraction/checkpoint evidence.
• Assert native work for admitted cases and explicit fallback for retained restrictions. Scan-only and constant-folded plans do not prove function execution.
• This audit adapts SQL arms from upstream tests. It is not a run of every unchanged upstream ITCase, every Table API arm, all Flink versions, original connector services or full failure/recovery deployments. Host-rejected SQL, external fixture prerequisites and nondeterministic outputs need separate accounting.
• Update the relevant operator/connector coverage documentation in each eventual implementation commit. No implementation is included in this issue.