Flink 1.18: connector and format parity matrix

[Flanderzz]

Current behavior

The admitted Flink 1.18 connector and format modules build and pass their selected test suites. Delta acceleration is unavailable on this line. Correct results alone do not prove that native decode/encode ran: the Kafka upstream suite still lacks blocking execution witnesses for those paths.

Shipped

• Per-line connector and format releases are selected by the flink-1.18 profile:
  flink-connector-kafka:3.2.0-1.18 with kafka-clients 3.4.0, paimon-flink-1.18:2.0.0, the
  1.18.1 formats, Calcite 1.32.0 and the protobuf 3.21.7 runtime that flink-protobuf:1.18.1
  generates against.
• All nine tagged module suites run on both lines in CI: Kafka, JSON, CSV, Raw, Avro,
  Avro-Confluent, Protobuf, Parquet and ORC.
• Paimon runs on both lines in CI for Parquet and ORC, and the upstream Paimon suite runs on both
  lines against its released 1.18 test contract.
• The upstream formats, parquet, orc and kafka suites run on both lines.
• Delta is decided: Audit legacy Delta SQL compatibility with released Flink 1.18 #197 audited the released legacy connector against Flink 1.18 and the
  Kernel-based implementation now belongs only to the 2.2 source root. There is no admitted
  native Delta connector on 1.18, and no 1.18 Delta payload is built or published. This is
  unavailable functionality, not a verified host fallback.
• Per-line format divergences are enumerated in docs/flink-compatibility.md: Avro legacy
  timestamp mapping only, Parquet map-key nullability in the footer, the 1.18 JSON decoder
  rejecting top-level arrays, the absent parser-selection option, and the older Jackson node
  factory's decimal spelling carried into JSON, CDC JSON and CSV encoders.

Remaining

• Add blocking native decode/encode work evidence for selected unchanged upstream Kafka source and sink/serialization cases on both 1.18 and 2.2. The 1.18 Kafka suite passes all 72 cases with real brokers, and native source tasks appear in its logs, but there are no per-invocation Kafka route contracts or mandatory Kafka native-work witnesses. Flink continues to own broker I/O; the witness must establish successful nonempty native codec work, not just planner installation or operator creation.
• Preserve the upstream SQL and assertions. Require exact reasons for intentional fallback and test that removing expected native execution fails the gate. Record native, expected-fallback and unclassified cases separately; coordinate with CI: account for native and fallback routes across upstream SQL tests #168. The general upstream format suites also do not yet require per-format native codec evidence, so their passing totals must not be presented as universal acceleration coverage.
• The released Kafka connector boundary and pinned client combinations are documented in docs/connectors/kafka.md by PR #289, commit 8ec4ec0c. This completes the documentation item, not the remaining blocking native codec witnesses.

Done when

The selected upstream Kafka cases fail CI if their expected native decode/encode path disappears, deliberate fallback is validated explicitly, and the released connector version boundary is documented. The dependency and result-parity groundwork is already shipped.

Verified on main 89da01cdad7342c4d825e36eaf4e8dc73bc7af9f: CI and upstream suites both completed successfully, with 47 successful jobs in total. Results were rechecked on September 20, 2026. The upstream coverage documentation records the unclassified Kafka invocations and the bounded Paimon 1.18 corpus.

Split from #182.