Skip to content

Native GROUP BY: admit TINYINT and SMALLINT SUM/MIN/MAX and reconcile coverage docs #229

Description

@liuyongvs

Verified on PR #220 after rebasing onto merged #222: commit 16ebb1e5a45cc5420abf90473b785d6c0b908a82, released Apache Flink 2.2.1, JDK 17, default compatibility settings. Each probe uses independent stock-Flink and native-enabled environments with bounded runtime DataStream input. Successful comparisons check output values and resolved result types before classifying execution.

Suggested priority: P2. This is a verified native support gap; the successful probes return equal results through explicit Flink fallback.

Runtime TINYINT/SMALLINT aggregates fall back even with ONE_PHASE selected, although the type table in docs/operators/group-by.md marks these types native. TWO_PHASE probes also fail the local/global partial-type gates. This is a routing and documentation discrepancy, not a wrong-result observation.

Runtime probe

Input type:

ROW<`f0` INT, `f1` TINYINT, `f2` SMALLINT, `f3` BIGINT, `f4` DECIMAL(20, 2), `f5` STRING, `f6` DATE, `f7` TIME(3), `f8` TIMESTAMP_LTZ(3), `f9` BOOLEAN> NOT NULL

Input: [+I[1, 3, 4, 5, 6.25, text, 2024-02-29, 12:34, 2024-02-29T12:34:56Z, true]]

SELECT f0, SUM(f1) FROM src GROUP BY f0;

Flink result: [[1, 3]]; resolved type: [INT, TINYINT]. Native substitutions: 0.

GROUP BY: SUM over an unsupported value type

Runtime probe

Input type:

ROW<`f0` INT, `f1` TINYINT, `f2` SMALLINT, `f3` BIGINT, `f4` DECIMAL(20, 2), `f5` STRING, `f6` DATE, `f7` TIME(3), `f8` TIMESTAMP_LTZ(3), `f9` BOOLEAN> NOT NULL

Input: [+I[1, 3, 4, 5, 6.25, text, 2024-02-29, 12:34, 2024-02-29T12:34:56Z, true]]

SELECT f0, MAX(f2) FROM src GROUP BY f0;

Flink result: [[1, 4]]; resolved type: [INT, SMALLINT]. Native substitutions: 0.

GROUP BY: MIN/MAX over an unsupported value type

Configuration for these examples: table.optimizer.agg-phase-strategy=ONE_PHASE. A second run enables mini-batching (size=5, latency=100 ms) and TWO_PHASE; both local and global gates reject the narrow partial types. Check wrapping SUM overflow, result widths, MIN/MAX, NULLs, retractions and checkpoint state before admission. Correct the docs if a shape remains intentionally excluded.

Acceptance

Preserve released Flink values, resolved types, NULL behavior and relevant error/short-circuit semantics. Add runtime-source SQL regressions that assert the specific native operator as well as result parity; do not use constant folding as proof of native support. Keep safe fallback for unverified forms and update the corresponding operator coverage documentation.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions