Skip to content

Parity tests: compare arrays nested in MAP and ROW results by value #238

Description

@liuyongvs

At PR #220 commit 16ebb1e5a45cc5420abf90473b785d6c0b908a82, src/test/java/tech/streamfusion/NativeParity.java normalizes direct arrays recursively but returns MAP and ROW containers unchanged. Java Map.equals compares an array-valued map entry by array object identity. Independent executions therefore produce false mismatches for equal MAP<STRING, ARRAY> results.

Suggested priority: P3, parity-test infrastructure for complex types. This is a test comparator defect, not a demonstrated native engine result error.

Reproduction

var left = Map.of("a", new Integer[] {1, 2});
var right = Map.of("a", new Integer[] {1, 2});
assertEquals(
    NativeParity.comparableValue(left),
    NativeParity.comparableValue(right));

The assertion fails even though the logical MAP/ARRAY contents are equal. Three adapted upstream MapFunctionITCase MAP_UNION SQL cases reproduced the false mismatch during the runtime audit. All had zero native substitutions and Calc: unsupported function/operator: MAP_UNION, so both executions used Flink. Normalizing nested containers recursively in the local audit helper made all 280 collection/map SQL cases complete without a result mismatch or native-only failure (201 fallback successes, 8 native successes and 71 both-error cases, which are excluded from successful parity).

Acceptance

  • Compare nested MAP keys/values, ROW fields and arrays by logical contents while retaining NULL keys/values, duplicate-sensitive array ordering, binary contents and declared result types.
  • Add positive equal-content and negative changed-content regressions, including maps containing arrays and rows containing maps.
  • Preserve the SQL type/schema check and avoid stringifying arbitrary containers or using Java object addresses as a result oracle.
  • Keep routing checks independent from value comparison, as tracked in CI: reconcile function fallback assertions with admission in #220 #223.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions