Skip to content

[Bug] collect_list(x ORDER BY ...) fails with an internal error #112

Open

Description

@andygrove

Describe the bug

collect_list with an ORDER BY inside the call fails with an internal error:

SELECT collect_list(a ORDER BY a DESC) FROM (VALUES (3), (1), (2)) t(a);
-- DataFusion error: Internal error: Assertion failed: values.len() == 1 (left: 2, right: 1): expects single batch.

Spark SQL has no ORDER BY inside collect_list, so there is no Spark result to match, but DataFusion's SQL accepts the syntax and an internal error is never the right answer.

Expected behavior

Either support the ordering (as DataFusion's array_agg(a ORDER BY ...) does), or reject it at planning time with a clear error. collect_set(a ORDER BY a DESC) does not fail: it ignores the ordering and returns the values in arbitrary order ([2, 1, 3] for the query above). Since a set has no order that is harmless, but rejecting it would be clearer.

Additional context

Found while documenting collect_list (#108).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions