Skip to content

[Bug] concat_ws panics when all arguments are scalars and number_rows > 1 #314

Open

Description

@andygrove

Describe the bug

concat_ws panics when every argument is a scalar and number_rows is greater than 1. spark_concat_ws (string/concat_ws.rs#L176) calls ColumnarValue::values_to_arrays, which returns 1-row arrays when no argument is an array. It then loops over 0..number_rows and indexes past the end of those arrays.

With the default optimizer this is hidden, because a literal-only call is constant-folded during planning and evaluated on a one-row batch. It panics when the expression is not folded, for example when an engine such as Comet builds the physical expression directly.

To Reproduce

Run with the simplify_expressions optimizer rule removed (Optimizer::new().rules without it, passed to SessionStateBuilder::with_optimizer_rules):

SELECT concat_ws(',', 'a', 'b') AS result FROM VALUES (1), (2), (3) AS t(x);
-- panic: Trying to access an element at index 1 from a StringArray of length 1

Expected behavior

Three rows of a,b. Either compute one row and return a ColumnarValue::Scalar when every argument is a scalar, or expand each argument with to_array(number_rows). #68 fixed the same pattern in hash and xxhash64.

Additional context

Found by the new evaluate mode of the sqllogictest runner, which runs the .slt files without simplify_expressions. That query is skipped in evaluate mode until this is fixed.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions