You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Bug] concat_ws panics when all arguments are scalars and number_rows > 1 #314
concat_ws panics when every argument is a scalar and number_rows is greater than 1. spark_concat_ws (string/concat_ws.rs#L176) calls ColumnarValue::values_to_arrays, which returns 1-row arrays when no argument is an array. It then loops over 0..number_rows and indexes past the end of those arrays.
With the default optimizer this is hidden, because a literal-only call is constant-folded during planning and evaluated on a one-row batch. It panics when the expression is not folded, for example when an engine such as Comet builds the physical expression directly.
To Reproduce
Run with the simplify_expressions optimizer rule removed (Optimizer::new().rules without it, passed to SessionStateBuilder::with_optimizer_rules):
SELECT concat_ws(',', 'a', 'b') AS result FROMVALUES (1), (2), (3) AS t(x);
-- panic: Trying to access an element at index 1 from a StringArray of length 1
Expected behavior
Three rows of a,b. Either compute one row and return a ColumnarValue::Scalar when every argument is a scalar, or expand each argument with to_array(number_rows). #68 fixed the same pattern in hash and xxhash64.
Additional context
Found by the new evaluate mode of the sqllogictest runner, which runs the .slt files without simplify_expressions. That query is skipped in evaluate mode until this is fixed.
Activity