Skip to content

Commit 60e0c38

Browse files
committed
docs(udf): update scalar UDF guide for ColumnarValue API
1 parent 34248fd commit 60e0c38

1 file changed

Lines changed: 49 additions & 10 deletions

File tree

‎docs/source/user-guide/scalar-udf.md‎

Lines changed: 49 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -19,9 +19,10 @@ under the License.
1919

2020
# Scalar UDFs
2121

22-
A scalar UDF is a Java-implemented SQL function that operates on one row at a
23-
time, expressed in vectorised form: each invocation receives a batch of input
24-
columns and returns a single output column of the same length.
22+
A scalar UDF is a Java-implemented SQL function that operates one row at a time,
23+
expressed in vectorised form: each invocation receives a batch of input columns
24+
and returns either a per-row output column of the same length (`Array`) or a
25+
single value broadcast to every row (`Scalar`).
2526

2627
## Implement
2728

@@ -32,10 +33,11 @@ per-batch `evaluate` body:
3233
```java
3334
import java.util.List;
3435
import org.apache.arrow.memory.BufferAllocator;
35-
import org.apache.arrow.vector.FieldVector;
3636
import org.apache.arrow.vector.IntVector;
3737
import org.apache.arrow.vector.types.pojo.ArrowType;
38+
import org.apache.datafusion.ColumnarValue;
3839
import org.apache.datafusion.ScalarFunction;
40+
import org.apache.datafusion.ScalarFunctionArgs;
3941
import org.apache.datafusion.Volatility;
4042

4143
public final class AddOne implements ScalarFunction {
@@ -47,24 +49,59 @@ public final class AddOne implements ScalarFunction {
4749
@Override public Volatility volatility() { return Volatility.IMMUTABLE; }
4850

4951
@Override
50-
public FieldVector evaluate(BufferAllocator allocator, List<FieldVector> args) {
51-
IntVector in = (IntVector) args.get(0);
52+
public ColumnarValue evaluate(BufferAllocator allocator, ScalarFunctionArgs args) {
53+
IntVector in = (IntVector) args.args().get(0).vector();
5254
IntVector out = new IntVector("add_one", allocator);
5355
out.allocateNew(in.getValueCount());
5456
for (int i = 0; i < in.getValueCount(); i++) {
5557
if (in.isNull(i)) out.setNull(i);
5658
else out.set(i, in.get(i) + 1);
5759
}
5860
out.setValueCount(in.getValueCount());
59-
return out;
61+
return ColumnarValue.array(out);
6062
}
6163
}
6264
```
6365

66+
Each entry in `args.args()` is a `ColumnarValue` — either `ColumnarValue.Array`
67+
(a per-row vector of length `args.rowCount()`) or `ColumnarValue.Scalar` (a
68+
length-1 vector representing a single literal or folded constant). Access the
69+
underlying Arrow vector with `.vector()`.
70+
6471
Allocate any new vectors — including the result — from the supplied
6572
`BufferAllocator`. The input vectors are read-only views; do not close them.
6673
Ownership of the returned vector transfers to the framework on return.
6774

75+
## Returning a Scalar
76+
77+
Functions that yield a single value (nullary constants like `pi()`, or any
78+
function that wants the framework to broadcast a result across the batch) can
79+
return `ColumnarValue.scalar(...)` over a length-1 vector:
80+
81+
```java
82+
public final class JavaPi implements ScalarFunction {
83+
private static final ArrowType FLOAT64 =
84+
new ArrowType.FloatingPoint(org.apache.arrow.vector.types.FloatingPointPrecision.DOUBLE);
85+
86+
@Override public String name() { return "java_pi"; }
87+
@Override public List<ArrowType> argTypes() { return List.of(); }
88+
@Override public ArrowType returnType() { return FLOAT64; }
89+
@Override public Volatility volatility() { return Volatility.VOLATILE; }
90+
91+
@Override
92+
public ColumnarValue evaluate(BufferAllocator allocator, ScalarFunctionArgs args) {
93+
org.apache.arrow.vector.Float8Vector out =
94+
new org.apache.arrow.vector.Float8Vector("pi", allocator);
95+
out.allocateNew(1);
96+
out.set(0, Math.PI);
97+
out.setValueCount(1);
98+
return ColumnarValue.scalar(out);
99+
}
100+
}
101+
```
102+
103+
The framework expands the scalar across `args.rowCount()` rows automatically.
104+
68105
## Register
69106

70107
Wrap the implementation in a `ScalarUdf` and pass it to
@@ -90,9 +127,11 @@ non-deterministic functions.
90127
## Errors
91128

92129
If the UDF throws, the exception class and message surface in the
93-
`RuntimeException` raised from `collect()`. If the returned vector is `null`,
94-
has the wrong row count, or the wrong type, the runtime raises a
95-
`RuntimeException` with a descriptive message.
130+
`RuntimeException` raised from `collect()`. If the returned `ColumnarValue` is
131+
`null`, an Array result's vector length does not equal `args.rowCount()`, or
132+
the result's Arrow type differs from the declared return type, the runtime
133+
raises a `RuntimeException` with a descriptive message. A Scalar result whose
134+
vector is not length-1 is rejected at the `ColumnarValue.scalar` factory.
96135

97136
## Threading
98137

0 commit comments

Comments
 (0)