Skip to content

Commit ab37f87

Browse files
committed
docs(contributor-guide): document proto-over-JNI convention
1 parent f13b51b commit ab37f87

1 file changed

Lines changed: 31 additions & 0 deletions

File tree

‎docs/source/contributor-guide/development.md‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,4 +74,35 @@ disk space.
7474
- `mvnw`, `mvnw.cmd` — bundled Maven wrapper.
7575
- `src/` — Java sources and tests.
7676
- `native/` — Rust crate (JNI + Arrow C Data Interface).
77+
- `proto/` — Protobuf definitions shared between Java and Rust.
7778
- `docs/` — Sphinx documentation source and build scripts.
79+
80+
## Passing structured options across the JNI boundary
81+
82+
When a JNI call needs to carry more than a handful of scalar arguments —
83+
for example, a struct of nullable knobs like `CsvReadOptions` or
84+
`SessionOptions` — encode the call's configuration as a protobuf message
85+
rather than expanding the JNI signature with one parameter per field.
86+
87+
Add a `.proto` file under `proto/`, declare `package datafusion_java;`,
88+
and follow the conventions already in use:
89+
90+
- One message per logical bundle of options.
91+
- Use `optional` for fields whose unset-ness must survive the boundary
92+
(so the Rust side can leave a DataFusion default in place).
93+
- Use proto enums for closed sets of choices instead of strings.
94+
- Keep large opaque payloads (Arrow IPC schemas, plan nodes) as separate
95+
`byte[]` JNI arguments next to the options proto, not inside it.
96+
- Suffix the message name with `Proto` if a sibling Java class would
97+
otherwise shadow it (e.g. `CsvReadOptionsProto` vs the public
98+
`CsvReadOptions`).
99+
100+
The proto is compiled by both `prost-build` (Rust, via `native/build.rs`)
101+
and the Maven `protobuf-maven-plugin` (Java). The Java side builds the
102+
message, serializes to bytes, and passes the byte array through JNI; the
103+
Rust side decodes once and folds the fields into the corresponding
104+
DataFusion struct.
105+
106+
This pattern keeps JNI signatures short, makes nullable and enum fields
107+
explicit in a single typed schema, and lets new fields be added without
108+
touching the signature on either side.

0 commit comments

Comments
 (0)