Skip to content

Commit 3ff4fda

Browse files
timsaucerclaude
andcommitted
fix: scope foundation docs/excludes to what 1/6 ships
Address review feedback on the workspace-foundation PR: - development.md: trim the repo-layout section to the crates this PR actually ships (native, native-common). It was forward-referencing spark/, spark/bridge, datafusion-spark-bridge, and examples/native -- none of which exist until later PRs in the stack -- and called the member list "three" while listing four. Later PRs (#105/#106/#107/#109) carry notes to re-add their own slice when those dirs land. - rat_exclude_files.txt: the Rust lockfile moved to the workspace root, so the stale native/Cargo.lock entry left the root Cargo.lock with no RAT exclude for the source-tarball check (check-rat-report.py). Point it at Cargo.lock. - native-common: dedupe the panic-payload downcast -- StreamingReader::next now calls errors::panic_message instead of repeating the String/&str match inline. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 2514f56 commit 3ff4fda

4 files changed

Lines changed: 17 additions & 23 deletions

File tree

‎dev/release/rat_exclude_files.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77
.mvn/wrapper/maven-wrapper.properties
88
mvnw
99
mvnw.cmd
10-
native/Cargo.lock
10+
Cargo.lock
1111
dev/release/rat_exclude_files.txt
1212
docs/source/_static/**
1313
docs/source/conf.py

‎docs/source/contributor-guide/development.md‎

Lines changed: 11 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -74,25 +74,22 @@ disk space.
7474

7575
The repository is a multi-module Maven build:
7676

77-
- `Cargo.toml` — Rust workspace root declaring the three crate members
78-
(`native`, `native-common`, `examples/native`, `spark/bridge`) and `[workspace.dependencies]`
79-
that pin shared versions in one place. Cargo writes artifacts to
80-
`rust-target/` (overridden in `.cargo/config.toml`) so `mvn clean` at the
81-
repo root does not nuke the Rust build cache.
82-
- `pom.xml` — parent POM declaring the `core`, `spark`, and `examples`
83-
modules and shared plugin/dependency versions.
77+
- `Cargo.toml` — Rust workspace root declaring the crate members
78+
(`native`, `native-common`) and `[workspace.dependencies]` that pin
79+
shared versions in one place. Cargo writes artifacts to `rust-target/`
80+
(overridden in `.cargo/config.toml`) so `mvn clean` at the repo root does
81+
not nuke the Rust build cache.
82+
- `pom.xml` — parent POM declaring the `core` and `examples` modules and
83+
shared plugin/dependency versions.
8484
- `core/` — `datafusion-java` library module (Java sources, tests, and
8585
generated protobuf classes).
86-
- `spark/` — `datafusion-java-spark` Spark DataSource V2 connector
87-
(Scala + Java, pure JVM) and its `spark/bridge/` Rust SDK crate
88-
(`datafusion-spark-bridge`: widening, scan machinery, `export_bridge!`).
8986
- `examples/` — `datafusion-java-examples` module containing runnable
9087
examples that depend on the library; built alongside the library so they
91-
cannot fall out of sync with the API. Includes `examples/native/`, a
92-
small `export_bridge!` cdylib used by the Spark connector demo
93-
(`ExampleBridgeProviderFactory` + the pyspark script under
94-
`examples/python/`).
88+
cannot fall out of sync with the API.
9589
- `native/` — `datafusion-jni` Rust crate (JNI + Arrow C Data Interface).
90+
- `native-common/` — `datafusion-jni-common` Rust crate: JNI plumbing
91+
shared across native crates (error→exception mapping, the per-cdylib
92+
Tokio runtime singleton, the async-stream→`FFI_ArrowArrayStream` bridge).
9693
- `proto/` — Protobuf definitions shared between Java and Rust.
9794
- `Makefile` — top-level build orchestration (`make test`, `make format`,
9895
`make tpch-data`).

‎native-common/src/errors.rs‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -164,7 +164,10 @@ fn throw(env: &mut JNIEnv, class: &str, message: &str) {
164164
let _ = env.throw_new(class, message);
165165
}
166166

167-
fn panic_message(panic: &Box<dyn Any + Send>) -> String {
167+
/// Best-effort extraction of a panic payload's message. `catch_unwind` hands
168+
/// back a `Box<dyn Any>`; the payload is a `String` or `&str` for ordinary
169+
/// `panic!`/`unwrap` sites, anything else is opaque.
170+
pub fn panic_message(panic: &Box<dyn Any + Send>) -> String {
168171
if let Some(s) = panic.downcast_ref::<String>() {
169172
s.clone()
170173
} else if let Some(s) = panic.downcast_ref::<&str>() {

‎native-common/src/lib.rs‎

Lines changed: 1 addition & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -82,13 +82,7 @@ impl Iterator for StreamingReader {
8282
match next {
8383
Ok(item) => item.map(|r| r.map_err(|e| ArrowError::ExternalError(Box::new(e)))),
8484
Err(panic) => {
85-
let msg = if let Some(s) = panic.downcast_ref::<String>() {
86-
s.clone()
87-
} else if let Some(s) = panic.downcast_ref::<&str>() {
88-
(*s).to_string()
89-
} else {
90-
"rust panic with non-string payload".to_string()
91-
};
85+
let msg = errors::panic_message(&panic);
9286
Some(Err(ArrowError::ExternalError(
9387
format!("panic in DataFrame stream: {msg}").into(),
9488
)))

0 commit comments

Comments
 (0)