You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
feat: expose disableSpill and maxTempDirectorySize on SessionContextBuilder #83
Is your feature request related to a problem or challenge?
PR #28 added tempDirectory(String) to SessionContextBuilder so callers can route DataFusion's spill files to a chosen directory. That setter is the only Java surface for DataFusion's RuntimeEnvBuilder disk-manager knobs today. Two real gaps remain — all three reachable on the Rust side, none reachable from Java:
No way to disable spill entirely. Upstream offers DiskManagerMode::Disabled for memory-only execution (queries that need spill fail with ResourcesExhausted rather than going to disk). Useful for pinning latency-sensitive queries to memory or for environments without writable disk.
No way to cap the spill volume size. Upstream's RuntimeEnvBuilder::with_max_temp_directory_size(u64) exists; without exposing it, a runaway sort or hash-aggregate can fill the spill disk — which on a multi-tenant node is a co-tenant outage, not just a query failure.
None of the two is reachable via setOption(...). setOption routes through DataFusion's ConfigOptions::set(key, value). The disk-manager configuration lives on RuntimeEnv construction, not in that namespace.
Describe the solution you'd like
Three new setters on SessionContextBuilder, sitting next to the existing tempDirectory(String):
// Force memory-only execution; queries that would need spill fail fast:SessionContext.builder()
.disableSpill()
.build();
// Single-dir + cap (the common case; the existing tempDirectory still works):SessionContext.builder()
.tempDirectory("/tmp/df-spill")
.maxTempDirectorySize(10L << 30)
.build();
changed the title [-]feat: full disk-manager configuration on `SessionContextBuilder` (multi-dir spill, disable, size cap)[/-][+]feat: full disk-manager configuration on `SessionContextBuilder` (disable, size cap)[/+]on May 21, 2026
changed the title [-]feat: full disk-manager configuration on `SessionContextBuilder` (disable, size cap)[/-][+]feat: expose disableSpill and maxTempDirectorySize on SessionContextBuilder[/+]on May 21, 2026
Activity
[-]feat: full disk-manager configuration on `SessionContextBuilder` (multi-dir spill, disable, size cap)[/-][+]feat: full disk-manager configuration on `SessionContextBuilder` (disable, size cap)[/+][-]feat: full disk-manager configuration on `SessionContextBuilder` (disable, size cap)[/-][+]feat: expose disableSpill and maxTempDirectorySize on SessionContextBuilder[/+]