You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Port Bloom filter expressions for runtime filtering from Comet #159
An LLM (Claude Code) created this issue. It compared the native expressions in Apache DataFusion Comet (native/spark-expr) with the functions in this crate. Check the details before you rely on them.
Is your feature request related to a problem or challenge?
Comet has a native, Spark-compatible implementation of Bloom filter expressions for runtime filtering that datafusion-spark does not have.
Spark's InjectRuntimeFilter rule adds BloomFilterAggregate and BloomFilterMightContain to plans. The serialized filter must be byte-compatible with Spark's.
Describe the solution you'd like
Port the implementation and its tests from Comet into datafusion-spark, and add .slt cases under crates/sqllogictest. Comet can then use the datafusion-spark version and remove its copy.
Activity
datafusion-sparkSpark Compatible Functions #5datafusion-sparkSpark Compatible Functions #5