From 80f84c5933967518bb1de5fac19480596f0b6a39 Mon Sep 17 00:00:00 2001 From: Andrew Lamb Date: Sun, 16 Aug 2026 07:29:53 -0400 Subject: [PATCH] Add draft blog post for DataFusion 55.0.0 release Co-Authored-By: Claude Fable 5 --- content/blog/2026-08-16-datafusion-55.0.0.md | 517 +++++++++++++++++++ 1 file changed, 517 insertions(+) create mode 100644 content/blog/2026-08-16-datafusion-55.0.0.md diff --git a/content/blog/2026-08-16-datafusion-55.0.0.md b/content/blog/2026-08-16-datafusion-55.0.0.md new file mode 100644 index 00000000..dd045de4 --- /dev/null +++ b/content/blog/2026-08-16-datafusion-55.0.0.md @@ -0,0 +1,517 @@ +--- +layout: post +title: Apache DataFusion 55.0.0 Released +date: 2026-08-16 +author: pmc +categories: [release] +--- + + + +[TOC] + +We are proud to announce the release of [DataFusion 55.0.0]. This post highlights +some of the major improvements since [DataFusion 54.0.0]. Notable additions +include range partitioning, `MERGE INTO` support, per-partition TopK for window +functions, and runtime row-group pruning for TopK queries, alongside significant +aggregation, function, and planning performance improvements. The complete list +of changes is available in the [changelog]. This release represents roughly 9 +weeks of development and 877 commits. Thanks to the [175 contributors] +(a new record!) for making it possible. + +[DataFusion 55.0.0]: https://crates.io/crates/datafusion/55.0.0 +[DataFusion 54.0.0]: https://datafusion.apache.org/blog/2026/06/12/datafusion-54.0.0/ +[changelog]: https://github.com/apache/datafusion/blob/branch-55/dev/changelog/55.0.0.md +[175 contributors]: https://github.com/apache/datafusion/blob/branch-55/dev/changelog/55.0.0.md#credits + +## Performance Improvements 🚀 + + + + +**Figure 1**: Average and median normalized execution times for DataFusion 55.0.0 on ClickBench queries, compared to previous releases. +Query times are normalized using the ClickBench definition. See the +[DataFusion Benchmarking Page](https://alamb.github.io/datafusion-benchmarking/) +for more details. + +We continue to make significant performance improvements in DataFusion, as +explained below. This release skips more work at runtime using statistics and +dynamic filters, and makes window functions, aggregation, and many built-in +functions faster. + +### Sort Pushdown: Runtime Row-Group Pruning for TopK Queries + +The multi-release [Sort Pushdown effort] makes `ORDER BY` and +`ORDER BY ... LIMIT` (TopK) queries on Parquet skip work end-to-end: skip the +sort, skip row groups via min/max statistics, and skip rows via dynamic filters. +DataFusion 55 lands the next phase: as a TopK query runs and its dynamic filter +threshold tightens, the Parquet reader now re-evaluates the threshold against +the remaining row groups at every row-group boundary and drops those that can +no longer contribute — zero IO and zero decode for the skipped row groups +([#22450]). This release also adds multi-column lexicographic statistics +reordering ([#23888]), so compound `ORDER BY` queries benefit too. In the +`topk_tpch` benchmark suite, 5 of 11 queries got 3-4x faster with no +regressions, reducing total suite runtime by 44%. +Thanks to [@zhuqi-lucas] for driving this work, with reviews from [@adriangb]. + +[Sort Pushdown effort]: https://github.com/apache/datafusion/issues/23036 + +### Per-Partition TopK for Window Functions + +A common analytics pattern selects the top N rows per group using a window +function: + +```sql +SELECT * FROM ( + SELECT ROW_NUMBER() OVER (PARTITION BY category ORDER BY revenue DESC) AS rn, * + FROM sales +) WHERE rn <= 5; +``` + +DataFusion previously sorted the *entire* input to evaluate the window +function, even though only a handful of rows per partition survive the filter. +DataFusion 55 recognizes this pattern and uses a new `PartitionedTopKExec` +operator that keeps only the top N rows per partition, dramatically reducing +sorting and memory for high-cardinality inputs. The optimization applies to +`ROW_NUMBER` and `RANK`, resolving a feature request first filed in 2023 +([#6899]). +Thanks to [@SubhamSinghal] for implementing this feature, with reviews from +[@2010YOUY01] and [@kosiew]. Related PRs: [#21479], [#22885], [#23096] + +### Aggregation Improvements + +**Complete Multi-Column `GROUP BY` Type Coverage**: +DataFusion's column-wise `GROUP BY` storage (`GroupValuesColumn`) has +type-specific fast paths, but previously any unsupported column type dragged +the entire grouping onto a slower row-encoded fallback. DataFusion 55 completes +the type coverage ([#22715]): new specializations were added for `Decimal256`, +`Float16`, and `Interval` ([#23849], [#23785], [#23786]), and a new generic +`Rows`-backed `GroupColumn` keeps mixed schemas on the column-wise path +([#23523]). +Thanks to [@zhuqi-lucas] and [@tohuya6] for this work. + +**Split Aggregation Streams**: +The hash aggregation state machine previously handled partial aggregation, +final aggregation, and streaming cases in one shared implementation. DataFusion +55 splits these semantically distinct paths into dedicated streams ([#22729]), +making the code easier to optimize and extend (part of epic [#22710]). +Thanks to [@2010YOUY01] for this work, with reviews from [@Rachelint] and +[@alamb]. + +### Faster Functions + +DataFusion ships hundreds of built-in functions, so speeding them up pays off +across many workloads. This release optimizes many, including [find_in_set] +(up to 24x faster), [trunc] (10x), [replace] (2x), [regexp_instr] (40%), +[regexp_match], [round], [date_trunc], [date_part], [get_field], [upper], and +[string_trim], plus dictionary-encoding preservation for many string functions +([#23743], [#23930], [#24100]) and a 100x improvement to [approx_distinct] for +low-cardinality inputs with many groups ([#22768]). The `approx_distinct` +aggregate also gained support for many more types, including `Decimal`, +`Interval`, `Duration`, `Struct`, `Map`, and `Union`, thanks to [@mkleen]. +Thanks to the many contributors who drove this work, especially [@andygrove], +[@neilconway], [@lyne7-sc], [@theirix], and [@haohuaijin]. + +**Faster `IN` List Evaluation**: +`IN` list membership checks can run millions of times during a scan, especially +with dynamic filter pushdown. DataFusion 55 adds exact lookup strategies +selected by type and list size, including bitmap filters for small integer +types and branchless filters for small primitive lists ([#19241]). +Thanks to [@geoffreyclaude] for driving this work, with contributions from +[@alamb]. Related PRs: [#23012], [#23014], [#23299] + +### Planner Improvements + +**Unified Distribution and Sorting Enforcement**: +The `EnforceDistribution` and `EnforceSorting` physical optimizer passes are +now merged into a single `EnsureRequirements` pass with idempotent sort +pushdown ([#21976]), fixing longstanding ordering issues between the two passes +and enabling the sort pushdown work described above. +Thanks to [@zhuqi-lucas] for this work, with reviews from [@2010YOUY01] and +[@alamb]. + +**Smarter Join Planning**: +DataFusion 55 converts inner joins to semi joins when equivalent ([#22652]), +eliminates `LEFT`/`RIGHT` joins with redundant sides ([#23566]), handles +intermediate projections in outer join elimination ([#22534]), and reorders +predicates in conjunctions using a cost heuristic ([#22343]). +Thanks to [@neilconway] and [@simonvandel] for this work. + +**Scalar UDF Strictness Metadata**: +Scalar UDFs can now declare that they are *strict* (they return `NULL` when any +input is `NULL`) ([#23148]). The optimizer uses this metadata to prove that +filters reject `NULL`s, unlocking outer join elimination for queries that +filter on the result of a function call. +Thanks to [@lyne7-sc] for implementing this feature, with reviews from [@alamb] +and [@kosiew]. + +**Faster Optimizer Passes**: +The logical optimizer now skips subquery traversal when there are no subqueries +and rewrites plans in place ([#22298]), collapses chained projections in a +single pass ([#22389]), and avoids re-inlining expensive common subexpressions +during projection pushdown ([#23459]). +Thanks to [@adriangb], [@Dandandan], and [@fordN] for this work. + +### Scan Improvements + +**Pruning Unread Parquet Leaves for Nested Columns**: +When a table declares a nested column narrower than the Parquet file's physical +type, DataFusion previously read every leaf of the column and dropped the extra +subfields in memory. DataFusion 55 derives the projection mask through casts, +so only the leaves that are actually accessed are read — one production query +reported by the [DataFusion Comet] project went from reading 1.35 TB to reading +only the required data ([#24090]). +Thanks to [@mbutrovich] for this work, with reviews from [@adriangb]. + +[DataFusion Comet]: https://datafusion.apache.org/comet/ + +**Other Scan Improvements**: +DataFusion 55 also skips loading the page index (and an expensive +`ParquetMetaData` clone) when a file has no page index ([#24150]), supports +file-level Parquet row selections ([#22940]), and lowers the default +`repartition_file_min_size` from 10 MiB to 1 MiB for better parallelism on +small files ([#22439]). +Thanks to [@alamb], [@haohuaijin], and [@adriangb]. + +## New Features ✨ + +### Range Partitioning + +DataFusion previously supported only hash and round-robin repartitioning. +DataFusion 55 adds *range partitioning*, where rows are distributed to +partitions based on ordered split points ([#22395], design discussion +[#21992]). Range partitioning preserves ordering across partitions, which is a +natural fit for pre-sorted data and for distributed engines that shuffle by +range. + +This release adds the new `Partitioning::Range` physical variant ([#22207]), +a logical representation ([#22777]), physical execution and planning +([#23231], [#23617]), and planner support so range-partitioned plans avoid +unnecessary repartitioning: range-partitioned inputs can now satisfy hash +joins ([#23484], [#23583]), window functions ([#23416]), and per-partition +TopK requirements ([#23290]). Dynamic filter pushdown also now works for +range-partitioned joins, routing probe-side filters to the correct build +partition using the range split points ([#23854]). + +Thanks to [@gene-bordegaray], [@jayshrivastava], [@saadtajwar], [@peterxcli], +[@gmhelmold], [@mattp5657], and [@mithuncy] for driving this substantial +community effort. + +### `MERGE INTO` + +`MERGE INTO` (SQL:2003) is a widely used DML statement for upsert and +conditional update workloads, and a key building block for table formats such +as Apache Iceberg and Delta Lake. DataFusion 55 adds the logical plan types +([#20763]) along with SQL planner, physical planner, and a new +`TableProvider::merge_into` hook ([#22988]) so table implementations can +execute merge operations: + +```sql +MERGE INTO target t +USING source s +ON t.id = s.id +WHEN MATCHED AND s.deleted THEN DELETE +WHEN MATCHED THEN UPDATE SET name = s.name +WHEN NOT MATCHED THEN INSERT (id, name) VALUES (s.id, s.name); +``` + +Built-in table providers do not yet implement the hook, but custom +`TableProvider` implementations (such as lakehouse table formats) can now plug +in their own merge execution. +Thanks to [@wirybeaver] for implementing this feature, with reviews from +[@alamb] and [@kosiew]. + +### Parquet Virtual Columns: `file_row_index` and `input_file_name` + +DataFusion 55 plumbs Parquet *virtual columns* through the scan ([#22026]), +building on arrow-rs support for computing row indexes during the read. Two new +functions expose this to queries: [file_row_index] returns the 0-based row +index of each row within its Parquet file ([#22604]), and [input_file_name] +returns the file each row came from ([#22978]). These are useful for building +change data capture, debugging, and Spark-compatible workloads (Spark exposes +similar metadata columns). +Thanks to [@mbutrovich] and [@AdamGS] for this work (reviving earlier work from +[@jkylling]), with reviews from [@adriangb], [@comphead], and [@niebayes]. + +### `unnest_outer` + +DataFusion's `unnest` drops rows whose input list is empty or `NULL`. The new +[unnest_outer] function ([#22100]) preserves them as `NULL` rows instead, +matching Spark's `explode_outer` semantics: + +```sql +SELECT id, unnest_outer(tags) +FROM VALUES (1, ['a', 'b']), (2, []), (3, NULL) AS t(id, tags); +-- 1, 'a' +-- 1, 'b' +-- 2, NULL +-- 3, NULL +``` + +Thanks to [@athlcode] for implementing this feature, with reviews from +[@comphead]. + +### Pluggable Spill Backends + +DataFusion spills to disk when a query exceeds its memory budget, but the spill +infrastructure was previously hardwired to OS-level temporary files. DataFusion +55 introduces a pluggable `SpillFile` trait and `TempFileFactory` ([#21882], +[#22230]) so hosts can route spill data through their own storage layers — for +example, Postgres extensions like ParadeDB can now direct spills through +Postgres `BufFile` APIs to respect `temp_tablespaces` and `temp_file_limit`. +Thanks to [@pantShrey] for this work, with reviews from [@alamb]. + +### Extensibility for Distributed Engines + +Several new APIs make it easier to build distributed systems such as +[datafusion-distributed], [DataFusion Ballista], and [DataFusion Python] on +top of DataFusion: + +- **Dynamic filter propagation across network boundaries**: new + `ExecutionPlan::apply_expressions` and + `ExecutionPlan::dynamic_expressions_produced` methods let engines discover + which plan nodes produce dynamic filters and re-wire them across stage + boundaries ([#24018], [#24068]). Thanks to [@jayshrivastava]. +- **`FFI_QueryPlanner`**: foreign libraries can now provide a custom query + planner over the FFI boundary — for example, connecting a distributed + planner to a `SessionContext` in Python ([#24028]). Thanks to [@timsaucer]. +- **Self-serializing execution plans**: built-in `ExecutionPlan`s were ported + to per-type `try_to_proto` / `try_from_proto` hooks ([#23494]), eliminating + the central downcast chain in `datafusion-proto` and putting built-in and + third-party plans on the same code path. Thanks to [@adriangb]. +- **Window accumulator state access**: `BoundedWindowAggExec` can now expose + finalized accumulator state to an observer callback, enabling incremental / + prefix-scan use cases ([#24035]). Thanks to [@avantgardnerio], with reviews + from [@alamb] and [@timsaucer]. + +[datafusion-distributed]: https://github.com/datafusion-contrib/datafusion-distributed +[DataFusion Ballista]: https://datafusion.apache.org/ballista/ +[DataFusion Python]: https://datafusion.apache.org/python/ + +### `EXPLAIN` Improvements + +DataFusion 55 adds a Postgres-style `EXPLAIN (...)` option list ([#21768]) and +a `pgjson` output format for `EXPLAIN ANALYZE` ([#21767]), making plan output +easier to consume with existing Postgres tooling such as plan visualizers. +Thanks to [@adriangb] for this work. + +### New Functions + +**SQL and Scalar Functions**: +DataFusion 55 adds new array math functions [array_scale], [array_add], +[array_subtract], [array_sum], and [array_avg], plus the higher-order +[array_first] function. +Thanks to [@crm26], [@SubhamSinghal], and [@EdsonPetry] for these +contributions. + +**Spark-Compatible Functions**: +The [datafusion-spark crate] gains new or improved Spark-compatible functions, +including [hypot], [atan2], [weekday], [monthname], and [concat_ws] with array +support, plus a new Spark SQL parser dialect config ([#22529]). +Thanks to the contributors who drove this work, especially +[@KarpagamKarthikeyan], [@sjhddh], [@JeelRajodiya], [@davidlghellin], and +[@kumarUjjawal]. + +## Upgrade Guide and Changelog 📖 + +Upgrading to 55.0.0 should be straightforward for most users, though there are +some breaking changes. See the [Upgrade Guide] for details and +migration snippets, and the [changelog] for the full list of changes. + +## About DataFusion + +[Apache DataFusion] is an extensible query engine, written in [Rust], that uses +[Apache Arrow] as its in-memory format. DataFusion is used by developers to +create new, fast, data-centric systems such as databases, dataframe libraries, +and machine learning and streaming applications. While [DataFusion's primary +design goal] is to accelerate the creation of other data-centric systems, it +provides a reasonable experience directly out of the box as a [dataframe +library], [Python library], and [command-line SQL tool]. + +DataFusion's core thesis is that, as a community, together we can build much +more advanced technology than any of us as individuals or companies could build +alone. Without DataFusion, highly performant vectorized query engines would +remain the domain of a few large companies and world-class research +institutions. With DataFusion, we can all build on top of a shared foundation +and focus on what makes our projects unique. + +## How to Get Involved + +DataFusion is not a project built or driven by a single person, company, or +foundation. Rather, our community of users and contributors works together to +build a shared technology that none of us could have built alone. + +If you are interested in joining us, we would love to have you. You can try out +DataFusion on some of your own data and projects and let us know how it goes, +contribute suggestions, documentation, bug reports, or a PR with documentation, +tests, or code. A list of open issues suitable for beginners is [here], and you +can find out how to reach us on the [communication doc]. + +[Apache DataFusion]: https://datafusion.apache.org/ +[Rust]: https://www.rust-lang.org/ +[Apache Arrow]: https://arrow.apache.org +[DataFusion's primary design goal]: https://datafusion.apache.org/user-guide/introduction.html#project-goals +[dataframe library]: https://datafusion.apache.org/user-guide/dataframe.html +[Python library]: https://datafusion.apache.org/python/ +[command-line SQL tool]: https://datafusion.apache.org/user-guide/cli/ +[Upgrade Guide]: https://datafusion.apache.org/library-user-guide/upgrading.html +[here]: https://github.com/apache/arrow-datafusion/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22 +[communication doc]: https://datafusion.apache.org/contributor-guide/communication.html + +[@2010YOUY01]: https://github.com/2010YOUY01 +[@AdamGS]: https://github.com/AdamGS +[@Dandandan]: https://github.com/Dandandan +[@EdsonPetry]: https://github.com/EdsonPetry +[@JeelRajodiya]: https://github.com/JeelRajodiya +[@KarpagamKarthikeyan]: https://github.com/KarpagamKarthikeyan +[@Rachelint]: https://github.com/Rachelint +[@SubhamSinghal]: https://github.com/SubhamSinghal +[@adriangb]: https://github.com/adriangb +[@alamb]: https://github.com/alamb +[@andygrove]: https://github.com/andygrove +[@athlcode]: https://github.com/athlcode +[@avantgardnerio]: https://github.com/avantgardnerio +[@comphead]: https://github.com/comphead +[@crm26]: https://github.com/crm26 +[@davidlghellin]: https://github.com/davidlghellin +[@fordN]: https://github.com/fordN +[@gene-bordegaray]: https://github.com/gene-bordegaray +[@geoffreyclaude]: https://github.com/geoffreyclaude +[@gmhelmold]: https://github.com/gmhelmold +[@haohuaijin]: https://github.com/haohuaijin +[@jayshrivastava]: https://github.com/jayshrivastava +[@jkylling]: https://github.com/jkylling +[@kosiew]: https://github.com/kosiew +[@kumarUjjawal]: https://github.com/kumarUjjawal +[@lyne7-sc]: https://github.com/lyne7-sc +[@mattp5657]: https://github.com/mattp5657 +[@mbutrovich]: https://github.com/mbutrovich +[@mithuncy]: https://github.com/mithuncy +[@mkleen]: https://github.com/mkleen +[@neilconway]: https://github.com/neilconway +[@niebayes]: https://github.com/niebayes +[@pantShrey]: https://github.com/pantShrey +[@peterxcli]: https://github.com/peterxcli +[@saadtajwar]: https://github.com/saadtajwar +[@simonvandel]: https://github.com/simonvandel +[@sjhddh]: https://github.com/sjhddh +[@theirix]: https://github.com/theirix +[@timsaucer]: https://github.com/timsaucer +[@tohuya6]: https://github.com/tohuya6 +[@wirybeaver]: https://github.com/wirybeaver +[@zhuqi-lucas]: https://github.com/zhuqi-lucas + +[find_in_set]: https://github.com/apache/datafusion/pull/23460 +[trunc]: https://github.com/apache/datafusion/pull/23593 +[replace]: https://github.com/apache/datafusion/pull/23589 +[regexp_instr]: https://github.com/apache/datafusion/pull/23540 +[regexp_match]: https://github.com/apache/datafusion/pull/23547 +[round]: https://github.com/apache/datafusion/pull/23471 +[date_trunc]: https://github.com/apache/datafusion/pull/23542 +[date_part]: https://github.com/apache/datafusion/pull/23491 +[get_field]: https://github.com/apache/datafusion/pull/23537 +[upper]: https://github.com/apache/datafusion/pull/23588 +[string_trim]: https://github.com/apache/datafusion/pull/23541 +[approx_distinct]: https://github.com/apache/datafusion/pull/22768 +[file_row_index]: https://github.com/apache/datafusion/pull/22604 +[input_file_name]: https://github.com/apache/datafusion/pull/22978 +[unnest_outer]: https://github.com/apache/datafusion/pull/22100 +[array_scale]: https://github.com/apache/datafusion/pull/22466 +[array_add]: https://github.com/apache/datafusion/pull/22459 +[array_subtract]: https://github.com/apache/datafusion/pull/22556 +[array_sum]: https://github.com/apache/datafusion/pull/22542 +[array_avg]: https://github.com/apache/datafusion/pull/23168 +[array_first]: https://github.com/apache/datafusion/pull/23267 +[hypot]: https://github.com/apache/datafusion/pull/23774 +[atan2]: https://github.com/apache/datafusion/pull/23962 +[weekday]: https://github.com/apache/datafusion/pull/22740 +[monthname]: https://github.com/apache/datafusion/pull/21639 +[concat_ws]: https://github.com/apache/datafusion/pull/20928 + +[#6899]: https://github.com/apache/datafusion/issues/6899 +[#19241]: https://github.com/apache/datafusion/issues/19241 +[#20763]: https://github.com/apache/datafusion/pull/20763 +[#21479]: https://github.com/apache/datafusion/pull/21479 +[#21768]: https://github.com/apache/datafusion/pull/21768 +[#21767]: https://github.com/apache/datafusion/pull/21767 +[#21882]: https://github.com/apache/datafusion/pull/21882 +[#21976]: https://github.com/apache/datafusion/pull/21976 +[#21992]: https://github.com/apache/datafusion/issues/21992 +[#22026]: https://github.com/apache/datafusion/pull/22026 +[#22100]: https://github.com/apache/datafusion/pull/22100 +[#22207]: https://github.com/apache/datafusion/pull/22207 +[#22230]: https://github.com/apache/datafusion/pull/22230 +[#22298]: https://github.com/apache/datafusion/pull/22298 +[#22343]: https://github.com/apache/datafusion/pull/22343 +[#22389]: https://github.com/apache/datafusion/pull/22389 +[#22439]: https://github.com/apache/datafusion/pull/22439 +[#22450]: https://github.com/apache/datafusion/pull/22450 +[#22529]: https://github.com/apache/datafusion/pull/22529 +[#22534]: https://github.com/apache/datafusion/pull/22534 +[#22604]: https://github.com/apache/datafusion/pull/22604 +[#22652]: https://github.com/apache/datafusion/pull/22652 +[#22715]: https://github.com/apache/datafusion/issues/22715 +[#22710]: https://github.com/apache/datafusion/issues/22710 +[#22729]: https://github.com/apache/datafusion/pull/22729 +[#22768]: https://github.com/apache/datafusion/pull/22768 +[#22777]: https://github.com/apache/datafusion/pull/22777 +[#22885]: https://github.com/apache/datafusion/pull/22885 +[#22940]: https://github.com/apache/datafusion/pull/22940 +[#22978]: https://github.com/apache/datafusion/pull/22978 +[#22988]: https://github.com/apache/datafusion/pull/22988 +[#23012]: https://github.com/apache/datafusion/pull/23012 +[#23014]: https://github.com/apache/datafusion/pull/23014 +[#23096]: https://github.com/apache/datafusion/pull/23096 +[#23148]: https://github.com/apache/datafusion/pull/23148 +[#23231]: https://github.com/apache/datafusion/pull/23231 +[#23290]: https://github.com/apache/datafusion/issues/23290 +[#23299]: https://github.com/apache/datafusion/pull/23299 +[#23416]: https://github.com/apache/datafusion/pull/23416 +[#23459]: https://github.com/apache/datafusion/pull/23459 +[#23484]: https://github.com/apache/datafusion/pull/23484 +[#23494]: https://github.com/apache/datafusion/issues/23494 +[#23523]: https://github.com/apache/datafusion/pull/23523 +[#23566]: https://github.com/apache/datafusion/pull/23566 +[#23583]: https://github.com/apache/datafusion/pull/23583 +[#23617]: https://github.com/apache/datafusion/pull/23617 +[#23743]: https://github.com/apache/datafusion/pull/23743 +[#23785]: https://github.com/apache/datafusion/pull/23785 +[#23786]: https://github.com/apache/datafusion/pull/23786 +[#23849]: https://github.com/apache/datafusion/pull/23849 +[#23854]: https://github.com/apache/datafusion/pull/23854 +[#23888]: https://github.com/apache/datafusion/pull/23888 +[#23930]: https://github.com/apache/datafusion/pull/23930 +[#23954]: https://github.com/apache/datafusion/pull/23954 +[#24018]: https://github.com/apache/datafusion/pull/24018 +[#24028]: https://github.com/apache/datafusion/pull/24028 +[#24035]: https://github.com/apache/datafusion/pull/24035 +[#24068]: https://github.com/apache/datafusion/pull/24068 +[#24090]: https://github.com/apache/datafusion/pull/24090 +[#24100]: https://github.com/apache/datafusion/pull/24100 +[#24150]: https://github.com/apache/datafusion/pull/24150 +[#22395]: https://github.com/apache/datafusion/issues/22395 + +[datafusion-spark crate]: https://docs.rs/datafusion-spark/latest/datafusion_spark/index.html