You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Static planner: null-aware anti join loses its probe-side coalesce under an aggregate #2849
With the static planner, a NOT IN subquery planned as a null-aware anti hash join can return wrong results, with no error, when an aggregate sits above it and the join's stage is split into tasks.
A null-aware anti join has to see the whole right side in one task, so lower_null_aware_join puts a CoalescePartitionsExec under its probe side. That coalesce sits inside the join's stage. When an aggregate is above the join, the planner runs EnsureRequirements over that stage again, and it removes the coalesce because the join does not ask for one. The join then runs in several tasks, each seeing only part of the right side.
Run a GROUP BY over a NOT IN subquery on a table read from several files. The counts came back as 1190/1155/1155, against 170/165/165 from DataFusion.
Expected behavior
The same counts as DataFusion.
Additional context
#2848 hits the same problem for nested-loop joins and avoids it by giving the probe side a one-partition stage of its own instead of a coalesce inside the join's stage. The same approach should work here.
Sorry @milenkovicm I deleted my last comment, I wasn't 100% sure this ticket can be deleted. I still hope that older issues can be cleaned-up, as it's hard to parse what's important in there.
Sorry @milenkovicm I deleted my last comment, I wasn't 100% sure this ticket can be deleted. I still hope that older issues can be cleaned-up, as it's hard to parse what's important in there.
Activity
milenkovicm commentedon Nov 16, 2026
Can't this issue be closed @ewoodbury ?
NoahKusaba commentedon Nov 16, 2026
Sorry @milenkovicm I deleted my last comment, I wasn't 100% sure this ticket can be deleted. I still hope that older issues can be cleaned-up, as it's hard to parse what's important in there.
milenkovicm commentedon Nov 16, 2026
No worries, tag me for other issues you find
ewoodbury commentedon Nov 19, 2026
Still an issue, just opened #2898 to address!