Skip to content

Open issues that can be closed: already fixed, duplicated, or out of date #2888

Closed

Description

@NoahKusaba

I went through the 147 open issues and checked each one against main (code search), the PRs that mention it, and its comment thread. Below are the ones that look safe to close, plus a few that need a quick check first. Nothing was rebuilt or benchmarked to confirm these, so please check before closing.

cc @milenkovicm, since you asked to be tagged on old issues that can be closed (#2849).

Already fixed

Issue Why it can be closed
#2193 NOT IN loses null semantics in SortMergeJoinExec DataFusion fixed this in apache/datafusion#22810, and main is on DataFusion 56. #2188 and #2199 also handle NOT IN on the Ballista side.
#1960 Surface column statistics via ShuffleReaderExec Done in the #2664 → #2671 stack. #2664 says "Closes #1960" but was closed rather than merged; its commits came in through #2671. state/aqe/mod.rs now reads output_column_stats.
#2063 AQE hangs when a replan cancels an in-flight stage #2192 now removes stages a replan cancels and ignores their late task updates. The "there are stages to be cancelled but its not implemented" warning is gone from main.
#348 Implement broadcast join optimization #342 (broadcast exchange) is closed, and #1900 and #1904 added broadcast joins.
#890 Add Iceberg table support #2638 (Iceberg scans) is merged, and the remaining Iceberg work is tracked in #2174.
#1151 Push-based shuffle / all-at-once stage scheduling Covered by the pipelined execution mode: #2657 is merged and the design issue #2652 is closed.
#408 Bubble execution mode Same as #1151: covered by pipelined mode (#2652, #2657).
#332 TaskGroups (share memory between task slots) #2038 added multi-partition tasks, so one task can run several partitions.
#406 Control collect_stat for the table provider #796 added the setting, and Ballista no longer overrides DataFusion's collect_statistics.
#119 ExecutorReservation during task scheduling ExecutorReservation was removed in #823.
#645 Source data cache layer A 2024 comment notes that its only unchecked item, #833, was merged.
#2162 S3 prefetching like Spark The comment on the issue says the benchmark gap behind it is resolved (721s vs 700s compared with Spark 3.4).
#2367 TPC-H "Failed to open partition file" The cause was a stale compiled Python extension in the local checkout, not a Ballista bug. The CI follow-up is tracked in #2372.
#2712 Nightly Docker image build failed Every nightly Docker build from Nov 12 to Nov 17 succeeded.
#2727 Run a stage once when several stages read it #2765 (which closed #1953) runs a repeated subplan only once under AQE, matching stages by a hash of their subtree. It would be good to confirm this covers physical plans that Comet submits.

Duplicates or covered by another issue

Issue Covered by
#553 Support multiple working dirs in Executor Same title as #196, and #553's body is the empty template.
#663 Make task reset policy configurable Same request as #189: a setting for how the scheduler recovers when an executor is lost.
#12 Prefer assigning a job's tasks to the same executor Covered by #2319 (shuffle affinity, open PR #2430).
#456 Support deltalake Covered by #1241 (open table formats). There is already an example extension at milenkovicm/ballista_delta.

Ask the reporter before closing

Probably out of date

These weren't checked against the code. They are a judgment call based on age and the APIs they refer to.

Can be closed once an open PR merges

Issue Open PR
#2882 #2784
#2840 #2867
#2839 #2886
#2804 #2809
#2767 #2775
#2691 #2698
#2622 #2818
#2578 #2774
#2454 #2501
#2319 #2430
#2854, #2866 #2852

#2371 could also be closed as won't fix, since the discussion on #2395 says the current behaviour is expected.

Look fixed but are still open

Activity

  1. milenkovicm commented on Nov 17, 2026

    @milenkovicm
    Contributor

    fair play to you @NoahKusaba thanks a lot
    will go through the list

  2. NoahKusaba commented on Nov 18, 2026

    @NoahKusaba
    ContributorAuthor

    fair play to you @NoahKusaba thanks a lot will go through the list

    Close this too if you don't need it anymore 😆

  3. milenkovicm commented on Nov 19, 2026

    @milenkovicm
    Contributor

    Thanks @NoahKusaba great cleanup. will close this now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions