Skip to content

Bug triage results: 2026-11-16 #7640

Open

Description

@andygrove

Bug triage pass for 2026-11-16.

  • Issues processed: 17 (8 bugs, 9 enhancements)
  • Bugs by priority: critical 2, high 0, medium 5, low 1
  • Skipped: 1

Criteria: Bug Triage Guide.

Labels have already been applied and requires-triage removed. Please spot-check the calls and close this issue when satisfied. Many issues arrived already carrying a type label; those were kept as they were.

Bugs

priority:critical

  • Native MergeRows can panic or produce wrong masks - arrow 59 and_not mishandles sliced arrays (#7596)
    • Area labels: none
    • Rationale: Besides the panic, and_not on a sliced array can return wrong masks with no error, so inserts and deletes land on the wrong rows, a silent wrong result.
  • Native string to decimal cast fails on a string with more than 38 digits: NULL or CAST_INVALID_INPUT where Spark returns a value (#7581)
    • Area labels: area:expressions
    • Rationale: Comet returns NULL where Spark returns a value (for example 1.00) with ANSI off, a silent wrong result.

priority:medium

  • fair_unified: consumers that have not started yet take an equal share, so the running sort or aggregate spills early (#7628)
    • Area labels: none
    • Rationale: Results are correct, but the extra spilling is a significant performance degradation (56 GB spilled against 19.7 GB for Spark in the report).
  • make_date can throw on unconsumed rows through PySpark and raw row iterators (#7616)
    • Area labels: area:expressions
    • Rationale: Comet raises an error on a row Spark never evaluates, only for partial consumers of raw row iterators; a visible failure, not a wrong result.
  • A failed native grow is only logged when the task makes no later try_grow (#7615)
    • Area labels: none
    • Rationale: A memory request Spark rejected is not surfaced, so the task can run past its grant; it needs a failed grow and no later try_grow, and does not return wrong data.
  • Native shuffle write fails when the local dir path contains .data; failed tasks also leave temp files behind (#7594)
    • Area labels: area:shuffle
    • Rationale: Native shuffle write fails for local dirs whose path contains .data, which can be worked around by renaming the directory, and failed tasks leave temp files behind.
  • lpad/rpad do not use withNullShortCircuit, so ANSI errors are raised on rows Spark skips (#7583)
    • Area labels: area:expressions
    • Rationale: Under ANSI, Comet raises an error on a NULL-input row where Spark returns NULL; a visible failure that needs an ANSI cast in a child expression.

priority:low

  • Native string to decimal cast raises CAST_INVALID_INPUT instead of NUMERIC_VALUE_OUT_OF_RANGE under ANSI when the scale adjustment is more than 38 (#7576)
    • Area labels: area:expressions
    • Rationale: Both engines fail under ANSI and return NULL otherwise; only the error class differs.

Enhancements

  • Native Iceberg writes: assert manifest metrics parity with iceberg-java for every data type (#7639)
    • Area labels: area:writer
    • Rationale: Asks for more parity tests before native Iceberg writes are enabled by default.
  • Native Iceberg writes: support hdfs data locations (#7637)
    • Area labels: area:writer
    • Rationale: Native Iceberg writes fall back to iceberg-java for hdfs by design; this asks for new support.
  • Support struct and array map keys in native MapSort (#7632)
    • Area labels: area:expressions
    • Rationale: Requests native evaluation for key types that already fall back.
  • Comet Native Executors: raise Spark's exceptions for cluster failures (#7621)
    • Area labels: none
    • Rationale: Requests a new structured error path; the limitation is already documented.
  • Migrate to the new apache/datafusion-spark repository (#7619)
    • Area labels: none
    • Rationale: Docs, skills and dependency housekeeping after an upstream move; main is unaffected.
  • Research using the Rust Allocator API (stabilized in 1.100) for memory-pool accounting (#7613)
    • Area labels: none
    • Rationale: A research task, not a defect.
  • Restore native row-group pruning for FLOAT and DOUBLE range filters in Iceberg scans (#7588)
  • Speculate string functions such as upper in NullShortCircuit by adding them to is_cheap (#7587)
    • Area labels: area:expressions
    • Rationale: A performance optimization request.
  • Consolidate the Scala "cannot fail" checks: arrays.cannotThrow and QueryPlanSerde.cannotFail (#7585)
    • Area labels: area:expressions
    • Rationale: A refactor to remove duplicated lists.

Escalations to consider

Skipped — needs more info

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions