Skip to content

Client hides plan-encoding errors behind an Internal error #2885

Open

Description

@NoahKusaba

Describe the bug

When the client can't encode a logical plan to send it to the scheduler, execute_query_params (ballista/core/src/execution_plans/distributed_query.rs) turns the error into a string:

let plan_message = T::try_from_logical_plan(plan, extension_codec).map_err(|e| {
    DataFusionError::Internal(format!("failed to serialize logical plan: {e:?}"))
})?;

That loses the error chain, so DataFusionError::find_root can't reach the cause, and it formats the cause with Debug, escaping its quotes and newlines. Because it's Internal, DataFusion also appends "This issue was likely caused by a bug in DataFusion's code. Please help us to resolve this by filing a bug report", even when the cause is the user's, such as a table that no longer exists.

To Reproduce

With ballista-iceberg, read a view whose Iceberg table was dropped from its catalog (#2784). The error is:

Internal error: failed to serialize logical plan: Context("Error serializing custom table at …/datafusion-proto-…/src/logical_plan/mod.rs:1534", Execution("no composed codec could encode it: codec 0: This feature is not implemented: LogicalExtensionCodec is not provided; codec 1: IcebergError(TableNotFound)\ncaused by\nExecution error: TableNotFound => No such table: TableIdent { namespace: NamespaceIdent([\"ns\"]), name: \"t\" }")).
This issue was likely caused by a bug in DataFusion's code. Please help us to resolve this by filing a bug report in our issue tracker: https://github.com/apache/datafusion/issues

Expected behavior

The cause is kept, with context added rather than replacing it:

.map_err(|e| e.context("failed to serialize logical plan"))

The message then reads "failed to serialize logical plan" followed by the cause, without the bug-report text, and find_root returns the cause. The try_encode error two lines below has the same shape.

Additional context

Raised in review of #2784. The cause also reaches this point flattened by ComposedLogicalExtensionCodec, which joins its codecs' errors into one string; that's a TODO on its collect, to fix when it moves to DataFusion.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions