Skip to content

[FEATURE] Support multi-line virtual tables for ease of reading #162

Description

@eliot1480

Summary

When it comes to representing large virtual tables with substrait-explain, it can sometimes get a little visually cluttered when everything has to be displayed on a single line. If we implemented a format to allow for every row of data to have it's own line, it'd be much easier to read.

Motivation

By having every row of data have it's own line instead of a single massive line, it makes things much easier to read

Proposed Solution

When converting from substrait-explain to protobuf, instead of parsing the entire massive table, we parse the rows line by line. And when we convert from protobuf to substrait-explain, we print the rows within the virtual tables line by line.

Additional Context

Going from this:

== Query
=== Plan
Root[id, name, age]
  Sort[($1, &DescNullsFirst) => $0, $1, $2]
    Read:Virtual[(1:i32, 'Alice':string?, 25:i32), (15:i32, 'Bob':string?, 30:i32), (5:i32, null:string?, 20:i32) => id:i32, name:string?, age:i32]

To this:

== Query
=== Plan
Root[id, name, age]
  Sort[($1, &DescNullsFirst) => $0, $1, $2]
    Read:Virtual[
    (1:i32, 'Alice':string?, 25:i32), 
    (15:i32, 'Bob':string?, 30:i32), 
    (5:i32, null:string?, 20:i32) 
    => id:i32, name:string?, age:i32]

Is much more readable.

There are several formats that can be done though:

== Query
=== Plan
Root[id, name, age]
  Sort[($1, &DescNullsFirst) => $0, $1, $2]
    Read:Virtual[
      (1:i32, 'Alice':string?, 25:i32), 
      (15:i32, 'Bob':string?, 30:i32), 
      (5:i32, null:string?, 20:i32) 
      => id:i32, name:string?, age:i32]

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions