Skip to content

change: fully deprecated the acceptance of value field in VirtualTables - #24346

Open
eliot1480 wants to merge 2 commits into
apache:mainfrom
eliot1480:change-virtualtable-consumer-to-only-accept-expressions
Open

change: fully deprecated the acceptance of value field in VirtualTables#24346
eliot1480 wants to merge 2 commits into
apache:mainfrom
eliot1480:change-virtualtable-consumer-to-only-accept-expressions

Conversation

@eliot1480

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

#23608

Rationale for this change

This PR is the final step in fully deprecating the VirtualTables.values, as we now only allow the consumption of expressions instead of the value field as well.

What changes are included in this PR?

These changes remove the deprecated VirtualTable.values consumer path, including the old literal-row conversion helper and the broad deprecation allowance around from_read_rel. The consumer now handles empty virtual tables, one-row empty relations, and normal virtual table rows using VirtualTable.expressions only.

Are these changes tested?

Yes.

Are there any user-facing changes?

No.

@github-actions github-actions Bot added the substrait Changes to the substrait crate label Aug 13, 2026
@codecov-commenter

codecov-commenter commented Aug 13, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 78.12500% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.31%. Comparing base (f171587) to head (8c9b893).
⚠️ Report is 98 commits behind head on main.

Files with missing lines Patch % Lines
...ubstrait/src/logical_plan/consumer/rel/read_rel.rs 78.12% 6 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #24346      +/-   ##
==========================================
+ Coverage   81.17%   81.31%   +0.13%     
==========================================
  Files        1109     1117       +8     
  Lines      388033   395873    +7840     
  Branches   388033   395873    +7840     
==========================================
+ Hits       314985   321903    +6918     
- Misses      54509    55148     +639     
- Partials    18539    18822     +283     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@kosiew kosiew left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eliot1480, thanks for working on this. This looks good to me.

The change cleanly removes support for consuming the deprecated VirtualTable.values field and the associated literal-row conversion helper, while consolidating virtual-table planning on VirtualTable.expressions. The empty-table and one-row EmptyRelation cases are also preserved using the expressions representation.

I don't have any blocking concerns or additional suggestions. 👍

@vbarua

vbarua commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

There is actually a user-facing change in this PR. If DataFusion consumes Substrait plans that set only the deprecated VirtualTable.values fields, those fields will be ignored.

Elliot's prior changes in #23672, now released in v55.0.0, make it so DataFusion produces plans using the VirtualTable.expressions fields. Any plans produced by DataFusion v55.0.0 onwards should be consumed correctly, which will allow users to update any stored plans they may have. Externally produced plans will also be affected by this change. Users sending plans to DataFusion will need to make sure they are setting VirtualTable.expressions in order for DataFusion to see them.

Comment thread datafusion/substrait/src/logical_plan/consumer/rel/read_rel.rs Outdated
Co-authored-by: Victor Barua <victor.barua@datadoghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

substrait Changes to the substrait crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove usages of deprecated VirtualTable.values field

4 participants