Skip to content

fix(workflow-operator): one validation contract for the four legacy Hugging Face operators - #9209

Open
PG1204 wants to merge 4 commits into
apache:mainfrom
PG1204:fix/hf-legacy-schema-validation
Open

PG1204 wants to merge 4 commits into
apache:mainfrom
PG1204:fix/hf-legacy-schema-validation

Conversation

@PG1204

@PG1204 PG1204 commented Oct 29, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

The four legacy Hugging Face operators (Spam Detection, Sentiment Analysis, Text Summarization and Iris Logistic Regression) now validate their configuration the same way in getOutputSchemas, through one shared helper, HuggingFaceSchemaValidation:

  • They throw and never return null, as @Ma77Ball recommended on Align schema validation across the four legacy Hugging Face operators #8728. Spam and Sentiment used to return null for an empty result column. The compiler runs getOutputSchemas inside a Try and stores a thrown error on the output port. A returned null counted as success, then failed one line later in PhysicalOp.propagateSchema (schemaMapping.foldLeft) with a NullPointerException, so the user saw an NPE instead of a message. Summarization and Iris already threw, but with generic messages. All four now name the field, e.g. Spam result attribute cannot be empty.
  • Input columns are checked at build time. The column each operator reads (Attribute for the three text operators, the two petal columns for Iris) must be set and present in the input table. The match is exact-case: Schema lookups ignore case, but the generated Python does not. The error lists the available columns, as fix(workflow-operator): report a clear error when a configured HF image/audio column is missing #7299's does. Iris's petal columns must also be numeric (integer, long or double), matching its existing frontend type rule.
  • Summarization reads its input schema by port id (inputSchemas(operatorInfo.inputPorts.head.id)), as the other three do, instead of inputSchemas.values.head.

Two corrections to the text of #8728. Iris reads two input columns (petal length and petal width), not four. And #7299's check runs in the generated Python at run time, so it is the precedent for the message wording, not for where the check lives.

Any related issues?

Closes #8728

How was this PR tested?

The tests were written first. 22 new or rewritten tests across the four specs failed against main for the expected reasons; Spam's and Sentiment's propagation tests failed with the NullPointerException itself. All 22 pass with the change, and reverting the four operators makes exactly those 22 fail again. The six tests that asserted null now assert the thrown message. The Spam and Iris happy-path inputs gained the columns those operators read, and their assertions are unchanged.

The full WorkflowOperator suite passes: 2756 tests, 0 failures, with 1 cancelled and 2 pending, none of them in the Hugging Face suites. The generated Python of all 117 operators compiles, and scalafmt and scalafix are clean.

Was this PR authored or co-authored using generative AI tooling?

Yes, this PR was co-authored with Claude in compliance with ASF policy.

@PG1204

PG1204 commented Oct 29, 2026

Copy link
Copy Markdown
Contributor Author

/request-review @Ma77Ball

@Yicong-Huang Yicong-Huang added the release/v1.3 back porting to release/v1.3 label Oct 29, 2026
@Yicong-Huang Yicong-Huang added the release/v1.2 back porting to release/v1.2 label Oct 29, 2026
@github-actions
github-actions Bot requested review from mengw15 and xuang7 October 29, 2026 06:01
@github-actions

Copy link
Copy Markdown
Contributor

Backport auto-label report

This fix: PR was checked against each actively-supported release branch. A release/* label nominates a backport target; the branch's release manager approving this PR is what sends the fix there. The required Backport Approvals check stays red until every label below is approved, so each manager either approves or removes their own label — which is why the labels left on a merged PR are exactly the branches it reached.

Release branch Analysis
✅ release/v1.3 Change detected on this branch — label added; this fix is queued to backport here. @mengw15 decides: approving sends the fix here, removing this label declines it. The merge waits on one or the other. Review requested.
✅ release/v1.2 Change detected on this branch — label added; this fix is queued to backport here. @xuang7 decides: approving sends the fix here, removing this label declines it. The merge waits on one or the other. Review requested.

Auto-label run.

@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • Contributors with relevant context: @kz930, @aglinxinyuan
    You can notify them by mentioning @kz930, @aglinxinyuan in a comment.

@codecov-commenter

codecov-commenter commented Oct 29, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.58%. Comparing base (7f9bdf3) to head (7ef67b6).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #9209      +/-   ##
============================================
+ Coverage     91.57%   91.58%   +0.01%     
+ Complexity     5158     5146      -12     
============================================
  Files          1300     1301       +1     
  Lines         58092    58109      +17     
  Branches       7615     7613       -2     
============================================
+ Hits          53195    53217      +22     
+ Misses         2959     2958       -1     
+ Partials       1938     1934       -4     
Flag Coverage Δ *Carryforward flag
access-control-service 77.44% <ø> (ø)
agent-service 99.56% <ø> (ø) Carriedforward from 4db10cf
amber 85.46% <100.00%> (+0.03%) ⬆️
computing-unit-managing-service 63.88% <ø> (ø)
config-service 87.50% <ø> (ø)
file-service 82.16% <ø> (ø)
frontend 96.45% <ø> (+<0.01%) ⬆️ Carriedforward from 4db10cf
notebook-migration-service 83.77% <ø> (ø)
pyamber 98.38% <ø> (ø) Carriedforward from 4db10cf
result-service 85.71% <ø> (ø)
workflow-compiling-service 78.30% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

⚠️ Benchmark changes need a look

🟢 2 better · 🔴 4 worse · ⚪ 9 noise (<±5%) · 0 without baseline

Compared against main 7f9bdf3 benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

config throughput MB/s latency max Δ latest / 7d
🔴 bs=10 sw=10 sl=64 598 0.365 15,348/25,109/25,109 us 🔴 +8.5% / 🔴 +94.0%
🔴 bs=100 sw=10 sl=64 1,299 0.793 77,910/83,630/83,630 us 🟢 -15.7% / 🟢 -17.6%
⚪ bs=1000 sw=10 sl=64 1,464 0.894 682,553/731,597/731,597 us ⚪ within ±5% / 🟢 -8.0%
Baseline details

Latest main 7f9bdf3 from same runner

config metric PR latest main 7d avg Δ latest Δ 7d
bs=10 sw=10 sl=64 throughput 598 tuples/sec 642 tuples/sec 1,092 tuples/sec -6.9% -45.2%
bs=10 sw=10 sl=64 MB/s 0.365 MB/s 0.392 MB/s 0.666 MB/s -6.9% -45.2%
bs=10 sw=10 sl=64 p50 15,348 us 14,146 us 9,398 us +8.5% +63.3%
bs=10 sw=10 sl=64 p95 25,109 us 23,972 us 12,941 us +4.7% +94.0%
bs=10 sw=10 sl=64 p99 25,109 us 23,972 us 15,496 us +4.7% +62.0%
bs=100 sw=10 sl=64 throughput 1,299 tuples/sec 1,327 tuples/sec 1,486 tuples/sec -2.1% -12.6%
bs=100 sw=10 sl=64 MB/s 0.793 MB/s 0.81 MB/s 0.907 MB/s -2.1% -12.6%
bs=100 sw=10 sl=64 p50 77,910 us 73,263 us 72,544 us +6.3% +7.4%
bs=100 sw=10 sl=64 p95 83,630 us 99,213 us 80,069 us -15.7% +4.4%
bs=100 sw=10 sl=64 p99 83,630 us 99,213 us 101,531 us -15.7% -17.6%
bs=1000 sw=10 sl=64 throughput 1,464 tuples/sec 1,449 tuples/sec 1,533 tuples/sec +1.0% -4.5%
bs=1000 sw=10 sl=64 MB/s 0.894 MB/s 0.885 MB/s 0.936 MB/s +1.0% -4.5%
bs=1000 sw=10 sl=64 p50 682,553 us 686,663 us 711,170 us -0.6% -4.0%
bs=1000 sw=10 sl=64 p95 731,597 us 735,144 us 749,276 us -0.5% -2.4%
bs=1000 sw=10 sl=64 p99 731,597 us 735,144 us 794,786 us -0.5% -8.0%
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,334.17,200,128000,598,0.365,15348.07,25109.16,25109.16
1,100,10,64,20,1539.80,2000,1280000,1299,0.793,77909.87,83630.30,83630.30
2,1000,10,64,20,13657.18,20000,12800000,1464,0.894,682553.26,731596.55,731596.55

@mengw15 mengw15 removed the release/v1.3 back porting to release/v1.3 label Nov 9, 2026

@Ma77Ball Ma77Ball left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review summary

This PR routes getOutputSchemas validation for the four legacy Hugging Face operators (Spam, Sentiment, Summarization, Iris) through one private[huggingFace] helper. The substantive fixes are correct and well targeted: Spam and Sentiment stop returning null and now throw a named-field error, which removes the real bug (a returned null counted as success in PhysicalOp.propagateSchema, then threw an opaque NullPointerException one line later in schemaMapping.foldLeft); all four messages now name the field; input columns are checked at build time with exact-case matching that mirrors the generated Python; and Summarization reads its schema by port id instead of values.head. The change is sound, and the test coverage is unusually thorough (negative, positive, case-sensitivity, numeric, and the end-to-end propagation path that proves the NPE is gone). No blocking issues, and nothing I would gate merge on.

Action items: Nothing blocking; no changes requested.


Findings

0 total: 0 P0, 0 P1, 0 P2, 0 P3.

No inline findings. The items I went looking for (a possible NoSuchElementException on the input-port key, a caller relying on the old null return, an over-strict case/title check, or error titles that don't match the UI) all checked out as safe; see Verification.


Verification

  • null to throw is safe. The sole caller of getOutputSchemas is PhysicalOp.propagateSchema (PhysicalOp.scala:497), which runs it inside Try and stores any thrown error on the output port; the previous null escaped that Try as the NullPointerException in schemaMapping.foldLeft. Grepped the backend: no caller compares a getOutputSchemas result to null, so no path depended on the old contract.
  • The new unconditional input-port access cannot throw NoSuchElementException. All four operators now read inputSchemas(operatorInfo.inputPorts.head.id) before any guard. propagateSchema only invokes the function when updatedOp.inputPorts.size == inputSchemas.size (PhysicalOp.scala:495), and the map is built from the operator's own input ports, so every input port id is present when the function runs.
  • Exact-case matching is correct, not over-strict. The generated Python indexes tuple_[$attribute] by exact name, while Schema.getIndex/containsAttribute lowercase both sides (Schema.scala:39,63,75). The helper's manual attributes.find(_.getName == column) is the deliberate case-sensitive workaround, so a workflow that only differs by case would fail at run time anyway and is correctly rejected at build time.
  • Error-message field titles match the rendered form labels. Operator schemas are generated with html5EnabledSchema, whose autoGenerateTitleForProperties = true is not overridden by texeraSchemaGeneratorConfig (OperatorMetadataGenerator.scala:87), and generateTitleFromPropertyName humanizes camelCase and capitalizes the first letter (JsonSchemaGenerator.scala:1485). So attribute to "Attribute", petalLengthCmAttribute to "Petal Length Cm Attribute", and prediction class name to "Prediction class name" all match the messages exactly, including the leading capital.
  • Not run locally. I did not execute the suite; I relied on the PR's reported 2756-pass run. At review time the build / amber (ubuntu-latest, 17) job was still pending, so the test job is not yet green on CI.

@Ma77Ball Ma77Ball left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@PG1204 ty for the pr. Overall, it LGTM!

@mengw15 / @xuang7 Please remove the v1.2 backport and review when available.

@xuang7

xuang7 commented Nov 16, 2026

Copy link
Copy Markdown
Contributor

Removing release/v1.2: error message improvement, and part of the code is not on that branch.

@xuang7 xuang7 removed the release/v1.2 back porting to release/v1.2 label Nov 16, 2026

@xuang7 xuang7 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. The field names in the messages match the form titles, and the exact-case column check lines up with how the generated Python reads the tuple. One optional note inline.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Align schema validation across the four legacy Hugging Face operators

6 participants