Repository navigation
Conversation
…ugging Face operators
|
/request-review @Ma77Ball |
Backport auto-label reportThis
|
Automated Reviewer SuggestionsBased on the
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #9209 +/- ##
============================================
+ Coverage 91.57% 91.58% +0.01%
+ Complexity 5158 5146 -12
============================================
Files 1300 1301 +1
Lines 58092 58109 +17
Branches 7615 7613 -2
============================================
+ Hits 53195 53217 +22
+ Misses 2959 2958 -1
+ Partials 1938 1934 -4
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🔴 | bs=10 sw=10 sl=64 | 598 | 0.365 | 15,348/25,109/25,109 us | 🔴 +8.5% / 🔴 +94.0% |
| 🔴 | bs=100 sw=10 sl=64 | 1,299 | 0.793 | 77,910/83,630/83,630 us | 🟢 -15.7% / 🟢 -17.6% |
| ⚪ | bs=1000 sw=10 sl=64 | 1,464 | 0.894 | 682,553/731,597/731,597 us | ⚪ within ±5% / 🟢 -8.0% |
Baseline details
Latest main 7f9bdf3 from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 598 tuples/sec | 642 tuples/sec | 1,092 tuples/sec | -6.9% | -45.2% |
| bs=10 sw=10 sl=64 | MB/s | 0.365 MB/s | 0.392 MB/s | 0.666 MB/s | -6.9% | -45.2% |
| bs=10 sw=10 sl=64 | p50 | 15,348 us | 14,146 us | 9,398 us | +8.5% | +63.3% |
| bs=10 sw=10 sl=64 | p95 | 25,109 us | 23,972 us | 12,941 us | +4.7% | +94.0% |
| bs=10 sw=10 sl=64 | p99 | 25,109 us | 23,972 us | 15,496 us | +4.7% | +62.0% |
| bs=100 sw=10 sl=64 | throughput | 1,299 tuples/sec | 1,327 tuples/sec | 1,486 tuples/sec | -2.1% | -12.6% |
| bs=100 sw=10 sl=64 | MB/s | 0.793 MB/s | 0.81 MB/s | 0.907 MB/s | -2.1% | -12.6% |
| bs=100 sw=10 sl=64 | p50 | 77,910 us | 73,263 us | 72,544 us | +6.3% | +7.4% |
| bs=100 sw=10 sl=64 | p95 | 83,630 us | 99,213 us | 80,069 us | -15.7% | +4.4% |
| bs=100 sw=10 sl=64 | p99 | 83,630 us | 99,213 us | 101,531 us | -15.7% | -17.6% |
| bs=1000 sw=10 sl=64 | throughput | 1,464 tuples/sec | 1,449 tuples/sec | 1,533 tuples/sec | +1.0% | -4.5% |
| bs=1000 sw=10 sl=64 | MB/s | 0.894 MB/s | 0.885 MB/s | 0.936 MB/s | +1.0% | -4.5% |
| bs=1000 sw=10 sl=64 | p50 | 682,553 us | 686,663 us | 711,170 us | -0.6% | -4.0% |
| bs=1000 sw=10 sl=64 | p95 | 731,597 us | 735,144 us | 749,276 us | -0.5% | -2.4% |
| bs=1000 sw=10 sl=64 | p99 | 731,597 us | 735,144 us | 794,786 us | -0.5% | -8.0% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,334.17,200,128000,598,0.365,15348.07,25109.16,25109.16
1,100,10,64,20,1539.80,2000,1280000,1299,0.793,77909.87,83630.30,83630.30
2,1000,10,64,20,13657.18,20000,12800000,1464,0.894,682553.26,731596.55,731596.55
Ma77Ball
left a comment
There was a problem hiding this comment.
Review summary
This PR routes getOutputSchemas validation for the four legacy Hugging Face operators (Spam, Sentiment, Summarization, Iris) through one private[huggingFace] helper. The substantive fixes are correct and well targeted: Spam and Sentiment stop returning null and now throw a named-field error, which removes the real bug (a returned null counted as success in PhysicalOp.propagateSchema, then threw an opaque NullPointerException one line later in schemaMapping.foldLeft); all four messages now name the field; input columns are checked at build time with exact-case matching that mirrors the generated Python; and Summarization reads its schema by port id instead of values.head. The change is sound, and the test coverage is unusually thorough (negative, positive, case-sensitivity, numeric, and the end-to-end propagation path that proves the NPE is gone). No blocking issues, and nothing I would gate merge on.
Action items: Nothing blocking; no changes requested.
Findings
0 total: 0 P0, 0 P1, 0 P2, 0 P3.
No inline findings. The items I went looking for (a possible NoSuchElementException on the input-port key, a caller relying on the old null return, an over-strict case/title check, or error titles that don't match the UI) all checked out as safe; see Verification.
Verification
- null to throw is safe. The sole caller of
getOutputSchemasisPhysicalOp.propagateSchema(PhysicalOp.scala:497), which runs it insideTryand stores any thrown error on the output port; the previousnullescaped thatTryas theNullPointerExceptioninschemaMapping.foldLeft. Grepped the backend: no caller compares agetOutputSchemasresult tonull, so no path depended on the old contract. - The new unconditional input-port access cannot throw
NoSuchElementException. All four operators now readinputSchemas(operatorInfo.inputPorts.head.id)before any guard.propagateSchemaonly invokes the function whenupdatedOp.inputPorts.size == inputSchemas.size(PhysicalOp.scala:495), and the map is built from the operator's own input ports, so every input port id is present when the function runs. - Exact-case matching is correct, not over-strict. The generated Python indexes
tuple_[$attribute]by exact name, whileSchema.getIndex/containsAttributelowercase both sides (Schema.scala:39,63,75). The helper's manualattributes.find(_.getName == column)is the deliberate case-sensitive workaround, so a workflow that only differs by case would fail at run time anyway and is correctly rejected at build time. - Error-message field titles match the rendered form labels. Operator schemas are generated with
html5EnabledSchema, whoseautoGenerateTitleForProperties = trueis not overridden bytexeraSchemaGeneratorConfig(OperatorMetadataGenerator.scala:87), andgenerateTitleFromPropertyNamehumanizes camelCase and capitalizes the first letter (JsonSchemaGenerator.scala:1485). Soattributeto "Attribute",petalLengthCmAttributeto "Petal Length Cm Attribute", andprediction class nameto "Prediction class name" all match the messages exactly, including the leading capital. - Not run locally. I did not execute the suite; I relied on the PR's reported 2756-pass run. At review time the
build / amber (ubuntu-latest, 17)job was still pending, so the test job is not yet green on CI.
|
Removing release/v1.2: error message improvement, and part of the code is not on that branch. |
xuang7
left a comment
There was a problem hiding this comment.
LGTM. The field names in the messages match the form titles, and the exact-case column check lines up with how the generated Python reads the tuple. One optional note inline.
What changes were proposed in this PR?
The four legacy Hugging Face operators (Spam Detection, Sentiment Analysis, Text Summarization and Iris Logistic Regression) now validate their configuration the same way in
getOutputSchemas, through one shared helper,HuggingFaceSchemaValidation:null, as @Ma77Ball recommended on Align schema validation across the four legacy Hugging Face operators #8728. Spam and Sentiment used to returnnullfor an empty result column. The compiler runsgetOutputSchemasinside aTryand stores a thrown error on the output port. A returnednullcounted as success, then failed one line later inPhysicalOp.propagateSchema(schemaMapping.foldLeft) with aNullPointerException, so the user saw an NPE instead of a message. Summarization and Iris already threw, but with generic messages. All four now name the field, e.g.Spam result attribute cannot be empty.Attributefor the three text operators, the two petal columns for Iris) must be set and present in the input table. The match is exact-case:Schemalookups ignore case, but the generated Python does not. The error lists the available columns, as fix(workflow-operator): report a clear error when a configured HF image/audio column is missing #7299's does. Iris's petal columns must also be numeric (integer, long or double), matching its existing frontend type rule.inputSchemas(operatorInfo.inputPorts.head.id)), as the other three do, instead ofinputSchemas.values.head.Two corrections to the text of #8728. Iris reads two input columns (petal length and petal width), not four. And #7299's check runs in the generated Python at run time, so it is the precedent for the message wording, not for where the check lives.
Any related issues?
Closes #8728
How was this PR tested?
The tests were written first. 22 new or rewritten tests across the four specs failed against
mainfor the expected reasons; Spam's and Sentiment's propagation tests failed with theNullPointerExceptionitself. All 22 pass with the change, and reverting the four operators makes exactly those 22 fail again. The six tests that assertednullnow assert the thrown message. The Spam and Iris happy-path inputs gained the columns those operators read, and their assertions are unchanged.The full
WorkflowOperatorsuite passes: 2756 tests, 0 failures, with 1 cancelled and 2 pending, none of them in the Hugging Face suites. The generated Python of all 117 operators compiles, and scalafmt and scalafix are clean.Was this PR authored or co-authored using generative AI tooling?
Yes, this PR was co-authored with Claude in compliance with ASF policy.