Repository navigation
Description
Part of #271 and apache/datafusion-comet#7300.
What the function does
xpath_int(xml, path) evaluates an XPath 1.0 expression, converts the result with XPath number() into a
Java Double, and narrows it with Double.intValue(), giving an INT. Both arguments are strings
(collation-aware from 4.0), and path must be foldable. The result is always nullable. Available since Spark
2.0.0.
The number() conversion is in the xpath_double issue. Parsing, DTD and entity handling, namespaces, NULL
handling and errors are in the xpath issue. This issue covers only the double-to-int step, which is Java's
narrowing conversion (int) d:
NaNbecomes0;- values are truncated toward zero;
- out-of-range values saturate at
Integer.MAX_VALUE/Integer.MIN_VALUE, and so do+/-Infinity.
This matches Rust's d as i32. Nothing raises an error, and ANSI mode has no effect.
Examples
SELECT xpath_int('<a><b>1</b><b>2</b></a>', 'sum(a/b)');
-- 3
SELECT xpath_int('<a>this is not a number</a>', 'a');
-- 0
SELECT xpath_int('<a>this 2 is not a number</a>', 'a');
-- 0
SELECT xpath_int('<a><b>2000000000</b><c>40000000000</c></a>', 'a/b * a/c');
-- 2147483647
SELECT xpath_int('<a>try a boolean</a>', 'a = 10');
-- 0
SELECT xpath_int('<a><b class="odd">1</b><b class="even">2</b><b class="odd">4</b><c>8</c></a>', 'a/b');
-- 1
SELECT xpath_int('<a><b class="odd">1</b><b class="even">2</b><b class="odd">4</b><c>8</c></a>', 'sum(a/*)');
-- 15
SELECT xpath_int('<a><b class="odd">100000</b><b class="even">2</b><b class="odd">4</b><c>8</c></a>', 'sum(a/b[@class="odd"])');
-- 100004These come from the ExpressionDescription, the Hive udf_xpath_int golden files run by
HiveCompatibilitySuite, and XPathExpressionSuite.
Edge cases to test
- Saturation at both ends:
8.0E19gives2147483647(above). By the same rule,-1 * a/b * a/cgives
-2147483648,1 div 0gives2147483647, and-1 div 0gives-2147483648. - Truncation:
<a>2.7</a>gives2and<a>-2.7</a>gives-2, by code reading. Note that XPathround()
andfloor()give different results, so testxpath_int(x, 'round(a)')as well. NaNgives0(non-numeric text,0 div 0, and the empty node-set, which isNaNby XPath 1.0 rules).
The usage string says "zero if no match is found, or a match is found but the value is non-numeric".- Values just at and beyond
2^31:2147483647,2147483648and-2147483649. NULLor''arguments give NULL. A non-literal path givesDATATYPE_MISMATCH.NON_FOLDABLE_INPUT.
Malformed XML and an invalid path raise runtime errors (see thexpathissue).
Differences across Spark versions
None found beyond the shared ones in the xpath issue.
Implementation notes
- Spark:
XPathIntinsql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/xml/xpath.scala.
It callsXPathIntEvaluator(ret.intValue()) in.../xml/XmlExpressionEvalUtils.scala. - DataFusion core has no equivalent.
- Reuse the shared
number()evaluator from thexpathandxpath_doubleissues in
crates/spark/src/function/xml/, and narrow withas i32. Rust's saturating float-to-int cast matches
Java's here. - Tests: add
crates/sqllogictest/test_files/xml/xpath_int.sltand document the function with#[user_doc].
This issue was generated by an LLM (Claude) from the Apache Spark source code and tests. Examples come from Spark's ExpressionDescription examples, test suites and .sql.out golden files, but please verify them against Spark before relying on them.
Activity