Repository navigation
Description
Part of #271 and apache/datafusion-comet#7300.
What the function does
xpath_boolean(xml, path) evaluates an XPath 1.0 expression and converts the result to BOOLEAN using the
XPath boolean() rules (XPathConstants.BOOLEAN). Both arguments are strings (collation-aware from 4.0), and
path must be foldable. The result is always nullable. Available since Spark 2.0.0. The expression is a
Predicate, so it can be used directly in WHERE.
Parsing, DTD and entity handling, namespaces, NULL and empty-string handling, invalid-XML and invalid-path
errors, and lazy path compilation are shared with the rest of the family. They are specified in the xpath
issue, and the implementation should reuse that evaluator. This issue covers only the boolean conversion.
Examples
SELECT xpath_boolean('<a><b>1</b></a>','a/b');
-- true
SELECT xpath_boolean('<a><b>b</b></a>', 'a/c');
-- false
SELECT xpath_boolean('<a><b>b</b></a>', 'a/b = "b"');
-- true
SELECT xpath_boolean('<a><b>b</b></a>', 'a/b = "c"');
-- false
SELECT xpath_boolean('<a><b>10</b></a>', 'a/b < 10');
-- false
SELECT xpath_boolean('<a><b>10</b></a>', 'a/b = 10');
-- trueThese come from the ExpressionDescription, XPathExpressionSuite and the Hive udf_xpath_boolean golden
files run by HiveCompatibilitySuite. In UDFXPathUtilSuite, a/b[1]/text() is true and a/b[4] (no match)
is false.
Edge cases to test
- Conversion rules from XPath 1.0
boolean():- a node-set is true if it is non-empty, whatever the text.
<b>false</b>witha/b/text()gives true; - a number is false for
0,-0andNaN, and true otherwise; - a string is true if it is non-empty, so
string(a/c)on a missing node gives false; - a boolean result is used as is.
- a node-set is true if it is non-empty, whatever the text.
- Comparisons between a node-set and a value are existential:
a/b = 2is true if anybequals 2. Also test
number versus string comparison (a/b = "10"versusa/b = 10with<b>10.0</b>). NULLor''for either argument gives NULL. A non-literalpathfails with
DATATYPE_MISMATCH.NON_FOLDABLE_INPUT.- Malformed XML and an invalid path raise the runtime errors described in the
xpathissue. There is no
falsefallback. - Use in a filter (
WHERE xpath_boolean(xml, 'a/b')) with NULL rows. NULL rows are filtered out.
Differences across Spark versions
None found beyond the shared ones listed in the xpath issue (4.0 moved to RuntimeReplaceable and added
collation support).
Implementation notes
- Spark:
XPathBooleaninsql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/xml/xpath.scala.
It callsXPathBooleanEvaluator, which callsUDFXPathUtil.evalBoolean. - DataFusion core has no equivalent.
- Put it in
crates/spark/src/function/xml/on top of the shared evaluator from thexpathissue.
The existingcrates/sqllogictest/test_files/xml/xpath_boolean.sltis a commented-out stub that should be
enabled and extended. Document the function with#[user_doc].
This issue was generated by an LLM (Claude) from the Apache Spark source code and tests. Examples come from Spark's ExpressionDescription examples, test suites and .sql.out golden files, but please verify them against Spark before relying on them.
Activity