Skip to content

Gandiva: Add missing SQL function implementations - #115

Closed
StevenMPhillips wants to merge 17 commits into
dremio:dremio_26.0_18.1.0from
StevenMPhillips:gandiva-functions-steven
Closed

StevenMPhillips wants to merge 17 commits into
dremio:dremio_26.0_18.1.0from
StevenMPhillips:gandiva-functions-steven

Conversation

@StevenMPhillips

Copy link
Copy Markdown

Summary

This PR adds native Gandiva implementations for several SQL functions that were previously Java-only in Dremio.

Functions Implemented

  • date_trunc() - All precision levels (year, quarter, month, week, day, hour, minute, second)
  • parse_url() - Extract URL components (HOST, PATH, QUERY, PROTOCOL, etc.)
  • pi() - Mathematical constant π
  • find_in_set() - Find string position in comma-separated list
  • array_length/cardinality/array_position - Array utility functions with proper NULL semantics
  • parse_timestamp() - Parse timestamp from string with SQL format patterns
  • current_date/current_time/current_timestamp/now() - Non-deterministic time functions
  • convert_timezone() - 2-arg and 3-arg timezone conversion
  • string_binary/binary_string - Binary/string reinterpret conversions
  • add_weeks/add_months/add_quarters/add_years - Date arithmetic (prototypes)

Testing

Each function includes unit tests. Requesting CI run to validate.

Notes

  • Built against Arrow 18.x (matching Dremio 26)
  • Some functions (regexp_like, to_char) have partial work but are not included in this PR

Add the mathematical constant pi() function to Gandiva.

- Returns the value of mathematical constant π (3.14159265358979...)
- No input parameters
- Returns double (float64)
- Always non-null

This enables Gandiva to natively execute queries using pi() without
falling back to Java execution.
Add the find_in_set(needle, haystack) function to Gandiva.

- Searches for a string in a comma-separated list
- Returns 1-indexed position if found, 0 if not found
- Example: find_in_set('b', 'a,b,c') returns 2

This enables Gandiva to natively execute queries using find_in_set()
without falling back to Java execution.
Formats a number with commas and fixed decimal places.
Examples:
- format_number(1234567.89, 2) -> "1,234,567.89"
- format_number(1000.0, 0) -> "1,000"
- format_number(-1234.5, 2) -> "-1,234.50"

Handles NaN, Infinity, negative numbers, and edge cases.
Generates a random UUID v4 string (RFC 4122) in the format:
xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

Uses thread-local mt19937_64 PRNG, sets proper version (4) and
variant (10xx) bits per the RFC specification.
- lshift(value, bits): left shift (zero-fill on right)
- rshift(value, bits): arithmetic right shift (sign-extends)
- rshiftunsigned(value, bits): logical right shift (zero-fill on left)

Supports int32 and int64 types. Handles edge cases to avoid C++ UB:
- bits < 0 or >= bitwidth returns 0 for lshift/rshiftunsigned
- rshift with bits >= bitwidth returns -1 for negative, 0 for non-negative
- rand_integer(max): returns random int in [0, max]
- rand_integer(min, max): returns random int in [min, max]

Uses thread-local mt19937 PRNG with uniform distribution.
Handles min > max by swapping arguments.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant