Skip to content

Commit 67efcd9

Browse files
fix(kernel): keep local path validation scoped to transfers
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
1 parent 3305d43 commit 67efcd9

8 files changed

Lines changed: 32 additions & 11 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# Release History
22

33
# Unreleased
4-
- Support SQL PUT, GET, and REMOVE file transfers with a native kernel exposing `Statement.set_staging_allowed_local_paths`; older wheels reject transfers and continue to support ordinary queries.
4+
- Support SQL PUT, GET, and REMOVE file transfers with a native kernel exposing `Statement.set_staging_allowed_local_paths`; older wheels reject transfers and continue to support ordinary queries. Streaming PUT remains unsupported in kernel mode.
55
- Fix inline parameters (`use_inline_params=True`) so a sequence directly after `IN` renders as a list again: `x IN %(ids)s` renders as `x IN (1,2,3)` instead of `x IN ARRAY(1,2,3)`, which Spark rejects. Inside the list, tuples render as rows for a multi-column `IN`, and a `set` or `frozenset` is accepted again. Sequences anywhere else still render as `ARRAY(...)` ([#629](https://github.com/databricks/databricks-sql-python/issues/629))
66

77
# 4.9.0 (2026-11-04)

‎CONNECTION_PARAMETERS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -171,7 +171,7 @@ to change without notice.
171171
| `use_inline_params` | `bool` \| `"silent"` | ✅ | ✅ | `False` | Render parameters inline (legacy) vs. native bound params (DBR 14.1+). Shared cursor code performs the rendering before dispatch, so both backends honor it identically. |
172172
| `ignore_transactions` | `bool` | ✅ | ✅ | `True` | When `True`: `commit()` is a no-op, `rollback()` raises `NotSupportedError`, and setting `autocommit` is a no-op. |
173173
| `fetch_autocommit_from_server`| `bool` | ✅ | ✅ | `False` | Query the server (`SET AUTOCOMMIT`) for autocommit state instead of returning the cached value. |
174-
| `staging_allowed_local_path` | `str` \| `List[str]` | ✅ | ✅ | `None` | Local path(s) permitted for Unity Catalog Volume `PUT`/`GET`. Kernel transfers require a wheel exposing `Statement.set_staging_allowed_local_paths` (kernel [#439](https://github.com/databricks/databricks-sql-kernel/pull/439)); older wheels reject transfer commands. |
174+
| `staging_allowed_local_path` | `str` \| `List[str]` | ✅ | ✅ | `None` | Local path(s) permitted for Unity Catalog Volume `PUT`/`GET`. Kernel transfers require a wheel exposing `Statement.set_staging_allowed_local_paths` (kernel [#439](https://github.com/databricks/databricks-sql-kernel/pull/439)); older wheels reject transfer commands. Kernel mode does not support streaming PUT. |
175175

176176
## Telemetry
177177

‎KERNEL_REV‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
02283500d50e1bcd61007e0cf04a7e1e3d16ea86
1+
2f317d97212f00d73b89d947f5261a2145ea7a10

‎src/databricks/sql/backend/kernel/_errors.py‎

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -17,8 +17,8 @@
1717
"only PEP 249 exception types cross the boundary". Wrapped in
1818
``OperationalError`` here.
1919
20-
Local missing transfer files deliberately retain the built-in
21-
``FileNotFoundError`` contract.
20+
Local missing transfer files use the built-in ``FileNotFoundError`` class;
21+
the OS ``errno`` and ``filename`` attributes are unavailable.
2222
2323
These primitives live in their own module so both ``client.py``
2424
(which orchestrates PyO3 calls) and ``result_set.py`` (which calls
@@ -192,7 +192,8 @@ def wrap_kernel_exception(
192192
"""Map a PyO3 exception to the connector's public error contract.
193193
194194
- ``KernelError`` → mapped class with structured attrs forwarded.
195-
A local missing transfer file maps to ``FileNotFoundError``.
195+
A local missing transfer file maps to ``FileNotFoundError`` without
196+
the OS ``errno`` or ``filename`` attributes.
196197
- Already-PEP-249 ``Error`` (e.g. raised by an inner caller that
197198
already mapped) → passed through unchanged.
198199
- Anything else (``TypeError`` / ``ValueError`` / etc. from PyO3

‎src/databricks/sql/backend/kernel/client.py‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -588,7 +588,11 @@ def execute_command(
588588
"Streaming PUT is not supported by the kernel backend."
589589
)
590590
stmt.set_sql(operation)
591-
allowed_paths = cursor.connection.staging_allowed_local_path
591+
allowed_paths = (
592+
cursor.connection.staging_allowed_local_path
593+
if _is_staging_statement(operation)
594+
else None
595+
)
592596
if isinstance(allowed_paths, (str, os.PathLike)):
593597
allowed_paths = [allowed_paths]
594598
set_paths = getattr(stmt, "set_staging_allowed_local_paths", None)

‎src/databricks/sql/client.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1394,7 +1394,7 @@ def _handle_staging_put(
13941394
session_id_hex=self.connection.get_session_id_hex(),
13951395
)
13961396

1397-
# The kernel completes configured transfers before reaching these handlers.
1397+
# Kernel backends transfer files natively and never dispatch to these handlers.
13981398
assert self.connection.http_client is not None
13991399
with open(local_file, "rb") as fh:
14001400
r = self.connection.http_client.request(

‎tests/e2e/test_kernel_backend.py‎

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -367,9 +367,7 @@ def test_schemas_with_empty_string_filter_matches_nothing(conn):
367367
assert cur.fetchall() == []
368368

369369

370-
@pytest.mark.parametrize(
371-
"empty_filter", ["schema_name", "table_name", "column_name"]
372-
)
370+
@pytest.mark.parametrize("empty_filter", ["schema_name", "table_name", "column_name"])
373371
def test_columns_with_empty_string_filter_matches_nothing(conn, empty_filter):
374372
filters = {
375373
"catalog_name": "system",

‎tests/unit/test_kernel_client.py‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -847,6 +847,24 @@ def test_execute_command_forwards_allowed_local_paths(allowed, expected, async_o
847847
(stmt.submit if async_op else stmt.execute).assert_called_once()
848848

849849

850+
@pytest.mark.parametrize("allowed", [123, [object()]])
851+
def test_non_transfer_query_ignores_malformed_local_paths(allowed):
852+
c = _make_client()
853+
c._kernel_session = MagicMock()
854+
cursor = MagicMock()
855+
cursor.connection.staging_allowed_local_path = allowed
856+
stmt = c._kernel_session.statement.return_value
857+
stmt.execute.return_value = MagicMock(
858+
statement_id="00000000-0000-0000-0000-000000000001",
859+
arrow_schema=MagicMock(return_value=pa.schema([("x", pa.int64())])),
860+
)
861+
c.execute_command(
862+
"SELECT 1", MagicMock(), 1, 1, False, cursor, False, [], False, False
863+
)
864+
stmt.set_staging_allowed_local_paths.assert_called_once_with([])
865+
stmt.execute.assert_called_once()
866+
867+
850868
@pytest.mark.parametrize(
851869
"query",
852870
[

0 commit comments

Comments
 (0)