Repository navigation
Complete configured Volume transfers through the native backend - #613
aakash-saravanan-db wants to merge 6 commits into
Conversation
|
Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase ( |
ad49367 to
7b276d7
Compare
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
Signed-off-by: Aakash Saravanan <aakash.saravanan@databricks.com>
7b276d7 to
12876fc
Compare
There was a problem hiding this comment.
Verdict: 1 Medium · 1 Low
Mostly sound, with one medium issue: on the kernel path, the new StagingError wrapping can also catch the statement's own execution errors, not just transfer errors. Nits: there is a stray blank line in the CHANGELOG entry, and the constructor comment in KernelOperationBackend still says cancel() "always routes to the cancellable execution". Follow-up (not caused by this PR): executeStatement waits for the transfer before it returns the operation, so a caller can only interrupt an in-flight transfer by closing the session.
| this.volumeCompleted = outcome === 'completed'; | ||
| } catch (err) { | ||
| const mapped = decodeNapiKernelError(err); | ||
| if ( |
There was a problem hiding this comment.
🟡 Medium — The try covers getFetchHandle() as well as completeVolumeOperation(). On the cancellable-execution path and the directResults Running arm, getFetchHandle() is where the statement's own execution error comes out (result() / awaitResult() rejecting). Kernel SQL-error envelopes can carry httpStatus (for example a SqlError with 400). In that case the kernelMetadata?.httpStatus branch turns a server SQL failure into a plain StagingError(message), which drops the OperationStateError class, sqlState, kernelMetadata and cause.
This also hits ordinary non-Volume queries. DBSQLSession now calls completeStagingOperation for every kernel statement that has stagingAllowedLocalPath set, so a failing SELECT run with that option gets the wrong error type.
Suggested fix: resolve the handle outside the try, and only reclassify errors that come from completeVolumeOperation() (the transfer's HTTP or path-validation failures). If you still wrap those, consider keeping cause/kernelMetadata on the StagingError.
| expect(executionCancelled).to.equal(false); | ||
| }); | ||
|
|
||
| for (const outcome of ['completed', 'not_volume']) { |
There was a problem hiding this comment.
🔵 Low — The new tests cover only the completed and not_volume outcomes and the cancel routing. These new branches in completeStagingOperation have no unit test:
- the
needs_input_stream→StagingErrorpath - reclassifying
ParameterErrorandhttpStatuserrors asStagingError - passing other errors through unchanged (for example local fs errors decoded with
code/path)
No test checks that DBSQLSession closes the operation when completion fails either. The error-classification branch is where F1 lives, so a test there would have caught it.
Feature or gap
Configured SQL Volume operations can return instruction rows without transferring files on the native backend.
What this PR addresses
Invoke shared completion when stagingAllowedLocalPath is supplied, preserve raw opt-out and error mapping, and route cancellation to active transfers. Retain direct results for configured async execution, whose facade already waits for transfer completion.
Dependencies
Shared implementation: kernel change. Requires a compatible published native package and updated platform dependency versions. These release versions remain unselected.
Tests and validation
The full unit suite, affected async/cancellation tests, build and type checks pass. Independent sync/async byte, deletion and error checks pass; five-run comparisons meet ±10%, including changed-byte async overwrite. Windows and published-package proof remain pending.