Repository navigation
Gather over frames without the warehouse package - #101
Conversation
A schema built from frames names no source, so the relational gather takes the branch that reads nothing from a warehouse. It still imported the data providers, and each of those imports sdm_connectors at module scope, so a gather over tables already in memory raised ModuleNotFoundError on a client installed without a driver extra. The package is in no base dependency, and every extra carrying it also carries a driver the route never uses. run.py already names the rule this breaks, keeping its own import of the package under TYPE_CHECKING because a core route over frames needs none of it, and test_core_route_without_connectors.py already asserted it. That test only ran the tabular route, whose materializer builds the request directly and never reaches the gather, so the one route that could break it was the one route it did not cover. The providers module now imports a provider on the first read of its name, through the module's own attribute lookup so a test that binds one still wins, and the test runs both routes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Manush Murali <manushm@nvidia.com>
|
No actionable comments were generated in the recent review. Walkthrough
Change: Bug fix Merge Risk:
|
The lazy read bound the class it imported into this module's globals. A test that swaps a driver out and back imports the provider module again inside that window, so the binding outlived the module it came from and a later run answered with a class the swap had made. Python's module cache makes the repeated import a dictionary read. Signed-off-by: Manush Murali <manushm@nvidia.com> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Add Carrot credits or activate Agent usage billing to use Autofix |
What breaks
On a client installed without a driver extra, a relational gather over tables
already in memory fails:
Reproduced on a clean install of this branch's base, with no connectors
package present:
The chain, every link at module scope:
relational.py:76is the branch taken whenschema.source is None, so theroute that reads nothing from a warehouse is the one that required the
warehouse package.
structured-data-models-connectorsis in no basedependency of the client, and every extra that brings it also brings a driver
this route never uses, so there is nothing a caller can install to mean "I
have frames".
Why it is a bug and not a dependency
_rust/run.py:46keeps its own import of the package underTYPE_CHECKING,saying so: the warehouse package is an optional dependency, and a core route
over frames needs none of it.
tests/test_core_route_without_connectors.pyalready asserted that invariant by name. It only ran the tabular route, whose
materializer builds the request directly and never reaches the gather, so the
only route that could break the rule was the only route the test did not run.
The change
_rust/providers/__init__.pyimports a provider module on the first read ofits name instead of when the package is imported.
for_sourceresolves thename through the module's own attribute lookup, so a test that binds a
provider still wins; a first attempt that called
__getattr__directlybypassed module globals and broke 32 tests that monkeypatch
providers.SnowflakeProviderand friends.__all__, from-imports and theAttributeErrorfor an unknown name areunchanged.
The test now runs both routes, parametrised.
Checks
[relational]case fails on the base and passes with the change;[tabular]passes either way.suite: 1 failed, 4355 passed. The failure diff is empty, and that one
failure (
test_materialize_payloadstimestamptz) is pre-existing onv2-candidate.ruff check,ruff format --check,mypyover the threesrctrees, allfive hygiene checks, and the architecture-policy and tooling tests
(170 passed) are green.
Found while verifying the in-process skill references in #76, which had to
tell readers to install the connectors package to work around this.
🤖 Generated with Claude Code