You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Insert SQLAlchemy compliance fixture rows once for read-only test classes #847
The SQLAlchemy compliance suites (just test sqla and just test sqla-async) issued about 65% of the CI queries in the sample recorded on #834. Much of that comes from fixture data rather than from the statements under test:
SQLAlchemy's TablesTest inserts its fixture rows before each test and deletes them after each test (run_inserts="each", run_deletes="each"). This also happens for tests that are then skipped by method-level requirements.
tests/sqlalchemy/conftest.py creates every table as Iceberg, so each insert and each DELETE FROM is a separate Iceberg commit that writes several S3 objects.
Since #837, these suites run on dialect changes, weekly, and on dispatch. Each run is still the most expensive part of CI.
Proposed change
Reduce fixture traffic in tests/sqlalchemy/test_suite.py without changing what the tests assert:
For compliance classes whose tests only read their fixture data, override run_inserts = "once" and run_deletes = None, so rows are inserted once per class.
Where possible, avoid inserting fixture rows for tests that the dialect's requirements skip.
Keep per-test inserts and deletes for classes whose tests modify the rows.
Consider whether read-only classes need Iceberg tables at all, or can use cheaper Hive tables.
Each override has to be checked against the SQLAlchemy class it modifies, because a class that looks read-only may still rely on fresh rows.
Validation plan (if implementing)
Run just test sqla and just test sqla-async against AWS, and confirm the same pass/skip results as before.
Count the queries issued by one compliance run before and after the change, for example with a temporary client-side counter or from the Athena query history of the run's databases.
Activity
Insert SQLAlchemy compliance fixture rows once for read-only classes