Materialise FK lists in ImportMetadataViewset to avoid mis-planned subqueries#14770
Open
rtibbles wants to merge 1 commit into
Open
Materialise FK lists in ImportMetadataViewset to avoid mis-planned subqueries#14770rtibbles wants to merge 1 commit into
rtibbles wants to merge 1 commit into
Conversation
Contributor
Build Artifacts
Smoke test screenshot |
Replace `__in=<queryset>` with materialised lists and the existing `__uuidin` lookup (which inlines IDs as SQL literals) for UUID-keyed columns. The subquery form gave postgres correlated subqueries it sometimes mis-planned on a fresh test database, producing multi-hour query times in CI. `__uuidin` also sidesteps SQLITE_MAX_VARIABLE_NUMBER. Language codes are dedup'd to a small set and use a plain literal IN. Catch EmptyResultSet from `__uuidin=[]` in the serialization loop. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
86cbf90 to
187330e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
__in=<queryset>inImportMetadataViewset._serializegave postgres correlated subqueries it mis-planned on fresh test DBs.__uuidinlookup (inlines IDs as SQL literals).References
Postgres CI hang observed intermittently across multiple recent PRs.
Investigation + fault-handler instrumentation temporarily on: PR Remove task-worker multiprocessing #14754 (now removed). Captured stack (CI run):
Same stack across 11 consecutive 120-s dumps over ~2 h before the suite recovered.
Reviewer guidance
test_public_api.pyexercises every_serializequery against postgres — that suite plus the existing channel-import flow is the primary check.__uuidinfilters (e.g. a tag-less channel) flow through theEmptyResultSetbranch — worth a glance at the loop'stry/except.Q(...) | Q(...)OR-subquery — confirmNoneis filtered and no codes are lost.AI usage
Used Claude Code for investigation, diagnosis (fault-handler instrumentation + CI stack capture), and drafting the patch. I guided the lookup choice (
__uuidin), reviewed the diff and drove revisions, and confirmedtest_public_api.pypasses against postgres locally.