Derive default-catalog product ids deterministically (issue #34) - #37
Open
aleksUIX wants to merge 1 commit into
Open
Derive default-catalog product ids deterministically (issue #34)#37aleksUIX wants to merge 1 commit into
aleksUIX wants to merge 1 commit into
Conversation
…b#34) build_static_product_catalog minted prod-{uuid4} ids per process, so any multi-worker deployment (the shipped Dockerfile runs uvicorn --workers 2) or a restart between GET /products and GET /products/{id} could 404 on an id the server itself had just returned. This is the surviving kernel of issue IABTechLab#34 observation 1; the CSV-mode side was fixed in v2.2.2. Derive ids with uuid5 over the config name instead: same prod-[0-9a-f]{8} shape, unique per product, identical in every process. Adds a regression test that rebuilds the catalog after a cache reset (the single-process proxy for a second worker) and asserts the id set is unchanged.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The last live piece of #34 observation 1. In default (non-CSV) mode,
build_static_product_catalogmintsprod-{uuid4().hex[:8]}ids at build time, and the catalog cache is per process. Any deployment with more than one process reproduces the reported symptom:GET /productsserved by worker A returns ids thatGET /products/{id}on worker B has never seen, so the lookup 404s. The repo's own image runsuvicorn --workers 2(infra/docker/Dockerfile:82), and a restart between list and get does the same thing. The CSV-mode side of this was already fixed in v2.2.2 by reading ids from the CSV.Change
Derive each id with
uuid5(NAMESPACE_URL, "ad-seller-product:{name}")instead ofuuid4. Ids keep the exactprod-[0-9a-f]{8}shape pinned bytest_default_mode_ids_stay_uuid_shaped_and_stable, stay unique per product, and are now identical in every process and across restarts.Adds
test_default_mode_ids_survive_cache_reset: rebuilds the catalog afterreset_catalog_cache()(the single-process proxy for a second worker) and asserts the id set is unchanged, plus uniqueness across the 13 configs.Verification
Ran the catalog build in two separate Python processes: identical id sets, all shape-conformant, 13 unique ids. The new test passes; note the full suite needs the
iab-agentic-primitivesdependency, which public clones cannot currently install.