feat(artifacts): add UserArtifactsIndex and backfill the keys it needs - #982
Merged
Merged
Conversation
Groundwork for serving the library's user-wide listing from an index
instead of a base-table Query. Nothing reads it yet — the query switch
ships separately, for the deploy-ordering reason below.
## Why an index at all
The base table is already partitioned by user, so this was never about
reachability: `list_for_user` can Query PK=USER#{uid} today. It is about
what that Query has to read. HEAD and version rows share the partition,
so it spans roughly 3x the rows it returns, then filters and date-sorts
them in memory — which is also what makes it unpaginable, since the page
boundary would fall in the wrong place. Only HEAD rows carry GSI2PK, so
the index holds one row per artifact, already newest-first.
## The backfill is the load-bearing half
`UserArtifactsIndex` is sparse. A HEAD row without GSI2PK is not stale
in it — it is absent from it forever, silently. The writer began
stamping the keys on 2026-09-04; every row written before that has
neither attribute, so switching the query without backfilling would drop
every older artifact from its owner's library with no error anywhere.
`backfill_artifact_user_index_keys.py` stamps them. Dry-run by default,
idempotent (`attribute_not_exists(GSI2PK)`, so it also yields to the
writer), and it never resurrects a row deleted mid-run
(`attribute_exists(SK)`).
It reads `updated_at` to build GSI2SK and never assigns it — that
attribute is embedded in both GSI sort keys and is writer-owned, the
same restraint `rename` observes. A HEAD row lacking `updated_at` is
reported by name rather than stamped with a fabricated timestamp that
would sort it wrongly forever.
Run on dev: 22 HEAD rows, 22 stamped, 0 skipped, 0 failed; re-run
reported 22 already-stamped, 0 stamped. Verified independently
afterwards: 0 rows missing GSI2PK, 0 version rows wrongly stamped, and
GSI1SK == GSI2SK on all 22 — GSI1SK is the *writer's* own
`ARTIFACT#{updated_at}#{aid}`, so that agreement checks the backfill's
format against production-written data rather than against a reading of
the code.
## Deploy notes
Adding this is one `UpdateTable`, and only one GSI may be added per
`UpdateTable` — the committed `gsi-inventory.json` gains exactly one
line here, so no release split is needed.
CFN reporting UPDATE_COMPLETE does NOT mean the index is ACTIVE;
DynamoDB backfills it asynchronously. That is why the query switch is a
separate PR: platform.yml and backend.yml share a concurrency group but
their order is not enforced, so shipping both together risks app-api
querying an index that does not exist yet.
Infrastructure: 784 passed. Backend: 186 passed across the artifact
suites, including 10 new tests for the script.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Groundwork for serving the library's user-wide listing from an index instead of a base-table Query. Nothing reads it yet — the query switch ships separately, for the deploy-ordering reason below.
Why an index at all
The base table is already partitioned by user, so this was never about reachability —
list_for_usercan QueryPK=USER#{uid}today. It's about what that Query has to read. HEAD and version rows share the partition, so it spans roughly 3× the rows it returns, then filters and date-sorts them in memory. That in-memory sort is also what makes it unpaginable: the page boundary would fall in the wrong place.Only HEAD rows carry
GSI2PK, so the index holds one row per artifact, already newest-first.The backfill is the load-bearing half
UserArtifactsIndexis sparse. A HEAD row withoutGSI2PKisn't stale in it — it's absent from it forever, silently. The writer began stamping the keys on 2026-09-04, so every row written before that has neither attribute. Switching the query without backfilling would drop every older artifact from its owner's library with no error anywhere.backfill_artifact_user_index_keys.pystamps them: dry-run by default, idempotent (attribute_not_exists(GSI2PK), so it also yields to the writer), and it never resurrects a row deleted mid-run (attribute_exists(SK)).It reads
updated_atto buildGSI2SKand never assigns it — that attribute is embedded in both GSI sort keys and is writer-owned, the same restraintrenameobserves. A HEAD row lackingupdated_atis reported by name rather than stamped with a fabricated timestamp that would sort it wrongly forever.Already run on dev
Verified independently afterwards: 0 rows missing
GSI2PK, 0 version rows wrongly stamped, andGSI1SK == GSI2SKon all 22. That last one is the useful check —GSI1SKis the writer's ownARTIFACT#{updated_at}#{aid}, so the agreement validates the backfill's format against production-written data rather than against my reading of the code.Deploy notes
UpdateTable, and only one GSI may be added perUpdateTable. The committedgsi-inventory.jsongains exactly one line, so no release split needed.UPDATE_COMPLETE≠ index ACTIVE — DynamoDB backfills asynchronously. This is why the query switch is a separate PR: platform.yml and backend.yml share a concurrency group but their order isn't enforced, so shipping both together risks app-api querying an index that doesn't exist yet.Tests
Infrastructure 784 passed. Backend 186 passed across the artifact suites, including 10 new for the script — weighted toward what it refuses to do (fabricate a sort key, touch
updated_at, stamp version rows, resurrect a deleted row), since those are the silent failures.🤖 Generated with Claude Code