CLUE-588: reconcile firestore.indexes.json with deployed production indexes

[scytacki]

Summary

The repo's firestore.indexes.json had drifted from what is actually deployed to the production Firestore project (collaborative-learning-ec215). Several indexes/field-overrides existed in production but were missing from the file, so deploying the file as-is would have attempted to delete live production indexes (including the summaries vector index).

This makes the file a superset of production: everything production has, plus the messages index that was added to firestore.indexes.json as part of an earlier PR. We needed to deploy this messages index to production, so we made this superset to do so.

This firestore.indexes.json has already been deployed to production as part of 7.3.0 release.

Changes

Adds back the three production-only entries that were missing from the file:

• documents index (unit, type): I added this so I could track down which types have units and which types don't.
• summaries index (context_id, investigation, problem, unit, key, numAiAgreements + summaryEmbedding vector 1536) : I'm pretty sure these are for the AI summary support.
• the arrayConfig: CONTAINS entry on the comments/uid field override : I think this might have been added to support the comment tags report. I think this allows finding all comments by list of authors.

The existing docs and messages indexes are unchanged.

Deploy process

firebase deploy --only firestore:indexes --project production

Notes

There are more indexes on staging. We did not include them here. They seem to be experimental. There are a summaries vector-only (1536 / 2048) indexes.

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 69.30%. Comparing base (4fc50e8) to head (1de7299).

❗ There is a different number of reports uploaded between BASE (4fc50e8) and HEAD (1de7299). Click for more details.

HEAD has 22 uploads less than BASE

Flag	BASE (4fc50e8)	HEAD (1de7299)
cypress-regression	15	0
cypress	7	0

Additional details and impacted files

@@             Coverage Diff             @@
##           master    #2931       +/-   ##
===========================================
- Coverage   85.86%   69.30%   -16.56%     
===========================================
  Files         957      952        -5     
  Lines       54527    54503       -24     
  Branches    14357    14355        -2     
===========================================
- Hits        46822    37776     -9046     
- Misses       7687    16693     +9006     
- Partials       18       34       +16     

Flag	Coverage Δ
cypress	?
cypress-regression	?
cypress-smoke	41.53% <ø> (-0.03%)	⬇️
jest	55.84% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[cypress]

collaborative-learning    Run #19535

Run Properties:   Passed #19535  •  1de7299174: CLUE-588: reconcile firestore.indexes.json with deployed production indexes

Project	collaborative-learning
Branch Review	CLUE-588-firestore-indexes
Run status	Passed #19535
Run duration	03m 38s
Commit	1de7299174: CLUE-588: reconcile firestore.indexes.json with deployed production indexes
Committer	Scott Cytacki
View all properties for this run ↗︎

---

Test results
Failures	0
Flaky	0
Pending	0
Skipped	0
Passing	4
View all changes introduced in this branch ↗︎

[scytacki]

This has been deployed to production already. It was necessary to get the messages index on production as part of the 7.3.0 release. This associated JIRA story is tagged for 7.4.0 because that is the tag that will contain these updated rules.

[dougmartin]

Looks good 👍

The JSON is valid and schema-conformant, and both new queryScope: COLLECTION values are right: documents is a subcollection under {appMode}/{rootId}/documents accessed only via .collection(path) (never collectionGroup), and summaries is a genuine root-level collection (functions-v2/src/on-document-summarized.ts:101). Making the file a superset of production is strictly safer than the pre-PR state, and since it's already deployed there's nothing to un-do.

A few non-blocking notes (take or leave):

• firestore.indexes.json (lines 17-30): the documents (unit, type) index has no supporting query in the repo, and wouldn't need a composite even if it did. The only unit filters are shared/update-class-data-docs.ts:56-64 (unit == + context_id ==) and src/models/stores/sorted-documents.ts:124 (unit in [...]); the only type filter is scripts/find-documents-missing-metadata.ts:128 (type not-in [...]). A two-equality query is served by single-field indexes via zigzag merge, so a composite only helps for where unit == X orderBy type or a range on type, neither of which exists. Since it's COLLECTION-scoped it's maintained for every documents subcollection across authed/*, demo/*, dev/*, qa/*, test/*. If the ad hoc investigation that prompted it is done, consider a follow-up to drop it from the file and from production; if it's meant to stay as a permanent analysis index, worth saying so somewhere durable so the next reconciliation doesn't delete it.

• firestore.indexes.json (lines 45-81): the summaries vector index has no reader anywhere in the repo. functions-v2/src/on-document-summarized.ts is the only file that touches summaries, and it does point reads/writes on a known doc id; there is no .collection("summaries") query and no findNearest call anywhere (FieldValue.vector at line 112 is a write). A vectorConfig index can only serve findNearest, so it's dead as far as this repo is concerned. The prefix set also looks off for a nearest-neighbor query, since Firestore requires equality on all prefix fields and this one includes key (filtering to a single document, which defeats similarity search) and numAiAgreements (an exact count). Worth confirming with whoever owns AI summaries what consumer this serves (likely an out-of-repo notebook or a console experiment) and recording that, rather than leaving "I'm pretty sure these are for the AI summary support" as the only justification.

• firestore.indexes.json (whole file), .firebaserc: this is now a superset of production but a subset of staging, and firebase.json points both projects at this single file, so the failure mode this PR fixes for production is now the live state for staging. The next person deploying indexes to staging gets prompted to delete the experimental vector indexes you mention in the Notes. Either add the staging-only entries here too, or state explicitly in a repo doc that staging holds untracked experiments and index deploys there need index-by-index review. Related: .firebaserc has no default alias, so a bare firebase deploy --only firestore:indexes targets whatever firebase use last selected.

• firestore.indexes.json (line 90): the stated rationale for the comments/uid CONTAINS entry looks incorrect. The list-of-authors query is scripts/export-comment-tags.ts:514 (collectionGroup("comments").where("uid", "in", uidBatch)), and Firestore runs in on a scalar field as a disjunction of equality lookups served by the ASCENDING COLLECTION_GROUP index already on line 91. uid is scalar (src/lib/firestore-schema.ts:42), every writer assigns a single string, and firestore.rules:56-58 enforces string(request.auth.token.platform_user_id) == request.resource.data.uid, which an array can never satisfy. There's no array-contains on uid anywhere. The entry itself is correct and free: it restores Firestore's automatic array index that the CLUE-549 field override (af59f2ea) had inadvertently disabled by enumerating only three of the four default single-field indexes. Just worth fixing the description so a future reconciliation doesn't conclude uid is sometimes an array.

• package.json (lines 93-94), README.md (lines 82-95): there's no deploy:firestore:indexes script, which is why this used a raw firebase deploy. Suggest adding "deploy:firestore:indexes": "firebase deploy --only firestore:indexes". Separately, README.md:92 references npm run deploy:firebase:rules, which doesn't exist (the real script is deploy:database:rules). Nothing in .github/workflows/ reads or validates this file, and git log shows only 4 commits ever against it versus 20+ for firestore.rules, so it's only touched when someone hits a missing-index error, which is how the drift accrued. A release-checklist step (or CI job) that diffs firebase firestore:indexes --project production against the committed file would catch the next one.

• firestore.indexes.json (lines 17-30, 45-81, 90): JSON allows no comments, so once this merges the reasoning for the back-filled entries lives only in the PR body, and two of the three are hedged ("I'm pretty sure", "I think this might have been"). A short "Firestore indexes" section in docs/firestore-schema.md listing each index, the query or script that needs it, and an owner for the ones with no in-repo reader would give the next reconciliation something better than "safe answer is keep it".

• firestore.indexes.json (lines 3-16, unchanged): while you're reconciling, the pre-existing docs COLLECTION_GROUP index appears orphaned. There's no collection("docs") or collectionGroup("docs") anywhere in src/, functions-v1/src/, functions-v2/src/, scripts/, or shared/, and firestore.rules has no match for docs at all, so clients hit the catch-all deny. Looks like a legacy name superseded by documents. Out of scope to remove here (it would trigger a production delete), but worth confirming and filing a follow-up.

[scytacki]

It is interesting that your AI thinks that the "the summaries vector index has no reader anywhere in the repo". My Claude found a reader via on-analysis-document-imaged.

I'm going to just merge this as is. I agree with most of the points, but it was just a clean up after a CLUE release. I don't have the time to do more.