fix(google): make the Google AI embedding provider usable - #124
Conversation
The Google provider could never complete setup. gemini-embedding-2 emits 3072 dimensions, but pgvector caps HNSW at 2000 for the `vector` type, so all four `CREATE INDEX ... USING hnsw` statements across setup-db-google.sql, setup-db-conversation-google.sql and setup-db-insights-google.sql errored out. EMBEDDING_PROVIDER=google is documented in the README, ENV docs, architecture and install guides, so following those instructions could not work. Request 1536 dimensions instead of storing 3072. gemini-embedding-2 is Matryoshka-trained and auto-normalizes truncated output, so cosine distance stays valid with no client-side renormalization, and MTEB across its dimension tiers spans only 67.99-68.17. This removes the failure mode rather than working around it: an alternative fix is to index a halfvec cast (indexable to 4000), but that is an expression index, so any later edit to hybrid_search() that dropped the cast would silently fall back to a sequential scan. At 1536 the Google schemas match the already-proven OpenAI shape and nothing is near a cap. Also migrate off @google/generative-ai, which is end-of-life (its repo is now google-gemini/deprecated-generative-ai-js). pnpm outdated does not flag it because 0.24.1 is the final frozen release. The replacement @google/genai is also what exposes outputDimensionality, so the two changes are one change. batchEmbedContents becomes models.embedContent with an array of Content, and readGoogleEmbeddings() validates the now-optional embeddings/values fields so a partial response fails at the call site instead of writing undefined vectors. Default GOOGLE_EMBEDDING_MODEL moves from gemini-embedding-2-preview to the GA gemini-embedding-2. Adds vector-index-dimensions.test.ts, which reads every setup-db*.sql and fails on an HNSW index over a >2000-dim vector column — verified to flag all four original indexes. Adds coverage for getEmbeddingDimensions(), which had none. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PDjKtobTDeToJnMHRbYjGp
…hono audio-processor.ts and image-processor.ts have had zero importers since they were added incidentally in c311a99 (a GCS storage commit). They are not referenced by any tool, route, doc, or roadmap item. Removing them also drops one of the two `openai` consumers, which makes the pending v4->v7 upgrade a single-file change. src/tools/index.ts was missing registerPgAnalyzeTools, the only tool registrar absent from the re-export surface. server.ts imports every tool directly, so nothing was broken by this — the barrel was just incomplete. hono was declared as a direct dependency but imported nowhere. Note that simply removing it does NOT clear its advisories: hono is also transitive via @modelcontextprotocol/sdk and @hono/node-server, so the pnpm override is what actually governs the installed version. Bumped that override from ^4.11.7 (resolving to a vulnerable 4.12.18) to ^4.12.33, which clears the HIGH CORS credential-reflection advisory and the hono/jsx cx() XSS advisory. @hono/node-server's <2.0.5 advisory (Windows serve-static path traversal) remains: it is pinned by the MCP SDK, forcing a major on it risks breaking the SDK, and nothing here serves static files through hono. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PDjKtobTDeToJnMHRbYjGp
Direct dependencies with advisories reachable from server or CLI code:
multer 2.1.1 -> 2.2.0 HIGH, DoS via deeply nested field names,
reachable from src/api/upload.ts
ws 8.20.1 -> 8.21.1 HIGH, memory-exhaustion DoS from tiny frames,
reachable from src/api/websocket.ts
sanitize-html 2.17.4 -> 2.17.6 MODERATE, incomplete javascript: URI
validation, used by the CLI HTML normalizer
Also refreshes @modelcontextprotocol/sdk 1.29.0 -> 1.30.0 and
@modelcontextprotocol/ext-apps 1.7.1 -> 1.7.5.
Two transitive pins in pnpm.overrides were stale and are bumped:
tar >=7.5.7 -> >=7.5.22 clears six advisories including the only
CRITICAL in the tree (decompression DoS,
vulnerable <=7.5.18)
body-parser (new) >=2.3.0 clears a low-severity DoS; transitive via
express under the MCP SDK
Audit totals: 81 advisories (1 critical / 34 high / 40 moderate / 6 low) ->
57 (0 critical / 30 high / 23 moderate / 4 low).
Not fixed, deliberately: @hono/node-server <2.0.5 (Windows serve-static path
traversal) is pinned by the MCP SDK, forcing a major on it risks breaking the
SDK, and nothing here serves static files through hono. The remaining highs are
in the dashboard/desktop workspaces (next, vite, sharp, electron-updater) and
are out of scope for this pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PDjKtobTDeToJnMHRbYjGp
INSIGHT_MODEL: claude-sonnet-4-6 -> claude-sonnet-5 (same $3/$15 list price). EXTRACTION_MODEL: claude-haiku-4-5-20251001 -> claude-haiku-4-5. Haiku 4.5 is still current, so that one is alias hygiene rather than a generation bump. The Sonnet 5 swap is not safe as a bare model-ID change. Sonnet 5 runs adaptive thinking when `thinking` is omitted (Sonnet 4.6 ran thinking-off), and max_tokens bounds thinking PLUS response text. synthesizeInsights() asks for a JSON array under max_tokens: 2000 and falls through to generateRuleBasedInsights whenever parsing fails — so the new default would have burned the budget before emitting JSON and silently downgraded every synthesis to the rule-based path. memory-extraction.ts already carries a comment about exactly this failure mode from a previous truncation bug. Guards both ways: thinking is explicitly disabled on that call, and max_tokens goes 2000 -> 4096 to absorb the Sonnet 5 tokenizer's ~30% higher token counts. Neither call site passes temperature/top_p/top_k/budget_tokens, all of which now 400 on Sonnet 5, so nothing else in the request shape needed changing. Docs also referenced `claude-sonnet-4-6-20250514` as the INSIGHT_MODEL default in docs/ENV.md and configuration.mdx. That model ID does not exist — the Sonnet 4.6 alias carries no date suffix — so anyone copying it would have hit a 404. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PDjKtobTDeToJnMHRbYjGp
The feature table advertised "Multimodal — Process images (Claude vision) and audio (Whisper transcription) alongside documents". No such code path exists or ever has. audio-processor.ts and image-processor.ts were added incidentally in c311a99 and were never imported by anything; `git log -S` shows the symbols appearing in that commit and disappearing when they were removed as dead code, with no wiring in between. The processor registry has handlers for text, pdf, docx, csv, xlsx, json and html only — there is no image or audio MIME type registered anywhere, so uploading either is rejected as an unsupported type. Multi-Format also listed PPTX. There is no pptx handler and no pptx converter; the string does not appear anywhere in src/ or scripts/. Replaced the list with what is actually registered, which was under-selling as much as over-selling — CSV, JSON, Markdown/text and ZIP archives were all supported but unlisted. Also removes images/audio from the intro and the privacy section, which described sending them to providers that were never called. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PDjKtobTDeToJnMHRbYjGp
The 'connect' handler fired the hnsw.* GUC query and dropped the promise. pg emits 'connect' synchronously and hands the client to the waiter without awaiting anything the handler started, so the first query on a fresh connection could run before the settings landed — falling back to pgvector's default ef_search=40 with iterative scans off. That combination is what makes filtered vector searches silently under-return, which is the exact failure hnsw.iterative_scan was added to prevent. With Neon scaling compute to zero, fresh connections are common rather than rare. Records the tuning promise per client in a WeakMap and awaits it in pgQuery, which every query path funnels through. This requires checking out the client explicitly instead of using pool.query(), since pool.query() never exposes the client the barrier is keyed to. Also replaces the DO ... EXCEPTION WHEN others THEN NULL wrapper with two independent SET statements. The old block swallowed every error server-side, so the client-side .catch() could never fire and there was no way to discover that tuning had failed. Now a failure rejects and is logged, an older pgvector that rejects one setting still gets the other, and the connection continues to serve queries with server defaults. Leaves HNSW_EF_SEARCH on the local envInt() helper rather than routing it through the zod config: nothing under src/db/ imports src/utils/config.js, and the value is already validated as a positive integer. Preserving that layer boundary is worth more than the consistency. Adds pg-client.test.ts covering the ordering guarantee, both GUCs, the tuning-failure path, and client release on error. Verified to fail when the barrier is removed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PDjKtobTDeToJnMHRbYjGp
openai 4.104 -> 7.3 three majors; a single call site
(embeddings.ts) since audio-processor.ts
was removed. Also clears the unmet
zod@^3 peer warning — v7 supports zod 4.
@anthropic-ai/sdk 0.71 -> 0.115 well behind, and directly relevant now
that INSIGHT_MODEL is claude-sonnet-5
express-rate-limit 7.5 -> 8.6 paired with rate-limit-redis 4 -> 6;
the store interface changes together
google-auth-library 10 -> 11
file-type 21 -> 22 magic-byte sniffing in the ZIP entry path
jsdom 29 -> 30
redis 5 -> 6
p-limit 5 -> 7, commander 12 -> 15, glob 10 -> 13
@types/node 22 -> 26, @types/jsdom 21 -> 28, @types/multer 1.4 -> 2.2
Plus current-line refreshes: biome 2.5.6, vitest 4.1.10, tsx 4.23.4, jose
6.2.7, helmet 8.3.0, unpdf 1.8.0, @google-cloud/storage 7.21, tasks 6.3,
mailparser 3.9.14, @types/pg, @types/supertest.
Two follow-on source changes:
- rateLimit.ts uses `limit` instead of `max`. Both work in v8, but `max` is
deprecated. `standardHeaders: true` is left alone deliberately — switching
draft versions would change the response headers clients see, which is a
behavioral change rather than an upgrade.
- Two test files are reformatted by biome 2.5.6's updated formatter. No
behavior change.
The removed v8 options (draft_polli_ratelimit_headers, onLimitReached) were not
in use. Full `pnpm verify` passes: 478 tests, typecheck, esbuild bundle and the
docs site build.
Not upgraded: typescript 5.7 -> 7.0 is a compiler rewrite, out of scope here and
worth its own pass. Workspace packages (next, react, tailwindcss, @vercel/*)
are likewise left for a separate change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PDjKtobTDeToJnMHRbYjGp
TypeScript 7 is the native compiler port, so this is a larger change than the version bump suggests — kept as its own commit to make it independently revertable. Clean on this codebase: no source changes were needed. `pnpm verify` passes end to end — typecheck, 478 tests, biome, markdownlint, the security/docs/ tool-sync scripts, the esbuild bundle, and the docs site build. The built dist/index.js was additionally smoke-tested by importing it directly. Incidental: the docs site build drops from ~15.5s to ~6.7s. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PDjKtobTDeToJnMHRbYjGp
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
WalkthroughThe change migrates Google embeddings to Sequence Diagram(s)sequenceDiagram
participant Application
participant EmbeddingsService
participant GoogleGenAI
Application->>EmbeddingsService: Request embeddings
EmbeddingsService->>GoogleGenAI: Request 1536-dimensional vectors
GoogleGenAI-->>EmbeddingsService: Return embedding values
EmbeddingsService-->>Application: Return validated vectors
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (1)
package.json (1)
116-117: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick winBound the
tarandbody-parseroverrides to compatible majors. The>=ranges admit future major releases. Use^7.5.22and^2.3.0, or exact patched versions, to prevent incompatible selections during lockfile refreshes.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@package.json` around lines 116 - 117, Update the tar and body-parser dependency overrides in package.json from unbounded minimum versions to compatible-major ranges, using ^7.5.22 for tar and ^2.3.0 for body-parser (or exact patched versions), so lockfile refreshes cannot select future incompatible majors.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/research/memory-systems.md`:
- Line 292: Update the “Embeddings” paragraph in memory-systems.md to accurately
document only the existing memory schema variants: OpenAI with VECTOR(1536) and
Ollama with VECTOR(1024). Remove or qualify the claims that Google AI and Ollama
v2 schema variants exist, unless corresponding setup-db-memory SQL schemas are
added.
In `@scripts/setup-db-google.sql`:
- Around line 19-25: Update the Option B migration example to recreate
chunks_embedding_idx after adding the embedding column back as vector(1536),
using the same HNSW index definition expected by hybrid_search and
semantic_search.
In `@scripts/setup-db-insights-google.sql`:
- Line 47: Update the database setup flow for proactive_insights.embedding to
migrate existing vector(3072) columns before HNSW index creation: use an
idempotent schema change that replaces or clears the old column with
vector(1536), preserving compatibility with CREATE TABLE IF NOT EXISTS and
requiring existing insights to be re-embedded.
In `@src/db/__tests__/pg-client.test.ts`:
- Line 43: Update the query mock in the pg-client test to explicitly reject when
the SQL is `BOOM`, while preserving the existing resolution behavior for other
non-`SET` queries. In the test covering `BOOM`, assert that `inFlight` rejects
before verifying the release count.
---
Nitpick comments:
In `@package.json`:
- Around line 116-117: Update the tar and body-parser dependency overrides in
package.json from unbounded minimum versions to compatible-major ranges, using
^7.5.22 for tar and ^2.3.0 for body-parser (or exact patched versions), so
lockfile refreshes cannot select future incompatible majors.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 3bac8386-dd6c-415e-a63c-ca841ee3e97a
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (34)
.env.exampleCLAUDE.mdREADME.mddocs/ENV.mddocs/architecture/embeddings.mdxdocs/architecture/index.mdxdocs/concepts/index.mdxdocs/getting-started/configuration.mdxdocs/getting-started/installation.mdxdocs/research/memory-systems.mdpackage.jsonscripts/migrations/2026-07-10-hybrid-search-filters.sqlscripts/setup-db-claims-google.sqlscripts/setup-db-claims-ollama-v2.sqlscripts/setup-db-claims-ollama.sqlscripts/setup-db-claims.sqlscripts/setup-db-conversation-google.sqlscripts/setup-db-google.sqlscripts/setup-db-insights-google.sqlsrc/api/__tests__/upload-process.test.tssrc/api/middleware/rateLimit.tssrc/db/__tests__/claims-schema.test.tssrc/db/__tests__/pg-client.test.tssrc/db/__tests__/vector-index-dimensions.test.tssrc/db/pg-client.tssrc/services/__tests__/embeddings.dimensions.test.tssrc/services/audio-processor.tssrc/services/embeddings.tssrc/services/image-processor.tssrc/services/insight-analysis.tssrc/services/processor/__tests__/archive-zip.test.tssrc/tools/index.tssrc/utils/__tests__/query-embedding-cache.test.tssrc/utils/config.ts
💤 Files with no reviewable changes (2)
- src/services/image-processor.ts
- src/services/audio-processor.ts
…potent Addresses four CodeRabbit findings on #124, all verified valid against a real PostgreSQL 16 + pgvector instance rather than by inspection. The premise is now confirmed empirically: applying the pre-fix setup-db-google.sql fails with ERROR: column cannot have more than 2000 dimensions for hnsw index and leaves the tables created but chunks_embedding_idx absent. That partial state is what made CodeRabbit's migration finding important — CREATE TABLE IF NOT EXISTS would not narrow the existing vector(3072) column, so re-running even the fixed script would have failed at exactly the same place. Changes: - setup-db-google.sql and setup-db-insights-google.sql now carry an idempotent resize migration, matching the pattern setup-db-conversation-google.sql already used. Each detects a mismatched atttypmod, drops the column and its index, and recreates them at vector(1536); the CREATE INDEX IF NOT EXISTS further down rebuilds the index. Verified to take a broken 3072 database to 1536 with the index present, in one run. - The Option B manual migration steps in setup-db-google.sql dropped chunks_embedding_idx and never recreated it, which would have left hybrid_search and semantic_search on a silent sequential scan. Replaced with the automatic migration above rather than patching the instructions. - CREATE POLICY was unguarded in all four setup-db-insights*.sql and all four setup-db-conversation*.sql variants, so a second run failed on "policy already exists". Added DROP POLICY IF EXISTS, the pattern the claims schemas already use. Three consecutive full re-runs are now clean. - docs/research/memory-systems.md claimed the setup-db-memory*.sql variants cover all four providers. Only two exist (OpenAI 1536, Ollama 1024); corrected. - pg-client.test.ts mocked every non-SET query as resolving, so the "releases the client even when the query throws" case never exercised a throw. The mock now rejects BOOM and the test asserts the rejection. End-to-end functional check on the same instance: hybrid_search returns correctly RRF-ranked results with source_type, content_type and tag filters pushed down; semantic_search returns exact 1.0/0.0 cosine similarity on orthogonal vectors; and EXPLAIN confirms "Index Scan using chunks_embedding_idx" rather than a sequential scan. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PDjKtobTDeToJnMHRbYjGp
There was a problem hiding this comment.
🧹 Nitpick comments (1)
src/db/__tests__/pg-client.test.ts (1)
43-43: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winUse
async/throwfor the rejection branch.Line 43 returns
Promise.reject(...). This TypeScript guideline requiresasync/awaitinstead of raw promises. Markqueryasasyncand thrownew Error('query failed')forBOOM.Proposed fix
- query: (sql: string) => { + query: async (sql: string) => { callLog.push(sql); if (sql.startsWith('SET')) { // existing tuning behavior } - if (sql === 'BOOM') return Promise.reject(new Error('query failed')); + if (sql === 'BOOM') throw new Error('query failed');As per coding guidelines, “Always use async/await over raw promises for asynchronous operations.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/db/__tests__/pg-client.test.ts` at line 43, Update the test’s query mock to be async and, for the “BOOM” input, throw the query failure error directly instead of returning Promise.reject; preserve the existing behavior for all other inputs.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@src/db/__tests__/pg-client.test.ts`:
- Line 43: Update the test’s query mock to be async and, for the “BOOM” input,
throw the query failure error directly instead of returning Promise.reject;
preserve the existing behavior for all other inputs.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: b0170dc0-a863-4d7a-8942-79563123385a
📒 Files selected for processing (11)
docs/research/memory-systems.mdscripts/setup-db-conversation-google.sqlscripts/setup-db-conversation-ollama-v2.sqlscripts/setup-db-conversation-ollama.sqlscripts/setup-db-conversation.sqlscripts/setup-db-google.sqlscripts/setup-db-insights-google.sqlscripts/setup-db-insights-ollama-v2.sqlscripts/setup-db-insights-ollama.sqlscripts/setup-db-insights.sqlsrc/db/__tests__/pg-client.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
- scripts/setup-db-conversation-google.sql
Follows the repo's TypeScript rule (.cursor/rules/typescript.mdc, "Always use async/await over raw promises"): the mock's query() is now async and throws instead of returning Promise.reject. The SET branch deliberately keeps its raw `new Promise`. That promise must stay pending until the test resolves it — it is how these tests control when the tuning lands relative to the query — so there is nothing to await. Added a comment so it does not read as an oversight. The executor still runs synchronously, so pendingTuning is populated before query() returns and the drain loop cannot deadlock. Re-verified the barrier test still fails when `await clientTuning.get(client)` is removed from pgQuery: making query() async delays settling by a microtask, but callLog.push happens synchronously on call, so the assertion still catches a missing barrier rather than passing vacuously. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PDjKtobTDeToJnMHRbYjGp
Summary
An Aug 2026 maintenance pass: verify the core primitives still work, and land the updates the project had drifted behind on.
The headline finding is that the Google AI embedding provider had never worked.
gemini-embedding-2emits 3072 dimensions, but pgvector caps HNSW at 2000 for thevectortype, so all fourCREATE INDEX ... USING hnswstatements across the Google schemas errored out.EMBEDDING_PROVIDER=googleis documented in the README, ENV docs, architecture and install guides — following those instructions could not succeed. Confirmed against a real PostgreSQL 16 + pgvector instance:The fix requests 1536 dimensions rather than storing 3072.
gemini-embedding-2is Matryoshka-trained and auto-normalizes truncated output, so cosine distance stays valid with no client-side renormalization, and MTEB across its dimension tiers spans only 67.99–68.17. Indexing ahalfveccast (indexable to 4000) was the alternative, but that is an expression index — any later edit tohybrid_search()that dropped the cast would silently fall back to a sequential scan. At 1536 the Google schemas match the already-proven OpenAI shape and nothing sits near a cap.Changes
outputDimensionality; rewrite the four Google schemas tovector(1536). Migrate off@google/generative-ai, which is end-of-life (its repo is nowdeprecated-generative-ai-js;pnpm outdatedmisses it because 0.24.1 is the final frozen release) to@google/genai, which is also what exposesoutputDimensionality. DefaultGOOGLE_EMBEDDING_MODELmoves to the GAgemini-embedding-2.setup-db-google.sqlandsetup-db-insights-google.sqlnow detect a mismatched dimension and recreate the column and index, matching the patternsetup-db-conversation-google.sqlalready used. This matters because the old script left tables created but unindexed, andCREATE TABLE IF NOT EXISTSwould not have narrowed that column — a re-run of even the fixed script would have failed identically.DROP POLICY IF EXISTSacross all foursetup-db-insights*.sqland all foursetup-db-conversation*.sql; a second run previously died onpolicy ... already exists.hnsw.ef_searchrace. The pool'sconnecthandler fired the GUC query and dropped the promise, so the first query on a fresh connection could run at pgvector's defaultef_search=40with iterative scans off — the exact condition that makes filtered vector searches silently under-return. Now recorded per client and awaited inpgQuery.INSIGHT_MODEL→claude-sonnet-5,EXTRACTION_MODEL→claude-haiku-4-5. Sonnet 5 runs adaptive thinking whenthinkingis omitted andmax_tokensbounds thinking plus output, so insight synthesis is pinned tothinking: disabledwith a raised ceiling — otherwise every synthesis would have silently degraded to the rule-based fallback.multer(HIGH, upload path),ws(HIGH, WebSocket API),sanitize-html; bumped staletarand newbody-parseroverrides. Notehonois transitive under the MCP SDK, so the override — not the direct dependency — is what governs it.openai4→7,@anthropic-ai/sdk0.71→0.115,express-rate-limit7→8 withrate-limit-redis4→6,google-auth-library10→11,file-type21→22,jsdom29→30,redis5→6,p-limit,commander,glob,@types/*, and TypeScript 5.7→7.0 (its own commit, independently revertable).audio-processor.ts/image-processor.ts(zero importers since they were added incidentally) and the unusedhonodependency. The README advertised "Multimodal — images (Claude vision) and audio (Whisper)" and PPTX support; none of that has ever had a code path, so the feature list now reflects what actually ships (which was also under-selling CSV, JSON, Markdown and ZIP).Type of Change
Checklist
pnpm typecheckwith no errorspnpm dev— see the note below on what was run instead.js, noconsole.log)On testing:
pnpm verifypasses end to end (478 tests, up from 444; biome, markdownlint, the security/docs/tool-sync scripts, the esbuild bundle and the docs site build), anddist/index.jswas smoke-tested by importing it. Rather thanpnpm dev, the database work was verified against a scratch PostgreSQL 16 + pgvector instance: the pre-fix schema was confirmed to fail; the fixed schema applies cleanly to both a fresh database and one already in the broken 3072/no-index state; three consecutive full re-runs are clean;hybrid_searchreturns correctly RRF-ranked results with all three filters pushed down;semantic_searchreturns exact 1.0/0.0 cosine similarity on orthogonal vectors; andEXPLAINconfirmsIndex Scan using chunks_embedding_idx.Not verified: a live end-to-end ingest against the Google API, which needs a real
GOOGLE_AI_API_KEY.Related Issues
None — this came out of a scheduled maintenance review rather than a filed issue.