feat(analytics): log LLM token usage on ai-gateway + generate-profile

[brooksRoley]

Summary

• Both /api/tools/ai-gateway and /api/tools/generate-profile have been unmetered since launch — no visibility into which providers/models/token volumes the site's two LLM routes actually use. This is the CFO cost-visibility gap first flagged 2026-08-28, re-flagged 2026-09-04, and standing at flag-2 unshipped as of yesterday's (2026-09-10) CTO brief. Per the routine's own anti-duplication rule ("flagged twice → ship it, don't re-recommend a third time"), this session builds it instead of writing the prompt again.
• New logLlmUsage() helper (src/lib/ai-providers/usageLog.ts) writes {provider, model, inputTokens, outputTokens} into the existing events table (event_type: 'llm_usage', page: the route name) after each call — reusing the table every other interaction on the site already logs into, rather than adding a new Neon table. Wired into ai-gateway.ts via streamText's onFinish callback and into generate-profile.ts via generateText's returned usage. Logging is best-effort (try/catch, never throws) so a DB hiccup can never affect an AI response already streaming or already sent.
• admin/analytics.ts gets a 6th query aggregating llm_usage events over the last 7 days, grouped by route/provider/model; /admin/analytics gets a new "LLM usage (7d)" section (3 summary tiles + a breakdown table) so this is actually visible, not just logged.
• Token counts only — no dollar conversion, since pricing varies per provider/model and both routes let the caller pick either (per the original CFO scoping note).

Livelihood stream

• SaaS micro-tool
• Consulting funnel / CTA
• Games
• Digital products
• Portfolio signal (hiring)
• Infrastructure / CI

(Closest fit: unit-economics visibility for the SaaS/AI-tools stream — Chat Sandbox and Model Arena's cost exposure has been qualitative-only for ~10 weeks per the CFO ledger; this closes the engineering-reachable half of that gap. The other half — real dollar figures — stays blocked on Brooks's Vercel/Anthropic/OpenRouter billing-API access, unchanged.)

Files changed

• src/lib/ai-providers/usageLog.ts (new) — shared best-effort logger
• src/pages/api/tools/ai-gateway.ts — onFinish hook on streamText
• src/pages/api/tools/generate-profile.ts — logs usage from generateText
• src/pages/api/admin/analytics.ts — new llm_usage aggregate query + response field
• src/pages/admin/analytics.tsx — new "LLM usage (7d)" dashboard section
• src/lib/ai-providers/__tests__/usageLog.test.ts (new)
• src/__apiTests__/tools/ai-gateway-usage-logging.test.ts (new)
• src/__apiTests__/tools/generate-profile-usage-logging.test.ts (new)
• src/__apiTests__/admin/analytics.test.ts, analytics-auth.test.ts — updated mock sequence for the new 6th query, plus one new assertion test

Test steps

1. yarn dev, log in at /login, visit /admin/analytics — the page loads with an empty "LLM usage (7d)" state until real traffic hits the AI routes (no POSTGRES_URL needed for this to render; it degrades like every other section here).
2. Real command output (this session, fresh sandbox — node_modules didn't exist until yarn install --frozen-lockfile, 64.09s):
   • yarn test → 843/843 passing, 76 files (previously 834/834 before this PR; 3 pre-existing tests in analytics-auth.test.ts broke from the new 6th SQL query until their mock sequences were updated — fixed in this PR, not a regression I introduced and left).
   • yarn lint → clean, 0 warnings.
   • npx tsc --noEmit → clean, no errors.
   • yarn build → clean, all routes compile (including /api/tools/ai-gateway, /api/tools/generate-profile, /admin/analytics).

Agent checklist

• yarn build passes locally
• yarn test passes (no regressions to game logic)
• No TODO, placeholder, or stub code
• No secrets, API keys, or .env values committed
• New pages include og:title, og:description, og:image, twitter:card — N/A, no new page (existing admin page, noindex)
• PreText, Reveal, TiltCard used for any new UI text/cards — N/A, admin dashboard uses its existing plain-table style, not the marketing-page components
• Lighthouse scores not regressed — N/A, gated /admin route, not indexed/public-funnel
• README updated — N/A, no new page or component, CLAUDE.md's existing ai-gateway.ts/generate-profile.ts/admin/analytics.ts route descriptions still accurately describe these files' purpose

Learn

The AI SDK v6 streamText's onFinish callback receives totalUsage (a LanguageModelUsage with inputTokens/outputTokens), and returning a promise from it lets the SDK await your side effect before the stream is considered fully finished — useful for "log after the response, but don't fire-and-forget into a serverless function that might already be torn down." generateText's non-streaming result carries the same usage shape directly on the returned object, no callback needed.

🤖 Generated with Claude Code

https://claude.ai/code/session_01UgHaXtpMZHY1pi3sGYv6zz

---

Generated by Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
zero-next	Ready	Preview	Sep 11, 2026 3:57pm UTC