up

KNOWN_RISKS.md

@@ -195,6 +195,24 @@
 
 ---
 
+## 13. Context Bloat — Token Cost Scales with Canon Size
+
+**Risk:** The `ContextAssembler` injects ALL canon entities into every generation prompt. This is already consuming ~1500–2000 tokens per call at 36 entities. At 100+ entities this grows linearly, exhausting daily token limits on budget provider tiers.
+
+**Trigger:** Every generation call. The current implementation has no cap on context entities.
+
+**Impact:** Token costs escalate. Groq daily TPD limit (100K tokens) gets exhausted after ~50 generations. Provider rate limits fire sooner. Unnecessary tokens inflate every prompt with entities irrelevant to the current generation type.
+
+**Mitigation:**
+- Implement `getRelevantContext(entityType, hints)` that caps at 15 entities
+- Always include: top 10 by relationship count, world-core.json, campaign context
+- Fill remaining slots with type-relevant neighbors
+- This reduces token overhead ~60% at current canon size
+
+**Status: PENDING — requires implementation before next generation run**
+
+---
+
 ## Summary of Urgency
 
 | Risk | Urgency | Needs Phase |

NEXT_STEPS.md

@@ -1,42 +1,38 @@
-# Next Steps — Phase 4 (Post-Deployment)
+# Next Steps — IMMEDIATE NEXT SESSION
 
-Campaign: "Echoes of the Northern Fracture" — Region: Northern Velkaris
-Status: Phase 3 complete. Phase 4 not started.
+## 1 task only: Fix context-assembler.ts token bloat
 
-## Options
+**Why:** `ContextAssembler` currently injects ALL 36 canon entities into every generation prompt. This uses ~1500–2000 tokens per call. At 100+ entities this will exhaust daily token limits on all providers.
 
-### A) Push to GitHub + configure secrets + deploy
-- Create GitHub repo (if not existing)
-- Push `main` branch
-- Add 4 secrets: CF_API_TOKEN, CF_ACCOUNT_ID, GEMINI_API_KEY, GROQ_API_KEY
-- Create Cloudflare Pages project `aetherion-archive`
-- Trigger first deploy via workflow_dispatch
+**Required changes in `src/engine/context-assembler.ts`:**
 
-### B) SEO improvements
-- Add sitemap.xml (`pages/sitemap.xml.ts`)
-- Add robots.txt
-- Add schema.org structured data (WebPage, CreativeWork) to entity pages
-- Add Open Graph / Twitter Card meta tags
+Add function:
+```typescript
+getRelevantContext(entityType: string, hints?: string[]): Entity[]
+```
 
-### C) Generate more entities (target 50+)
-- Run `generate-batch` locally or via workflow_dispatch
-- Target thin entity types (religions, races, regions)
-- Add missing metadata to close warnings (school, power level, threat level, lifespan)
-
-### D) Content quality pass
-- Rewrite entities with thin descriptions or content
-- Fill in attributes that are empty strings
-- Standardize lore tone across all 11 types
+- Max **15 entities** injected into prompt (not all canon)
+- Top 10 by relationship count always included
+- `world-core.json` always included
+- Campaign / generation context always included
+- Remaining slots filled with type-relevant neighbors
 
 ---
 
 ### Canon status
 - 36 unique entities across 11 types
-- 174 relationships, 0 orphans, 0 bidirectional gaps
+- 182 relationships, 0 orphans, 0 bidirectional gaps
 - 1 error (forbidden-moon-rituals placeholder)
 - 18 warnings (non-blocking metadata)
 
-### Astro site status
-- 48 pages, static output, ~3.9s build time
+### Site
+- LIVE at aetherion-archive.pages.dev
+- 49 pages, static output, ~5s build time
 - Tailwind v4, dark fantasy theme
-- GitHub Actions CI/CD ready (pending secrets)
+- Cloudflare Pages auto-deploys on push to master
+
+### Generation
+- 3x/day via GitHub Actions (2am/9am/3pm UTC)
+- Rotating provider strategy (balanced/cheap/quality)
+- 429 → backoff → fall through
+- Manual trigger with provider + count options

PROJECT_STATE.md

@@ -1,11 +1,11 @@
 # Project State
 
-## Phase: 3 — CI/CD & Deployment (COMPLETE)
 ## Phase: 2 — Astro Frontend (COMPLETE)
+## Phase: 3 — CI/CD & Deployment (COMPLETE)
 ## Phase 1C — Sandbox Canon Generation (COMPLETE)
 ## Pre-Phase 2 Cleanup (COMPLETE)
 ## Version: 0.1.x
-## Status: 36 unique entities, 174 relationships, 0 orphans, 0 bidirectional gaps, 1 remaining error (pre-existing). Generation system: 3 runs/day, rotating provider strategy (balanced/cheap/quality). Phase 3 complete — GitHub Actions workflows ready, SEO layer complete. Deployed on Cloudflare Pages.
+## Status: 36 unique entities, 182 relationships, 0 orphans, 0 bidirectional gaps, 1 remaining error (pre-existing). Site LIVE at aetherion-archive.pages.dev. Generation: 3x/day rotating provider strategy (balanced/cheap/quality). Known issue: context-assembler injects all 36 entities per generation → token cost too high. Fix PENDING.
 
 ### What exists
 - Full generation pipeline (context → AI → parse → validate → retry → register → JSONL append → markdown → memory snapshot)
@@ -34,7 +34,7 @@
 - **Regions (1)**: northern-velkaris
 
 ### Relationships
-- 174 total, 0 orphans, 0 bidirectional gaps
+- 182 total, 0 orphans, 0 bidirectional gaps
 - 1 remaining error (forbidden-moon-rituals placeholder — pre-existing, separate entity)
 - 18 warnings (all non-blocking — missing school/power level/threat level/lifespan metadata)
 
@@ -77,9 +77,9 @@
 
 ### Phase 3 — CI/CD & Deployment (COMPLETE)
 - deploy.yml: install → typecheck → validate → build-and-deploy (Cloudflare Pages)
+- Site LIVE at aetherion-archive.pages.dev
 - generate.yml: 3 daily runs (2am/9am/3pm UTC), rotating strategy (balanced/cheap/quality), generate-batch → commit → triggers deploy
 - DEPLOYMENT_SETUP.md: created, includes search engine submission guide
-- Status: READY FOR DEPLOYMENT (pending: GitHub Secrets + CF project creation)
 
 ### SEO Layer (COMPLETE)
 - sitemap-index.xml (auto-generated via @astrojs/sitemap)
@@ -92,7 +92,7 @@
 
 ### What does NOT exist yet
 - No search, no embeddings, no graph visualization
-- No deployment (production URL, domain, hosting) — pending GitHub Secrets + CF project
+- No custom domain (using default pages.dev)
 - No analytics
 
 ### Graph health
@@ -109,5 +109,8 @@
 - 3 runs/day (2am, 9am, 3pm UTC)
 - Rotating provider: balanced→cheap→quality
 - BATCH_COUNT=1 per run (token efficient)
-- 429 → immediate skip to next provider
+- 429 → exponential backoff + skip to next provider
 - Manual trigger: workflow_dispatch with provider + count options
+
+### Known issues
+- Context-assembler injects all 36 entities per generation → ~1500–2000 tokens per call. Fix PENDING: cap at 15 entities (top 10 by relationship count + world-core + campaign).