California plan comparison, a source registry, and plan-search fixes

CHANGELOG.md

@@ -18,6 +18,63 @@
 - The database was seeded once from the local dump — 1,568 chunks, 35,756 SBC
   chunks, 3,276 plans, 56,280 ZIP-county rows and the `bm25` index row — with
   zero restore errors and every count matching the source.
+- Design for California plan comparison and a source registry
+  (`docs/superpowers/specs/2026-09-24-ca-plan-catalog-design.md`), the first of
+  five California sub-projects. Covered California plans come from CMS's
+  state-based exchange PUF, since the Marketplace API has none. Their SBC links
+  are blank there, and Covered California's terms forbid automated access, so
+  it is linked, never read.
+- Implementation plan for California plan comparison
+  (`docs/superpowers/plans/2026-09-24-ca-plan-catalog.md`). The design is
+  corrected from checks against the real 2026 file: a plan is sold only where it
+  has a filed rate, four Los Angeles ZIPs with no CMS rating area show
+  unpriced, and only in-network individual cost shares are loaded.
+- `src/core/exchanges.py` and `src/core/plan_year.py`: which exchange sells
+  each state's plans (Covered California for California, HealthCare.gov for
+  the 30 API states), and the plan year on sale, which is next year from
+  1 November.
+- A committed source registry (`src/ingestion/sources/registry.toml`, ADR
+  0025): every source with its licence, permission status, `robots.txt` status
+  and removal steps. It is validated when loaded, only approved sources run,
+  and Covered California is recorded as link-only because its Terms of Use
+  forbid automated access.
+- Migration `15bb18a57fac` for a filed-rate plan catalog:
+  - `plans.catalog_source`, `cms_api` or `ca_sbe_puf`, checked, so a source can
+    be audited or removed;
+  - `plan_counties.zipcodes`, for a plan sold in only some of a county's ZIPs;
+  - new `rating_areas`, `plan_rates` (ages 14 to 64, checked) and
+    `catalog_loads`, with the file's label and sha256.
+  API-sourced rows are unchanged.
+- `make ingest-ca-plans YEAR=` (ADR 0024): loads Covered California's plans
+  from CMS's state-based exchange PUF, because the Marketplace API has none.
+  - **2026:** 190 plans from 11 issuers and 28,101 filed rates.
+  - **Validated first:** the whole file is checked before one transaction
+    writes it, so a bad file changes nothing.
+  - **Sold only where rated:** a plan is sold only in counties where it has a
+    filed rate. Western Health Advantage files one service area over two
+    rating areas.
+  - **Hand-transcribed:** rating areas and issuer names, cross-checked against
+    the data. Four Los Angeles ZIPs that CMS gives no rating area are reported
+    and shown unpriced.
+  - **Docs:** `docs/findings/ca-sbe-puf.md` and `docs/runbooks/california.md`.
+- California plan comparison:
+  - **Pricing:** plan search prices California plans from CMS's filed rates for
+    the ZIP's rating area and the person's age, in SQL, and never calls CMS for
+    them.
+  - **Partial counties:** a plan sold in part of a county is listed only for its
+    ZIPs.
+  - **Wording:** answers say premiums are CMS's published rates before any tax
+    credit or state help, and name Covered California.
+  - **Out-of-date year:** when the plan year is no longer on sale, the answer
+    opens with a server-written notice saying so. It is not model-written,
+    because asked to write it, the model also wrote it where it was untrue.
+  - **Profile:** says plan comparison is available in California only once its
+    plans are loaded.
+  - **Plan table:** shows the plan year and describes each exchange's premiums
+    when one answer mixes states.
+- README and CLAUDE.md now cover California. The scope includes Covered
+  California, and the README documents `make ingest-ca-plans`, the source
+  registry, and the PLAN SEARCH and CONCEPTS eval sections.
 - A `Dockerfile` for the API, and a CI job that builds and scans the image
   (ADR 0022) — the first of ADR 0002's two deferred items. Multi-stage, runs
   as a non-root user, `torch` from PyTorch's CPU index on Linux so no CUDA
@@ -390,6 +447,21 @@
 
 ### Fixed
 
+- "Compare the gold plans." and "Compare silver plans for me." skipped the plan
+  search in about half of runs, in every state. The prompt left it open whether
+  comparing plans of a metal level meant real plans or the level in general,
+  and the model spent its turn deciding. The prompt now says a request to
+  compare, list or show plans means searching; only what a level means stays
+  a corpus question.
+  - **Before, on `main`'s prompt:** 2 of 4 runs searched.
+  - **After:** 35 of 35 search requests searched, and 0 of 10 concept
+    questions did.
+  - **Eval:** a new floor for concept questions that must not search.
+- A Bronze plan search now includes Expanded Bronze plans, which it silently
+  missed; every California bronze plan is one.
+- Plan answers, the "no Summary of Benefits link" reason and the plan table name
+  the exchange that sells the state's plans. They used to send everyone to
+  HealthCare.gov, which is wrong for the 21 states and DC that run their own.
 - The API never configured logging. Every command-line entry point calls
   `setup_logging()`; `create_app()` did not, so in the Flask process the root
   logger kept Python's default of WARNING with no handler: nothing the request

CLAUDE.md

@@ -4,7 +4,7 @@
 
 PolicyPal lets a logged-in user ask natural-language questions about any kind of insurance — health, auto, life, home, and more — and returns grounded answers generated by an LLM using context retrieved from a vector store.
 
-**Built:** comparison of real purchasable health plans from the CMS Marketplace API. That half is narrower than the Q&A — ACA individual and family plans, in the 30 states on HealthCare.gov. PolicyPal compares plans, never recommends one. All five phases in [docs/plans](docs/plans/) are complete; keeping the plan documents current is a routine, in [docs/runbooks/sbc.md](docs/runbooks/sbc.md).
+**Built:** comparison of real purchasable health plans from the CMS Marketplace API, and for California from CMS's state-based exchange PUF (ADR 0024). That half is narrower than the Q&A — ACA individual and family plans, in the 30 states on HealthCare.gov plus California (Covered California). PolicyPal compares plans, never recommends one. All five phases in [docs/plans](docs/plans/) are complete; keeping the plan documents current is a routine, in [docs/runbooks/sbc.md](docs/runbooks/sbc.md).
 
 ## Stack
 

Makefile

@@ -1,4 +1,4 @@
-.PHONY: ingest build-index ingest-plans ingest-sbc refresh-sbc sbc-report migrate api \
+.PHONY: ingest build-index ingest-plans ingest-ca-plans ingest-sbc refresh-sbc sbc-report migrate api \
         ui-dev ui-build ui-lint ui-preview ui-test test lint check
 
 # The plan year to work on; the commands default to the calendar year, which
@@ -21,6 +21,14 @@ ingest-plans:
 	@test -n "$(STATES)" || { echo "STATES is required, e.g. make ingest-plans STATES=TX,FL (or STATES=ALL)"; exit 2; }
 	uv run python -m src.ingestion.plans --states $(STATES) $(YEAR_ARG)
 
+# California's plans, from CMS's state-based exchange PUF (ADR 0024); the
+# Marketplace API has none. YEAR is required: a plan year's file appears May
+# to August of that year, and until then this exits 3. ZIP=path loads a file
+# downloaded by hand; REFRESH=1 downloads it again. e.g. make ingest-ca-plans YEAR=2026
+ingest-ca-plans:
+	@test -n "$(YEAR)" || { echo "YEAR is required, e.g. make ingest-ca-plans YEAR=2026"; exit 2; }
+	uv run python -m src.ingestion.ca_puf --year $(YEAR) $(if $(ZIP),--zip $(ZIP)) $(if $(REFRESH),--refresh)
+
 # Summary of Benefits PDFs for the catalog plans in STATES; run ingest-plans
 # first. Required for the same reason. e.g. make ingest-sbc STATES=NH,DE
 # TOP_ISSUERS=1 reads only the largest parent companies' plans, ISSUERS=40788,66252

README.md

@@ -4,7 +4,7 @@
 
 PolicyPal lets a logged-in user ask natural-language questions about any kind of insurance — health, auto, life, home, and more — and returns grounded answers generated by an LLM using context retrieved from a vector store. Every answer cites the source passages it was built from.
 
-**Built:** comparison of real purchasable health plans drawn from the CMS Marketplace API. That half is deliberately narrower than the Q&A — ACA individual and family plans, in the 30 states served by HealthCare.gov. The 21 state-based exchanges, employer coverage, Medicare and Medicaid have no equivalent public catalog and are out of scope. All five phases of [docs/plans](docs/plans/) are complete; the catalog is loaded for eighteen states (3,276 plans, 137 issuers), and 2,274 of those plans (69.4%) have their Summary of Benefits read here — 2,136 read whole and 138 in part (ADR 0017). Keeping the documents current is a routine now, not a phase: [docs/runbooks/sbc.md](docs/runbooks/sbc.md).
+**Built:** comparison of real purchasable health plans drawn from the CMS Marketplace API, and for California from CMS's state-based exchange data (ADR 0024). That half is deliberately narrower than the Q&A — ACA individual and family plans, in the 30 states served by HealthCare.gov plus California's Covered California (190 plans from 11 issuers for 2026, priced from CMS's filed rates). The other 20 state-based exchanges, employer coverage, Medicare and Medicaid are out of scope. All five phases of [docs/plans](docs/plans/) are complete; the catalog is loaded for eighteen states (3,276 plans, 137 issuers), and 2,274 of those plans (69.4%) have their Summary of Benefits read here — 2,136 read whole and 138 in part (ADR 0017). Keeping the documents current is a routine now, not a phase: [docs/runbooks/sbc.md](docs/runbooks/sbc.md).
 
 PolicyPal will compare plans; it will not recommend one. Advice-shaped questions are refused by design, and that refusal is a measured property with its own regression floor (ADR 0004).
 
@@ -17,7 +17,7 @@ Because HealthCare.gov is the authoritative consumer source, answers about healt
 - **Conversational Q&A**, organized into threads, over a curated insurance knowledge base
 - **Hybrid RAG pipeline** — BM25 + pgvector semantic search, cross-encoder reranked and relevance-filtered before reaching the LLM, so weakly-relevant matches never become context
 - **Cited answers** — every reply lists the source passages and their relevance score, and the plans it compared in a side-by-side table; both survive reloading the conversation. Each plan in the table says whether its Summary of Benefits was read, and why not (ADR 0017)
-- **Real plan comparison** — ask about ACA Marketplace plans; the model searches the ingested catalog and CMS prices the plans live for your age (ADR 0010). It compares plans and never recommends one
+- **Real plan comparison** — ask about ACA Marketplace plans; the model searches the ingested catalog and CMS prices the plans live for your age (ADR 0010). In California, premiums are CMS's filed rates for your age and ZIP's rating area, and answers name Covered California (ADR 0024). It compares plans and never recommends one
 - **A profile, kept off the model** — signup takes a ZIP code, date of birth and county, and plan questions use them without their ever being sent to the LLM. Nobody under 13 can sign up (ADR 0012)
 - **JWT-authenticated API** — only a signed-in user can query, and only ever sees their own threads
 - **Reproducible ingestion** — a single command runs fetch → normalize → chunk → embed → store, across every registered source
@@ -248,6 +248,34 @@ ingested before that table existed, re-run once. Needs `CMS_MARKETPLACE_API_KEY`
   explicitly: `uv run python -m src.ingestion.plans --states TX --year 2027`.
 
 The county list is cached per plan year in `data/plans/raw/`; delete it to refresh.
+
+#### California (Covered California)
+
+```bash
+make ingest-ca-plans YEAR=2026      # ZIP=path for a file downloaded by hand, REFRESH=1 to re-download
+```
+
+The Marketplace API has no California plans, so they come from CMS's
+state-based exchange PUF, one zip per plan year, published May to August of
+that year (ADR 0024). The loader:
+
+- writes the same tables, plus `rating_areas`, `plan_rates` and `catalog_loads`;
+- validates the whole file before writing, in one transaction, so a bad file
+  changes nothing;
+- keeps a plan only in counties where it has a filed rate.
+
+Premiums are read from `plan_rates` in SQL; CMS is never called for California.
+Rating areas and issuer names are transcribed by hand and re-checked yearly:
+[`docs/runbooks/california.md`](docs/runbooks/california.md). What the file
+holds is in [`docs/findings/ca-sbe-puf.md`](docs/findings/ca-sbe-puf.md).
+
+#### Source registry
+
+Every source PolicyPal reads or links is recorded in
+[`src/ingestion/sources/registry.toml`](src/ingestion/sources/registry.toml),
+with its licence, permission status, `robots.txt` status and removal steps
+(ADR 0025). Only approved sources run. Covered California is recorded as
+link-only: its Terms of Use forbid automated access.
 Verified API behaviour is recorded in
 [`docs/findings/cms-marketplace-api.md`](docs/findings/cms-marketplace-api.md).
 
@@ -434,6 +462,13 @@ cite general material or state a figure for it. Its floor is full marks, and eve
 COVERAGE answer is held to the same rule. It also needs the NC catalog:
 `make ingest-plans STATES=NC` and `make ingest-sbc STATES=NC TOP_ISSUERS=1`.
 
+PLAN SEARCH requires every plan question to reach `search_plans`. That includes
+comparisons by metal level ("Compare the gold plans."), which used to skip it,
+and two California questions that must name Covered California, never
+HealthCare.gov. They need the California catalog: `make ingest-ca-plans
+YEAR=2026`. CONCEPTS requires the opposite for a question about what a level
+means ("What is a silver plan?"), which is answered from the corpus.
+
 `uv run python -m scripts.eval_sbc_ranking` checks, without calling a model, that the
 section answering each of ten questions is among the four `plan_coverage` ranks
 highest, for two plans per issuer with stored SBCs. Run it before and after a parser