Keep eligibility assessment enabled when CoT is disabled

CHANGELOG.md

@@ -4,6 +4,25 @@ All notable changes to TrialMatchAI are documented here. The format follows
 [Keep a Changelog](https://keepachangelog.com/) and the project adheres to
 [Semantic Versioning](https://semver.org/).
 
+## [Unreleased]
+
+### Fixed
+- Eligibility assessment is controlled by `rag.enabled` independently of the CoT
+  prompt setting. Disabling `use_cot_reasoning` now selects direct JSON assessment.
+- Ranked output records assessment controls and availability. Reports explicitly
+  label retrieval-only runs and hide assessments from prior enabled runs.
+- Assessment-control changes invalidate cached matching and per-trial assessment
+  reuse. Legacy matches without mode metadata are recomputed on resume.
+- Shortlist-budget changes invalidate cached assessments. Partial, unavailable,
+  missing, or corrupt assessment outputs remain retryable; fresh attempts isolate
+  their outputs so an early return or failed write cannot revive old verdicts.
+- The flowchart places eligibility assessment on the default path and labels the
+  explicit retrieval-only bypass. Both SVGs state its default-on behavior.
+
+### Migration
+- Set `rag.enabled: false` to skip assessment. `use_cot_reasoning: false` alone no
+  longer disables it. No new GPU or clinical-accuracy qualification is claimed.
+
 ## [0.9.0] — 2026-09-12
 
 ### Added

README.md

@@ -15,9 +15,8 @@
 </div>
 
 TrialMatchAI imports patient information, searches a local clinical-trial index,
-and produces ranked results and portable HTML reports. Optional local models add
-biomedical entity extraction, query expansion, reranking, and criterion-level
-eligibility assessments.
+and produces ranked results and portable HTML reports. Configured local models perform biomedical entity extraction, reranking, and
+criterion-level eligibility assessment. Query expansion can be enabled separately.
 
 **Release status: beta.** The automated release gates exercise the installed CLI
 with synthetic data and real CPU search. They do not qualify GPU inference,
@@ -57,7 +56,7 @@ benchmark.
 | :--- | :--- | :--- |
 | Patient input | Text, FHIR, Phenopacket, and OMOP importers; canonical profiles and summaries | Format coverage is partial; validate mappings against your source data |
 | Retrieval | Local LanceDB tables, BM25/vector/hybrid search, multi-channel fusion, structured filters | Retrieval quality depends on corpus, embeddings, filters, and candidate budgets |
-| Model stages | Entity extraction, optional query expansion, LLM reranking, criterion assessments | Requires extra dependencies, model access, and runtime qualification |
+| Model stages | Entity extraction, default-on reranking and eligibility assessment, optional query expansion | Requires extra dependencies, model access, and runtime qualification |
 | Review | Ranked JSON, per-trial outputs, individual and multi-patient HTML reports | No hosted clinical workspace, authentication, review queue, or sign-off system |
 | Operations | CLI stages, registry updater, checksum tools, synthetic e2e, verified package publishing | Complete cache invalidation, GPU qualification, and application deployment remain open |
 | Agent behavior | A fixed sequence of configurable stages, with optional LLM expansion | Bounded retrieval/review agents are planned; autonomous task planning is not implemented |
@@ -68,17 +67,41 @@ across ranking quality, clinical workflow, agents, and delivery.
 
 ## How matching works
 
-<img src="https://raw.githubusercontent.com/cbib/TrialMatchAI/main/docs/assets/matching-flow.svg" alt="Matching flow: patient profile, trial retrieval, criterion retrieval, final ranking, and human review; optional query expansion, reranking, and eligibility assessment" width="1120">
+<img src="https://raw.githubusercontent.com/cbib/TrialMatchAI/main/docs/assets/matching-flow.svg" alt="Default matching flow: patient profile, trial retrieval, criterion retrieval, eligibility assessment, final ranking, and human review; explicit retrieval-only bypass" width="1120">
 
 
 The build path prepares trial records and criterion rows, then indexes them.
 During matching, first-level retrieval gathers trial candidates across query
-channels; second-level retrieval selects criterion evidence. Enabled reranking,
-constraint scoring, and eligibility assessment contribute to the final output.
+channels; second-level retrieval selects criterion evidence. Reranking, constraint scoring, and default-on eligibility assessment contribute
+to the final output.
 The code calls its generated eligibility explanations **CoT reasoning**; these
 are model outputs to inspect alongside source evidence, not verified clinical
 reasoning or a guarantee that every criterion has been covered.
 
+In source checkouts after 0.9.0, eligibility assessment and CoT prompt style have
+independent switches (the change is currently unreleased):
+
+| `rag.enabled` | `use_cot_reasoning` | Result |
+| :--- | :--- | :--- |
+| `true` (default) | `true` (default) | Eligibility assessment using the CoT prompt |
+| `true` | `false` | Eligibility assessment using the direct JSON prompt |
+| `false` | Either | Retrieval-only ranking; no eligibility assessment |
+
+Both assessment prompts request criterion classifications and evidence-based
+justifications. `rag.no_think` separately controls supported models' thinking-mode
+settings. Rankings record their mode and assessment availability; reports label
+retrieval-only results and suppress assessments left over from earlier runs.
+Missing or unusable assessment outputs are not an eligibility verdict.
+
+**Migration from 0.9.0:** to disable assessment, explicitly set `rag.enabled: false`.
+Setting only `use_cot_reasoning: false` now keeps assessment enabled. Legacy matches
+without the new mode metadata are recomputed when matching resumes; changing the
+assessment switches or `rag.max_trials_rag` also invalidates their cached results.
+Incomplete assessments remain pending on resume, reusing compatible successful
+trial outputs and retrying missing or unusable ones. Aborted attempts cannot reuse
+verdicts from a prior configuration. This does not establish
+complete patient/model/corpus cache identity, which remains roadmap work.
+
 The default configuration selects `BAAI/bge-m3` embeddings,
 `fastino/gliner2-base-v1` entity extraction, `google/gemma-2-2b-it` reranking, and
 `microsoft/phi-4` eligibility reasoning with TrialMatchAI adapters. Query expansion

docs/index.md

@@ -4,8 +4,8 @@
 
 **A configurable CLI for patient-to-trial retrieval and eligibility review.**
 TrialMatchAI imports patient information, searches local LanceDB tables, and writes
-ranked results and HTML reports. Optional models add entity extraction, expansion,
-reranking, and criterion-level assessments.
+ranked results and HTML reports. The default model workflow includes entity extraction, reranking, and
+criterion-level eligibility assessment. Query expansion is separately configurable.
 
 !!! info "Beta release — declared validation scope"
     The release gates test the installed CLI with synthetic data and real CPU search.

docs/pipeline.md

@@ -99,3 +99,30 @@ run_pipeline(ctx, from_stage="index", to_stage="match")
 
 See the [API reference](api.md) for `StageContext`, `Stage`, `select_stages`, and
 `run_pipeline`.
+
+## Assessment modes
+
+This section describes the unreleased changes on `main` after 0.9.0.
+
+Eligibility assessment is enabled by default. `rag.enabled` controls whether it
+runs; `use_cot_reasoning` selects the CoT prompt (`true`) or direct JSON prompt
+(`false`). Both prompts request per-criterion classifications and justifications.
+Set `rag.enabled: false` for retrieval-only execution, as the synthetic CPU demo does.
+`rag.no_think` remains a separate model thinking-mode setting.
+
+`ranked_trials.json` now carries a `Run` object with assessment controls, result mode,
+output availability, and assessed trial IDs. Reports show retrieval-only and partial
+assessment states, and do not join old assessments into a retrieval-only result.
+These fields describe available outputs, not verified clinical completeness.
+
+When upgrading from 0.9.0, configurations that disabled CoT to skip the whole stage
+must explicitly disable `rag.enabled`. Legacy results without this metadata are
+recomputed on resume. Assessment-control changes, including `rag.max_trials_rag`,
+invalidate both match completion and per-trial assessment reuse. Unavailable or
+partial assessments stay pending; a resumed attempt reuses compatible successful
+outputs and retries the remaining trials. Missing or corrupt saved assessments
+also trigger retries. An empty shortlist or explicitly disabled assessment is a
+completed run. Each attempt stages its outputs separately, so an early return or
+failed write cannot associate old verdicts with new controls. Full
+evidence/config/model identity and atomic publication of an entire patient run
+remain P02 work.

docs/production-validation.md

@@ -45,3 +45,23 @@ Hosted CI and release execution are recorded on [PR #31](https://github.com/cbib
 and the [0.9.0 release](https://github.com/cbib/TrialMatchAI/releases/tag/v0.9.0).
 A package release is separate from model migration and application deployment;
 neither of those is included.
+
+## Assessment controls follow-up (unreleased)
+
+- Full local suite: **498 passed**. Includes regression coverage for
+  independent assessment/CoT flags, preflight, both backend dispatch paths, mode and
+  prompt-style changes during resume, result provenance, stale-output suppression,
+  missing/partial assessment display, preservation of metadata when reranking,
+  shortlist-budget changes, retries of incomplete assessments, and isolation of
+  stale outputs after early returns or failed writes.
+- Isolated installed-wheel CLI e2e: passed, including the explicit retrieval-only
+  mode in ranked JSON and HTML, report repair, unchanged resume output, and checksums.
+- Ruff and strict MkDocs build: passed. The revised SVGs were rendered and inspected;
+  the synthetic retrieval-only HTML report was inspected in headless Firefox.
+- Model dispatch regressions use deterministic test processors. These checks do not
+  qualify actual GPU inference, clinical completeness, or benchmark improvements.
+
+Configurations that previously used only `use_cot_reasoning: false` to disable
+assessment must now set `rag.enabled: false`. Legacy matches without `Run` metadata
+are recomputed on resume. This follow-up binds assessment controls, not complete
+patient/corpus/model evidence identity.

scripts/installed_smoke.py

@@ -64,6 +64,10 @@ def run(*arguments):
     assert "demo-patient" in report and "NCT00000001" in report
     assert '"reasoning_available": false' in report
     ranked = demo / "results/demo-patient/ranked_trials.json"
+    payload = json.loads(ranked.read_text())
+    assert payload["Run"]["mode"] == "retrieval_only"
+    assert payload["Run"]["assessment_status"] == "disabled"
+    assert '"mode": "retrieval_only"' in report and "Retrieval-only" in report
     original = (ranked.read_bytes(), ranked.stat().st_mtime_ns)
     run("demo", "--workdir", str(demo), "--resume")
     assert (ranked.read_bytes(), ranked.stat().st_mtime_ns) == original

src/trialmatchai/config/config.json

@@ -128,6 +128,7 @@
   },
   "use_cot_reasoning": true,
   "rag": {
+    "enabled": true,
     "batch_size": 4,
     "max_trials_rag": 20
   },