review: 2026-09-08 review-and-improvement pass (property laws, runner fail-fast, gate honesty, metrics)

ISA.md

@@ -2,7 +2,7 @@
 project: cogant
 phase: current-readiness
 mode: source-grounded
-updated: 2026-06-07
+updated: 2026-09-08
 ---
 
 # COGANT Ideal State Artifact
@@ -54,7 +54,7 @@ uv run --directory cogant python ../tools/regenerate_metrics.py
 uv run python scripts/z_generate_manuscript_variables.py --strict
 uv run python tools/check_metrics_fresh.py
 uv run python tools/audit_docs_constants.py
-uv run python tools/audit_stage_list.py
+uv run --directory cogant python ../tools/audit_stage_list.py
 uv run python tools/audit_manuscript_numbers.py --output /tmp/cogant_number_audit.md
 uv run python tools/audit_manuscript_markdown_links.py
 uv run python tools/audit_manuscript_crossrefs.py
@@ -107,6 +107,15 @@ target.
 
 ## Changelog
 
+- **2026-09-08 — Review-and-improvement pass.** Property-law tests re-pinned
+  to the fail-closed renderer contract (laws 4/5/6 + matrix stochasticity);
+  `run_all` fail-fast now persists the partial manifest/summary, capture
+  steps are timeout-bounded, interrupted git clones self-repair; math-adjacency
+  and coverage-table gates fail loud on unavailable inputs; cog-p1-02 closed
+  (parser capability registry + fallback contract tests); METRICS.yaml
+  regenerated against commit-bound HEAD and the manuscript chain re-rendered
+  (publication readiness: ready). Full suite 9612 passed / 0 failed / 47
+  skipped, coverage 94.61%. See `REVIEW_LOG_2026-09-08.md`.
 - **2026-06-08 — Post-review hardening + held-out eval (round 2).**
   - **Guard audit (durable):** added `tools/audit_manuscript_math_adjacency.py`
     + `tests/test_audit_manuscript_math_adjacency.py` and wired it into the gate

README.md

@@ -51,8 +51,7 @@ Audit drift-sensitive docs and stubs:
 ```bash
 uv run python tools/audit_docs_constants.py
 uv run python tools/audit_folder_docs.py
-uv run python tools/audit_pyi_exports.py
-uv run python tools/audit_stage_list.py
+uv run --directory cogant python ../tools/audit_stage_list.py
 uv run python tools/audit_manuscript_crossrefs.py
 uv run python tools/audit_manuscript_citations.py
 uv run python tools/audit_manuscript_numbers.py

REVIEW_LOG_2026-09-08.md

@@ -0,0 +1,134 @@
+# COGANT review-and-improvement pass log — 2026-09-08
+
+Fleet review-and-improvement pass on `ActiveInferenceInstitute/COGANT`
+(default branch `main`, base `39b961b` "chore(metrics): regenerate METRICS
+against commit-bound HEAD"). Clean worktree at start; no pre-existing dirty
+files.
+
+## Phase 0 — Preflight
+
+- Baseline gates verified at HEAD: manuscript audit battery green
+  (crossrefs 126 ids / 515 refs, citations 155/155, numbers 0 mismatches,
+  markdown links 0, math adjacency 0, claim scope 0, robustness 8/8,
+  formalisms, synthetic surfaces, module refs 88/0 unresolved, figure
+  renderers 14+3, coverage table, metrics freshness, folder docs, docs
+  constants, `.pyi` exports, publication readiness `verdict=ready`).
+- Full package suite baseline: `uv run pytest tests/ -q` → 1 failed with
+  `-x` (property law 4), and `.pytest_cache/lastfailed` listed 39 stale
+  node-ids (most already fixed; cache had not been cleared).
+
+## Phase 1 — Review findings (7 parallel dimension scouts)
+
+Ranked outcomes per dimension:
+
+1. **Package tests** — 6 genuinely failing property tests
+   (`test_correctness_laws.py` laws 4/5/6, `test_law_matrix_stochasticity.py`
+   5a/5b/5c) pinned a removed contract: they built source models with empty
+   A/B/C/D and expected `render_matrices_module` to invent defaults. The
+   renderer is now deliberately fail-closed ("no matrix is invented when the
+   source is incomplete"), so the tests contradicted the implemented
+   contract. The other lastfailed clusters (config loaders, language
+   fallbacks, package-init) were already green — stale cache.
+2. **run_all orchestration** — `--fail-fast` returned mid-loop before
+   `run_manifest.json`/`summary.*` were written, discarding all completed
+   targets' evidence; `run_cmd_capture` had no timeout (a hung child blocks
+   the batch forever); an interrupted git clone (dest without `.git`)
+   poisoned every re-run; `export-gnn:{tid}` check label did not match the
+   recorded `export_gnn:{tid}` step name.
+3. **Tools/scripts hygiene** — two silent-fallback paths weakened gates:
+   `audit_manuscript_math_adjacency._resolve` swallowed resolution failures
+   (auditing unsubstituted text = silent pass) and
+   `check_coverage_table._load_metrics_yaml` returned `{}` on missing or
+   broken METRICS.yaml. Entry points are otherwise thin with `__main__`
+   guards; no dead tools found; duplication findings noted as taste, not
+   fixed.
+4. **Manuscript** — prose quality high, cross-refs/figures/citations clean;
+   the injected copy under `output/manuscript/` was stale relative to
+   METRICS.yaml (9560/53/48 vs 9606/6/47 and older timestamps). Fixed by
+   re-running the documented generator (regenerated outputs are disposable).
+   The audit battery contradicts the scout's "7 uncited bib entries"
+   observation: 155 keys defined, 155 used, 0 orphans.
+5. **Plans vs implemented state** — cog-p1-02 (parser capability registry)
+   is fully implemented and accepted → closed; cog-p1-01 substantially
+   implemented but the schema-version migration mechanism is partial → kept
+   active; tasks.yaml has overdue `in_progress` rows (server-managed file,
+   left untouched, flagged to the owner); TODO.md cited a regression test at
+   a path that does not exist (historical section, left as dated history).
+6. **REVIEW_LOG follow-through** — sole log `REVIEW_LOG_2026-08-02.md`; of
+   its three open items, the audit-test failure was already fixed
+   (5/5 green) and METRICS had been regenerated on 2026-08-19, but the
+   TODO checkboxes were never ticked. Notebook stubs remain deliberately
+   open. Both resolved items are now recorded; METRICS regenerated again
+   against this pass's commit-bound HEAD.
+7. **Figures** — 18/18 registered figures pass strict copy + visual QA; no
+   orphaned manuscript references; only cosmetic extras (unregistered audit
+   PNG; missing source sidecars for eval fig1–fig4, tolerated by the copier).
+
+## Phase 2 — Implementation (commits)
+
+1. `9ac186d` — test(property): laws 4/5/6 + matrix-stochasticity strategies
+   supply complete valid source matrices; law4/law5 exercise the degenerate
+   aggregate broadcast (D `[1.0]` → uniform over declared cardinality,
+   single-column A → column-stochastic expansion). Template-contract test
+   accepts a sidecar clone named `COGANT`.
+2. `68ce23e` — fix(runner): `FailFastExit` aborts after persisting the
+   partial manifest + summary; 1800s timeouts on capture steps; incomplete
+   git clones repaired; `export-gnn` label aligned.
+3. `c14e1d9` — fix(tools): math-adjacency and coverage-table gates fail
+   loud when their inputs are unavailable.
+4. (this pass) — TODO.md reconciliation (cog-p1-02 closed, dated pass
+   section, open items ticked), REVIEW_LOG (this file), README
+   `audit_stage_list` invocation corrected to the inner-package env form
+   matching CI, METRICS.yaml + manuscript/figure/report artifacts
+   regenerated against commit-bound HEAD.
+
+## Phase 3 — Verification
+
+- Full package suite: **9612 passed / 0 failed / 47 skipped**, coverage
+  94.61% against the 89% line gate (12m09s).
+- Root tools lane (`uv run pytest tests/` at the project root, minus the
+  three tests that require the inner package env — those pass 15/15 under
+  `uv run --directory cogant`): the template-contract test is fixed; 10
+  remaining failures are pre-existing at HEAD (verified by stash) and
+  environmental: the root compatibility-shell venv intentionally installs
+  only `pyyaml`+`pytest`, while four test files invoke audit tools that
+  import `cogant.*` internals (pydantic). CI never runs this lane — every
+  audit gate runs via `uv run --directory cogant` (`.github/workflows/ci.yml`),
+  which is green.
+- `./run_all.sh --dry-run` → exit 0 (24 targets); batch dashboard
+  regeneration → exit 0.
+- Manuscript generator re-run → publication readiness `verdict=ready`,
+  blockers=0; visual QA 18/18.
+- Release gate (`tools/release_gate.py`) lane matrix, root venv
+  (Python 3.14.6): package-tests **passed**, ruff **passed**, mypy
+  **passed**, rust-format **passed**; rust lanes require
+  `PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1` (PyO3 0.22.6 caps at 3.13) and
+  then rust-check / rust-test / rust-clippy all **pass**; wheel-smoke
+  **fails** only because no pydantic_core binary supports 3.14. Under the
+  inner 3.12 env the wheel toolchain is supported but the gate's
+  package-tests lane exceeds its 1800s cap (inner addopts add coverage +
+  verbose output and the gate runs rust lanes concurrently). All failures
+  are environmental (interpreter/toolchain), not code regressions: this
+  pass touches no Rust, wheel, or dependency code, and the identical
+  pre-pass HEAD fails the same lanes under the same root venv.
+
+## Open / deferred
+
+- **Root-lane environmental test failures** (10, pre-existing at HEAD):
+  `tests/test_audit_stage_list.py` (4), `tests/test_audit_synthetic_surfaces.py` (2),
+  `tests/test_check_metrics_fresh.py` (3), `tests/test_manuscript_figures.py` (1).
+  Fix options: skip-if-`cogant.config`-unavailable markers, or run the tools
+  under the inner env. Not caused by this pass; out of every configured CI gate.
+- **tasks.yaml date slips** (owner/taskboard server): six `in_progress` rows
+  with past end dates and `cog-p4-01` carrying a future `start` while active.
+- **Notebook stubs** (`cogant/docs/notebooks/*.md`, 12 pages) — unchanged,
+  deliberate placeholders.
+- **cog-p1-01** schema-version migration mechanism remains partial (field
+  exists; no version-to-version migration code).
+- **Pre-existing ruff I001/SIM115 in `tools/run_all_runner.py`** — outside
+  the configured lint scope (CI lints `py/cogant/ tests/` in the inner
+  package only); left untouched.
+- **Release gate wheel-smoke under Python 3.14** — needs a PyO3/pydantic
+  toolchain that supports 3.14, or running the gate from a ≤3.13
+  interpreter with a package-tests timeout budget that accommodates the
+  inner env's coverage addopts. Environmental; out of scope for this pass.

TODO.md

@@ -2,7 +2,10 @@
 
 - Status: Active (P0–P5 roadmap below).
 - Owner: DAF.
-- Last reviewed: 2026-08-02 (docs-deep review pass: Minor/Medium findings implemented; see "Completed/Closed — 2026-08-02 docs-deep review pass").
+- Last reviewed: 2026-09-08 (review-and-improvement pass: property-law tests
+  re-pinned to the fail-closed renderer contract, runner fail-fast evidence
+  persistence, gate honesty fixes, METRICS regenerated against commit-bound
+  HEAD; see "Completed/Closed — 2026-09-08 review pass").
 
 This is the future-only execution backlog. Completed work belongs in Git
 history, generated evidence, audit reports, or the taskboard; it must not be
@@ -87,15 +90,6 @@ Source-of-truth rules:
   - Acceptance: compatibility fixtures prove precedence
     `defaults < preset < file < environment < CLI`; no parallel registry remains.
 
-- [~] **cog-p1-02 — Explicit parser capabilities.**
-  - Priority: P1.
-  - Depends on: cog-p0-01, cog-p0-03.
-  - Deliverable: registry-only parser selection with language, extensions,
-    implementation, optional dependency, fallback/degraded status, and source
-    provenance.
-  - Acceptance: tree-sitter is selected when installed; fallback is observable;
-    unavailable-parser errors and capability tests cover every registered language.
-
 - [~] **cog-p1-03 — Fail-closed pipeline contracts.**
   - Priority: P1.
   - Depends on: cog-p1-01, cog-p1-02.
@@ -509,11 +503,48 @@ layout, and METRICS.yaml before editing. Commits: `d86528c`, `d2d7156`.
   "(planned)" placeholders with one-line descriptions; filling them needs the
   Jupyter toolchain and is a major effort. Left as-is (they are honestly labeled
   in the nav).
-- [ ] **Pre-existing audit test failure** —
-  `tests/test_audit_docs_constants.py::test_roundtrip_claim_audit_accepts_current_ledger_claim`
-  fails at HEAD without this pass's changes (verified by stash); not caused by the
-  docs pass. Owner should reconcile the test fixture with the current qualifier
-  regex.
-- [ ] **METRICS regeneration** — `METRICS.yaml` must be regenerated against the
-  new HEAD (`tools/regenerate_metrics.py`) before the metrics-fresh / release
-  gates can claim commit-bound freshness (see `cog-p0-04`).
+- [x] **Pre-existing audit test failure** — resolved: the 2026-08-19 fleet
+  review pass reconciled the qualifier regex; the targeted test now passes
+  5/5 (`uv run pytest -q --no-cov tests/test_audit_docs_constants.py`).
+- [x] **METRICS regeneration** — done in the 2026-09-08 pass:
+  `tools/regenerate_metrics.py` ran against commit-bound HEAD and
+  `check_metrics_fresh.py --fail-on-dirty` is green.
+
+## Completed/Closed — 2026-09-08 review-and-improvement pass
+
+Items implemented and closed this pass (verified by targeted runs; the full
+package suite is green — 9612 passed / 0 failed / 47 skipped, coverage
+94.61% against the 89% line gate):
+
+- [x] **Property-law tests re-pinned to the fail-closed renderer contract**
+  (`9ac186d`): laws 4/5/6 and the matrix-stochasticity laws built source
+  models with empty A/B/C/D and expected the renderer to invent defaults —
+  a contract `render_matrices_module` removed when matrix synthesis went
+  fail-closed. The strategies now supply complete valid matrices, and
+  law4/law5 exercise the degenerate aggregate broadcast (D `[1.0]` → uniform
+  over the declared cardinality; single-column A → column-stochastic
+  expansion).
+- [x] **Runner fail-fast evidence persistence** (`68ce23e`): `--fail-fast`
+  previously returned mid-loop before `run_manifest.json`/summary were
+  written, discarding every completed target's evidence. `FailFastExit`
+  now aborts after persisting the partial manifest and summary; capture
+  steps gained a 1800s timeout; interrupted git clones (dest without
+  `.git`) are repaired instead of poisoning every re-run; the `export-gnn`
+  check label now matches the recorded `export_gnn:{tid}` step name.
+- [x] **Gate honesty fixes** (`c14e1d9`): `audit_manuscript_math_adjacency`
+  no longer silently audits unsubstituted text when variable resolution
+  fails; `check_coverage_table` raises on missing/unparseable METRICS.yaml
+  instead of degrading benchmark sidecar comparisons.
+- [x] **cog-p1-02 — Explicit parser capabilities**: closed. The canonical
+  registry (`cogant/py/cogant/parsers/registry.py` with per-language
+  modules and `.pyi` stubs) plus the fallback/unavailable contract tests
+  (`tests/unit/test_ingest_language_detection_fallbacks.py`, 13/13 green)
+  satisfy the deliverable and acceptance. cog-p1-01 remains active: the
+  schema-version migration mechanism is still partial.
+- [x] **Template-contract test de-coupled from checkout directory case**
+  (`9ac186d`): a sidecar clone named `COGANT` no longer fails
+  `test_project_root_points_to_cogant_staging_root`.
+- [x] **REVIEW_LOG_2026-08-02 open items**: the audit-test failure was
+  already fixed (see the ticked 2026-08-02 item above); METRICS.yaml
+  regenerated against commit-bound HEAD this pass; the 12 notebook stubs
+  remain deliberately open.

cogant/evaluation/METRICS.yaml

@@ -3,17 +3,17 @@
 # DO NOT EDIT BY HAND. Run: cd cogant && uv run python ../tools/regenerate_metrics.py
 
 schema_version: '1.0'
-generated_at: '2026-08-19T15:55:54.237757Z'
-generator_git_sha: 36371f163920e1f5e13e4c93d65e4d09f6a8317c
+generated_at: '2026-09-08T22:48:12.814800Z'
+generator_git_sha: c14e1d9887cff323702329b5b9d2995d015346e5
 cogant_version: 0.6.0
 package:
   name: cogant
   version: 0.6.0
   python_min: '3.11'
 testing:
   test_count_total: 9659
-  test_count_passing: 9606
-  test_count_failing: 6
+  test_count_passing: 9612
+  test_count_failing: 0
   test_count_skipped: 47
   test_count_xfailed: 0
   test_count_xpassed: 0
@@ -517,7 +517,7 @@ literature:
   source: cogant/docs/evaluation/LITERATURE.md
   bibliography_entries: 104
 benchmark:
-  suite_runtime_s: 446.5
+  suite_runtime_s: 608.32
   shipped_fixture_count: 12
   benchmark_suite_file: suite_20260818.md
   benchmark_python_version: 3.12.13