PR 6.2 Copilot review pass: fold all four real findings back in

Copilot's review of the PR 6.2 self-review surfaced four still-real
findings (two others were marked outdated — already addressed by the
self-review pass).  Folded all four back in before requesting human
review.

  - (COPILOT-1) Grammar: "reflect a honest production model" →
    "reflect an honest production model" in NB04 §2 narrative.
  - (COPILOT-3) The threshold-selection comment claimed the K-th
    highest probability + ``mask = lr_probs >= threshold`` admits
    *exactly* K leads, "ties resolved by score order".  The
    inclusive comparison can admit more than K when leads share the
    threshold's probability — and there is no implicit tie-break.
    Rewrote the comment to be honest about the semantics ("admits
    AT LEAST K via probs >= threshold; ties at the threshold can
    inflate the slate; ``actually_above`` makes the realised count
    visible").  Kept the threshold-based selection rather than
    switching to a true top-K via ``argsort`` because the
    pedagogical point of section 6 is *threshold selection*, not
    *rank cutoff*.
  - (COPILOT-4) Bootstrap loop comment said "Degenerate resample —
    re-roll" but the implementation writes NaN and continues.
    Rewrote the comment to match what the code does (mark NaN, let
    ``_summary`` filter it out) and added the actual probability
    bound — with n_test=750 and base rate ~22 %, the all-positive
    or all-negative draw probability is ~10⁻¹⁰⁰, so the branch is
    dead in practice and exists only as a defensive safety net for
    tiny test sets.  Implementing a real re-roll loop would never
    execute on this dataset.
  - (COPILOT-6) The top-level ``_doc`` in
    ``release/notebooks/_release_targets.json`` claimed the file
    contains only "cross-seed-median metric values", which became
    inaccurate after the PR 6.2 cohort_shift block (single-seed,
    seed 42).  Rewrote the docstring to call out the mixed
    structure: per-tier blocks hold cross-seed medians; cohort_shift
    block holds single-seed values from
    ``validation_report.cohort_shift``.  Audit-sync test continues
    to enforce both cases via separate loops.

  - (COPILOT-2, COPILOT-5) outdated; already addressed by the
    self-review fix-up commit (25b9ec1) — the 70/30-vs-85/15
    narrative whiplash and the wrong summary numbers (0.19 / 2.6×
    vs the actual 0.13 / 2.75×) are both fixed there.  Resolved as
    already-treated.

Net: 28/28 notebook builder + execution + audit-sync tests pass;
ruff + mypy clean; both notebooks still execute end-to-end in
<10 s each.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Author: shaypal5

release/notebooks/04_lift_calibration_value_ranking.ipynb

@@ -63,7 +63,7 @@
    "cell_type": "markdown",
    "id": "cell_003",
    "metadata": {},
-   "source": "## 2. Train the headline LR + GBM panel\n\nSame preprocessing as notebooks 01 / 02 (mirrors\n`leadforge.validation.release_quality._build_pipeline`).\nWe drop the documented leakage trap `total_touches_all`\nhere so the calibration / lift / value plots in sections\n3–6 reflect a honest production model. The cohort-shift\nsection in section 7 uses the validator's full-panel\nposture (trap kept) so its number is comparable to the\npublished validation report."
+   "source": "## 2. Train the headline LR + GBM panel\n\nSame preprocessing as notebooks 01 / 02 (mirrors\n`leadforge.validation.release_quality._build_pipeline`).\nWe drop the documented leakage trap `total_touches_all`\nhere so the calibration / lift / value plots in sections\n3–6 reflect an honest production model. The cohort-shift\nsection in section 7 uses the validator's full-panel\nposture (trap kept) so its number is comparable to the\npublished validation report."
   },
   {
    "cell_type": "code",
@@ -333,7 +333,14 @@
     "\n",
     "sorted_probs = np.sort(lr_probs)[::-1]\n",
     "# The K-th highest probability is the smallest threshold that\n",
-    "# admits exactly K leads (ties resolved by score order).\n",
+    "# admits AT LEAST K leads via ``probs >= threshold``.  If\n",
+    "# several leads share that probability, the inclusive\n",
+    "# comparison can admit more than K — that's a property of\n",
+    "# threshold-based selection, not a bug.  The\n",
+    "# ``actually_above`` readout below makes the realised count\n",
+    "# visible so the operator can see when ties are inflating\n",
+    "# the slate (and decide whether to break them with a\n",
+    "# secondary score).\n",
     "threshold = float(sorted_probs[CAPACITY - 1])\n",
     "mask = lr_probs >= threshold\n",
     "n_above = int(mask.sum())\n",
@@ -508,7 +515,13 @@
     "for i in range(N_BOOT):\n",
     "    idx = rng.integers(0, n_test, n_test)\n",
     "    if y_test[idx].sum() == 0 or y_test[idx].sum() == n_test:\n",
-    "        # Degenerate resample — re-roll.\n",
+    "        # Degenerate resample (all-positive or all-negative)\n",
+    "        # — ``roc_auc_score`` is undefined here.  We mark\n",
+    "        # the iteration NaN and let ``_summary`` filter it\n",
+    "        # out; with n_test=750 and base rate ~22 %, the\n",
+    "        # probability of a degenerate draw is ~10⁻¹⁰⁰, so\n",
+    "        # this branch is dead in practice.  Kept as a\n",
+    "        # defensive safety net for tiny test sets.\n",
     "        boot_lr_auc[i] = np.nan\n",
     "        boot_gbm_auc[i] = np.nan\n",
     "        boot_lr_ap[i] = np.nan\n",

release/notebooks/_release_targets.json

@@ -1,5 +1,5 @@
 {
- "_doc": "Cross-seed-median metric values from release/validation/validation_report.json, sliced to the metrics the release notebooks pin via assert_within_tolerance. Audited against the report by tests/release/notebooks/test_release_targets_match_report.py — if you change a value here, the test will fail unless the corresponding median in the validation report changes to match.",
+ "_doc": "Reproduction targets that release notebooks pin via assert_within_tolerance, sourced from release/validation/validation_report.json. Mixed structure: per-tier blocks (intermediate, ...) hold cross-seed-median metrics from validation_report.tiers.<tier>.medians; the cohort_shift block holds single-seed (seed 42) metrics from validation_report.cohort_shift.<tier>, since the report runs cohort-shift on seed 42 only. Audited against the report by tests/release/notebooks/test_release_targets_match_report.py — if you change a value here, the test will fail unless the corresponding source value in the validation report changes to match.",
  "cohort_shift": {
   "_doc": "Per-tier cohort-shift metrics from validation_report.cohort_shift (single-seed values; the report runs cohort-shift only on seed 42). Notebook 04 reproduces these via a chronological resplit and pins them via assert_within_tolerance.",
   "intermediate": {

scripts/build_release_notebook_04.py

@@ -128,7 +128,7 @@ def cells() -> list[nbf.NotebookNode]:
             `leadforge.validation.release_quality._build_pipeline`).
             We drop the documented leakage trap `total_touches_all`
             here so the calibration / lift / value plots in sections
-            3–6 reflect a honest production model. The cohort-shift
+            3–6 reflect an honest production model. The cohort-shift
             section in section 7 uses the validator's full-panel
             posture (trap kept) so its number is comparable to the
             published validation report.
@@ -421,7 +421,14 @@ def acv_capture(use_value: bool, k: int) -> float:
 
             sorted_probs = np.sort(lr_probs)[::-1]
             # The K-th highest probability is the smallest threshold that
-            # admits exactly K leads (ties resolved by score order).
+            # admits AT LEAST K leads via ``probs >= threshold``.  If
+            # several leads share that probability, the inclusive
+            # comparison can admit more than K — that's a property of
+            # threshold-based selection, not a bug.  The
+            # ``actually_above`` readout below makes the realised count
+            # visible so the operator can see when ties are inflating
+            # the slate (and decide whether to break them with a
+            # secondary score).
             threshold = float(sorted_probs[CAPACITY - 1])
             mask = lr_probs >= threshold
             n_above = int(mask.sum())
@@ -644,7 +651,13 @@ def _gbm_pipeline_for_cohort() -> Pipeline:
             for i in range(N_BOOT):
                 idx = rng.integers(0, n_test, n_test)
                 if y_test[idx].sum() == 0 or y_test[idx].sum() == n_test:
-                    # Degenerate resample — re-roll.
+                    # Degenerate resample (all-positive or all-negative)
+                    # — ``roc_auc_score`` is undefined here.  We mark
+                    # the iteration NaN and let ``_summary`` filter it
+                    # out; with n_test=750 and base rate ~22 %, the
+                    # probability of a degenerate draw is ~10⁻¹⁰⁰, so
+                    # this branch is dead in practice.  Kept as a
+                    # defensive safety net for tiny test sets.
                     boot_lr_auc[i] = np.nan
                     boot_gbm_auc[i] = np.nan
                     boot_lr_ap[i] = np.nan