From b5eb613b28663a8e5c6d84f8c2a5feea2fc3f290 Mon Sep 17 00:00:00 2001 From: "R. Mark Sharp" Date: Mon, 27 Jul 2026 12:12:28 -0500 Subject: [PATCH] =?UTF-8?q?docs(session-runner):=20Learning=20#13=20?= =?UTF-8?q?=E2=80=94=20a=20handoff's=20predictions=20decay?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Appends one row to the Learnings table (was 1-12; append only, never renumber — this file is bin/_manifest.py-distributed). A handoff carries predictions as well as facts, and a prediction about state the handoff's own session later changed is the least reliable kind. The existing rows do not cover this: #6 and FM #11 are about claims written from memory, and #7/#10/#12 are about cross-references that go stale in the corpus. Neither addresses a forward-looking claim whose expiry is caused by the writing session's own later commits. Motivating case is this repo's own v3.6 ship. The predecessor receipt said "expect one CHANGELOG.md union conflict"; the merge produced seven, because Layer 8 and the pre-PR review fixes were committed to the branch after the point the fork had been ported from it. Five files the prediction never named had diverged, including both dashboard twins and the unit suite. A wrong prediction is worse than no prediction: it licenses the successor to resolve quickly rather than carefully, and a merge resolved on the belief that "only the ledger differs" discards code silently, with no failing check to catch it. Cross-reference sweep per Learnings #7/#10: no live count-claim about the table exists in the corpus — the "table was 1-11"/"table is 1-12" tokens are confined to dated CHANGELOG entries and HANDOFFS receipts, left verbatim by design, and the caption states no size. Phase 3C of fork session S17, whose durable receipt is the HANDOFFS.md S17 block on the fork; no new receipt is opened here. Verified: bin/tests.sh 84/84, bin/check-links OK, 197/197 unit, Learnings table reads 1-13 with every row 4-column. Co-Authored-By: Claude Opus 5 (1M context) --- CHANGELOG.md | 29 +++++++++++++++++++++++++++++ starter-kit/SESSION_RUNNER.md | 1 + 2 files changed, 30 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index bbc7c0e..9b2c110 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -32,6 +32,35 @@ Reverse-chronological, newest on top; prepend-only. Promote to `## YYYY-MM` sect --- +### 2026-07-27 · [ad hoc] New `SESSION_RUNNER.md` Learning #13 — a handoff's predictions decay + +- **Change:** one row appended to the `starter-kit/SESSION_RUNNER.md` Learnings table (table was + **1–12**; appended, never renumbered — that file is `bin/_manifest.py`-**distributed**, so + adopters receive this via `bin/sync`). *A handoff carries predictions as well as facts, and a + prediction about state the handoff's own session later changed is the least reliable kind.* +- **Why the existing rows did not cover it.** Learning #6 and FM #11 cover claims written *from + memory*; #7, #10 and #12 cover cross-references that go stale in the *corpus*. None covers the + **forward-looking** claim — a state description whose expiry is caused by the writing session's + own subsequent commits. +- **The motivating case is this repo's own v3.6 ship.** The predecessor receipt said "expect one + `CHANGELOG.md` union conflict"; `git merge upstream/main` produced **seven**, because Layer 8 and + the pre-PR review fixes were committed to the branch *after* the point the fork had been ported + from it. Five files the prediction never named had diverged, including both dashboard twins and + the unit suite. The prediction was true when written and false when used. +- **Why it is worth a row rather than a gotcha:** a wrong prediction is worse than no prediction, + because it licenses the successor to resolve *quickly* rather than *carefully* — and a merge + resolved on the belief that "only the ledger differs" discards code silently, with no failing + check to catch it. The countermeasure is mechanical: diff the branch against the commit the other + side was **ported from**, not against its tip. +- **Cross-reference sweep (Learnings #7/#10, applied to this change).** No live count-claim about + the Learnings table exists anywhere in the corpus; the `table was 1-11` / `table is 1-12` tokens + are confined to dated `CHANGELOG.md` entries and `HANDOFFS.md` receipts, which this repo leaves + verbatim by design. The table caption states no size. Nothing else needed updating. +- **Session:** fork S17 (the v3.6 ship session) — this is its Phase 3C, so the durable receipt is + the `HANDOFFS.md` S17 block on the fork rather than a new receipt here. +- **Verified:** `bin/tests.sh` 84/84 · `bin/check-links` OK · 197/197 unit · Learnings table reads + 1–13 with every row 4-column. + ### 2026-07-26 · [ad hoc] Dashboard signal-integrity campaign lands upstream — the scanner's signals now mean what they say (v3.6) - **Change:** the portfolio health scanner's signals are reconciled with what they actually measure, closing upstream issues [#59](https://github.com/KJ5HST/methodology/issues/59), diff --git a/starter-kit/SESSION_RUNNER.md b/starter-kit/SESSION_RUNNER.md index c8d627a..79df54e 100644 --- a/starter-kit/SESSION_RUNNER.md +++ b/starter-kit/SESSION_RUNNER.md @@ -375,6 +375,7 @@ These are documented tendencies. The agent must actively guard against them. | 10 | **A review pass scoped to the diff has a blind spot: what the change made stale *outside* it.** Learning #7 (cross-reference completeness) and Learning #8 (close-out-gate propagation) are authoring-time self-checks — they catch drift the *author* introduces. But an adversarial review or audit is pointed at the diff by default, so "what did this change make stale elsewhere in the corpus?" is nobody's job. The v3.3 close-out-receipt work (#52/#53) shipped a clean 6-lens adversarial review, yet left `HOW_TO_USE.md`, the `README.md` repo tree, and three tutorials describing close-out *without* the new receipt — none were in the diff, so no diff-scoped lens looked at them. A human had to ask "has documentation been fully updated?" (fixed in #54). | Upstream issue #55 (completeness-critic proposal), motivated by the v3.3 doc-completeness gap fixed in PR #54 | Promote Learning #7/#8 from an authoring self-check to a review-time lens: when a change adds, renames, or removes a concept, artifact, file, step, or numbered-set member, grep the *whole corpus* (not just the diff) for file/artifact enumerations, worked examples/tutorials, indexes/cross-references, and count/size claims ("N failure modes", "the six requirements") that now lag. See `workstreams/AUDIT_WORKSTREAM.md`'s Verification Checklist for the operative step. | | 11 | **Heterogeneous capability tiers within one slice are a division of labor, not a new gate.** Splitting a pre-declared vertical slice's layers across agents of different capability tiers — with the strongest tier reviewing every lighter-tier layer's output before that layer's checkpoint commit lands — satisfies the existing per-boundary verification gates (c)/(d); it does not add a fifth gate or substitute for the build/test matrix. The risk is a plausible-sounding misreading: treating cross-tier review as proof a layer is correct on its own, or batching review to the end of the slice instead of per boundary — either one reopens failure mode #26 (mega-session masquerading as a vertical slice) wearing a division-of-labor costume. | The close-out-receipt slice's own hybrid capability-tier build (`HANDOFFS.md` session S1; concrete agent/model names live in `RECOMMENDED_SKILLS.md`), formalized as the §Vertical Slice Sessions "capability-tiered review" addition (BL-7) | When a slice's layers will be built or reviewed by agents of different capability tiers: state the tier assignment in the gate (a) contract, delegate a layer only where an objective gate (test suite, exhaustive grep, mechanical spec) proves its correctness, and confirm review happens per boundary before each checkpoint commit — never as one pass at the end. | | 12 | **When an invariant is mechanical, encode it as a test — a review-time grep is a human step that silently stops happening.** Learnings #7 and #10 catch stale cross-references by *grepping at review time*, and that is the right tool for a relation between prose and meaning. But where the invariant is a relation between two **enumerable sets**, the grep is a weak substitute for an assertion: it depends on someone remembering, and its absence leaves no evidence. The motivating case: `HANDOFFS.md` joined the distributed corpus in v3.3 and was never added to the health scanner's methodology checklist, so the checklist silently stopped describing the corpus it scores — and the gap survived *two* releases of exactly the greps #7 and #10 prescribe. Stated as a property of two lists ("every distributed adopter-root destination is either on the checklist or explicitly exempt with a stated reason"), it is a few lines of test code — plus a second test that guards the guard, asserting every exemption names a real distributed file and states a reason, so a stale exemption cannot quietly hollow the invariant out. Once written, the whole class is closed: the next distributed artifact fails the suite instead of going uncounted. **The precondition is non-negotiable — drive the new assertion RED against unpatched code and watch it fail before trusting it.** A mechanized invariant that was never seen to fail is the same trap one level up, and this campaign hit it twice: an absence-check filtering a *dict* by `"Large files" in r` tested keys and passed vacuously, and a fixture whose render toolchain short-circuited the counts under test was structurally incapable of failing while looking like coverage. Pair every absence assertion with a presence control, and ask what a fixture makes *unreachable*, not only what it asserts. | Dashboard signal-integrity campaign (S9–S16) — the manifest-vs-checklist guard prototyped in S9 and shipped in Layer 1; the two vacuous tests caught in Layer 7 | When a change adds a member to an enumerable distributed set (a shipped file, a checklist item, a numbered set), or when you are about to rely on #7/#10 to keep two lists in step: ask whether the invariant is a relation between two sets you can enumerate in code. If it is, write the assertion — then drive it RED before trusting it. Keep the grep for what cannot be enumerated. | +| 13 | **A handoff carries predictions as well as facts — and a prediction about state the handoff's own session later changed is the least reliable kind.** Learning #6 and FM #11 cover claims written *from memory*; #7, #10 and #12 cover cross-references that go stale in the *corpus*. Neither covers the forward-looking claim. A handoff that says "expect one merge conflict", "these are the files you will touch", or "this should be quick" describes a state that was true when it was written — and the single most common thing that invalidates it is the writing session's own later commits. Concrete case: the v3.6 ship session inherited "expect one `CHANGELOG.md` union conflict" and got **seven**, because the predecessor had committed two further layers to the branch *after* the point the fork had been ported from it, so five files the prediction never mentioned had diverged. A wrong prediction is worse than no prediction: it licenses the successor to resolve quickly rather than carefully, and a merge resolved on the belief that "only the ledger differs" discards code silently, with no failing check to catch it. | v3.6 ship session — a predecessor-predicted one-conflict merge that produced seven | Treat every forward-looking claim in a handoff as a hypothesis with an expiry date and re-derive it from current state before acting on it. For a merge specifically, diff the branch against the commit the other side was **ported from**, not against its tip. And when you *write* a prediction, name the state it depends on, so your successor can tell at a glance whether it still holds. | ---