diff --git a/CHANGELOG.md b/CHANGELOG.md index bbc7c0e..9b2c110 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -32,6 +32,35 @@ Reverse-chronological, newest on top; prepend-only. Promote to `## YYYY-MM` sect --- +### 2026-07-27 · [ad hoc] New `SESSION_RUNNER.md` Learning #13 — a handoff's predictions decay + +- **Change:** one row appended to the `starter-kit/SESSION_RUNNER.md` Learnings table (table was + **1–12**; appended, never renumbered — that file is `bin/_manifest.py`-**distributed**, so + adopters receive this via `bin/sync`). *A handoff carries predictions as well as facts, and a + prediction about state the handoff's own session later changed is the least reliable kind.* +- **Why the existing rows did not cover it.** Learning #6 and FM #11 cover claims written *from + memory*; #7, #10 and #12 cover cross-references that go stale in the *corpus*. None covers the + **forward-looking** claim — a state description whose expiry is caused by the writing session's + own subsequent commits. +- **The motivating case is this repo's own v3.6 ship.** The predecessor receipt said "expect one + `CHANGELOG.md` union conflict"; `git merge upstream/main` produced **seven**, because Layer 8 and + the pre-PR review fixes were committed to the branch *after* the point the fork had been ported + from it. Five files the prediction never named had diverged, including both dashboard twins and + the unit suite. The prediction was true when written and false when used. +- **Why it is worth a row rather than a gotcha:** a wrong prediction is worse than no prediction, + because it licenses the successor to resolve *quickly* rather than *carefully* — and a merge + resolved on the belief that "only the ledger differs" discards code silently, with no failing + check to catch it. The countermeasure is mechanical: diff the branch against the commit the other + side was **ported from**, not against its tip. +- **Cross-reference sweep (Learnings #7/#10, applied to this change).** No live count-claim about + the Learnings table exists anywhere in the corpus; the `table was 1-11` / `table is 1-12` tokens + are confined to dated `CHANGELOG.md` entries and `HANDOFFS.md` receipts, which this repo leaves + verbatim by design. The table caption states no size. Nothing else needed updating. +- **Session:** fork S17 (the v3.6 ship session) — this is its Phase 3C, so the durable receipt is + the `HANDOFFS.md` S17 block on the fork rather than a new receipt here. +- **Verified:** `bin/tests.sh` 84/84 · `bin/check-links` OK · 197/197 unit · Learnings table reads + 1–13 with every row 4-column. + ### 2026-07-26 · [ad hoc] Dashboard signal-integrity campaign lands upstream — the scanner's signals now mean what they say (v3.6) - **Change:** the portfolio health scanner's signals are reconciled with what they actually measure, closing upstream issues [#59](https://github.com/KJ5HST/methodology/issues/59), diff --git a/starter-kit/SESSION_RUNNER.md b/starter-kit/SESSION_RUNNER.md index c8d627a..79df54e 100644 --- a/starter-kit/SESSION_RUNNER.md +++ b/starter-kit/SESSION_RUNNER.md @@ -375,6 +375,7 @@ These are documented tendencies. The agent must actively guard against them. | 10 | **A review pass scoped to the diff has a blind spot: what the change made stale *outside* it.** Learning #7 (cross-reference completeness) and Learning #8 (close-out-gate propagation) are authoring-time self-checks — they catch drift the *author* introduces. But an adversarial review or audit is pointed at the diff by default, so "what did this change make stale elsewhere in the corpus?" is nobody's job. The v3.3 close-out-receipt work (#52/#53) shipped a clean 6-lens adversarial review, yet left `HOW_TO_USE.md`, the `README.md` repo tree, and three tutorials describing close-out *without* the new receipt — none were in the diff, so no diff-scoped lens looked at them. A human had to ask "has documentation been fully updated?" (fixed in #54). | Upstream issue #55 (completeness-critic proposal), motivated by the v3.3 doc-completeness gap fixed in PR #54 | Promote Learning #7/#8 from an authoring self-check to a review-time lens: when a change adds, renames, or removes a concept, artifact, file, step, or numbered-set member, grep the *whole corpus* (not just the diff) for file/artifact enumerations, worked examples/tutorials, indexes/cross-references, and count/size claims ("N failure modes", "the six requirements") that now lag. See `workstreams/AUDIT_WORKSTREAM.md`'s Verification Checklist for the operative step. | | 11 | **Heterogeneous capability tiers within one slice are a division of labor, not a new gate.** Splitting a pre-declared vertical slice's layers across agents of different capability tiers — with the strongest tier reviewing every lighter-tier layer's output before that layer's checkpoint commit lands — satisfies the existing per-boundary verification gates (c)/(d); it does not add a fifth gate or substitute for the build/test matrix. The risk is a plausible-sounding misreading: treating cross-tier review as proof a layer is correct on its own, or batching review to the end of the slice instead of per boundary — either one reopens failure mode #26 (mega-session masquerading as a vertical slice) wearing a division-of-labor costume. | The close-out-receipt slice's own hybrid capability-tier build (`HANDOFFS.md` session S1; concrete agent/model names live in `RECOMMENDED_SKILLS.md`), formalized as the §Vertical Slice Sessions "capability-tiered review" addition (BL-7) | When a slice's layers will be built or reviewed by agents of different capability tiers: state the tier assignment in the gate (a) contract, delegate a layer only where an objective gate (test suite, exhaustive grep, mechanical spec) proves its correctness, and confirm review happens per boundary before each checkpoint commit — never as one pass at the end. | | 12 | **When an invariant is mechanical, encode it as a test — a review-time grep is a human step that silently stops happening.** Learnings #7 and #10 catch stale cross-references by *grepping at review time*, and that is the right tool for a relation between prose and meaning. But where the invariant is a relation between two **enumerable sets**, the grep is a weak substitute for an assertion: it depends on someone remembering, and its absence leaves no evidence. The motivating case: `HANDOFFS.md` joined the distributed corpus in v3.3 and was never added to the health scanner's methodology checklist, so the checklist silently stopped describing the corpus it scores — and the gap survived *two* releases of exactly the greps #7 and #10 prescribe. Stated as a property of two lists ("every distributed adopter-root destination is either on the checklist or explicitly exempt with a stated reason"), it is a few lines of test code — plus a second test that guards the guard, asserting every exemption names a real distributed file and states a reason, so a stale exemption cannot quietly hollow the invariant out. Once written, the whole class is closed: the next distributed artifact fails the suite instead of going uncounted. **The precondition is non-negotiable — drive the new assertion RED against unpatched code and watch it fail before trusting it.** A mechanized invariant that was never seen to fail is the same trap one level up, and this campaign hit it twice: an absence-check filtering a *dict* by `"Large files" in r` tested keys and passed vacuously, and a fixture whose render toolchain short-circuited the counts under test was structurally incapable of failing while looking like coverage. Pair every absence assertion with a presence control, and ask what a fixture makes *unreachable*, not only what it asserts. | Dashboard signal-integrity campaign (S9–S16) — the manifest-vs-checklist guard prototyped in S9 and shipped in Layer 1; the two vacuous tests caught in Layer 7 | When a change adds a member to an enumerable distributed set (a shipped file, a checklist item, a numbered set), or when you are about to rely on #7/#10 to keep two lists in step: ask whether the invariant is a relation between two sets you can enumerate in code. If it is, write the assertion — then drive it RED before trusting it. Keep the grep for what cannot be enumerated. | +| 13 | **A handoff carries predictions as well as facts — and a prediction about state the handoff's own session later changed is the least reliable kind.** Learning #6 and FM #11 cover claims written *from memory*; #7, #10 and #12 cover cross-references that go stale in the *corpus*. Neither covers the forward-looking claim. A handoff that says "expect one merge conflict", "these are the files you will touch", or "this should be quick" describes a state that was true when it was written — and the single most common thing that invalidates it is the writing session's own later commits. Concrete case: the v3.6 ship session inherited "expect one `CHANGELOG.md` union conflict" and got **seven**, because the predecessor had committed two further layers to the branch *after* the point the fork had been ported from it, so five files the prediction never mentioned had diverged. A wrong prediction is worse than no prediction: it licenses the successor to resolve quickly rather than carefully, and a merge resolved on the belief that "only the ledger differs" discards code silently, with no failing check to catch it. | v3.6 ship session — a predecessor-predicted one-conflict merge that produced seven | Treat every forward-looking claim in a handoff as a hypothesis with an expiry date and re-derive it from current state before acting on it. For a merge specifically, diff the branch against the commit the other side was **ported from**, not against its tip. And when you *write* a prediction, name the state it depends on, so your successor can tell at a glance whether it still holds. | ---