Skip to content

Learning #13 — a handoff's predictions decay - #63

Open
rmsharp wants to merge 1 commit into
KJ5HST:mainfrom
rmsharp:docs/learning-13-handoff-predictions
Open

Learning #13 — a handoff's predictions decay#63
rmsharp wants to merge 1 commit into
KJ5HST:mainfrom
rmsharp:docs/learning-13-handoff-predictions

Conversation

@rmsharp

@rmsharp rmsharp commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

What this is

One row appended to the starter-kit/SESSION_RUNNER.md Learnings table (was 1–12; appended, never renumbered). No other change to that file, and no other file touched except the ledger entry.

A handoff carries predictions as well as facts — and a prediction about state the handoff's own session later changed is the least reliable kind.

This is the Phase 3C output of the session that shipped v3.6 (PR #62). It is a docs-only addition: no principle, phase, gate, workstream or failure mode changes, and the FM count stays 27.

Why the existing rows don't cover it

Existing row What it covers
Learning #6, FM #11 claims written from memory
Learnings #7, #10, #12 cross-references that go stale in the corpus

Neither covers the forward-looking claim — a state description whose expiry is caused by the writing session's own subsequent commits. That is a distinct failure surface, and it bit this repo directly.

The motivating case is this repo's own v3.6 ship

The predecessor receipt told the ship session to "expect one CHANGELOG.md union conflict". git merge upstream/main produced seven.

The cause is worth stating precisely, because it generalizes: Layer 8 and the pre-PR review fixes were committed to the PR branch after the point the fork had been ported from it. So five files the prediction never named had diverged — both dashboard twins, the unit suite, CLAUDE.md and README.md. The prediction was true when written and false when used, and the session that wrote it is the session that invalidated it.

Why this is worth a table row rather than a gotcha: a wrong prediction is worse than no prediction. It licenses the successor to resolve quickly rather than carefully — and a merge resolved on the belief that "only the ledger differs" discards code silently, with no failing check to catch it. The countermeasure is mechanical and cheap: diff the branch against the commit the other side was ported from, not against its tip.

The row also asks the writer of a handoff to name the state a prediction depends on, so the reader can tell at a glance whether it still holds.

Cross-reference sweep (Learnings #7/#10, applied to this change)

Changing the size of a numbered set is exactly the trigger those rows describe, so the sweep was run before commit:

  • No live count-claim about the Learnings table exists anywhere in the corpus. The table was 1-11 / table is 1-12 tokens are confined to dated CHANGELOG.md entries and HANDOFFS.md receipts, which this repo leaves verbatim by design.
  • The table caption states no size.
  • Nothing else required updating.

Verification

  • bin/tests.sh84 passed, 0 failed
  • bin/check-links — OK, 82 relative links across 21 distributed markdown files
  • tools/test_methodology_dashboard.py197/197 OK
  • Learnings table parses as rows 1–13, every row 4-column

Notes

starter-kit/SESSION_RUNNER.md is bin/_manifest.py-distributed, so adopters receive this via bin/sync.

No new HANDOFFS.md receipt is opened here. This is Phase 3C of fork session S17, whose durable receipt already exists on the fork; opening a second receipt for one session would double-count it. The CHANGELOG.md entry records the action and points at that receipt.

Whether this warrants a version event is yours to call — it is a single distributed-doc row, and v3.6 was tagged the same day.

🤖 Generated with Claude Code

Appends one row to the Learnings table (was 1-12; append only, never
renumber — this file is bin/_manifest.py-distributed). A handoff carries
predictions as well as facts, and a prediction about state the handoff's
own session later changed is the least reliable kind.

The existing rows do not cover this: #6 and FM KJ5HST#11 are about claims
written from memory, and KJ5HST#7/KJ5HST#10/KJ5HST#12 are about cross-references that go
stale in the corpus. Neither addresses a forward-looking claim whose
expiry is caused by the writing session's own later commits.

Motivating case is this repo's own v3.6 ship. The predecessor receipt
said "expect one CHANGELOG.md union conflict"; the merge produced seven,
because Layer 8 and the pre-PR review fixes were committed to the branch
after the point the fork had been ported from it. Five files the
prediction never named had diverged, including both dashboard twins and
the unit suite.

A wrong prediction is worse than no prediction: it licenses the
successor to resolve quickly rather than carefully, and a merge resolved
on the belief that "only the ledger differs" discards code silently,
with no failing check to catch it.

Cross-reference sweep per Learnings KJ5HST#7/KJ5HST#10: no live count-claim about
the table exists in the corpus — the "table was 1-11"/"table is 1-12"
tokens are confined to dated CHANGELOG entries and HANDOFFS receipts,
left verbatim by design, and the caption states no size.

Phase 3C of fork session S17, whose durable receipt is the HANDOFFS.md
S17 block on the fork; no new receipt is opened here.

Verified: bin/tests.sh 84/84, bin/check-links OK, 197/197 unit,
Learnings table reads 1-13 with every row 4-column.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
rmsharp added a commit to rmsharp/methodology that referenced this pull request Jul 27, 2026
Completes the one close-out step a5dc925 left open, and records the PR
open as a non-commit action the ledger owes an entry for (FM KJ5HST#27).

The learning itself does not land here. starter-kit/SESSION_RUNNER.md is
bin/_manifest.py-distributed, so committing an unreleased change to it on
fork main would break the invariant this session had just verified and
reported — that fork and upstream differ only by fork-only
docs/planning/* plus this repo's own ledger and receipts. The branch was
cut from upstream/main (0 behind) per the standing rule for clean
single-topic upstream PRs.

The S17 receipt is amended in place rather than duplicated: this is the
same session's Phase 3C, not a new session, and a second receipt for one
session would double-count it in the ledger built to prevent that.

Verified: bin/check-handoff OK. On the branch: bin/tests.sh 84/84,
bin/check-links OK, 197/197 unit, Learnings table 1-13 all 4-column.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
rmsharp added a commit to rmsharp/methodology that referenced this pull request Jul 30, 2026
Ratified plan for recording which model(s) ran each session/action.
No dedicated field exists today in either CHANGELOG.md or HANDOFFS.md;
model info currently leaks out only incidentally (free-text prose on
capability-tiered sessions, or git commit co-author trailers).

Design panel (3 candidates, 3 judges x 4 lenses, synthesis, adversarial
completeness critique) surfaced the load-bearing finding: git commit
trailers can directly misattribute capability-tiered work. S1's own
receipt says Sonnet 5 built P2/P4, but all six of S1's checkpoint
commits — including those two — are trailer-tagged Opus 4.8. Verified
independently three times (design panel, synthesis, this session).

Ratified design (operator decisions D1-D4, all via AskUserQuestion):
optional **Model:** bullet in CHANGELOG.md's per-action entry (scales
to multi-tier for free, since the ledger is already per-layer) + a
formalized free-text convention in HANDOFFS.md for session-level
lookup + a SESSION_RUNNER.md Phase 3F propagation clause + a
canonical-only bin/model-report tool that treats git trailers as
disclaimed corroboration only, never authoritative. No change to
bin/check-handoff or REQUIRED_KEYS.

Two defects the completeness critic found in the synthesis are
corrected in place: a broken regression-check regex/file-scope claim,
and a mis-anchored insertion point in SESSION_RUNNER.md's Phase 3F
bullet. Two more numeric claims (RECOMMENDED_SKILLS.md brand-token
count, git trailer coverage count) were independently re-verified and
corrected before ratification.

Implementation (P1-P3) is a separate session per "1 and done"; P3 is
additionally gated on upstream PR KJ5HST#63 merging first, to avoid a
Learnings-table numbering collision.

Closes out S18: HANDOFFS.md receipt completed (self_score 9,
predecessor_score 9), CHANGELOG.md ledger entry recorded.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
rmsharp added a commit to rmsharp/methodology that referenced this pull request Jul 30, 2026
Implements Phase 1 of the ratified docs/planning/model-use-provenance-plan.md
(S18): an optional **Model:** bullet in starter-kit/CHANGELOG.md's format
section (single-tier + capability-tiered two-entry examples), a formalized
free-text model-naming convention in starter-kit/HANDOFFS.md's "How to write
a receipt" section, and a brand-neutral propagation clause appended to
starter-kit/SESSION_RUNNER.md's Phase 3F action-ledger bullet. No new
bin/check-handoff key, no hard gate anywhere in the design.

A 3-lens adversarial review before commit caught and fixed 3 real defects
the mechanical completion-criteria greps had missed: a direct
self-contradiction between the CHANGELOG.md and HANDOFFS.md edits on
whether to name the model in both places for a single-tier session, a
backwards "documented above" cross-reference, and a missing pair of
concrete worked examples the plan's own Phase 1 task called for.

Scoped to Phase 1 only per the plan's own "(one session each)" phasing.
Phase 2 (bin/model-report + Test 23) is the next session; Phase 3 stays
blocked on upstream PR KJ5HST#63 merging (confirmed OPEN at commit time).

Verified: bin/tests.sh 84/84, bin/check-links OK (82/21), bin/check-handoff
OK, all 4 plan-specified completion-criteria greps pass, brand-neutrality
regression check empty.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
rmsharp added a commit to rmsharp/methodology that referenced this pull request Jul 30, 2026
… 23 (S20)

Implements Phase 2 of the ratified docs/planning/model-use-provenance-plan.md
(S18/S19): a canonical-only bin/model-report tool reading three sources —
CHANGELOG.md **Model:** bullets (primary/structured), HANDOFFS.md free-text
"model" mentions (secondary/best-effort, regex-fuzzy), and git Co-Authored-By
trailers (corroboration-only, never authoritative, hard-disclaimer citing
S1's real trailer-vs-prose mismatch) — kept visually/structurally separate,
never merged. bin/tests.sh gains Test 23 (8 assertions), written RED-first
per Learning KJ5HST#12: a deliberately naive single-merged-list draft was run and
confirmed to fail before the real implementation replaced it.

A 4-lens adversarial-review workflow before commit caught and fixed 5 real
defects: (1) HIGH — the CHANGELOG parser had no code-fence awareness, so it
fabricated pseudo-entries out of starter-kit/CHANGELOG.md's own permanent
illustrative examples, a live default-path exposure for every future adopter
since that file is SEED-copied verbatim; (2) Test 23 originally always
passed --no-git, never exercising the trailer source; (3) the module
docstring misattributed S1's tier-split sentence to "prose" when it lives in
the fenced what_was_done field, contradicting the tool's own live output;
(4) a garbled section-reference typo; (5) a completeness sweep found the
README repo-structure tree and both starter-kit files' model-naming
sections didn't yet mention the new tool. Also fixed a **Model:** bullet
continuation-line truncation bug found during the same pass.

Scoped to Phase 2 only, per the plan's own "(one session each)" phasing.
Phase 3 (new Learnings row) stays blocked on upstream PR KJ5HST#63 merging
(confirmed still OPEN at commit time).

Verified: bin/tests.sh 92/92, bin/check-links OK (82/21), bin/check-handoff
OK, brand-neutrality regression check empty, python3 bin/model-report runs
clean against this fork's real 20-session history.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant