Skip to content

fix(aidd-dev): a review appends a round and scores the whole plan - #977

Merged
blafourcade merged 1 commit into
nextfrom
fix/review-phase-scope
Oct 10, 2026
Merged

blafourcade merged 1 commit into
nextfrom
fix/review-phase-scope

Conversation

@blafourcade

@blafourcade blafourcade commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

🎯 What & why

One phase of a 7-phase plan, one diff, three models: 100% (6/6) on 7 blocks, 100% (6/6) on 1 block, 53% (16/34) where 16/34 is 47%, and boxes copied from the previous run. 05-review never said what a phase outside the diff becomes, how the percent is computed, or where a past round lives.

🛠️ How it works

A review is appended to, one section per round, and never rewritten. A round carries its date, its author from git config user.name, its diff, the axes that ran, a verdict and a score. Its Criteria list holds the plan's criteria, its Findings list the defects no criterion covers.

The score replaces the percent that had no definition: it counts over the plan's whole criteria total, in three numbers that sum to it — 1/6 met, 1 unmet, 4 out of the diff. One denominator, so a plan covered at a third can no longer read as finished. A phase the diff leaves untouched gets one Out of the diff: line instead of a block of empty boxes.

The skill now follows the authoring contract aidd-context:04-skill-generate applies to itself: the router holds the flow, the action table and one transversal rule, and nothing else. 01-prepare resolves the diff once for the three axes and opens the round; 05-finalize judges it and checks its shape. references/report-contract.md holds where the report lives, what diff is reviewed and how a round joins it. SKILL.md, the only always-on text, goes from 446 words to 189.

🧪 How to verify

node scripts/check-tests-leave-git-alone.js -- node --test 'scripts/__tests__/**/*.test.js'   # 580 pass, 0 fail
claude -p --plugin-dir "$PWD/plugins/aidd-dev" "Run the aidd-dev:05-review skill, functional axis only, on <plan.md>, for the diff HEAD~1...HEAD."
  • Guard: 26 assertions, 34 mutations, each reddening one assertion alone.
  • Volume: 55 runs over 11 fixtures, every fixture unanimous. A plan of 10 phases and 34 criteria whose diff touches 2 phases reads 3/34 met, 5 unmet, 26 out of the diff, 5 runs of 5.
  • Against next, same fixtures, same prompts, counting checked boxes against the truth: 6 wrong runs of 18 for next, 0 of 45 here. next both over-credits (6 boxes where 5 hold, crediting a stock: "untouched" label) and under-credits (a criterion evidenced by a context line, a not-applicable one).
  • Callers: aidd-orchestrator:01-sdlc, 00-async-dev and the ship-a-feature recipe name the skill and its findings, never a section or a percent. Nothing downstream reads the artifact's shape.

⚠️ Heads-up

  • The artifact changes shape. review.md becomes a log of rounds. That is not what fix(aidd-dev): review leaves out-of-diff phases and the verified percent undefined #858 asked for, and the issue names "the verified percent" that the Score line replaces — to settle before merge.
  • Four rules come from the volume campaign, each turning a split between runs into unanimity: a context line of the diff is evidence; a criterion whose subject is absent is a gap, not not-applicable; a phase projecting no file is traced, not out of the diff; a label echoing the criterion's words is not evidence. They cost 03-review-functional 97 words, and they are the only words in it whose effect is measured.
  • The skill is 1983 words against 1785 on next. The always-on router shrank by 58%; the total grew because 01-prepare and 05-finalize own work that was implicit, and because the contract requires a | Case | Pass | table per action.
  • Cross-model is unresolved. On haiku, both designs produce a report about half the time and neither is clearly better on the landed runs. haiku did expose a real defect, now fixed: nothing in the skill named review.md any more, and a reference named review-report.md sat one hyphen away, so haiku wrote its report there.
  • references/review-rubric.md keeps 🟡 warning in a scale whose neighbours are critical and minor. major would be consistent; it touches 04-audit, 03-assert and 07-refactor, so it belongs to its own change.

🔗 Linked issue

Refs #858

✅ I certify

  • I DO CERTIFY I READ EACH LINE OF THE PULL REQUEST BECAUSE I AM A SOFTWARE ENGINEER, NOT A AI PUPPY.

🤖 Generated with Claude Code

https://claude.ai/code/session_01VE16AxC1V9wAMGmGgefe4z

@blafourcade
blafourcade force-pushed the fix/review-phase-scope branch from d639634 to 72ef2e1 Compare October 8, 2026 04:19
@blafourcade blafourcade changed the title fix(aidd-dev): a review says what happens outside the diff and how the percent is computed fix(aidd-dev): a review keeps its rounds, its phase scope and a computable percent Oct 8, 2026
@blafourcade
blafourcade force-pushed the fix/review-phase-scope branch 2 times, most recently from c9c182f to 5157028 Compare October 9, 2026 08:25
@blafourcade blafourcade changed the title fix(aidd-dev): a review keeps its rounds, its phase scope and a computable percent fix(aidd-dev): a review appends a round and scores the whole plan Oct 9, 2026
@blafourcade
blafourcade force-pushed the fix/review-phase-scope branch 4 times, most recently from 1f054fa to 8f3ebda Compare October 10, 2026 12:12
One phase of a 7-phase plan, one diff, three models: `100% (6/6)` on 7 blocks,
`100% (6/6)` on 1 block, `53% (16/34)` where 16/34 is 47%, and boxes copied from
the previous run. The skill never said what a phase outside the diff becomes, how
the percent is computed, or where a past round lives.

A review now appends one section per round and is never rewritten. A round carries
its date, its author from `git config user.name`, its diff, the axes that ran, a
verdict and a score; its `Criteria` list holds the plan's criteria and its
`Findings` list the defects no criterion covers. The score counts over the plan's
whole criteria total, in three numbers that sum to it, so a plan covered at a
third can no longer read as finished.

The skill follows `skill-generate`'s authoring contract: the router holds the
flow, the five actions and two transversal rules, and nothing else. `01-prepare`
resolves the diff once for the three axes and opens the round; `05-finalize`
judges it and checks its shape. `references/report-contract.md` holds where the
report lives, what diff is reviewed and how a round joins it. An action says what
to decide; the template owns the names and the form, the validator the field set.
The skill reads 1583 words, against 1785 on `next`.

Severity reads `critical`, `major`, `minor`: one scale, where `warning` put a log
level between two gravities. Three files outside this skill still say `warning`.
A critical is the harm a merge would do to production, its users, their data or
what depends on it, never how strictly a rule is worded. Before, a broken
declared rule read critical in 7 of 12 runs and a crash in production blocked in
3 of 5; now 0 of 10 and 5 of 5, over 50 runs on ten defect fixtures with every
verdict unanimous.

Measured, each with the mutation that proves it: 31 assertions, 39 mutations, each
reddening one assertion alone. The assertions spell a path the way the documents
spell it, so Windows and macOS read the same tree. 71 real runs on the final
text: 50 of 50 functional runs over 10 fixtures scored right, including 10 phases
and 34 criteria reading `3/34 met, 5 unmet, 26 out of the diff`; 3 of 3 runs
without a plan read `not scored`; 33 of 33 planted defects found across the three
axes; no run patched a file. Against `next` on the same fixtures, counting checked
boxes against the truth: 6 wrong runs of 18.

Four rules come from those campaigns, each turning a split between runs into
unanimity: a context line of the diff is evidence, a criterion whose subject is
absent is a gap rather than `not-applicable`, a phase projecting no file is traced
rather than out of the diff, and a label echoing the criterion's words is not
evidence.

Refs #858

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VE16AxC1V9wAMGmGgefe4z
AIDD-Session-Id: bf713ca4-3c1b-491d-ac0a-4d4533330b5c
@blafourcade
blafourcade force-pushed the fix/review-phase-scope branch from 8f3ebda to 390fa30 Compare October 10, 2026 18:45
@blafourcade
blafourcade marked this pull request as ready for review October 10, 2026 21:36
@blafourcade
blafourcade requested a review from a team as a code owner October 10, 2026 21:36
@blafourcade
blafourcade merged commit b25ae36 into next Oct 10, 2026
38 of 40 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants