[agent-experience] Remove redundancy and fold back eval iteration 7 - #16
Open
spoenemann wants to merge 3 commits into
Open
spoenemann wants to merge 3 commits into
spoenemann wants to merge 3 commits into
Conversation
… templates Give every rule one owner file: SKILL.md stays the spine (goal, load pointer, done-when per phase) and now defines affordances and variety reduction once; audit-playbook.md owns the Phase 6 proof protocol and claim check; docs-structure.md owns routing, triggers, and lifecycle; the asset templates own artifact shape; agents-md.md owns the evidence numbers; the check_docs.py docstring owns the list of checks. Fixes copies that had already drifted (100- vs 150-line map, four- vs five-part self-correction messages, an inventory list missing the docs site, a design-docs row missing two triggers, a missing design-docs pointer in the AGENTS.md template) and trims repeated reinforcement to a rule at each decision point plus one rationale. Verified by a sentence-level diff (every removed unit traces to a surviving owner), check_docs.py runs, and a pointer-resolution check; evals.json reviewed, no assertion depends on the changed wording. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Skill: revert the greenfield day-one build order to its explicit list (the pointer to the remediation ordering had widened day-one scope to CI); restore the routing symptom words (bug-vs-intended, a promise that keeps being broken) in the consult bullet and the ADR template header; make the Phase 6 litter check mechanical (a tree-wide search for caches comes back empty) and generalize the playbook's cache example; name the agent tools in play at Phase 0 so Phase 5 projects to that list. Evals: README-overlap budgets count sentences, not wrapped lines; parameterised commands count as executed once one instantiation ran; consistency assertions name config files and code comments as in scope; the bundled promise assertion is enumerated into clauses a-c. Tags and the assertion count (51) are unchanged. Items 2, 5 and 6 of the report are held as post-hoc fits to single runs; 3 kept as is; 4, 10 and the proposed additions batched for the next suite revision. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Fold-backs from eval reports, field runs, and PR reviews have produced the same three faults repeatedly: a rule restated in several skill files until the copies drift, new content that contradicts the skill's existing structure or vocabulary, and one project's specifics encoded as if they were the general rule. AGENTS.md now fences each with one incident-cited convention: one owner per rule, term, and number (widening the former jargon-once line); new content fits the skill it lands in; a finding enters as the general rule it instances, with ungeneralizable findings held and listed as held in the commit message. The definition of done gains a grep-and-re-read check for redundancy and consistency, and the layout line states that SKILL.md carries the procedure and points to the reference owning each detail. Verified: check_docs passes on the final state; every SKILL.md names every reference it ships; a duplicate-sentence scan across the seven skills finds one two-owner case (idiomatic-go's goroutine exit rule), left for a separate change. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Removes redundancy from the agent-experience skill so every rule has one owner file, then folds back the findings of eval iteration 7 (dedupe branch vs. main on Sonnet: 44/51 vs. 45/51, within the noise band).
Changes to the skill
audit-playbook.mdonly. The always-loaded file shrinks from 27.7 KB to 24.8 KB.docs-structure.mdowns triggers and lifecycle. Template headers are short pointers, not copies of the rubric.agents-md.md; the check list only incheck_docs.py's docstring. No more diverging enumerations.Eval spec
evals.json: README-overlap budgets count sentences instead of wrapped lines, parameterised commands count as executed once one instantiation ran, consistency assertions name config files and code comments as in scope, and the bundled promise assertion is enumerated into clauses. Tags and the assertion count (51) are unchanged.Verification
Sentence-level diff of the originals (every removed unit traces to a surviving owner),
check_docs.pyrepo-wide run passes, pointer resolution checked, eval iteration 7 report inskills/agent-experience-workspace/iteration-7/.🤖 Generated with Claude Code