guidelines: four lessons from auditing a production agent fleet

[pallaoro]

Four additions to the guidelines, all from a single debugging session on a live multi-agent system. Worth stating the common shape up front, because it's what makes them worth adding: a customer-visible failure turned out to be three separate instances of the same problem — a correct rule that existed, was read in the same session, and wasn't followed. None of them was a missing rule.

§4 — A test that has never failed is not a regression test

Writing the test after the fix is the normal order when the bug came from production, but a green test proves only that the implementation does what it does. Revert the fix, watch it fail with the signature you expect, restore.

In the session behind this, doing exactly that turned 7 passing tests into 5 failures — the headline one reproducing the incident's signature verbatim. Without that step I'd have shipped tests that asserted my own implementation.

§6 — The DRY test applies to rules, not just skills

An instruction duplicated across a prompt, a config file and a skill is three copies that drift, and the agent reading all three can't tell which is current. One banned construct in that system was specified in three places that disagreed with each other, including one that endorsed the opposite. When a rule is being violated, the first move is to count the copies — not to write a clearer one.

§6 — A rule that isn't enforced isn't a rule

If an instruction is written down, read in the very session that broke it, and still ignored, more prose won't change that. Either something else the agent reads contradicts it, or it needs a mechanical guard instead of a sentence.

§6 — Examples outrank rules; audit them against each other

The observed cause of the above. A skill banned a construct in a rule near the bottom of an 18KB file while demonstrating it 44 times in the examples above — including inside the "good examples" it told the agent to emulate. The examples won, because they're concrete. Hard rules go before examples, and every example gets checked against the rule it illustrates.

§7 — A working copy is a cache, not the source

Refresh before reasoning about repo state. A local clone goes stale silently, and faster when platform tooling commits on your behalf. I read a three-days-stale clone during that session and reported its contents as current — a confident, well-grounded answer about a file nobody had anymore.

---

Mechanics: CLAUDE.md is the source; ran npm run build to regenerate the six fanned-out rule files. npm run check-sync (what CI runs) passes, and all four additions verified present in each of the six copies.

Scope note: these are sub-rules inside existing principles, not new principles, so the README's "Seven Principles" table and detail sections stay accurate and are untouched. Say the word if you'd rather the detail sections mention the new material too.

[pallaoro]

Closing: this was a misread on my part. The ask was to update the installed plugin, not to propose guideline changes.