Skip to content

[DOCS] sweep: automation-mode.md contradicts SKILL.md on open_ids for --only / --min-severity runs #824

Description

@tronghieu

What type of documentation issue is this?

Outdated content

Documentation location

src/bmad_loop/data/skills/bmad-loop-sweep/automation-mode.md, the "Validation rules the orchestrator enforces" list (lines 54-59 at v0.12.0, unchanged on main).

What's the issue?

#651 added --only and --min-severity. SKILL.md was updated for them. automation-mode.md was not. The two files now give the triage agent opposite instructions for a subset sweep.

SKILL.md:

If the invocation carries --only DW-1,DW-2,..., those ids are the complete triage universe for this session. Read the full ledger, but verify and partition exactly those named open entries; do not add other raw-open entries to open_ids or any result category. The orchestrator has already applied the operator's named-subset or severity-floor selector and validates your result against this exact scope.

automation-mode.md:

Validation rules the orchestrator enforces (a violation fails the whole result and burns a retry):

  • open_ids must list exactly the ledger's status: open entries — the orchestrator parses the ledger itself and compares.
  • Every open id appears in exactly ONE of already_resolved / bundles / blocked / skip / decisions. No misses, no duplicates, no invented ids.

The engine agrees with SKILL.md. In sweep.py at v0.12.0, the run loop builds selected_ids from selection.selected and passes it to _cycle. That set reaches validate_triage as expected_open_ids. A subset sweep is validated against the selected ids, not against every open entry.

So this is not a runtime bug. The risk is the agent. automation-mode.md is the mandatory contract, and it says a violation burns a retry. An agent that obeys it on a --only run lists every open entry in open_ids. Validation then fails with "open_ids do not match the ledger's open entries", and the retry is burned.

An automated PR reviewer read the pair as unsatisfiable while reviewing a 0.12.0 upgrade. A triage agent can read it the same way.

Suggested improvement

Scope both bullets to the session's triage universe. For example:

  • open_ids must list exactly the open entries in this session's triage universe: every status: open entry, or only the named ids when the invocation carries --only. The orchestrator computes the same set and compares.
  • Every id in that universe appears in exactly ONE of already_resolved / bundles / blocked / skip / decisions. No misses, no duplicates, no invented ids.

bmad-loop Version (if applicable)

0.12.0

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions