fix(babysit): name the PR by URL in the guidance, since that is what gates the loop

[chenmingwei23]

Problem / Motivation

A monitor loop is observation-gated only when its instruction names a pull request
by explicit public URL. Inference deliberately refuses a bare PR #123 and the
owner/name#123 shorthand, because a loop's own subject and a blocker it merely
mentions cannot be told apart, and gating on the wrong one retires a loop whose
real work is unfinished.

Both places that teach an agent to draft that instruction showed the subject as a
bare number:

• src/kiro_crew/config/prompt.md -- the monitor_start guidance injected into
  every agent's prompt: Check PR #123 for new CI results ...
• src/kiro_crew/builtin_skills/kirocrew-dev/babysit/SKILL.md -- the decision
  table said only "name the pull request in the instruction", and the worked
  example armed message="Check PR #247. ..."

The tool description already said to use a full URL, but the tool description is
read when calling the tool; the examples are what get copied when composing the
message. So agents wrote the form that refuses gating.

Why it matters

Measured on a live gateway with the feature merged: of six loops that had asked
to be gated, five had written a bare number and were running on the plain timer,
spending a turn every interval. Only the one loop whose instruction carried a URL
was actually gated -- ten quiet ticks, zero wakes charged to the fallback path,
zero gate fallbacks.

The mechanism worked exactly as designed while the saving read as one sixth of
what was available. That is the same failure mode this line of work exists to
prevent: a correct feature whose adoption is quietly zero. Here the counters
surfaced it, which is the only reason it was found.

What changed

Guidance only -- no behaviour change, no production code touched.

1. config/prompt.md: the example names the subject by URL, and one clause says
   why -- the URL is what makes the loop gated, a bare PR #123 leaves it on the
   plain timer, and the user will usually say "babysit PR revert: remove project agents feature (backend + frontend) #123" so you write the
   URL.
2. babysit/SKILL.md: the decision-table rule now names the required form
   instead of saying "name the pull request", and the worked example arms a URL
   while keeping the user's realistic bare-number request above it -- the point
   being the translation from what the user says to what you arm.
3. test/test_babysit_guidance_gates.py: a ratchet that extracts the URLs from
   the shipped text and asserts probes.targets.infer accepts them, so a reword
   back into a non-gating form fails instead of silently costing turns. A third
   test guards the guard: it pins that a bare reference is still refused, so the
   other two cannot start passing vacuously if inference ever loosened.

One detail worth recording because it nearly shipped: the placeholder style
matters. https://github.com/<owner>/<repo>/pull/123 does NOT infer -- the angle
brackets are not valid in an owner or repo segment -- so an example written that
way is as ungated as the bare number it replaced. The samples use bracket-free
placeholders (owner/repo), which infer, and the ratchet pins that too.

Tests

• test/test_babysit_guidance_gates.py -- 3 new assertions, each
  mutation-verified: reverting prompt.md's example to PR #123 reddens the
  first, reverting the skill example to PR #247 reddens the second, and both
  return green when restored.
• 286 passed across test_babysit_guidance_gates.py, test_acp_prompt_blocks.py,
  test_builtin_skill_packaging.py, test_builtin_skill_scope.py,
  test_builtin_skill_sync_safety.py, test_babysit_pr_watch.py and
  test_probe_targets.py.
• flake8, black --check and isort --check-only clean on the new file.

Manual verification

Ran probes.targets.infer against the real strings rather than reasoning about
the regex: the old prompt example is refused, the new one selects
github.com, the babysit example still gates with a bare 247 appearing later
in the same message (same number, so no ambiguity refusal), a URL alongside a
DIFFERENT bare PR #99 is refused as designed, and one of the five live loop
instructions is refused verbatim.

Related Issues

Refs #7634, which shipped the gate this guidance failed to demonstrate.

Pattern harvest

Rule candidate: when a feature's behaviour depends on how an instruction is
WORDED, the guidance that teaches the wording must be asserted against the real
predicate, not reviewed by eye -- and the EXAMPLE is what gets copied, so fixing
the prose while leaving a wrong example fixes nothing.

An example is an interface, and it outranks the prose beside it. The
monitor_start tool description already required a full URL. It lost anyway,
because the agent reads the tool description when CALLING the tool and copies the
EXAMPLE when composing the argument. Five of six live loops copied the example.
When a correct rule and a wrong example disagree, assume the example wins and fix
the example.

When behaviour depends on how a string is worded, test the shipped words
against the real predicate. Reviewing guidance by eye is what let this survive:
Check PR #123 looks like it names a pull request, and a human reader cannot see
that inference refuses it. The new test does not assert a spelling; it runs
probes.targets.infer over the text that ships, so the guard tracks the predicate
even if the predicate changes. The third test guards the guard by pinning that a
bare reference is still refused -- without it, loosening inference would make the
other two pass vacuously.

A placeholder can fail the predicate it is illustrating.
https://github.com/<owner>/<repo>/pull/123 reads as a perfectly good generic
URL and does not infer, because angle brackets are not valid in an owner or repo
segment. A placeholder that cannot survive being copied is the same defect as the
bare number, one level more subtle. Bracket-free placeholders (owner/repo)
both read as generic and satisfy the predicate.

A feature can be correct, adopted, and still saving nothing. The gate worked
exactly as designed from the day it merged; what was missing was that anything
reached it. The counters shipped in #7634 are the only reason the one-in-six ratio
was visible at all -- without a number that distinguishes "gated and quiet" from
"never gated", this would have read as a feature that simply did not help.

[github-actions]

Design Review (Fable 5) — ✅ PASS

Design-level review of a7dc0a116ffb6836ce5923b56478faca6b824564 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: PASS

A measured adoption failure fixed at its true cause — the copied example — with a ratchet asserting the shipped text against the real predicate, not a spelling.

The alternative — loosening probes.targets.infer to accept bare PR #123 — is correctly rejected: the ambiguity rationale in infer's docstring (own subject vs. mentioned blocker) is load-bearing, and the third test deliberately pins that refusal. Guidance-only, no production surface, fully reversible.

[DESIGN-REVIEWED] a7dc0a1

[github-actions]

Opus 4.8 Review — ✅ no blocking findings

Reviewed a7dc0a116ffb6836ce5923b56478faca6b824564 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] a7dc0a1

Verdict parsed from the review's SHA-scoped output markers for commit a7dc0a116ffb6836ce5923b56478faca6b824564.

False positive or not applicable? A repository writer can comment:
/ai-review override fable a7dc0a116ffb6836ce5923b56478faca6b824564: <one-sentence reason>

[github-actions]

First Principles Review (Fable 5) — 🟡 CONCERNS

Premise-level review of a7dc0a116ffb6836ce5923b56478faca6b824564 — why this exists and whether the shipped surface is the smallest honest version. Updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

All evidence is gathered; here is the review.

First-Principles-Verdict: CONCERNS

The fix is aimed at the true cause — examples outrank prose — but a third copyable example still teaches the bare form, uncovered by the new ratchet.

What this change ships

Intent: make babysit loops actually observation-gated by fixing the examples agents copy when arming monitor_start. This is a FIX.

1. prompt.md monitor_start example names the subject by full PR URL, with the why — justified
2. Babysit decision table now says "by full URL" and names the refused forms — justified
3. Babysit worked example arms a URL; note explains the bare-number→URL translation — justified
4. New ratchet test runs the shipped guidance text through real infer — justified, declared
5. Guard-the-guard test pinning bare-number and angle-bracket refusal — declared; bare half already pinned elsewhere

Watch

• One counted unfixed sibling. The description says "Both places that teach an agent to draft that instruction showed the subject as a bare number," but grepping monitor_start( worked examples under src/kiro_crew/builtin_skills finds a third: prepare-pr/SKILL.md:360 arms message="Re-poll PR #<n> with pr_status.py …" — copied, that is exactly the bare form inference refuses, so every prepare-pr watch loop stays on the plain timer (plausibly among the five-of-six ungated loops the author measured). The fix is one URL in that example plus one more anchor in test_babysit_guidance_gates.py; if prepare-pr's loop is deliberately ungated, nothing in that skill says so.

Subtractions

• Drop the first assert of test_a_bare_number_is_still_refused_so_the_ratchet_means_something — bare-reference refusal is already pinned three times in test/test_probe_targets.py:86,159,160; keep only the angle-bracket-placeholder assert, which is pinned nowhere else.

[FIRST-PRINCIPLES-REVIEWED] a7dc0a1

[github-actions]

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of a7dc0a116ffb6836ce5923b56478faca6b824564 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] a7dc0a1

False positive or not applicable? A repository writer can comment:
/ai-review override gpt a7dc0a116ffb6836ce5923b56478faca6b824564: <one-sentence reason>

[chenmingwei23]

Everything on this PR that is mine is green: all five review lanes (GPT 5.6, Opus
4.8, Design, First Principles, CodeQL) and PR Hygiene.

The one remaining red, Dependency Audit / Audit Production Dependencies, is
REPO-WIDE and not caused by this change. Evidence:

• The failure is a tool timeout, not a finding:
  ERROR: production dependency audit failed closed: npm audit timed out after 120s for website/package-lock.json. It failed closed four times with that same
  message, so nothing was ever adjudicated about the lockfile.
• This PR's diff touches ZERO dependency files -- it is two markdown guidance
  files and one test.
• Every other open pull request sampled just now (feat(sandbox): give each agent scope an owning-instance slice #8351 through fix(dashboard): stop the PR panel re-reading unchanged pull requests #8356, six of
  them) carries the same failure on that gate.

I am deliberately NOT adding a .vulnerability-exceptions.json entry, which the
gate's own guidance offers. That file is for a real vulnerability awaiting
remediation; using it to cover a timeout would convert "we do not know" into a
pass, on a security gate, repo-wide. The honest state is that the audit could not
run, and that belongs with whoever owns the audit step -- either a longer timeout
or a registry that answers inside it.

Nothing else is outstanding. Not merging.

For the record on PR Hygiene, since it took me two tries: the gate wants a section
matching ^##+ *Pattern harvest AND a line matching
^ *(Rule candidate|Not generalizable) *:. The second is anchored at line start,
so a bolded **Rule candidate:** does not satisfy it -- the emphasis markers sit
where the anchor expects the word. Also, gh run rerun replays the original event
payload and therefore re-reads the OLD body, so a body fix only becomes visible to
that gate on a fresh edit.

[iamwhatever]

Tier 1 auto-approve: fix (3 files). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, SAST annotations clean, security checklist all-NO, AI reviewers green. Category: babysit guidance named the watched PR by number, but the gated monitor keys on the full PR URL, so the guidance is corrected to name the URL -- SKILL.md + prompt.md wording plus a test pinning the gate.