From da871939e692daefedd7127b407146fa004b94a8 Mon Sep 17 00:00:00 2001 From: Nic Crane Date: Fri, 16 Oct 2026 13:57:39 +0100 Subject: [PATCH] fix(issue-backlog-stats): expect marketplace-form actions in step-4 evals The step-4-recommend fixtures expected `top_action` as a bare skill name (`issue-triage`, `issue-stale-sweep`), but the skill's rules table emits the marketplace form (`/magpie-issue:triage`, `/magpie-issue:stale-sweep`) and says so explicitly, and the step's output spec calls the field a slash command string. The bare names are a leftover from #1195: its first commit de-stuttered the fixtures to bare names, and a later commit in the same PR put the slash form back into the stats dashboards because a bare name is not a pasteable command on any install path. The fixtures were not re-synced after that second commit, so the two cases failed deterministically against a live model. Generated-by: Claude Code (Fable 5.1) --- .../fixtures/case-1-high-untriaged/expected.json | 2 +- .../step-4-recommend/fixtures/case-2-high-stale/expected.json | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/skill-evals/evals/issue-backlog-stats/step-4-recommend/fixtures/case-1-high-untriaged/expected.json b/tools/skill-evals/evals/issue-backlog-stats/step-4-recommend/fixtures/case-1-high-untriaged/expected.json index 5ef68989b..a5ee42c7d 100644 --- a/tools/skill-evals/evals/issue-backlog-stats/step-4-recommend/fixtures/case-1-high-untriaged/expected.json +++ b/tools/skill-evals/evals/issue-backlog-stats/step-4-recommend/fixtures/case-1-high-untriaged/expected.json @@ -1 +1 @@ -{"fired_rules": ["R1", "R4"], "top_priority": "high", "top_action": "issue-triage"} +{"fired_rules": ["R1", "R4"], "top_priority": "high", "top_action": "/magpie-issue:triage"} diff --git a/tools/skill-evals/evals/issue-backlog-stats/step-4-recommend/fixtures/case-2-high-stale/expected.json b/tools/skill-evals/evals/issue-backlog-stats/step-4-recommend/fixtures/case-2-high-stale/expected.json index 89c67440f..b8ae9c8f4 100644 --- a/tools/skill-evals/evals/issue-backlog-stats/step-4-recommend/fixtures/case-2-high-stale/expected.json +++ b/tools/skill-evals/evals/issue-backlog-stats/step-4-recommend/fixtures/case-2-high-stale/expected.json @@ -1 +1 @@ -{"fired_rules": ["R2", "R5", "R6"], "top_priority": "high", "top_action": "issue-stale-sweep"} +{"fired_rules": ["R2", "R5", "R6"], "top_priority": "high", "top_action": "/magpie-issue:stale-sweep"}