Repository navigation
feat(setup): first-run welcome guide for the utilities family (#1663) - #1669
Conversation
4f2d089 to
da44b7d
Compare
|
Implemented first-run welcome guide for the utilities family (#1663) and eval fixtures. |
da44b7d to
db1b9c6
Compare
potiuk
left a comment
There was a problem hiding this comment.
Thanks — the structure, wiring and fixtures follow #1653 closely. Two of the five skills are described inaccurately in the guide, which matters for a first-run page that teaches users what to ask for; details inline. Please also clear the CodeQL alert (suggestion inline) and regenerate case-10-utilities-first-run/report.md after the guide text changes.
This review was drafted by an AI-assisted tool and
confirmed by an Apache Magpie maintainer. After you've
addressed the points above and pushed an update, an Apache Magpie
maintainer — a real person — will take the next look
at the PR. The findings cite the project's review criteria;
if you think one of them is mis-applied, please reply on the
PR and a maintainer will weigh in.More on how Apache Magpie handles maintainer review:
Contributing guide.
potiuk
left a comment
There was a problem hiding this comment.
Thanks for this. It follows the #1653 pattern closely and the fixture and test changes are solid. The inline comments are mostly about two places where the guide describes list-skills and skill-reconciler differently from what those skills do. Because the guide's example phrases route users to skills, "reconcile my local skills against the snapshot" would send them to the wrong one. Two smaller points: the new text-symlink fallback in welcome.py needs a test, and the stdout reconfigure change needs a reason or its own PR.
I've rebased the branch onto current main, so please pull before pushing again.
Guide accuracy (plugins/magpie-setup/skills/setup/setup_preflight/guides/utilities.md:10)
list-skills lists only the skills installed for this repository (its description: "Print a human-readable index of every skill installed for this repository…"). It doesn't read the marketplace.
1. 📋 **List** the skills installed here, by family — *"list skills"*
Guide accuracy (plugins/magpie-setup/skills/setup/setup_preflight/guides/utilities.md:14)
skill-reconciler doesn't reconcile a repo against its snapshot. Its description: "Compare two near-duplicate skills — typically an ASF variant and a non-ASF or multi-project variant — and classify every difference as ALLOWED, DRIFT, or SAFETY-BASELINE… read-only". Checking local configuration against the installed framework is /magpie-setup:setup reconcile.
5. 🔁 **Reconcile** two near-duplicate skills and classify their differences — *"reconcile skill-A and skill-B"*
Guide accuracy (plugins/magpie-setup/skills/setup/setup_preflight/guides/utilities.md:35)
Same issue as line 14: the guide's example phrases route users to skills, and "reconcile my local skills against the snapshot" would send them to skill-reconciler, which compares two skills with each other. An example that matches the skill is something like "reconcile the ASF and generic variants of the triage skill".
Smaller observations
- See inline comments on
plugins/magpie-setup/skills/setup/setup_preflight/guides/utilities.md:43,plugins/magpie-setup/skills/setup/setup_preflight/welcome.py:438,plugins/magpie-setup/skills/setup/setup_preflight/welcome.py:460.
This review was drafted by an AI-assisted tool and
confirmed by an Apache Magpie maintainer. After you've
addressed the points above and pushed an update, an Apache Magpie
maintainer — a real person — will take the next look
at the PR. The findings cite the project's review criteria;
if you think one of them is mis-applied, please reply on the
PR and a maintainer will weigh in.More on how Apache Magpie handles maintainer review:
Contributing guide.
potiuk
left a comment
There was a problem hiding this comment.
Thanks @Kaap10. To get this landed, I pushed 05b044d onto your branch with the points from my review applied:
- The guide now describes
list-skillsas listing the skills installed here, grouped by family, with no marketplace claim. The override example became "group the skill list by capability instead of family". skill-reconcileris now described as comparing two near-duplicate skills and classifying where they drifted. The "Just ask" example is now "have these two skill copies drifted?".- The empty
except OSError: passthat CodeQL flagged is nowcontextlib.suppress(OSError), with a comment explaining the text-file mirror fallback. - I re-rendered
case-10-utilities-first-run/report.md(render-preflight-fixtures.py --checkis clean) and rantest_welcome.py(48 passed).
The Windows fallback and the stdout.reconfigure change still have no test. A small follow-up PR covering them would be welcome. Approving and merging.
This review was drafted by an AI-assisted tool and
confirmed by an Apache Magpie maintainer. The maintainer
approving this PR has read the findings and signed off. If
something feels off, please reply on the PR and a maintainer
will follow up.More on how Apache Magpie handles maintainer review:
Contributing guide.
…#1663) Adds a concise first-run welcome guide for the utilities skill family (list-skills, write-skill, optimize-skill, report-framework-issue, and skill-reconciler), following the pattern established in apache#1653. - Adds plugins/magpie-setup/skills/setup/setup_preflight/guides/utilities.md - Highlights the typical utilities flow and notes zero configuration needed - Links the guide from docs/utilities/README.md under Install & first runs - Supports personal layer reconciled state in render-preflight-fixtures.py - Adds case-10-utilities-first-run preflight-floor fixture and updates case-9 - Expands test_welcome.py test coverage and adds cross-platform symlink robustness
… guide list-skills indexes installed skills only, and skill-reconciler compares two near-duplicate skills rather than a repo against its snapshot. Also replaces the empty `except OSError: pass` flagged by CodeQL with `contextlib.suppress`, and re-renders the case-10 fixture. Generated-by: Claude Opus 5
scan_families() now has a test for a mirror entry checked out as a plain file holding its target path, as git writes it without symlink support, including an entry whose target does not exist. The stdout.reconfigure() in main() is unrelated to the utilities guide and is removed from this change. Generated-by: Claude Opus 5
05b044d to
e2c9a14
Compare
|
Thanks @potiuk for polishing the skill descriptions and adding the plain-file mirror fallback test! |
Summary
utilitiesskill family (plugins/magpie-setup/skills/setup/setup_preflight/guides/utilities.md), following the template established in feat(setup): first-run welcome guides, general and for the security skills #1653.list-skills→write-skill→optimize-skill→report-framework-issue→skill-reconciler), explicitly highlighting that meta-skills require no initial project configuration to run.case-10-utilities-first-run, updatescase-9-first-magpie-run, links the guide indocs/utilities/README.md, and expands test coverage intest_welcome.py.Type of change
.claude/skills/<name>/) — eval fixtures updated belowtools/<system>/*.md)tools/*/withpyproject.toml)docs/,README.md,CONTRIBUTING.md)projects/_template/)prek, workflows, validators)Test plan
prek run --all-filespasses (markdownlint, mixed-line-ending, doctoc, add-license-headers)uv run pytest/ruff check/mypypasses(
PYTHONPATH=tools/skill-evals/src python3 -m skill_evals.runner tools/skill-evals/evals/<skill>/)(a regression test for the bug fixed / the behaviour added — see CONTRIBUTING.md)
python tools/dev/render-preflight-fixtures.py --checkpasses with zero drift;pytest tools/setup-preflight/tests/test_welcome.pypasses (49 tests).RFC-AI-0004 compliance
<PROJECT>,<tracker>,<upstream>,<security-list>) used in all skill / tool prose (thecheck-placeholdersprek hook is the mechanical gate)Linked issues
Closes #1663 (child of #1654)
Notes for reviewers (optional)
render-preflight-fixtures.pynow supports optionalreconciledstate instate.jsonto allow testing family-specific welcome guides where the general guide was previously seen.scan_families()inwelcome.pyfalls back to reading askills/mirror entry as a text file holding its target path. That is how git checks out a symlink whencore.symlinks=false, so mirrored skills resolve the same way on a Windows checkout as on Linux CI.test_a_mirror_entry_checked_out_as_a_text_file_still_resolvescovers it, including an entry whose target is missing.stdoutinwelcome.main()for Windows consoles. That change is unrelated to the guide, so it was dropped from this PR.list-skillsandskill-reconcilerdescriptions and examples in the guide, cleared the CodeQL empty-exceptalert, added the fallback test, and rebased ontomain.