Repository navigation
feat(nvidia-skill-finder): use the Hermes built-in catalog under Hermes - #820
Conversation
156190d to
9109ad1
Compare
|
/nvskills-ci |
|
Hey Jason, nice work on this. I tested it on Hermes 0.22.7 and the flow works well: it picked up the skill on its own, used the Hermes search instead of npx, and gave the uppercase NVIDIA/ install commands. The casing thing checks out too: lowercase really does come back as community trust. One thing I hit: Small one: maybe mention that Happy to approve once that's in and re-signed. |
…ult 25 hermes skills search returns 25 results by default, which silently truncates the broad single-keyword queries this guidance asks for: jetson matches 39 skills, so jetson-quick-start was cut off and never recommended. Pass --limit 100 on every search. Also note that --source nvidia requires Hermes 0.22 or newer, and give the fallback for older installs. Reported by teknium1 on #820. Signed-off-by: Jason Dudash <jdudash@nvidia.com>
|
Thanks Jason, the --limit fix works. I re-ran it on Hermes 0.22.7: |
The finder told every agent to run `npx skills add nvidia/skills --list` for catalog lookup and `npx skills add ... --global --yes` for install. On Hermes that bypasses the built-in catalog and the skill security scan, and `--global` writes the default profile's skills directory regardless of the active profile. Branch both the catalog lookup and the install on the agent the skill is running in. Under Hermes use `hermes skills search <query> --source nvidia` and `hermes skills install NVIDIA/skills/<skill-name>`; every other agent keeps the existing `npx` path. Identifier casing is significant -- `NVIDIA/...` resolves to the trusted first-party source while `nvidia/...` resolves to a community-trust mirror -- so the skill is told to copy the identifier from search output verbatim. Hermes search matches every word in the query, so note that one or two keywords work where a natural-language phrase returns nothing. Also update the plugin README, which still said Hermes support was coming soon, and release the plugin as 1.5.0. Signed-off-by: Jason Dudash <jdudash@nvidia.com>
`hermes skills search` requires every word in the query to match, so a descriptive phrase usually returns nothing: `jetson pinmux` returns no rows even though `jetson-customize-pinmux` exists, and `OpenUSD` does not return `omniverse-usd-performance-tuning`. Across ten realistic lookups only six found the right skill. That is a real regression against the `npx` path, where the model sees all 431 names and descriptions and does the matching itself. Searching one broad product, family, or lane keyword at a time recovers every one of those misses, so require single-keyword queries and tell the model to run two or three of them and choose from the full descriptions `--json` returns rather than trusting the first row. Add a full-catalog fallback for when those searches still come up empty, so a narrow search term cannot hide a skill that exists. `hermes skills browse` is not used for this: it truncates both names and descriptions and costs more context than the catalog listing it would replace. Signed-off-by: Jason Dudash <jdudash@nvidia.com>
…ence The Hermes branch had grown to roughly 50 inline lines, most of it recovery detail for the substring-match behaviour of hermes skills search. Keep the correctness-critical parts inline -- the two commands, the NVIDIA/ vs nvidia/ casing rule, and the guard against running hermes outside Hermes -- and move the recall tactics, full-catalog fallback, and identifier confirmation to references/hermes-catalog.md, loaded only when searches come up empty. This follows the progressive-disclosure pattern the skill already uses for references/taxonomy-routing.md. Signed-off-by: Jason Dudash <jdudash@nvidia.com>
…ermes Put the npx commands back first and unbroken so the common path reads as the default, and reduce the Hermes branch to one bullet in each of the lookup and install sections pointing at references/hermes-catalog.md. The reference now carries the whole Hermes flow rather than only the recovery tactics. Reorders the plugin README the same way. Signed-off-by: Jason Dudash <jdudash@nvidia.com>
Keep 'When No Skills are Found' exactly as it was on main; the Hermes variant of the ecosystem search does not need to be called out there. Signed-off-by: Jason Dudash <jdudash@nvidia.com>
generate_versions.py tracks the last commit touching each skill directory, so editing the skill leaves versions.json stale. Signed-off-by: Jason Dudash <jdudash@nvidia.com>
Signed-off-by: nvskills-svc-account <svc-nvskills-signing@nvidia.com>
…ult 25 hermes skills search returns 25 results by default, which silently truncates the broad single-keyword queries this guidance asks for: jetson matches 39 skills, so jetson-quick-start was cut off and never recommended. Pass --limit 100 on every search. Also note that --source nvidia requires Hermes 0.22 or newer, and give the fallback for older installs. Reported by teknium1 on #820. Signed-off-by: Jason Dudash <jdudash@nvidia.com>
Signed-off-by: Jason Dudash <jdudash@nvidia.com>
Signed-off-by: Jason Dudash <jdudash@nvidia.com>
Signed-off-by: Jason Dudash <jdudash@nvidia.com>
Signed-off-by: Jason Dudash <jdudash@nvidia.com>
… searching Hermes search matches the query as a plain case-insensitive substring over each entry's joined fields, so it misses phrases and truncates broad keywords at the default limit of 25. Working around that took ~8 searches per question, which cost more context than reading the whole catalog and could still miss the right skill. Read the 437-entry catalog listing once and choose from it, then confirm the identifier with a single narrow search before installing through Hermes, which keeps the security scan and profile awareness. Across the 22-question product sweep this cut searches from 176 to 21 and transcripts from 250KB to 46KB at equal accuracy. Signed-off-by: Jason Dudash <jdudash@nvidia.com>
Signed-off-by: Jason Dudash <jdudash@nvidia.com>
645d745 to
f73b929
Compare
|
Thanks Moshe. Your What your finding led to. The 25-result cap was a symptom. Hermes search matches the query as a plain case-insensitive substring over each entry's joined fields, so it also misses phrases entirely —
Searching was the most expensive option and the only incomplete one. Since the whole point of the Hermes branch was avoiding npx's context cost, that undercut the rationale for it. What it does now. Read the 437-entry catalog listing once and pick from it, then confirm the chosen skill's identifier with one narrow search before installing through Hermes: curl -sL .../metadata.json | jq -r '.skills[] | "\(.name): \(.description[0:110])"'
hermes skills search <skill-name> --source nvidia --json --limit 100
hermes skills install <identifier>Install still goes through Hermes, so the security scan and profile awareness are unchanged. Your Results — same 22 product questions, forced activation, Hermes 0.22.5 on a clean box:
Orin Nano / Caveats:
Ask: could you independently verify this before approving? My numbers come from my own harness and my own choice of 22 questions, so they're worth confirming against a different setup. Most useful checks would be your Orin Nano prompt that previously missed |
|
/nvskills-ci |
Signed-off-by: nvskills-svc-account <svc-nvskills-signing@nvidia.com>
The python3 -c one-liner was broken two ways. Backslash escapes inside an f-string expression are only legal from Python 3.12, so it raised SyntaxError on macOS's system 3.9 and Ubuntu 22.04's 3.10. Separately, Hermes's approval layer classifies inline interpreter scripts as "script execution via -e/-c flag" and refuses to run them at all. Use python3 -m json.tool piped through grep and cut instead, which works unchanged on 3.9 and clears the same approval check, and tell the model not to reach for python3 -c. Thanks to @mosheabr for catching the version incompatibility in review. Signed-off-by: Jason Dudash <jdudash@nvidia.com>
|
Thanks Moshe — per your Slack thread on Python version bug. It fails on every Python version, not just 3.11 and older. You're right about the PEP 701 rule, but this particular one-liner never worked anywhere. Because the snippet is wrapped in shell single quotes, the So the fallback was dead on arrival. Thanks for pulling the thread. There was a second problem with it too. Hermes's approval layer classifies inline interpreter scripts as "script execution via -e/-c flag" and refuses to run them, so even a correctly-quoted curl -sL https://raw.githubusercontent.com/NVIDIA/skills/main/.github/scripts/marketplace/metadata.json \
| python3 -m json.tool | grep -E '"(name|description)":' | cut -c1-130Verified byte-identical output (898 lines) on macOS's system 3.9.6 and on 3.14.7. Please don't take my word for the behaviour claims — the approval-layer block and the catalog-pull flow are both worth an independent run on your side before this merges. On the red /nvskills-ci |
|
/nvskills-ci |
Signed-off-by: nvskills-svc-account <svc-nvskills-signing@nvidia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Moshe Abramovitch <moshea@nvidia.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Moshe Abramovitch <moshea@nvidia.com>
…ult 25 hermes skills search returns 25 results by default, which silently truncates the broad single-keyword queries this guidance asks for: jetson matches 39 skills, so jetson-quick-start was cut off and never recommended. Pass --limit 100 on every search. Also note that --source nvidia requires Hermes 0.22 or newer, and give the fallback for older installs. Reported by teknium1 on #820. Signed-off-by: Jason Dudash <jdudash@nvidia.com>
NVIDIA/skills#820 landed on main, so repin to the commit that attaches the NVSkills signatures for it. The plugin at the old pin still reached for a python3 -c JSON fallback that Hermes's approval layer refuses as "script execution via -e/-c flag" (and that was a SyntaxError on Python 3.11 and older); the new pin replaces it with a json.tool pipeline. Also bumps the cosmetic version label to match plugin.json (1.5.0) and moves the banner URL to the same commit. Signed-off-by: Jason Dudash <jdudash@nvidia.com>
NVIDIA/skills#820 landed on main, so repin to the commit that attaches the NVSkills signatures for it. The plugin at the old pin still reached for a python3 -c JSON fallback that Hermes's approval layer refuses as "script execution via -e/-c flag" (and that was a SyntaxError on Python 3.11 and older); the new pin replaces it with a json.tool pipeline. Also bumps the cosmetic version label to match plugin.json (1.5.0) and moves the banner URL to the same commit. Signed-off-by: Jason Dudash <jdudash@nvidia.com>
Summary
Teaches
nvidia-skill-finderto use Hermes Agent's built-in skills catalog instead ofnpx, and bumpsnvidia-skillsto 1.5.0. This addresses review feedback on NousResearch/hermes-agent#146865, which flagged that the skill runs an unpinnednpxcommand and installs with--global --yesin a way that bypasses Hermes's security scan and is not profile-aware.What changed
SKILL.mdkeeps thenpxpath first and unbroken as the default, with Hermes as a single pointer in each of the lookup and install sections.references/hermes-catalog.md(new) carries the whole Hermes flow: search, the single-keyword strategy, the full-catalog fallback, install, and identifier rules. This follows the progressive-disclosure pattern the skill already uses forreferences/taxonomy-routing.md.plugins/nvidia-skills/README.mddrops "Hermes support is coming soon" and documents what the finder runs per harness.1.4.1→1.5.0.Two findings worth flagging
Identifier casing determines trust.
hermes skills install nvidia/skills/<name>(lowercase) resolves to a community-trust skills.sh mirror with a placeholder description, whileNVIDIA/skills/<name>resolves to the trusted first-party source. Segment count does not matter —NVIDIA/skills/<name>andNVIDIA/skills/skills/<name>both resolve to the same trusted skill with the same scan hash. The skill now specifies the uppercase form and tells the model to copy theIdentifierfrom search output verbatim.hermes skills searchmatches substrings, not tokens.Testing
See comments below for updated testing details