Skip to content

Add NVIDIA SkillSpector as a complementary skill security scan - #38

Merged
criptogus merged 1 commit into
mainfrom
claude/keen-meitner-qDzRn
May 30, 2026
Merged

Add NVIDIA SkillSpector as a complementary skill security scan#38
criptogus merged 1 commit into
mainfrom
claude/keen-meitner-qDzRn

Conversation

@criptogus

Copy link
Copy Markdown
Owner

What

Layers NVIDIA SkillSpector on top of the existing audit:skills gate as an independent, advisory second opinion for every marketplace package.

audit:skills stays the curated, high-precision blocking gate. SkillSpector adds NVIDIA's broader catalogue of vulnerability patterns (prompt injection, data exfiltration, privilege escalation, supply-chain, excessive agency, MCP tool poisoning, …) plus AST/YARA behavioural detection — without changing existing behavior.

How

Marketplace packages are structured YAML, but SkillSpector expects a skill directory (a SKILL.md). The wrapper renders each package's LLM-readable fields into a throwaway SKILL.md, runs SkillSpector's static analysis, and aggregates the results.

  • scripts/scan-skillspector.mjs — renders → scans → aggregates risk scores/issues, emits merged SARIF, and skips gracefully (exit 0) when SkillSpector isn't installed, so local installs and unrelated CI stay green. --block makes it fail on any package at/above the risk threshold (default 50, SKILLSPECTOR_THRESHOLD to override).
  • .github/workflows/skillspector.yml — advisory CI scan of changed packages on PRs touching content/. Installs SkillSpector (pinned ref, Python 3.12 via uv), runs static-only (no LLM key needed), and uploads SARIF to the repo's Security tab. continue-on-error → does not block merges.
  • package.jsonnpm run scan:skillspector.
  • CONTRIBUTING.md / SECURITY.md — document the layered scanning model.

Validation

Tested locally end-to-end against real packages (skills/playbooks/souls/integrations):

  • Clean skills → score 0, exit 0.
  • Crafted malicious skill (credential read + curl | bash + injection) → score 100, DO_NOT_INSTALL, 6 findings, --block exits 1.
  • Tool-missing path → graceful skip (exit 0), or exit 2 with --require.
  • Merged SARIF 2.1.0 validates; eslint + prettier clean.

Notes / decisions for review

  • Advisory by default. I kept SkillSpector non-blocking so it augments rather than competes with the curated gate, and so external-tool/network flakiness can't block PRs. To make it blocking later: drop continue-on-error in the workflow and pass --block.
  • Static-only in CI (no provider key/secrets). Set SKILLSPECTOR_LLM=1 + a provider key to enable semantic analysis locally.

https://claude.ai/code/session_017yCVEShyDmhzgJXNVpMBoG


Generated by Claude Code

Layer NVIDIA SkillSpector on top of the existing audit:skills gate as an
independent, advisory second opinion for marketplace packages.

- scripts/scan-skillspector.mjs: renders each YAML package to a throwaway
  SKILL.md, runs SkillSpector static analysis, aggregates risk scores/issues,
  emits merged SARIF, and skips gracefully when the tool is not installed.
- .github/workflows/skillspector.yml: advisory CI scan of changed packages
  that uploads SARIF to the Security tab (non-blocking).
- package.json: add scan:skillspector script.
- Document the layered scanning model in CONTRIBUTING.md and SECURITY.md.
@criptogus
criptogus marked this pull request as ready for review May 30, 2026 21:08
@criptogus
criptogus merged commit 18e1b8f into main May 30, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants