Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 86 additions & 0 deletions .github/workflows/skillspector.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
name: SkillSpector scan

# Complementary, ADVISORY security scan for marketplace packages, powered by
# NVIDIA SkillSpector (https://github.com/NVIDIA/skillspector). It runs
# alongside — not instead of — the curated, blocking `audit:skills` gate in
# validate-content.yml. SkillSpector adds NVIDIA's broader catalogue of
# vulnerability patterns plus AST/YARA behavioural detection, and findings are
# surfaced in the repo's Security tab as a SARIF report.
#
# Findings here do NOT block merges (continue-on-error). To make it blocking,
# drop `continue-on-error` from the scan step and pass `--block` to the wrapper.

on:
pull_request:
paths:
- "content/skills/**"
- "content/playbooks/**"
- "content/souls/**"
- "content/integrations/**"
- "scripts/scan-skillspector.mjs"
- ".github/workflows/skillspector.yml"

permissions:
contents: read
security-events: write # required to upload SARIF to code scanning

# Pin SkillSpector to a known-good ref so a surprise upstream change can't break
# or silently weaken the gate. Bump deliberately.
env:
SKILLSPECTOR_REF: main

jobs:
scan:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0

- uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- name: Install JS dependencies
run: bun install --frozen-lockfile

- name: Install SkillSpector
run: |
python3 --version
git clone --depth 1 --branch "$SKILLSPECTOR_REF" https://github.com/NVIDIA/skillspector.git "$RUNNER_TEMP/skillspector"
pipx install uv || python3 -m pip install --user uv
uv tool install --python 3.12 "$RUNNER_TEMP/skillspector"
echo "$HOME/.local/bin" >> "$GITHUB_PATH"

- name: Determine changed packages
id: changed
run: |
base="${{ github.event.pull_request.base.sha }}"
files=$(git diff --name-only "$base" HEAD -- \
'content/skills/*.yaml' 'content/skills/*.yml' \
'content/playbooks/*.yaml' 'content/playbooks/*.yml' \
'content/souls/*.yaml' 'content/souls/*.yml' \
'content/integrations/*.yaml' 'content/integrations/*.yml' \
| grep -v '/_' || true)
{
echo "files<<EOF"
echo "$files"
echo "EOF"
} >> "$GITHUB_OUTPUT"
echo "Changed packages:"; echo "$files"

- name: Run SkillSpector (advisory)
if: steps.changed.outputs.files != ''
continue-on-error: true
env:
SKILLSPECTOR_LOG_LEVEL: ERROR
run: |
# Static analysis only — no LLM provider key needed in CI.
node scripts/scan-skillspector.mjs ${{ steps.changed.outputs.files }} \
--sarif skillspector.sarif

- name: Upload SARIF to code scanning
if: steps.changed.outputs.files != '' && hashFiles('skillspector.sarif') != ''
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: skillspector.sarif
category: skillspector
24 changes: 24 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,30 @@ non-allowlisted hosts, hardcoded keys, and obfuscated payloads. A package is
tag so its quoted example payloads are treated as data, not as attacks. Both
scripts run in CI on every PR that touches `content/`.

#### Optional second opinion — SkillSpector

```bash
npm run scan:skillspector # all packages
npm run scan:skillspector content/skills/your-skill.yaml # one package
```

`scan:skillspector` layers [NVIDIA SkillSpector](https://github.com/NVIDIA/skillspector)
on top of `audit:skills` as an independent, advisory scan. It renders each YAML
package into a throwaway `SKILL.md` and runs SkillSpector's broader catalogue of
vulnerability patterns (prompt injection, data exfiltration, privilege
escalation, supply-chain, excessive agency, MCP tool poisoning, …) plus AST/YARA
behavioural detection. It is **optional**: if SkillSpector is not installed the
command skips gracefully, and in CI its findings are reported in the repo's
Security tab without blocking merges. Install it once with:

```bash
git clone https://github.com/NVIDIA/skillspector && cd skillspector
uv tool install --python 3.12 .
```

Pass `--block` to fail on any package scoring at/above the risk threshold
(default 50; override with `SKILLSPECTOR_THRESHOLD`).

### Step 5 — Open a pull request

Use the **Package submission** PR template. Include:
Expand Down
18 changes: 18 additions & 0 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,4 +28,22 @@ Out of scope:
- Vulnerabilities in upstream dependencies already tracked by their maintainers
- Denial-of-service via unrealistic input volume

## How packages are scanned

Every package published to the marketplace passes through layered scanning
before it can be synced or released:

1. **`validate:content`** — schema, slug uniqueness, file naming, example count.
2. **`audit:skills`** (blocking gate) — a high-precision, schema-aware scan for
prompt-injection / jailbreak signals (shared with the runtime guard) and
malicious "functions" embedded in instructions (RCE, credential exfiltration,
reverse shells, beacons, hardcoded keys, obfuscated payloads). Runs in CI on
every PR touching `content/`.
3. **`scan:skillspector`** (advisory) — an independent second opinion from
[NVIDIA SkillSpector](https://github.com/NVIDIA/skillspector). Each package
is rendered to a `SKILL.md` and scanned against NVIDIA's broader catalogue of
vulnerability patterns plus AST/YARA behavioural detection. Findings are
uploaded to the repo's Security tab as SARIF; they do not block merges by
default. See `CONTRIBUTING.md` for setup.

Thank you for helping keep the ecosystem safe.
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
"format": "prettier --write .",
"validate:content": "node --experimental-strip-types scripts/validate-content.mjs",
"audit:skills": "node --experimental-strip-types scripts/audit-skills.mjs",
"scan:skillspector": "node scripts/scan-skillspector.mjs",
"sync:content": "node scripts/sync-content-to-registry.mjs",
"sync:adversarial": "node scripts/sync-adversarial-cases.mjs",
"eval:adversarial": "node scripts/eval-adversarial.mjs",
Expand Down
Loading
Loading