Skill Roast

A brutally useful quality checker for AI Agent Skills.

Skill Roast analyzes a SKILL.md, AGENTS.md, or system prompt directly in the browser and produces a shareable 0–100 report card.

The web app is currently in private preview. The source, rubric, and v0.2 benchmark are public; the free public URL will open for the launch campaign.

Private preview · Public v0.2 benchmark · Report feedback

What it checks

• Clarity — Can an agent follow the instructions without guessing?
• Activation — Does the file explain when it should and should not be used?
• Safety — Are sensitive data, destructive actions, permissions, and verification addressed?
• Efficiency — Does every token earn its place?
• Portability — Can the workflow travel across agent environments?

Every report includes dimension scores, prioritized findings, concrete fixes, evidence lines when available, a shareable result URL, and a downloadable social card.

Privacy

Analysis runs entirely in the browser. Pasted instructions are not uploaded or stored.

Share URLs and prefilled feedback issues contain only generated report data. They never include the original pasted instructions. Do not paste proprietary instructions into GitHub issues.

Supported inputs

• SKILL.md validates YAML frontmatter as well as the instruction body.
• AGENTS.md / prompt evaluates the instruction body without requiring Skill metadata.

The rubric is optimized for English instructions. It also recognizes common Chinese workflow and safety signals and uses a CJK-aware token estimate.

Public benchmark

The launch benchmark scores 20 public Skills from five repositories. Every file is pinned to a commit SHA, linked to its source and license status, and processed by the same v0.2 analyzer used in the browser.

Read the leaderboard and methodology, or reproduce it locally:

npm run benchmark

Scores are heuristic diagnostics, not certifications or judgments of the authors. v0.2 cannot inspect a Skill's referenced files, scripts, or assets.

Local development

Requirements: Node.js >=22.13.0.

npm install
npm run dev
npm test
npm run lint

How scoring works

Skill Roast v0.2 uses deterministic, testable heuristics. It parses frontmatter only from a valid opening YAML block, distinguishes ordinary product mentions from exclusive tool lock-in, and applies separate activation rules to Skills and general instruction files.

The score is a fast diagnostic. Always evaluate a Skill against real tasks before production use.

Feedback and roadmap

Use the issue forms for an incorrect score, a missed rule, a bug, or a focused feature request. The next rubric version will be prioritized from repeated false positives and false negatives, not raw feature votes.

• Multi-file Skill folder analysis (SKILL.md plus references and scripts)
• Expanded multilingual rubric coverage
• Exportable remediation checklist

Tech and license

React, vinext, Vite, and OpenAI Sites. No application backend or database is required.

Released under the MIT License. Third-party benchmark sources remain under their respective licenses and are not redistributed in this repository.