Skip to content

feat(checks): add model-unknown warn rule for the model frontmatter field#28

Merged
protosphinx merged 1 commit into
mainfrom
bot/model-unknown-check
Jun 7, 2026
Merged

feat(checks): add model-unknown warn rule for the model frontmatter field#28
protosphinx merged 1 commit into
mainfrom
bot/model-unknown-check

Conversation

@protosphinx
Copy link
Copy Markdown
Member

Why

Skills can specify a model: field in their frontmatter to pin the Claude
model used at runtime. A typo there (claude-sonet-4-6 instead of
claude-sonnet-4-6) causes the skill to silently run on the wrong model or
fall back to a default, with no feedback from the model output. This is
exactly the class of silent bug skillcheck is designed to catch.

This PR adds a model-unknown warn rule (not error, because the model list
evolves with each Anthropic release) that validates the model: field
against a curated set of known Claude model IDs.

Also fixes a docs omission: the existing empty-body rule was not listed
in the README "What it checks" table.

What

  • New constant KNOWN_CLAUDE_MODELS in src/checks.ts covering Claude 3.x
    and 4.x series model IDs
  • New check function checkModelUnknown called in the per-skill loop inside
    runChecks
  • 5 new tests covering: unrecognized model, recognized model, absent field,
    message content, and a 3.x series model
  • README table updated with empty-body and model-unknown rows

Tests

  • test/checks.test.ts: 5 new tests for model-unknown
  • All 104 existing tests continue to pass
  • npm run lint && npm run build && npm test all green locally

Self-merge gate

  • all CI checks pass
  • LOC delta < 250 (84 lines added, 0 removed)
  • no public-API surface change (does not touch src/index.ts)
  • no runtime-dependency additions
  • no workflow file changes
  • tests added or extended

Generated by Claude Code

…ield

Validates the optional model: field against a set of known Claude model
IDs. Issues a warn (not error) so CI does not break when new models ship.
Also documents empty-body and model-unknown in the README rules table.
@protosphinx protosphinx added the automated Opened by the daily bot label Jun 7, 2026 — with Claude
@protosphinx protosphinx merged commit 89d0d13 into main Jun 7, 2026
3 checks passed
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 66e4d9d01f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/checks.ts

// Known Claude model IDs. Severity is warn (not error) because the list
// evolves with each Anthropic release; update this set when new models ship.
const KNOWN_CLAUDE_MODELS: ReadonlySet<string> = new Set([
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Accept documented model aliases and inherit

This whitelist makes valid skill frontmatter produce model-unknown warnings: I checked the Claude Code skills frontmatter docs, which say model accepts the same values as /model, or inherit, and the CLI reference documents /model aliases such as sonnet and opus. A skill using model: inherit or model: sonnet will now be reported as a typo, and in --strict CI that false positive becomes a failing run, so the accepted set should include the documented non-ID values (or delegate validation to Claude Code's model parser).

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automated Opened by the daily bot

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant