ipi-check — Indirect Prompt injection and skills security scanner

A SAST scanner that detects indirect prompt injection (OWASP LLM01) in AI agent instruction files, documents and source code, and performs agent skill security auditing — detecting malicious behavior (credential theft, remote execution, privilege abuse, secrecy) in AI agent skill directories (SKILL.md-based). It combines deterministic static analysis with an optional LLM classifier and emits results as SARIF v2.1.0 — ready to upload to GitHub Code Scanning, GitLab SAST, or any SARIF-aware viewer.

Overview

LLM-powered coding agents read project-local instruction files such as AGENTS.md, .cursorrules, CLAUDE.md, and .windsurfrules. An attacker who controls those files (or smuggles content into source-code comments) can hijack the agent — exfiltrate secrets, run destructive commands, or rewrite its behavior. ipi-check scans a repository for the byte- and language-level signals of such attacks.

The scanner is built around a three-phase pipeline:

Phase A — Static analysis (always on). Byte-level inspection, regex pattern matching, and semantic heuristics produce per-file static results with deterministic confidence scores.
Phase B — LLM classification (optional). When an LLM provider is configured, sanitized file content is forwarded to the model for a structured classification verdict.
Phase C — Skill security audit (automatic). When SKILL.md files are discovered, skill units are analyzed with dedicated malware-focused static patterns (IPI401–411) and optional LLM classification targeting malicious behavior.

A deterministic confidence-fusion matrix combines all phases into a final verdict: PASS, REVIEW_REQUIRED, or BLOCK.

Features

Byte-level hidden-content detection — ANSI escapes, Unicode tag characters (E0000–E007F), variation selectors, bidi overrides, zero-width characters, Private Use Area codepoints, homoglyphs.
Regex injection patterns — instruction overrides (multilingual: EN/RU/CN/FR/ES/DE/JP/KR), authority claims (including CVE-2025-53773 chat.tools.autoApprove), destructive shell commands, data exfiltration (including conversation leakage), shell injection, jailbreak personas (STAN/DUDE/token system/role-play), social engineering, obfuscation (base64 decode / payload splitting), and instruction contradiction (intra-file rule negation).
Semantic heuristics — Shannon entropy, invisible-content ratio, instruction density, polarity contradiction detection.
Optional LLM classification via LiteLLM — works with OpenAI, Anthropic, Azure, Bedrock, Ollama, or any LiteLLM provider.
SARIF v2.1.0 output — compatible with GitHub Code Scanning, GitLab, VS Code SARIF Viewer, and Sonar.
AI-agent file awareness — automatically discovers .cursorrules, .windsurfrules, .clinerules, AGENTS.md, CLAUDE.md, copilot-instructions.md, .cursor/**/*.mdc, and more.
SVG file scanning — SVG files are included as scan targets to detect prompt injection in <desc>, <title>, and <metadata> elements.
Pre-LLM sanitization — neutralizes invisible characters, ANSI escapes, base64-encoded blocks, and ROT13-obfuscated text before LLM processing; prevents the classifier itself from being prompt-injected.
Batch LLM classification for source code files with automatic chunking of oversized files and exponential-backoff retry on partial failures.
Agent skill security auditing — automatically detects SKILL.md-based skill directories and scans them for credential harvesting, remote code execution, privilege escalation, secrecy directives, and other malicious behavior.
Skill-specific detection patterns (IPI401–411) — dedicated regex patterns for obfuscated code, dynamic context abuse, hidden HTML instructions, and excessive permissions.
Skill LLM classification — dedicated skill classifier prompt focuses on shadow-feature detection (behavior not inferable from the skill description).
Path-traversal protection for symbolic links.
Pygments-based code extraction isolates comments and string literals from source files for targeted scanning.

Installation

From PyPI

# TODO: pip install ipi-check

From source

git clone https://github.com/v0lka/ipi-check.git
cd ipi-check
pip install -e .

Docker

# TODO: docker pull ghcr.io/v0lka/ipi-check:latest
# or build locally:
docker build -t ipi-check .

Requires Python 3.12+.

Quick Start

Case 1 — Static analysis only

No LLM provider; uses byte, pattern, and heuristic stages.

ipi-check scan ./my-repo

Case 2 — Static analysis + LLM classification

ipi-check scan ./my-repo \
  --llm-model gpt-4o-mini \
  --llm-api-token "${OPENAI_API_KEY}"

Write SARIF to a file instead of stdout:

ipi-check scan ./my-repo --output results.sarif

The --llm-api-token value supports ${VAR_NAME} expansion so credentials never have to appear on the command line literally.

CLI Reference

ipi-check scan REPO_PATH [OPTIONS]

Argument / Option	Type	Default	Description
`REPO_PATH`	path	(required)	Repository directory to scan. Supports `${VAR}` expansion.
`--llm-base-url URL`	string	`None`	LiteLLM base URL (for self-hosted / proxy endpoints).
`--llm-model NAME`	string	`None`	LLM model name, e.g. `gpt-4o-mini`, `claude-3-5-sonnet`. Falls back to `IPI_CHECK_LLM_MODEL`. Required (together with an API key) to enable Stage 2 — LiteLLM has no default model.
`--llm-api-token TOKEN`	string	`None`	API token for the LLM provider. Supports `${VAR}` expansion.
`--output PATH`	path	stdout	Write SARIF to a file (compact JSON). Warns if extension is not `.sarif`.
`--quiet`	flag	`false`	Suppress banner, progress, and summary lines on stderr.
`--no-gitignore`	flag	`false`	Disable `.gitignore`-aware file exclusion.
`--exclude PAT`	string	(none)	Glob pattern to exclude (gitignore syntax). Repeatable.
`--fail-on POLICY`	enum	`none`	Exit-code policy: `none` always exits `0` on a completed scan; `block` exits `3` on any BLOCK verdict; `review` exits `3` on BLOCK and `4` when only REVIEW_REQUIRED verdicts are present. See Exit Codes.
`--version`	flag	—	Print version and exit.
`-h`, `--help`	flag	—	Show help and exit.

Exit Codes

Code	Meaning
`0`	Scan completed successfully. SARIF emitted. Findings (if any) are in SARIF.
`1`	Runtime error (I/O failure, pipeline crash, etc.). See stderr.
`2`	Usage error (invalid path, missing argument, bad output directory).
`3`	`--fail-on block` or `review` policy tripped by a BLOCK verdict.
`4`	`--fail-on review` tripped by a REVIEW_REQUIRED verdict (no BLOCK).

With the default policy (--fail-on none) ipi-check always exits 0 on a clean scan regardless of verdicts — the SARIF report carries the findings, leaving gating decisions to your CI. Pass --fail-on block (or review) to opt into non-zero exit codes for CI gates that prefer them; the emitted SARIF is identical under every policy.

Output Format

Output is SARIF v2.1.0. Each finding includes the file URI, the offending rule ID, the static confidence, the LLM verdict (when present), and a human-readable message.

{
  "$schema": "https://json.schemastore.org/sarif-2.1.0.json",
  "version": "2.1.0",
  "runs": [
    {
      "tool": {
        "driver": {
          "name": "ipi-check",
          "version": "0.1.0",
          "semanticVersion": "0.1.0",
          "informationUri": "https://github.com/v0lka/ipi-check",
          "rules": [
            {
              "id": "IPI101",
              "name": "IPI101",
              "shortDescription": {
                "text": "Instruction override pattern — attempts to bypass rules"
              },
              "fullDescription": {
                "text": "Instruction override pattern — attempts to bypass rules"
              },
              "defaultConfiguration": { "level": "warning" },
              "helpUri": "https://github.com/v0lka/ipi-check"
            }
          ]
        }
      },
      "invocations": [
        {
          "executionSuccessful": true,
          "startTimeUtc": "2026-06-06T12:00:00Z",
          "endTimeUtc": "2026-06-06T12:00:05Z"
        }
      ],
      "results": [
        {
          "ruleId": "IPI101",
          "level": "error",
          "message": {
            "text": "Instruction override pattern — attempts to bypass rules",
            "markdown": "**IPI101** at line 3, column 1: Instruction override pattern — attempts to bypass rules (matched: `ignore all previous instructions`)"
          },
          "locations": [
            {
              "physicalLocation": {
                "artifactLocation": { "uri": ".cursorrules" },
                "region": {
                  "startLine": 3,
                  "startColumn": 1
                }
              }
            }
          ]
        }
      ]
    }
  ]
}

Local Git Hook (blocking `post-checkout`)

ipi-check itself always exits 0 under the default --fail-on none policy, so its SARIF can be consumed by CI gates. For local development the shipped wrapper runs the scan with an explicit --fail-on policy and maps the scanner's exit code (3 BLOCK / 4 REVIEW_REQUIRED) to the hook status — it never parses the human-readable stderr summary. The wrapper is shipped at scripts/ipi-check-hook.sh and is suitable for any client-side hook (post-checkout, post-merge, post-rewrite, pre-commit, …).

About blocking semantics. post-checkout runs after the working tree has been updated, so a non-zero exit cannot roll the checkout back. It does, however, propagate as the exit status of git checkout, which is enough for shells, IDE integrations, and chained automation to treat the checkout as failed and halt subsequent steps. If you need to prevent a state change (rather than flag it after the fact), install the same wrapper as a pre-commit or pre-push hook — those are truly preventive.

Install globally (once per machine)

Use Git's core.hooksPath to point every repository on your machine at a single hooks directory:

# 1. Pick a directory for your global hooks.
mkdir -p ~/.githooks

# 2. Drop the wrapper in as the post-checkout hook (must be named exactly
#    'post-checkout' and be executable).
curl -fsSL https://raw.githubusercontent.com/v0lka/ipi-check/main/scripts/ipi-check-hook.sh \
    -o ~/.githooks/post-checkout
chmod +x ~/.githooks/post-checkout

# 3. Tell Git to use it for every repository on this machine.
git config --global core.hooksPath ~/.githooks

If you have a local clone of this repository instead, copy the script directly:

cp scripts/ipi-check-hook.sh ~/.githooks/post-checkout
chmod +x ~/.githooks/post-checkout

What the hook does

On every branch checkout (branch_flag == 1) the wrapper:

Skips when not inside a git work tree, when IPI_CHECK_HOOK_DISABLE=1 is set, or when the ipi-check binary is unavailable (fail-open).
Runs ipi-check scan "$REPO_ROOT" --output .git/ipi-check-last.sarif.
Re-emits the scanner banner and summary to your terminal.
Parses the BLOCK: N, REVIEW_REQUIRED: M, PASS: K summary line.
Exits 1 when BLOCK > 0, leaving the SARIF report at .git/ipi-check-last.sarif for inspection.

Tuning the hook

The wrapper honours a few environment variables:

Variable	Default	Effect
`IPI_CHECK_HOOK_DISABLE`	`0`	Set to `1` to skip the scan entirely (one-shot opt-out).
`IPI_CHECK_BLOCK_ON_REVIEW`	`0`	Set to `1` to also fail when `REVIEW_REQUIRED > 0`.
`IPI_CHECK_FAIL_ON`	—	Explicit `--fail-on` value (`none`/`block`/`review`); overrides the shortcut above.
`IPI_CHECK_BIN`	`ipi-check`	Override the binary path, e.g. to a virtualenv or pinned version.

The hook invokes ipi-check scan without LLM flags, so hook runs are static-only by default. To enable LLM classification for them, export an API key (OPENAI_API_KEY / LITELLM_API_KEY / ANTHROPIC_API_KEY) and IPI_CHECK_LLM_MODEL — the hook cannot pass --llm-model, and a key without a model leaves the scan static-only (reported on stderr).

Example — temporarily bypass the hook for one checkout:

IPI_CHECK_HOOK_DISABLE=1 git checkout some-branch

Example — make REVIEW_REQUIRED blocking for stricter local gating:

git config --global hook.ipi-check.blockOnReview true   # for documentation
export IPI_CHECK_BLOCK_ON_REVIEW=1                      # actually used by the hook

Disable for a single repository

Per-repo core.hooksPath overrides the global setting:

cd /path/to/repo
git config --local core.hooksPath .git/hooks   # restore the default

CI/CD Integration

GitHub — Code Scanning upload

name: ipi-check
on: [push, pull_request]
jobs:
  scan:
    runs-on: ubuntu-latest
    permissions:
      security-events: write
      contents: read
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install ipi-check
      - run: ipi-check scan . --output ipi-check.sarif
      - uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: ipi-check.sarif

GitLab — SAST report

ipi-check:
  image: python:3.12-slim
  stage: test
  script:
    - pip install ipi-check
    - ipi-check scan . --output gl-sast-report.sarif
  artifacts:
    reports:
      sast: gl-sast-report.sarif

Docker Usage

The image bundles the scanner together with its LLM client (LiteLLM), so the optional --llm-* classifier works with no extra install. Mount the repository to scan as a volume — /repo is the image's working directory.

Static scan (no LLM)

docker run --rm -v "$(pwd):/repo" ipi-check scan /repo

Add --output to persist the report inside the mounted volume:

docker run --rm -v "$(pwd):/repo" ipi-check scan /repo --output /repo/results.sarif

Scan with LLM classification

The classifier is enabled when a credential and a model are both available — the credential via --llm-api-token or one of the provider environment variables (OPENAI_API_KEY, ANTHROPIC_API_KEY, LITELLM_API_KEY) forwarded with -e, and the model via --llm-model or IPI_CHECK_LLM_MODEL. Passing either half alone leaves the scan static-only (exit 0, with a stderr warning naming what is missing).

OpenAI (or any LiteLLM-native provider) using an environment variable:

docker run --rm \
  -v "$(pwd):/repo" \
  -e OPENAI_API_KEY \
  ipi-check scan /repo \
  --llm-model gpt-4o-mini \
  --llm-api-token '${OPENAI_API_KEY}' \
  --output /repo/results.sarif

Every string argument supports ${VAR} expansion, so --llm-api-token can be fed from the container environment without the credential appearing literally on the command line.

Self-hosted / proxy endpoints (`--llm-base-url`)

Point --llm-base-url at any OpenAI-compatible endpoint — a LiteLLM proxy, Ollama, vLLM, Azure OpenAI, … — and prefix the model with its provider when LiteLLM needs the hint (openai/…, ollama/…, azure/…):

docker run --rm \
  -v "$(pwd):/repo" \
  -e LITELLM_API_KEY \
  ipi-check scan /repo \
  --llm-base-url 'http://host.docker.internal:4000/v1' \
  --llm-model 'openai/gpt-4o-mini' \
  --llm-api-token '${LITELLM_API_KEY}' \
  --output /repo/results.sarif

When the endpoint runs on the Docker host, reach it from the container with host.docker.internal (Docker Desktop resolves this automatically; on Linux pass --add-host=host.docker.internal:host-gateway) or, on Linux, with --network host and http://127.0.0.1:….

--format selects the report shape: the examples above emit the default SARIF v2.1.0, while --format md, --format table and --format json produce human-readable or flat-JSON reports instead.

Development

git clone https://github.com/v0lka/ipi-check.git
cd ipi-check
python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"

Run the test suite, linter, and type-checker:

pytest tests/ -v --tb=short
ruff check src/ tests/
mypy src/

Architecture

ipi-check is organized into a small set of focused modules:

core/types.py — shared data types, enums, and dataclasses.
scanner/file_discovery.py — locate AI-agent files and source files.
scanner/byte_analysis.py — detect hidden/abusive bytes.
scanner/pattern_matching.py — regex-based injection signatures + skill-specific patterns.
scanner/semantic_heuristics.py — entropy, density, invisibility ratios.
scanner/static_result.py — assemble static results and compute severity (per-file and per-skill).
scanner/code_extractor.py — Pygments-driven comment/string extraction.
scanner/llm_sanitizer.py — neutralize hostile content (invisible chars, ANSI, base64, ROT13) before the LLM call.
scanner/token_counter.py — tiktoken-based token counting for batch assembly.
scanner/llm_classifier.py — LiteLLM-backed classifier (single-file, batch, and per-skill).
scanner/confidence_fusion.py — deterministic verdict matrix (per-file and per-skill).
scanner/pipeline.py — end-to-end orchestration (non-skill files + skill audit).
reporter/sarif_reporter.py — SARIF v2.1.0 emission.
cli/main.py — argparse CLI entry point.

The full architecture and decision records live in specs/: system-overview.md, security-model.md, and the ADRs in specs/decisions/.

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
images		images
samples		samples
scripts		scripts
specs		specs
src/ipi_check		src/ipi_check
tests		tests
.gitignore		.gitignore
AGENTS.md		AGENTS.md
Dockerfile		Dockerfile
LICENSE		LICENSE
README.md		README.md
SECURITY.md		SECURITY.md
pyproject.toml		pyproject.toml
uv.lock		uv.lock

Navigation Menu

Repository navigation

About

Repository files navigation

ipi-check — Indirect Prompt injection and skills security scanner

Overview

Features

Installation

From PyPI

From source

Docker

Quick Start

Case 1 — Static analysis only

Case 2 — Static analysis + LLM classification

CLI Reference

Exit Codes

Output Format

Local Git Hook (blocking `post-checkout`)

Install globally (once per machine)

What the hook does

Tuning the hook

Disable for a single repository

CI/CD Integration

GitHub — Code Scanning upload

GitLab — SAST report

Docker Usage

Static scan (no LLM)

Scan with LLM classification

Self-hosted / proxy endpoints (`--llm-base-url`)

Development

Architecture

License

About

Releases

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

ipi-check — Indirect Prompt injection and skills security scanner

Overview

Features

Installation

From PyPI

From source

Docker

Quick Start

Case 1 — Static analysis only

Case 2 — Static analysis + LLM classification

CLI Reference

Exit Codes

Output Format

Local Git Hook (blocking post-checkout)

Install globally (once per machine)

What the hook does

Tuning the hook

Disable for a single repository

CI/CD Integration

GitHub — Code Scanning upload

GitLab — SAST report

Docker Usage

Static scan (no LLM)

Scan with LLM classification

Self-hosted / proxy endpoints (--llm-base-url)

Development

Architecture

License

About

Topics

Resources

Security policy

Stars

Watchers

Forks

Releases

Contributors

Languages

Local Git Hook (blocking `post-checkout`)

Self-hosted / proxy endpoints (`--llm-base-url`)