From db784c4d2c58310299def8659ae5a1e96ed1e1d7 Mon Sep 17 00:00:00 2001 From: akanthed Date: Sun, 16 Aug 2026 21:57:18 +0530 Subject: [PATCH 1/6] docs: add design spec for LiteLLM config scanner (LLC001-003) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit First of two planned LiteLLM-related features for the next release — static config scanning now, runtime guardrail hook designed separately later. Co-Authored-By: Claude Sonnet 5 --- ...026-08-16-litellm-config-scanner-design.md | 81 +++++++++++++++++++ 1 file changed, 81 insertions(+) create mode 100644 docs/superpowers/specs/2026-08-16-litellm-config-scanner-design.md diff --git a/docs/superpowers/specs/2026-08-16-litellm-config-scanner-design.md b/docs/superpowers/specs/2026-08-16-litellm-config-scanner-design.md new file mode 100644 index 0000000..860e498 --- /dev/null +++ b/docs/superpowers/specs/2026-08-16-litellm-config-scanner-design.md @@ -0,0 +1,81 @@ +# LiteLLM Config Scanner — Design + +## Context + +`secureai-scan` currently references LiteLLM in two places only: as a detected LLM SDK client (regex patterns in `python-scanner.ts`) and as a DEP003 dependency-advisory target (CVEs in `advisories-generated.ts`). It has no awareness of LiteLLM's own proxy configuration (`config.yaml`) the way it already does for MCP client configs (`.mcp.json`, `claude_desktop_config.json`, etc. via `mcp-config-scanner.ts`). + +This is the first of two planned LiteLLM-related sub-projects. The second — a runtime guardrail hook that plugs into LiteLLM Proxy's `CustomGuardrail` interface to block/flag requests live — is out of scope for this spec and will get its own design later. + +## Goal + +Statically scan LiteLLM proxy `config.yaml` files for parsed, factual misconfigurations: hardcoded secrets, plaintext transport to remote providers, and (as a low-confidence nudge only) the complete absence of a `guardrails:` section. This is the literal thing the user asked for ("something that works in litellm for guardrails") in its static form — a first pass that ships fast, in this repo's existing architecture, before the larger runtime component. + +## Non-goals + +- No runtime request-time behavior (that's the separate guardrail-hook sub-project). +- No general LiteLLM proxy config linting (auth toggles, database settings, spend-log config, etc.) — anything not specifically an LLM-security-relevant signal is out of scope, per this repo's fixed LLM/MCP/RAG detection scope (`CLAUDE.md` hard requirement #3). +- No evaluation of *which* guardrail provider is configured or whether it's well-configured beyond "does the section exist" — that would require provider-specific knowledge this scanner doesn't have and risks false confidence. + +## Architecture + +New self-contained module `src/scanner/litellm-config-scanner.ts`, following the exact pattern of `src/scanner/mcp-config-scanner.ts`: + +- Off-disk file discovery and parsing — not part of the ts-morph `Project`, not AST-based. +- Walks the repo using the same skip-dir list as the MCP config scanner (`node_modules`, `.git`, `dist`, `build`, `out`, `.next`, `.venv`, `venv`, `__pycache__`), respecting `skipPaths` from policy config. +- Candidate files: any `*.yaml`/`*.yml`. +- **Structural gate before treating a file as a LiteLLM config**: the parsed YAML must have a top-level `model_list` key whose value is an array, where at least one entry has a `litellm_params` object. This is LiteLLM's own proxy config schema shape and is distinctive enough to avoid false-positiving on unrelated YAML files (e.g. a Kubernetes `config.yaml`, a generic app config). Files that parse but don't match this shape are silently skipped — no finding, no error. +- Parsing library: **new dependency `js-yaml`**, using its default schema (`load`, not `loadAll`/unsafe custom-tag schemas) — no JS-object/function deserialization tags, consistent with this codebase's existing concern about unsafe YAML deserialization (SKL010, skill-bundle metadata). +- Wired into `scan.ts` (`scanRepositoryDetailed`) and `src/scanner/rules/index.ts` the same way `CONFIG_RULE_IDS` is today (new `LITELLM_CONFIG_RULE_IDS` array feeding `AVAILABLE_RULE_IDS`). + +## Rules + +New rule-ID prefix: `LLC` (LiteLLM Config) — chosen to avoid colliding with the `LLMxx:2026` OWASP taxonomy labels already used in report output for the OWASP LLM Top 10 mapping (those are a separate `owasp` field, not the `rule_id`, but a distinct prefix keeps them unambiguous at a glance). + +### LLC001 — Hardcoded secret in LiteLLM config +- **Evidence: `proven`, severity: critical.** +- Fires when any of these fields, under a `litellm_params` block or `general_settings`, hold a literal string value instead of an `os.environ/VAR_NAME` reference (LiteLLM's own convention for pulling secrets from the environment): + - `api_key` + - `master_key` + - `salt_key` + - other known integration credential fields following the same `*_api_key` / `*_key` naming used by LiteLLM's bundled integrations (Langfuse, Aporia, etc.) +- A value is treated as a literal secret if it does not match `^os\.environ/` and is not empty/placeholder-shaped (reuse the same minimum-length + non-placeholder heuristic MCP005 uses for `ENV_REFERENCE`/length gating, adapted to LiteLLM's `os.environ/X` syntax instead of `${...}`). +- Directly mirrors MCP005 (`src/scanner/mcp-config-scanner.ts`). + +### LLC002 — Plaintext HTTP provider endpoint +- **Evidence: `proven`, severity: high.** +- Fires when an `api_base` value under `litellm_params` uses `http://` and the host is not localhost/loopback (same locality check as MCP006: `localhost`, `127.0.0.1`, `0.0.0.0`, `::1`). +- Directly mirrors MCP006. + +### LLC003 — No guardrails configured +- **Evidence: `heuristic`, severity: low, hidden unless `--paranoid`.** +- Fires when a file passes the LiteLLM-config structural gate (real `model_list` with actual provider entries) and has no top-level `guardrails:` key at all. +- Deliberately the lowest evidence tier: absence of an optional feature is not itself a vulnerability, only a nudge. This keeps it out of default output and off the precision gate's `proven`/`likely` corpus requirement, consistent with `CLAUDE.md`'s evidence-tier contract (heuristic = pattern/proximity signal only, never asserted as a proven fact). + +## Data flow + +``` +config.yaml (found via same walk pattern as MCP scanner) + → js-yaml load() + → structural gate (model_list + litellm_params present?) + → no → skip file, no finding + → yes → walk model_list entries for LLC001/LLC002 + → check top-level guardrails key for LLC003 + → Finding[] (rule_id, file, line via same lineOf() line-anchoring approach as mcp-config-scanner.ts) +``` + +Line numbers: reuse the `lineOf(lines, needle)` approach from `mcp-config-scanner.ts` (find first line containing the matched key/value string) rather than tracking real YAML AST positions — js-yaml's default `load()` doesn't give node positions without extra options, and this repo's MCP scanner already accepts this approximation for JSON. + +## Testing + +Same checklist as any new rule, per `CLAUDE.md`'s "Adding a new rule": + +1. Vulnerable fixtures: `test-fixtures/vulnerable/litellm-config-secret.yaml` (LLC001), `test-fixtures/vulnerable/litellm-config-http.yaml` (LLC002), `test-fixtures/vulnerable/litellm-config-no-guardrails.yaml` (LLC003, only visible under `--paranoid`). +2. Safe fixture: `test-fixtures/safe/litellm-config-clean.yaml` — all secrets as `os.environ/VAR`, `api_base` on `https://`, a populated `guardrails:` section. Must produce zero `proven`/`likely` findings. +3. A safe fixture for the structural gate itself: an unrelated `config.yaml` (e.g. a generic app config with a coincidental `model_list` string field, or no `litellm_params`) that must produce zero findings — proves the gate isn't name-matching alone. +4. Catalog entries (`src/scanner/catalog.ts`: title/owasp/impact/fix, `FRAMEWORK_MAP` if an MCP/ASI mapping is defensible — likely not, this is provider-gateway config, not MCP) and explainer entries (`src/scanner/explainer.ts`'s `DEFAULT_EXPLANATIONS`) for LLC001–003. +5. Extend `EXPECTED_VULNERABLE` in `test/corpus.test.js`. +6. `npm run build && npm test`, then `npm run regression` before considering the rule done (a real-world LiteLLM proxy repo would be a good regression-scan candidate if one exists in the current `.regression-cache/` set or can be added). + +## Open item carried to implementation + +The exact list of "known integration credential fields" for LLC001 beyond `api_key`/`master_key`/`salt_key` should be finalized by looking at LiteLLM's actual documented `litellm_params`/`general_settings` schema during implementation, not guessed here — this keeps the field list a citable fact rather than a guess (consistent with `CLAUDE.md`'s DEP003 advisory bar: "a plausible guess is not the same as a citable fact"). From ae94a806778ea6efc5f2f9851c2e89bde1e737f0 Mon Sep 17 00:00:00 2001 From: akanthed Date: Sun, 16 Aug 2026 23:45:58 +0530 Subject: [PATCH 2/6] feat: add LiteLLM config scanner with rules for hardcoded secrets, HTTP endpoints, and guardrails - Implemented scanning for LiteLLM proxy config files to detect hardcoded secrets (LLC001), plaintext HTTP endpoints (LLC002), and missing guardrails (LLC003). - Added new rules to the rule catalog and updated the threat model to include LiteLLM configurations. - Created test fixtures for both safe and vulnerable LiteLLM configurations. - Added unit tests to validate the functionality of the LiteLLM config scanner. - Updated package dependencies to include js-yaml for YAML parsing. - Enhanced sitemap with new documentation links for LiteLLM scanning. --- docs/index.html | 2 +- docs/llms.txt | 4 + docs/mcp-tool-poisoning.html | 198 +++++++++++++++ docs/prompt-injection-detection.html | 201 +++++++++++++++ docs/sitemap.xml | 8 + llms.txt | 18 ++ package-lock.json | 39 +++ package.json | 2 + src/scanner/catalog.ts | 26 ++ src/scanner/explainer.ts | 46 ++++ src/scanner/litellm-config-scanner.ts | 238 ++++++++++++++++++ src/scanner/rules/index.ts | 5 + src/scanner/scan.ts | 9 + src/scanner/threat-model.ts | 4 + test-fixtures/safe/litellm-config-clean.yaml | 14 ++ .../safe/litellm-config-not-litellm.yaml | 7 + .../vulnerable/litellm-config-http.yaml | 11 + .../litellm-config-no-guardrails.yaml | 6 + .../vulnerable/litellm-config-secret.yaml | 12 + test/corpus.test.js | 2 + test/litellm-config.test.js | 79 ++++++ test/run-tests.js | 1 + 22 files changed, 931 insertions(+), 1 deletion(-) create mode 100644 docs/mcp-tool-poisoning.html create mode 100644 docs/prompt-injection-detection.html create mode 100644 llms.txt create mode 100644 src/scanner/litellm-config-scanner.ts create mode 100644 test-fixtures/safe/litellm-config-clean.yaml create mode 100644 test-fixtures/safe/litellm-config-not-litellm.yaml create mode 100644 test-fixtures/vulnerable/litellm-config-http.yaml create mode 100644 test-fixtures/vulnerable/litellm-config-no-guardrails.yaml create mode 100644 test-fixtures/vulnerable/litellm-config-secret.yaml create mode 100644 test/litellm-config.test.js diff --git a/docs/index.html b/docs/index.html index 9bc5f44..174ae91 100644 --- a/docs/index.html +++ b/docs/index.html @@ -506,7 +506,7 @@

Known-bad packages

-

For your own repository, run the full scanner: npx secureai-scan@0.9.0 scan . — traces prompt injection, RAG poisoning, and agent-privilege issues across your codebase, not just what you paste here.

+

For your own repository, run the full scanner: npx secureai-scan@0.9.0 scan . — traces prompt injection, RAG poisoning, and agent-privilege issues across your codebase, not just what you paste here. Read more: how to detect MCP tool poisoning · how to detect prompt injection.

SecureAI-Scan on GitHub
diff --git a/docs/llms.txt b/docs/llms.txt index 2b571dd..ddc95ec 100644 --- a/docs/llms.txt +++ b/docs/llms.txt @@ -9,6 +9,10 @@ The TypeScript/JavaScript engine uses ts-morph AST analysis. The Python engine u Project: https://github.com/akanthed/SecureAI-Scan Package: https://www.npmjs.com/package/secureai-scan Browser MCP scanner: https://akanthed.github.io/SecureAI-Scan/ +How to detect MCP tool poisoning: https://akanthed.github.io/SecureAI-Scan/mcp-tool-poisoning.html +How to detect prompt injection in code: https://akanthed.github.io/SecureAI-Scan/prompt-injection-detection.html +Real-world findings (regression scan against public repos): https://github.com/akanthed/SecureAI-Scan/blob/main/docs/RealWorldFindings.md +FAQ: https://github.com/akanthed/SecureAI-Scan/blob/main/docs/FAQ.md Release assurance: https://github.com/akanthed/SecureAI-Scan/blob/main/docs/ReleaseAssurance.md Latest benchmark: https://github.com/akanthed/SecureAI-Scan/blob/main/docs/benchmarks/v0.9.0.json Security policy: https://github.com/akanthed/SecureAI-Scan/blob/main/SECURITY.md \ No newline at end of file diff --git a/docs/mcp-tool-poisoning.html b/docs/mcp-tool-poisoning.html new file mode 100644 index 0000000..5839609 --- /dev/null +++ b/docs/mcp-tool-poisoning.html @@ -0,0 +1,198 @@ + + + + + +How to Detect MCP Tool Poisoning — Checklist & Free Scanner + + + + + + + + + + + + + + + + +
+ +
+
SecureAI-SCAN · docs
+ ← MCP X-Ray tool +
+ +
+
MCP security
+

How to detect MCP tool poisoning

+

MCP tool poisoning hides an instruction to the agent inside a tool's description field — the text a model reads to decide how to call a tool, not the code you'd normally review. Here's what to look for and how to check it before you run a server.

+
+ +
+
+

Short answer: read every tool description for invisible Unicode, phrasing directed at the agent instead of the user ("ignore previous instructions," "don't tell the user"), and instructions to read credential files or send data to a URL. Do this before the server runs — the payload activates the moment an agent loads the description into context, regardless of whether you ever call the tool it's attached to.

+
+ +

Why this bypasses normal code review

+

A tool's handler function can be completely benign while its description string carries the attack. MCP clients pass that description straight into the model's context on every turn so the agent knows when the tool is relevant — which means the description is executable in the sense that matters: a capable model treats plausible-sounding instructions in its context as instructions, regardless of which field they came from. Reviewing only the function body misses this entirely, because nothing in the handler changed.

+ +

The three signals to check for

+
+
+ Invisible Unicode +

Characters your eyes skip, the model doesn't

+

Zero-width joiners (U+200B–U+200D), bidirectional override characters (U+202A–U+202E), and the Unicode tags block (U+E0000–U+E007F) can splice hidden text into an otherwise normal-looking description. It renders as nothing or as ordinary text in most editors, but the tokenizer still sees it.

+
+
+ Agent-directed phrasing +

Instructions aimed at the model, not you

+

Real documentation explains what a tool does. Poisoned descriptions instead say things like "ignore previous instructions," "do not mention this to the user," or wrap directives in pseudo-system tags like <IMPORTANT> — language with no reason to exist except to steer the agent's next action.

+
+
+ Credential reads / exfil instructions +

The actual payload

+

Look for instructions to read ~/.ssh/id_rsa, .env, or .aws/credentials, paired with an instruction to send the result to an external URL. This pairing — read, then send — is the shape of nearly every real tool-poisoning incident, not two unrelated red flags.

+
+
+ +

Check it yourself

+

Paste a tool description, an .mcp.json, or server code into MCP X-Ray — a free browser tool that runs these three checks client-side, nothing you paste leaves the page. For scanning an entire repository or MCP server before installing it, the SecureAI-Scan CLI runs the same checks (plus prompt-injection dataflow tracing and dependency advisories) with a single command:

+

npx secureai-scan@0.9.0 mcp owner/repo

+ +
+

Try it now — paste a tool description and see the checks run in real time, entirely in your browser.

+ Open MCP X-Ray → +
+ +

Frequently asked questions

+
+
+

What is MCP tool poisoning?

+

An attack where an MCP server's tool description contains hidden instructions aimed at the agent reading it, not documentation for the human operator. Because the description loads into the model's context on every relevant call, it can steer an agent into reading credentials or exfiltrating data without the payload ever touching the tool's actual code.

+
+
+

Can this happen without changing the tool's code?

+

Yes — that's what makes it hard to catch with normal code review. The handler function can stay untouched while the description string carries the entire attack.

+
+
+

Is this the same as the postmark-mcp backdoor?

+

Related but different. postmark-mcp was a malicious code change shipped in a package update. Tool poisoning is the description-field attack specifically — no code change required.

+
+
+
+ + + +
+ + diff --git a/docs/prompt-injection-detection.html b/docs/prompt-injection-detection.html new file mode 100644 index 0000000..1723c72 --- /dev/null +++ b/docs/prompt-injection-detection.html @@ -0,0 +1,201 @@ + + + + + +How to Detect Prompt Injection in Code — Static Analysis Guide + + + + + + + + + + + + + + + + +
+ +
+
SecureAI-SCAN · docs
+ ← MCP X-Ray tool +
+ +
+
LLM01:2026 · Prompt Injection
+

How to detect prompt injection in code

+

Prompt injection isn't a keyword — it's a dataflow. It exists wherever user-controlled input reaches an LLM call in a position the model treats as an instruction. Here's how to trace that statically, and what a real finding looks like.

+
+ +
+
+

Short answer: trace data from a request source (HTTP body, form field, websocket message) forward through your code to see whether it lands in an LLM call's system-role message or gets concatenated into a system prompt string. If it resolves through a real import to a known LLM SDK and reaches that position unscoped, that's the vulnerability — independent of whether any specific attack phrase is present.

+
+ +

Why keyword matching doesn't work here

+

Scanning source code for phrases like "ignore previous instructions" catches text that's already sitting in a file — useful for checking untrusted content like an MCP tool description or a document you're about to feed to a RAG pipeline. It does not catch the underlying code vulnerability, because an attacker doesn't need to plant a famous phrase in your codebase — they need the code path to exist so that whatever they type at runtime lands somewhere the model treats as authoritative. The vulnerability is structural, not lexical.

+ +

What a traced finding looks like

+

A dataflow-based finding shows the actual path from an untrusted source, through however many function calls, to the exact LLM SDK call site it reaches:

+ 
HIGH  Prompt injection via user input
+PROVEN  LLM01:2026 Prompt Injection
+
+source  src/chat.ts:8   request data `req.body.input`
+flow    src/chat.ts:13  passed as `systemPrompt`
+sink    src/chat.ts:10  openai.chat.completions.create — system role (OpenAI)
+
+fix     Keep system prompts static; pass user input as a user-role message.
+

Each line is checkable against the source file — that's what separates a traced finding from a keyword hit. Three things have to be true for this to count as a real (not heuristic) finding:

+
    +
  • The sink is import-resolved. The call has to trace through an actual import to a known LLM SDK (OpenAI, Anthropic, the Vercel AI SDK, LangChain, Bedrock, etc.) — not just a function named something like chat() or complete().
    • +
  • The source is untrusted. Request bodies, query params, form inputs, websocket messages — not a hardcoded string or a value that only ever comes from your own config.
    • +
  • The position matters. Landing in a system-role message or being concatenated into a system-prompt string carries more trust than landing in a user-role message, which is the whole point — the model treats the two positions differently.
    • +
+ +

Trace it yourself

+
+
1. SourceFind where request-controlled data enters your code — an Express handler's req.body, a Next.js route's parsed form data, a webhook payload.
+
2. FlowFollow it through variable assignments, function parameters, and string concatenation — does it get merged into a prompt-building string anywhere along the way?
+
3. SinkCheck where it lands in the eventual LLM SDK call — a system-role message or a top-level system prompt is the dangerous position; a user-role message is the correct one.
+
+ +

Doing this by hand across a real codebase is slow and easy to miss — a value can pass through several functions before it reaches the sink. SecureAI-Scan automates exactly this trace via AST analysis (ts-morph for TypeScript/JavaScript, tree-sitter for Python), only flagging calls that resolve through real imports:

+

npx secureai-scan@0.9.0 scan .

+ +
+

Check an MCP tool description or config instead? That's a related but different check — invisible Unicode and injection phrasing in untrusted text, not code dataflow.

+ How to detect MCP tool poisoning → +
+ +

Frequently asked questions

+
+
+

What is prompt injection?

+

Untrusted input reaching an LLM call in a position the model treats as an instruction — a system prompt or similarly elevated-trust position — instead of being clearly scoped as data. It's OWASP's LLM01:2026 category.

+
+
+

Does a "clean" scan mean the code is safe?

+

It means no traced dataflow from an untrusted source to a resolved LLM sink was found by the specific checks that ran. Static analysis is a filter, not proof of runtime safety — it can't see behavior that only emerges from how a model actually responds to a given input.

+
+
+

What's the fix?

+

Keep system prompts static; pass user-controlled input as a user-role message rather than concatenating it into the system prompt.

+
+
+
+ + + +
+ + diff --git a/docs/sitemap.xml b/docs/sitemap.xml index 717dd9a..ca31f22 100644 --- a/docs/sitemap.xml +++ b/docs/sitemap.xml @@ -4,4 +4,12 @@ https://akanthed.github.io/SecureAI-Scan/ 2026-08-05 + + https://akanthed.github.io/SecureAI-Scan/mcp-tool-poisoning.html + 2026-08-14 + + + https://akanthed.github.io/SecureAI-Scan/prompt-injection-detection.html + 2026-08-14 + \ No newline at end of file diff --git a/llms.txt b/llms.txt new file mode 100644 index 0000000..ddc95ec --- /dev/null +++ b/llms.txt @@ -0,0 +1,18 @@ +# SecureAI-Scan + +SecureAI-Scan is a local-first static AI security scanner for TypeScript, JavaScript, Python, MCP configurations, and Agent Skill bundles. + +Primary capabilities: prompt-injection dataflow, MCP tool poisoning, Agent Skill poisoning, RAG and vector-store risks, unsafe LLM output handling, AI-BOM generation, SARIF reporting, and version-aware AI dependency advisories. + +The TypeScript/JavaScript engine uses ts-morph AST analysis. The Python engine uses tree-sitter AST nodes with local taint propagation. Findings are separated into proven, likely, and heuristic evidence tiers. Static analysis is a filter, not proof of runtime safety. + +Project: https://github.com/akanthed/SecureAI-Scan +Package: https://www.npmjs.com/package/secureai-scan +Browser MCP scanner: https://akanthed.github.io/SecureAI-Scan/ +How to detect MCP tool poisoning: https://akanthed.github.io/SecureAI-Scan/mcp-tool-poisoning.html +How to detect prompt injection in code: https://akanthed.github.io/SecureAI-Scan/prompt-injection-detection.html +Real-world findings (regression scan against public repos): https://github.com/akanthed/SecureAI-Scan/blob/main/docs/RealWorldFindings.md +FAQ: https://github.com/akanthed/SecureAI-Scan/blob/main/docs/FAQ.md +Release assurance: https://github.com/akanthed/SecureAI-Scan/blob/main/docs/ReleaseAssurance.md +Latest benchmark: https://github.com/akanthed/SecureAI-Scan/blob/main/docs/benchmarks/v0.9.0.json +Security policy: https://github.com/akanthed/SecureAI-Scan/blob/main/SECURITY.md \ No newline at end of file diff --git a/package-lock.json b/package-lock.json index 807b9df..cb56035 100644 --- a/package-lock.json +++ b/package-lock.json @@ -10,6 +10,7 @@ "license": "MIT", "dependencies": { "commander": "^15.0.0", + "js-yaml": "^5.3.0", "tree-sitter": "^0.25.1", "tree-sitter-python": "^0.25.0", "ts-morph": "^28.0.0" @@ -18,6 +19,7 @@ "secureai-scan": "dist/index.js" }, "devDependencies": { + "@types/js-yaml": "^4.0.9", "@types/node": "^26.1.2", "c8": "^12.0.0", "typescript": "^7.0.2" @@ -92,6 +94,13 @@ "dev": true, "license": "MIT" }, + "node_modules/@types/js-yaml": { + "version": "4.0.9", + "resolved": "https://registry.npmjs.org/@types/js-yaml/-/js-yaml-4.0.9.tgz", + "integrity": "sha512-k4MGaQl5TGo/iipqb2UDG2UwjXziSWkh0uysQelTlJpX1qGlpUZYm8PnO4DxG1qBomtJUdYJ6qR6xdIah10JLg==", + "dev": true, + "license": "MIT" + }, "node_modules/@types/node": { "version": "26.1.2", "resolved": "https://registry.npmjs.org/@types/node/-/node-26.1.2.tgz", @@ -468,6 +477,12 @@ "url": "https://github.com/chalk/ansi-styles?sponsor=1" } }, + "node_modules/argparse": { + "version": "2.0.1", + "resolved": "https://registry.npmjs.org/argparse/-/argparse-2.0.1.tgz", + "integrity": "sha512-8+9WqebbFzpX9OR+Wa6O29asIogeRMzcGtAINdpMHHyAg10f05aSFVBbcEqGf/PXw1EjAZ+q2/bEBg3DvurK3Q==", + "license": "Python-2.0" + }, "node_modules/balanced-match": { "version": "4.0.4", "resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-4.0.4.tgz", @@ -765,6 +780,28 @@ "node": ">=8" } }, + "node_modules/js-yaml": { + "version": "5.3.0", + "resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-5.3.0.tgz", + "integrity": "sha512-muutsYr+e2+d3rTgUGslq5rxbBlUy3cJ61IsHag2QNDQV+7zXWjkUpmALIajhrlLlrgRUiymj6U3zUr/TMK84Q==", + "funding": [ + { + "type": "github", + "url": "https://github.com/sponsors/puzrin" + }, + { + "type": "github", + "url": "https://github.com/sponsors/nodeca" + } + ], + "license": "MIT", + "dependencies": { + "argparse": "^2.0.1" + }, + "bin": { + "js-yaml": "bin/js-yaml.mjs" + } + }, "node_modules/locate-path": { "version": "6.0.0", "resolved": "https://registry.npmjs.org/locate-path/-/locate-path-6.0.0.tgz", @@ -932,6 +969,7 @@ "resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.4.tgz", "integrity": "sha512-QP88BAKvMam/3NxH6vj2o21R6MjxZUAd6nlwAS/pnGvN9IVLocLHxGYIzFhg6fUQ+5th6P4dv4eW9jX3DSIj7A==", "license": "MIT", + "peer": true, "engines": { "node": ">=12" }, @@ -1071,6 +1109,7 @@ "integrity": "sha512-mrcEdkYtHfrK1A6fs3O6FxkBo0Qig5XUXqHhxUOQu0bmPo00QF4XaSx4edpazdHwxnSCjlGKGgIqWdaN4dvTLA==", "hasInstallScript": true, "license": "MIT", + "peer": true, "dependencies": { "node-addon-api": "^8.5.0", "node-gyp-build": "^4.8.4" diff --git a/package.json b/package.json index 389b775..c486e7e 100644 --- a/package.json +++ b/package.json @@ -79,11 +79,13 @@ }, "dependencies": { "commander": "^15.0.0", + "js-yaml": "^5.3.0", "tree-sitter": "^0.25.1", "tree-sitter-python": "^0.25.0", "ts-morph": "^28.0.0" }, "devDependencies": { + "@types/js-yaml": "^4.0.9", "@types/node": "^26.1.2", "c8": "^12.0.0", "typescript": "^7.0.2" diff --git a/src/scanner/catalog.ts b/src/scanner/catalog.ts index 70bf94c..0847e9e 100644 --- a/src/scanner/catalog.ts +++ b/src/scanner/catalog.ts @@ -482,6 +482,32 @@ export const RULE_CATALOG: Record = { "Remove or update the package per the advisory; rotate credentials it could have accessed.", "Art. 15 (cybersecurity)", ), + LLC001: entry( + "LLC001", + "Hardcoded secret in LiteLLM config", + "critical", + "LLM02", + "Provider API keys and proxy master/salt keys written directly into a shared config.yaml are exposed to everyone with repo access.", + "Reference credentials via LiteLLM's os.environ/VAR_NAME convention instead of inlining values; rotate the exposed credential.", + "Art. 15 (cybersecurity)", + ), + LLC002: entry( + "LLC002", + "Plaintext HTTP provider endpoint", + "high", + "LLM04", + "Requests, responses, and header credentials to a non-localhost provider api_base travel unencrypted.", + "Use https:// for all non-localhost provider api_base URLs.", + "Art. 15 (cybersecurity)", + ), + LLC003: entry( + "LLC003", + "No guardrails configured", + "low", + "LLM03", + "A LiteLLM proxy with no guardrails: section has no pre/post-call PII, prompt-injection, or content-moderation checks in front of routed models.", + "Add a guardrails: section (top-level or under litellm_settings) if this proxy handles untrusted input.", + ), }; export function catalogFor(ruleId: string): RuleCatalogEntry | undefined { diff --git a/src/scanner/explainer.ts b/src/scanner/explainer.ts index 49f3c08..5e10a5e 100644 --- a/src/scanner/explainer.ts +++ b/src/scanner/explainer.ts @@ -617,6 +617,52 @@ setup: "run npm install"`, // Good — advisory-checked alternative, pinned "dependencies": { "postmark": "4.0.5" }`, }, + LLC001: { + summary: "A LiteLLM proxy config.yaml has a literal secret value instead of an env reference.", + whyRisky: + "LiteLLM proxy config files are routinely committed and shared across a team. A credential written directly into the config is exposed to everyone with repo access and every process that reads the file.", + howExploited: + "Anyone who clones the repo (or any tool that reads config.yaml) obtains a live provider API key, proxy master key, or salt key.", + howToFix: + "Reference the environment via LiteLLM's os.environ/VAR_NAME convention instead of inlining, and rotate the exposed credential immediately.", + codeExample: `# Bad (config.yaml) +litellm_params: + api_key: "sk-live-4f9a8b7c6d5e4f3a2b1c" + +# Good +litellm_params: + api_key: os.environ/OPENAI_API_KEY`, + }, + LLC002: { + summary: "A LiteLLM proxy routes to a provider api_base over plaintext HTTP.", + whyRisky: + "Requests, responses, and header credentials to a non-localhost endpoint travel unencrypted. An on-path attacker can read the traffic or tamper with it.", + howExploited: + "On a shared network, an attacker intercepts the HTTP traffic between the proxy and the provider, reading API keys and prompt/completion content.", + howToFix: "Use https:// for every non-localhost provider api_base URL.", + codeExample: `# Bad (config.yaml) +litellm_params: + api_base: "http://internal-llm.example.com/v1" + +# Good +litellm_params: + api_base: "https://internal-llm.example.com/v1"`, + }, + LLC003: { + summary: "A LiteLLM proxy config.yaml has no guardrails: section.", + whyRisky: + "This proxy routes models with no pre/post-call checks for PII, prompt injection, or content moderation. Absence of an optional feature is a nudge, not a proven gap — shown only with --paranoid.", + howExploited: + "Not directly exploitable on its own; it's the absence of a mitigating control that a proxy handling untrusted input would benefit from.", + howToFix: + "Add a guardrails: section (top-level or under litellm_settings) if this proxy handles untrusted input.", + codeExample: `# Good (config.yaml) +guardrails: + - guardrail_name: "pii-mask" + litellm_params: + guardrail: presidio + mode: pre_call`, + }, }; export class StaticExplainer implements Explainer { diff --git a/src/scanner/litellm-config-scanner.ts b/src/scanner/litellm-config-scanner.ts new file mode 100644 index 0000000..6e137bc --- /dev/null +++ b/src/scanner/litellm-config-scanner.ts @@ -0,0 +1,238 @@ +import fs from "node:fs"; +import path from "node:path"; +import { load as yamlLoad } from "js-yaml"; +import type { Finding } from "./types.js"; +import { evidenceConfidence } from "./confidence.js"; +import { stripBom } from "../utils/text.js"; + +/** + * Scans LiteLLM proxy `config.yaml` files for parsed, factual + * misconfigurations. Off-disk, not AST-based — same pattern as + * mcp-config-scanner.ts. + * + * Rules: + * LLC001 — hardcoded secret in litellm_params/general_settings + * LLC002 — plaintext HTTP provider endpoint (non-localhost) + * LLC003 — no guardrails configured (heuristic, --paranoid only) + */ + +const SKIP_DIRS = new Set([ + "node_modules", + ".git", + "dist", + "build", + "out", + ".next", + ".venv", + "venv", + "__pycache__", +]); + +const YAML_EXTENSIONS = new Set([".yaml", ".yml"]); + +export interface LiteLlmModelEntry { + model_name?: string; + litellm_params?: Record; +} + +export interface LiteLlmConfig { + model_list: LiteLlmModelEntry[]; + general_settings?: Record; + hasGuardrails: boolean; +} + +export function findLiteLlmConfigFiles(rootPath: string, skipPaths?: string[]): string[] { + const results: string[] = []; + const resolvedRoot = path.resolve(rootPath); + const skips = (skipPaths ?? []).map((p) => path.resolve(resolvedRoot, p)); + + function walk(dir: string, depth: number) { + if (depth > 6) return; + let entries: fs.Dirent[]; + try { + entries = fs.readdirSync(dir, { withFileTypes: true }); + } catch { + return; + } + for (const entry of entries) { + const full = path.join(dir, entry.name); + if (skips.some((s) => full === s || full.startsWith(s + path.sep))) continue; + if (entry.isDirectory()) { + if (SKIP_DIRS.has(entry.name)) continue; + walk(full, depth + 1); + } else if (entry.isFile()) { + if (YAML_EXTENSIONS.has(path.extname(entry.name).toLowerCase())) { + results.push(full); + } + } + } + } + + walk(resolvedRoot, 0); + return results; +} + +/** + * Structural gate: only treat a YAML file as a LiteLLM proxy config when it + * has a top-level `model_list` array with at least one entry carrying a real + * `litellm_params` object. This is LiteLLM's own proxy config shape and is + * distinctive enough to avoid false-positiving on unrelated YAML (k8s config, + * generic app config, etc). Anything that doesn't match is silently skipped. + */ +export function parseLiteLlmConfig(raw: string): LiteLlmConfig | null { + let parsed: unknown; + try { + parsed = yamlLoad(raw); + } catch { + return null; + } + if (typeof parsed !== "object" || parsed === null) return null; + + const obj = parsed as Record; + const rawModelList = obj.model_list; + if (!Array.isArray(rawModelList)) return null; + + const modelList: LiteLlmModelEntry[] = []; + let hasLitellmParams = false; + for (const rawEntry of rawModelList) { + if (typeof rawEntry !== "object" || rawEntry === null) continue; + const entry = rawEntry as Record; + const litellmParams = + typeof entry.litellm_params === "object" && entry.litellm_params !== null + ? (entry.litellm_params as Record) + : undefined; + if (litellmParams) hasLitellmParams = true; + modelList.push({ + model_name: typeof entry.model_name === "string" ? entry.model_name : undefined, + litellm_params: litellmParams, + }); + } + + if (!hasLitellmParams) return null; + + const generalSettings = + typeof obj.general_settings === "object" && obj.general_settings !== null + ? (obj.general_settings as Record) + : undefined; + + // guardrails can live top-level or nested under litellm_settings. + const litellmSettings = + typeof obj.litellm_settings === "object" && obj.litellm_settings !== null + ? (obj.litellm_settings as Record) + : undefined; + const hasGuardrails = + "guardrails" in obj || (litellmSettings !== undefined && "guardrails" in litellmSettings); + + return { model_list: modelList, general_settings: generalSettings, hasGuardrails }; +} + +/** Find the 1-based line where a string first appears, for report anchoring. */ +function lineOf(lines: string[], needle: string): number { + const idx = lines.findIndex((l) => l.includes(needle)); + return idx >= 0 ? idx + 1 : 1; +} + +// Known LiteLLM credential fields — api_key/master_key/salt_key are documented +// top-level litellm_params/general_settings fields; the suffix pattern covers +// bundled integration credentials (Langfuse, Aporia, etc.) that follow the +// same *_api_key/*_key/*_token naming convention. +const CREDENTIAL_FIELD = /(^api_key$|^master_key$|^salt_key$|(_api_key|_key|_token)$)/i; +const ENV_REFERENCE = /^os\.environ\//; + +function isLiteralSecret(key: string, value: unknown): value is string { + if (typeof value !== "string") return false; + if (!CREDENTIAL_FIELD.test(key)) return false; + if (value.trim().length < 8) return false; + if (ENV_REFERENCE.test(value.trim())) return false; + return true; +} + +export function scanLiteLlmConfigs(rootPath: string, skipPaths?: string[]): Finding[] { + const findings: Finding[] = []; + const resolvedRoot = path.resolve(rootPath); + + for (const configPath of findLiteLlmConfigFiles(resolvedRoot, skipPaths)) { + let raw: string; + try { + raw = stripBom(fs.readFileSync(configPath, "utf-8")); + } catch { + continue; + } + const relFile = path.relative(resolvedRoot, configPath); + const config = parseLiteLlmConfig(raw); + if (!config) continue; + const lines = raw.split(/\r?\n/); + + const credentialBlocks: Array<{ label: string; block: Record | undefined }> = [ + ...config.model_list.map((entry, i) => ({ + label: entry.model_name ?? `model_list[${i}]`, + block: entry.litellm_params, + })), + { label: "general_settings", block: config.general_settings }, + ]; + + for (const { label, block } of credentialBlocks) { + if (!block) continue; + + // LLC001 — hardcoded secret + for (const [key, value] of Object.entries(block)) { + if (!isLiteralSecret(key, value)) continue; + findings.push({ + rule_id: "LLC001", + title: "Hardcoded secret in LiteLLM config", + severity: "critical", + file: relFile, + line: lineOf(lines, key), + summary: `"${label}" has a literal value for "${key}" instead of an os.environ/ reference.`, + description: + "LiteLLM proxy config files are routinely committed and shared across a team. A credential written directly into the config, rather than referenced via LiteLLM's os.environ/VAR_NAME convention, is exposed to everyone with repo access and to any process that reads the file.", + recommendation: `Reference the environment instead of inlining the value, e.g. "${key}": "os.environ/${key.toUpperCase()}", and rotate the exposed credential now.`, + confidence: evidenceConfidence("proven"), + evidence: "proven", + }); + } + + // LLC002 — plaintext HTTP endpoint + const apiBase = block.api_base; + if (typeof apiBase === "string" && /^http:\/\//i.test(apiBase)) { + const host = apiBase.replace(/^http:\/\//i, "").split(/[/:]/)[0].toLowerCase(); + const isLocal = host === "localhost" || host === "127.0.0.1" || host === "0.0.0.0" || host === "::1"; + if (!isLocal) { + findings.push({ + rule_id: "LLC002", + title: "Plaintext HTTP provider endpoint", + severity: "high", + file: relFile, + line: lineOf(lines, apiBase), + summary: `"${label}" reaches ${apiBase} without TLS.`, + description: + "Requests and responses — including API keys sent as headers and prompt/completion content — travel unencrypted. An on-path attacker can read the traffic or tamper with it.", + recommendation: "Use https:// for all non-localhost provider api_base URLs.", + confidence: evidenceConfidence("proven"), + evidence: "proven", + }); + } + } + } + + // LLC003 — no guardrails configured (heuristic: absence is only a nudge) + if (!config.hasGuardrails) { + findings.push({ + rule_id: "LLC003", + title: "No guardrails configured", + severity: "low", + file: relFile, + line: 1, + summary: "LiteLLM proxy config has no guardrails: section.", + description: + "This config defines model routing but no guardrails. Absence of an optional feature is not itself a vulnerability — this is a nudge, not a proven gap — but LiteLLM Proxy supports pre/post-call guardrails for PII, prompt injection, and content moderation that this config isn't using.", + recommendation: + "Consider adding a guardrails: section (top-level or under litellm_settings) if this proxy handles untrusted input.", + confidence: evidenceConfidence("heuristic"), + evidence: "heuristic", + }); + } + } + + return findings; +} diff --git a/src/scanner/rules/index.ts b/src/scanner/rules/index.ts index 58108c2..9d638c4 100644 --- a/src/scanner/rules/index.ts +++ b/src/scanner/rules/index.ts @@ -62,6 +62,10 @@ export const RULES: Rule[] = [ // Config-file rules implemented outside the AST rule engine (mcp-config-scanner). export const CONFIG_RULE_IDS = ["MCP004", "MCP005", "MCP006"]; +// LiteLLM proxy config.yaml rules implemented outside the AST rule engine +// (litellm-config-scanner). +export const LITELLM_CONFIG_RULE_IDS = ["LLC001", "LLC002", "LLC003"]; + // Agent Skill (SKILL.md) rules implemented outside the AST rule engine (skill-scanner). export const SKILL_RULE_IDS = [ "SKL001", "SKL002", "SKL003", "SKL004", "SKL005", "SKL006", "SKL007", "SKL008", "SKL009", "SKL010", @@ -75,6 +79,7 @@ export const DEPENDENCY_RULE_IDS = ["DEP001", "DEP002", "DEP003"]; export const AVAILABLE_RULE_IDS = [ ...RULES.map((rule) => rule.id), ...CONFIG_RULE_IDS, + ...LITELLM_CONFIG_RULE_IDS, ...SKILL_RULE_IDS, ...DEPENDENCY_RULE_IDS, ]; diff --git a/src/scanner/scan.ts b/src/scanner/scan.ts index 1333fa8..27d2f5c 100644 --- a/src/scanner/scan.ts +++ b/src/scanner/scan.ts @@ -5,6 +5,7 @@ import { createScanProject } from "./project.js"; import { selectRules } from "./filters.js"; import { scanPythonFiles } from "./python-scanner.js"; import { scanMcpConfigs } from "./mcp-config-scanner.js"; +import { scanLiteLlmConfigs } from "./litellm-config-scanner.js"; import { scanSkillFiles } from "./skill-scanner.js"; import type { SourceFile } from "ts-morph"; @@ -57,6 +58,14 @@ export function scanRepositoryDetailed( ); findings.push(...mcpConfigFindings); + // LiteLLM proxy config files (config.yaml) + const liteLlmConfigFindings = scanLiteLlmConfigs(rootPath, options?.skipPaths).filter( + (f) => + (!options?.rules || options.rules.includes(f.rule_id)) && + !options?.blockedRules?.includes(f.rule_id), + ); + findings.push(...liteLlmConfigFindings); + // Agent Skill files (SKILL.md) const skillFindings = scanSkillFiles(rootPath, options?.skipPaths).filter( (f) => diff --git a/src/scanner/threat-model.ts b/src/scanner/threat-model.ts index 427cc5c..9f79e6e 100644 --- a/src/scanner/threat-model.ts +++ b/src/scanner/threat-model.ts @@ -20,6 +20,7 @@ const CATEGORY_LABELS: Record = { SKL: "Agent Skills", VEC: "Vector / RAG Pipeline", LLM: "LLM SDK Usage", + LLC: "LiteLLM Proxy Config", }; function ruleCategory(ruleId: string): string { @@ -59,6 +60,9 @@ function buildTrustBoundaries(findings: Finding[]): ThreatBoundary[] { } else if (f.rule_id.startsWith("VEC")) { from = "Vector Store / Document Pipeline"; to = "LLM Context / RAG"; + } else if (f.rule_id.startsWith("LLC")) { + from = "LiteLLM Proxy Config"; + to = "LLM Provider"; } else if (f.rule_id === "AI003") { from = "Unauthenticated Request"; to = "LLM Model"; diff --git a/test-fixtures/safe/litellm-config-clean.yaml b/test-fixtures/safe/litellm-config-clean.yaml new file mode 100644 index 0000000..8b8e9a5 --- /dev/null +++ b/test-fixtures/safe/litellm-config-clean.yaml @@ -0,0 +1,14 @@ +model_list: + - model_name: gpt-4 + litellm_params: + model: openai/gpt-4 + api_key: os.environ/OPENAI_API_KEY + api_base: https://api.openai.com/v1 +general_settings: + master_key: os.environ/LITELLM_MASTER_KEY + salt_key: os.environ/LITELLM_SALT_KEY +guardrails: + - guardrail_name: pii-mask + litellm_params: + guardrail: presidio + mode: pre_call diff --git a/test-fixtures/safe/litellm-config-not-litellm.yaml b/test-fixtures/safe/litellm-config-not-litellm.yaml new file mode 100644 index 0000000..8c3fea2 --- /dev/null +++ b/test-fixtures/safe/litellm-config-not-litellm.yaml @@ -0,0 +1,7 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: app-config +model_list: "not the litellm proxy schema, just a string field" +data: + api_key: this-would-be-a-hit-if-the-structural-gate-were-name-matching-only diff --git a/test-fixtures/vulnerable/litellm-config-http.yaml b/test-fixtures/vulnerable/litellm-config-http.yaml new file mode 100644 index 0000000..b795d30 --- /dev/null +++ b/test-fixtures/vulnerable/litellm-config-http.yaml @@ -0,0 +1,11 @@ +model_list: + - model_name: internal-llm + litellm_params: + model: openai/internal-llm + api_key: os.environ/INTERNAL_LLM_API_KEY + api_base: http://internal-llm.example.com/v1 +guardrails: + - guardrail_name: pii-mask + litellm_params: + guardrail: presidio + mode: pre_call diff --git a/test-fixtures/vulnerable/litellm-config-no-guardrails.yaml b/test-fixtures/vulnerable/litellm-config-no-guardrails.yaml new file mode 100644 index 0000000..92f7a45 --- /dev/null +++ b/test-fixtures/vulnerable/litellm-config-no-guardrails.yaml @@ -0,0 +1,6 @@ +model_list: + - model_name: gpt-4 + litellm_params: + model: openai/gpt-4 + api_key: os.environ/OPENAI_API_KEY + api_base: https://api.openai.com/v1 diff --git a/test-fixtures/vulnerable/litellm-config-secret.yaml b/test-fixtures/vulnerable/litellm-config-secret.yaml new file mode 100644 index 0000000..114fe56 --- /dev/null +++ b/test-fixtures/vulnerable/litellm-config-secret.yaml @@ -0,0 +1,12 @@ +model_list: + - model_name: gpt-4 + litellm_params: + model: openai/gpt-4 + api_key: sk-live-4f9a8b7c6d5e4f3a2b1c +general_settings: + master_key: sk-master-abcdef1234567890 +guardrails: + - guardrail_name: pii-mask + litellm_params: + guardrail: presidio + mode: pre_call diff --git a/test/corpus.test.js b/test/corpus.test.js index bb52ef2..6aab52c 100644 --- a/test/corpus.test.js +++ b/test/corpus.test.js @@ -39,6 +39,8 @@ const EXPECTED_VULNERABLE = [ ["MCP004", "vulnerable/mcp/.mcp.json"], ["MCP005", "vulnerable/mcp/.mcp.json"], ["MCP006", "vulnerable/mcp/.mcp.json"], + ["LLC001", "vulnerable/litellm-config-secret.yaml"], + ["LLC002", "vulnerable/litellm-config-http.yaml"], ["MCP007", "vulnerable/tool_poisoning.ts"], ["MCP008", "vulnerable/tool_poisoning.ts"], ["MCP009", "vulnerable/tool_poisoning.ts"], diff --git a/test/litellm-config.test.js b/test/litellm-config.test.js new file mode 100644 index 0000000..c0075f7 --- /dev/null +++ b/test/litellm-config.test.js @@ -0,0 +1,79 @@ +import test from "node:test"; +import assert from "node:assert/strict"; +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; +import { scanLiteLlmConfigs } from "../dist/scanner/litellm-config-scanner.js"; + +test("litellm config scanner flags hardcoded secret, http endpoint, and missing guardrails", () => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), "secureai-litellm-")); + fs.writeFileSync( + path.join(dir, "config.yaml"), + [ + "model_list:", + " - model_name: gpt-4", + " litellm_params:", + " model: openai/gpt-4", + " api_key: sk-live-4f9a8b7c6d5e4f3a2b1c", + " api_base: http://internal-llm.example.com/v1", + "general_settings:", + " master_key: sk-master-abcdef1234567890", + ].join("\n"), + ); + + const findings = scanLiteLlmConfigs(dir); + const ids = findings.map((f) => f.rule_id).sort(); + assert.deepEqual(ids, ["LLC001", "LLC001", "LLC002", "LLC003"]); + assert.ok(findings.filter((f) => f.rule_id !== "LLC003").every((f) => f.evidence === "proven")); + assert.equal(findings.find((f) => f.rule_id === "LLC003").evidence, "heuristic"); +}); + +test("litellm config scanner accepts env references, https, and a populated guardrails section", () => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), "secureai-litellm-safe-")); + fs.writeFileSync( + path.join(dir, "config.yaml"), + [ + "model_list:", + " - model_name: gpt-4", + " litellm_params:", + " model: openai/gpt-4", + " api_key: os.environ/OPENAI_API_KEY", + " api_base: https://internal-llm.example.com/v1", + "general_settings:", + " master_key: os.environ/LITELLM_MASTER_KEY", + "guardrails:", + " - guardrail_name: pii-mask", + " litellm_params:", + " guardrail: presidio", + " mode: pre_call", + ].join("\n"), + ); + + assert.deepEqual(scanLiteLlmConfigs(dir), []); +}); + +test("litellm config scanner ignores an unrelated YAML file with a coincidental model_list field", () => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), "secureai-litellm-gate-")); + fs.writeFileSync( + path.join(dir, "config.yaml"), + [ + "apiVersion: v1", + "kind: ConfigMap", + "model_list: not-an-array", + "data:", + " api_key: hardcoded-but-irrelevant-here", + ].join("\n"), + ); + + assert.deepEqual(scanLiteLlmConfigs(dir), []); +}); + +test("litellm config scanner ignores model_list entries without litellm_params", () => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), "secureai-litellm-gate2-")); + fs.writeFileSync( + path.join(dir, "config.yaml"), + ["model_list:", " - model_name: gpt-4", " - model_name: gpt-3.5"].join("\n"), + ); + + assert.deepEqual(scanLiteLlmConfigs(dir), []); +}); diff --git a/test/run-tests.js b/test/run-tests.js index 62171db..0a71541 100644 --- a/test/run-tests.js +++ b/test/run-tests.js @@ -6,6 +6,7 @@ import "./ignore-annotations.test.js"; import "./new-ai-rules.test.js"; import "./reporter-snippet.test.js"; import "./mcp-config.test.js"; +import "./litellm-config.test.js"; import "./python-ast.test.js"; import "./skill-scanner.test.js"; import "./corpus.test.js"; From cb13e5c26e6967f2b48576b84a9cadd97277b30b Mon Sep 17 00:00:00 2001 From: akanthed Date: Wed, 19 Aug 2026 00:37:13 +0530 Subject: [PATCH 3/6] Add new litellm config --- README.md | 13 +++++-- scripts/regression-scan.js | 4 ++ src/scanner/litellm-config-scanner.ts | 55 +++++++++++++++++++++++++-- src/scanner/scan.ts | 13 ++++++- test/litellm-config.test.js | 29 ++++++++++++++ 5 files changed, 105 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index 1ee8f13..f45a70b 100644 --- a/README.md +++ b/README.md @@ -9,11 +9,11 @@ [![Node](https://img.shields.io/badge/node-%3E%3D22.12-brightgreen)](https://nodejs.org) [![OWASP](https://img.shields.io/badge/OWASP-LLM%20%C2%B7%20ASI%20%C2%B7%20MCP%20Top%2010-000000)](#rules) -**The AI security scanner that proves its findings.** +**Offline CLI that scans TypeScript, JavaScript, and Python for LLM, MCP, Agent Skill, and RAG risks — import-resolved dataflow evidence, zero default false positives, mapped to OWASP LLM/ASI/MCP Top 10.** -SecureAI-Scan finds LLM, MCP, Agent Skill, and RAG vulnerabilities in **TypeScript, JavaScript, and Python** — and shows you the evidence: the exact source → flow → sink path for every dataflow finding, resolved through real imports, not keyword matching. +Most scanners in this space pattern-match a keyword and call it a finding. SecureAI-Scan traces the actual source → flow → sink path through real, import-resolved code — and a default scan shows you only what it can prove. No account, no cloud upload, nothing leaves your machine. -It provides launch-week support for the official [OWASP Top 10 for LLM Applications 2026](https://genai.owasp.org/resource/owasp-genai-llm-top-10-2026/), alongside the [Top 10 for Agentic Applications (2026)](https://genai.owasp.org/) and the [MCP Top 10](https://owasp.org/www-project-mcp-top-10/). Every threat model distinguishes static coverage from runtime concerns. +Covers the official [OWASP Top 10 for LLM Applications 2026](https://genai.owasp.org/resource/owasp-genai-llm-top-10-2026/), [Top 10 for Agentic Applications (2026)](https://genai.owasp.org/), and the [MCP Top 10](https://owasp.org/www-project-mcp-top-10/) from launch week. ## Get started in 30 seconds @@ -38,6 +38,8 @@ No account, cloud upload, Python interpreter, or configuration required. TypeScr **Is this for you?** SecureAI-Scan is scoped deliberately to LLM, MCP, and RAG/agent risks — prompt injection, tool poisoning, unsafe output handling, vector-store access control, agent-skill poisoning. It is not a general SAST or secrets scanner, and doesn't try to be one; a known-malicious package with no LLM-shaped payload (e.g. a hardcoded exfiltration address in an email API call) is caught by the offline advisory list (`DEP003`), not a pattern rule. If your codebase talks to an LLM, an MCP server, a vector store, or ships Agent Skills, this is built for you. +New: static config scanning for LiteLLM Proxy (`config.yaml`) — hardcoded secrets, plaintext provider endpoints, missing guardrails. See [Rules](#rules) (LLC001–LLC003). + ## Contents - [Why this scanner is different](#why-this-scanner-is-different) @@ -205,7 +207,7 @@ Scanning clean? Add the badge to your own README: ## Rules -**39 rules**, mapped to the official OWASP Top 10 for LLM Applications (2026) — plus, where applicable, the OWASP Top 10 for Agentic Applications (2026, ASI), the OWASP MCP Top 10 (2025), and an EU AI Act article. See the [versioned 2026 coverage and limits](docs/OWASP2026.md); `threat-model` renders the matrix for each scanned project. +**42 rules**, mapped to the official OWASP Top 10 for LLM Applications (2026) — plus, where applicable, the OWASP Top 10 for Agentic Applications (2026, ASI), the OWASP MCP Top 10 (2025), and an EU AI Act article. See the [versioned 2026 coverage and limits](docs/OWASP2026.md); `threat-model` renders the matrix for each scanned project. | Rule | What it proves | OWASP | |------|----------------|-------| @@ -248,6 +250,9 @@ Scanning clean? Add the badge to your own README: | DEP001 | Dependency name not found in the registry (opt-in `--check-dependencies`) | LLM04 | | DEP002 | Dependency name one edit away from a popular package (opt-in) | LLM04 | | DEP003 | Dependency with a documented malicious release or critical CVE — checked offline on every scan, version-range aware (postmark-mcp, mcp-remote CVE-2025-6514, …) | LLM04 · MCP04 | +| LLC001 | Hardcoded secret in a LiteLLM proxy `config.yaml` | LLM02 | +| LLC002 | LiteLLM proxy `api_base` reachable over plaintext HTTP | LLM04 | +| LLC003 | LiteLLM proxy config has no `guardrails:` section (heuristic, `--paranoid` only) | LLM03 | `secureai-scan explain ` gives the exploit walkthrough and a before/after code example for any rule. diff --git a/scripts/regression-scan.js b/scripts/regression-scan.js index cf79f9e..628794f 100644 --- a/scripts/regression-scan.js +++ b/scripts/regression-scan.js @@ -56,6 +56,10 @@ const REPOS = [ { name: "llama_index", url: "https://github.com/run-llama/llama_index.git" }, { name: "anthropic-skills", url: "https://github.com/anthropics/skills.git" }, { name: "cisco-skill-scanner", url: "https://github.com/cisco-ai-defense/skill-scanner.git" }, + // litellm added for LLC001-003 (litellm-config-scanner): the official repo + // ships real proxy config.yaml examples under litellm/proxy/example_config_yaml + // and docs, the only repo in this set that exercises those rules at all. + { name: "litellm", url: "https://github.com/BerriAI/litellm.git" }, ]; const fresh = process.argv.includes("--fresh"); diff --git a/src/scanner/litellm-config-scanner.ts b/src/scanner/litellm-config-scanner.ts index 6e137bc..00a60c5 100644 --- a/src/scanner/litellm-config-scanner.ts +++ b/src/scanner/litellm-config-scanner.ts @@ -2,7 +2,7 @@ import fs from "node:fs"; import path from "node:path"; import { load as yamlLoad } from "js-yaml"; import type { Finding } from "./types.js"; -import { evidenceConfidence } from "./confidence.js"; +import { evidenceConfidence, identifierTokens } from "./confidence.js"; import { stripBom } from "../utils/text.js"; /** @@ -139,11 +139,54 @@ function lineOf(lines: string[], needle: string): number { const CREDENTIAL_FIELD = /(^api_key$|^master_key$|^salt_key$|(_api_key|_key|_token)$)/i; const ENV_REFERENCE = /^os\.environ\//; +// LiteLLM's own docs/tests inline placeholder values like "fake-key", +// "my-fake-key", "sk-lar1-demo" to demonstrate config shape — not real +// secrets. Found scanning BerriAI/litellm itself (regression). A value built +// entirely from ordinary placeholder words, with no other content, isn't +// evidence of a leaked credential. +const PLACEHOLDER_TOKENS = new Set([ + "fake", + "dummy", + "test", + "tests", + "demo", + "sample", + "example", + "examples", + "placeholder", + "changeme", + "todo", + "tbd", + "xxx", + "redacted", + "mock", + "stub", + "insert", + "notreal", + "your", +]); + +function isPlaceholderValue(value: string): boolean { + const tokens = identifierTokens(value); + if (tokens.length > 0 && tokens.every((t) => PLACEHOLDER_TOKENS.has(t))) return true; + + // Real credentials are one long, effectively random alphanumeric run + // (sometimes with a short vendor prefix like "sk-"/"AKIA" split off by a + // hyphen). Hyphen/underscore-joined human phrases — "sk-lar1-demo" — never + // produce a long unbroken run even when no individual word is on the deny + // list above. Below this length, treat it as not credential-shaped. + const runs = value.match(/[A-Za-z0-9]+/g) ?? []; + const longestRun = Math.max(0, ...runs.map((r) => r.length)); + return longestRun < 12; +} + function isLiteralSecret(key: string, value: unknown): value is string { if (typeof value !== "string") return false; if (!CREDENTIAL_FIELD.test(key)) return false; - if (value.trim().length < 8) return false; - if (ENV_REFERENCE.test(value.trim())) return false; + const trimmed = value.trim(); + if (trimmed.length < 8) return false; + if (ENV_REFERENCE.test(trimmed)) return false; + if (isPlaceholderValue(trimmed)) return false; return true; } @@ -182,7 +225,11 @@ export function scanLiteLlmConfigs(rootPath: string, skipPaths?: string[]): Find title: "Hardcoded secret in LiteLLM config", severity: "critical", file: relFile, - line: lineOf(lines, key), + // Search by the literal value, not the key: key names like "api_key" + // repeat across every model_list entry, so a key-only search can + // anchor the finding to an unrelated (possibly safe) line with the + // same key. The flagged value itself is what's unique. + line: lineOf(lines, value), summary: `"${label}" has a literal value for "${key}" instead of an os.environ/ reference.`, description: "LiteLLM proxy config files are routinely committed and shared across a team. A credential written directly into the config, rather than referenced via LiteLLM's os.environ/VAR_NAME convention, is exposed to everyone with repo access and to any process that reads the file.", diff --git a/src/scanner/scan.ts b/src/scanner/scan.ts index 27d2f5c..cb9f924 100644 --- a/src/scanner/scan.ts +++ b/src/scanner/scan.ts @@ -39,7 +39,18 @@ export function scanRepositoryDetailed( const activeRules = selectRules(RULES, options?.rules, options?.blockedRules); for (const rule of activeRules) { - findings.push(...rule.run(context)); + try { + findings.push(...rule.run(context)); + } catch (err) { + // A crash in one rule (e.g. a ts-morph type-checker failure on an + // unusual file in a large multi-tsconfig monorepo) must not silently + // discard every other rule's findings for the whole repository — + // found scanning BerriAI/litellm, where a single dashboard file + // crashed AI001 and took the entire scan down with it. + console.error( + `Warning: rule ${rule.id} failed and was skipped: ${err instanceof Error ? err.message : String(err)}`, + ); + } } // Python scanning — merged into same findings list diff --git a/test/litellm-config.test.js b/test/litellm-config.test.js index c0075f7..9a2838d 100644 --- a/test/litellm-config.test.js +++ b/test/litellm-config.test.js @@ -77,3 +77,32 @@ test("litellm config scanner ignores model_list entries without litellm_params", assert.deepEqual(scanLiteLlmConfigs(dir), []); }); + +// Regression: found scanning BerriAI/litellm itself. A repo with many +// model_list entries repeats the key name "api_key" dozens of times; the +// finding must anchor to the line holding the actual literal secret, not +// the first line in the file containing the word "api_key". +test("litellm config scanner anchors LLC001 to the offending value's line, not the first key match in the file", () => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), "secureai-litellm-lineattrib-")); + fs.writeFileSync( + path.join(dir, "config.yaml"), + [ + "model_list:", + " - model_name: safe-one", + " litellm_params:", + " model: azure/gpt-4", + " api_key: os.environ/AZURE_API_KEY", + " - model_name: leaky-one", + " litellm_params:", + " model: openai/gpt-4", + " api_key: sk-live-4f9a8b7c6d5e4f3a2b1c", + "guardrails:", + " - guardrail_name: pii-mask", + ].join("\n"), + ); + + const findings = scanLiteLlmConfigs(dir); + const llc001 = findings.filter((f) => f.rule_id === "LLC001"); + assert.equal(llc001.length, 1); + assert.equal(llc001[0].line, 9, "must point at the literal-secret line, not the env-ref line above it"); +}); From 7a8465982a732d8eb84dd3319348c97134f8baa0 Mon Sep 17 00:00:00 2001 From: akanthed Date: Wed, 19 Aug 2026 01:00:33 +0530 Subject: [PATCH 4/6] feat: enhance scanning rules for LiteLLM to improve detection of user-controlled inputs and context handling --- src/scanner/python-scanner.ts | 46 +++++++++++++- src/scanner/rules/mcp-dynamic-server-url.ts | 15 +++-- .../safe/ai003_fastapi_depends_param_auth.py | 27 ++++++++ .../safe/mcp001_module_scope_false_context.py | 61 +++++++++++++++++++ test-fixtures/safe/mcp_url_parser_utility.ts | 17 ++++++ .../safe/re_search_not_vector_search.py | 12 ++++ 6 files changed, 172 insertions(+), 6 deletions(-) create mode 100644 test-fixtures/safe/ai003_fastapi_depends_param_auth.py create mode 100644 test-fixtures/safe/mcp001_module_scope_false_context.py create mode 100644 test-fixtures/safe/mcp_url_parser_utility.ts create mode 100644 test-fixtures/safe/re_search_not_vector_search.py diff --git a/src/scanner/python-scanner.ts b/src/scanner/python-scanner.ts index 744c540..4555717 100644 --- a/src/scanner/python-scanner.ts +++ b/src/scanner/python-scanner.ts @@ -124,7 +124,12 @@ const VECTOR_SEARCH_PATTERNS = [ /\.\s*as_retriever\s*\(/, /index\s*\.\s*query\s*\(/, /collection\s*\.\s*query\s*\(/, - /\.\s*search\s*\(\s*[^)]*vector/i, + // Excludes `re.search(...)`/`regex.search(...)` — Python's stdlib regex + // search, unrelated to vector stores. Found scanning BerriAI/litellm: + // `re.search(r"/vector_stores/([^/]+)/", path)` (URL-path parsing) matched + // only because the *regex pattern string* happened to contain the + // substring "vector", nothing to do with a vector-store client. + /(? matchesAny(decorator.text, AUTH_DECORATORS))) return null; if (matchesAny(fn.body.text, AUTH_DECORATORS)) return null; + // The parameter list is where FastAPI auth most commonly lives — a + // `Depends(...)` default value, invisible to the decorator/body checks + // above. + const parameters = fn.node.childForFieldName("parameters"); + if (parameters && matchesAny(parameters.text, AUTH_DECORATORS)) return null; return { ...findingBase("AI003", "LLM call before authentication", "critical", file, i + 1), @@ -787,6 +808,27 @@ function descriptionValueAtLine(src: PythonSource, line: number): PythonNode | u }); } +// `pythonScope` falls back to the whole module when a node isn't inside a +// function — fine for taint scoping, but wrong here: a module-level dict +// literal (e.g. an admin-UI settings schema) in a large file inherits "the +// entire file mentions MCP somewhere" as context, which is true of nearly +// any sizeable proxy/server file that also implements real MCP endpoints. +// Found scanning BerriAI/litellm's 17k-line proxy_server.py, where an +// unrelated settings-schema description ("...adds cache_control to the +// system prompt...") matched only because "mcp_tools" appears elsewhere in +// the same file. Cap the module-level fallback to a small line window +// around the description instead of the full file. +const MCP001_MODULE_WINDOW_LINES = 40; + +function mcp001Context(src: PythonSource, description: PythonNode, line: number): string { + const fn = src.ast.enclosingFunction(description); + if (fn) return fn.body.text; + const lines = src.ast.root.text.split(/\r?\n/); + const start = Math.max(0, line - MCP001_MODULE_WINDOW_LINES); + const end = Math.min(lines.length, line + 5); + return lines.slice(start, end).join("\n"); +} + function checkMCP001(src: PythonSource, i: number, file: string): Finding | null { const description = descriptionValueAtLine(src, i); if (!description) return null; @@ -794,7 +836,7 @@ function checkMCP001(src: PythonSource, i: number, file: string): Finding | null const matched = INJECTION_PHRASES.find((phrase) => lower.includes(phrase)); if (!matched) return null; - const ctx = pythonScope(src, description).text; + const ctx = mcp001Context(src, description, i); if (!MCP_LISTING_HINT.test(ctx)) return null; return { diff --git a/src/scanner/rules/mcp-dynamic-server-url.ts b/src/scanner/rules/mcp-dynamic-server-url.ts index 959e0a0..fd85fc0 100644 --- a/src/scanner/rules/mcp-dynamic-server-url.ts +++ b/src/scanner/rules/mcp-dynamic-server-url.ts @@ -68,10 +68,17 @@ export function isUserControlledValue(valueNode: Node, taintedVars: Set) export function collectRequestDerivedVars(fnNode: Node): Set { const tainted = new Set(); - for (const param of "getParameters" in fnNode ? (fnNode as any).getParameters() : []) { - const nameNode = param.getNameNode?.(); - if (nameNode && Node.isIdentifier(nameNode)) tainted.add(nameNode.getText()); - } + // Deliberately does NOT seed `tainted` from the function's own parameter + // names. That treated every parameter of every function as "user + // input" regardless of the function's role — found scanning + // BerriAI/litellm's ui/litellm-dashboard: extractMCPToken(url: string), a + // pure URL-parsing utility with no request boundary anywhere nearby, was + // flagged purely because it has a parameter named "url" that ends up in + // an object literal under a `baseUrl` key. The known-vulnerable fixture + // (mcp_dynamic_url.ts) doesn't rely on this: it matches `req.body.serverUrl` + // directly via REQUEST_SOURCES text below. Only actual evidence — a + // variable initialized from a request-shaped expression, or propagated + // from another tainted variable — should seed this set. for (const decl of fnNode.getDescendantsOfKind(SyntaxKind.VariableDeclaration)) { const init = decl.getInitializer(); if (!init) continue; diff --git a/test-fixtures/safe/ai003_fastapi_depends_param_auth.py b/test-fixtures/safe/ai003_fastapi_depends_param_auth.py new file mode 100644 index 0000000..0100373 --- /dev/null +++ b/test-fixtures/safe/ai003_fastapi_depends_param_auth.py @@ -0,0 +1,27 @@ +# Regression fixture: found scanning BerriAI/litellm. FastAPI's idiomatic +# per-route auth is a `Depends(...)` dependency injected via a parameter +# default (and/or the decorator's `dependencies=[...]` kwarg) — not a +# decorator by itself, and not something written in the function body. +# AI003 previously never looked at the parameter list at all. +import openai +from fastapi import APIRouter, Depends + +router = APIRouter() + + +async def user_api_key_auth(): + ... + + +@router.get( + "/health/services", + dependencies=[Depends(user_api_key_auth)], +) +async def health_services_endpoint( + user_api_key_dict=Depends(user_api_key_auth), +): + client = openai.OpenAI() + return client.chat.completions.create( + model="gpt-4", + messages=[{"role": "user", "content": "ping"}], + ) diff --git a/test-fixtures/safe/mcp001_module_scope_false_context.py b/test-fixtures/safe/mcp001_module_scope_false_context.py new file mode 100644 index 0000000..58268c2 --- /dev/null +++ b/test-fixtures/safe/mcp001_module_scope_false_context.py @@ -0,0 +1,61 @@ +# Regression fixture: found scanning BerriAI/litellm's 17k-line +# proxy_server.py. A module-level settings-schema dict (no enclosing +# function) with an injection-phrase-shaped description must not be treated +# as MCP tool metadata just because an unrelated MCP listing endpoint exists +# far away in the same file — the module-level context window is capped, +# not "the whole file". +UI_SETTINGS_SCHEMA = { + "enable_anthropic_prompt_caching": { + "type": "Boolean", + "description": ( + "Auto-adds cache_control to the system prompt and trailing turn " + "for supported Anthropic and Bedrock Claude models." + ), + }, +} + + +def _filler_1(): + return 1 + + +def _filler_2(): + return 2 + + +def _filler_3(): + return 3 + + +def _filler_4(): + return 4 + + +def _filler_5(): + return 5 + + +def _filler_6(): + return 6 + + +def _filler_7(): + return 7 + + +def _filler_8(): + return 8 + + +def _filler_9(): + return 9 + + +def _filler_10(): + return 10 + + +# Real MCP tool listing lives far below — outside the settings-dict's +# context window, so it must not retroactively justify the finding above. +def list_mcp_tools(): + return [] diff --git a/test-fixtures/safe/mcp_url_parser_utility.ts b/test-fixtures/safe/mcp_url_parser_utility.ts new file mode 100644 index 0000000..a677d83 --- /dev/null +++ b/test-fixtures/safe/mcp_url_parser_utility.ts @@ -0,0 +1,17 @@ +// Safe: found scanning BerriAI/litellm's ui/litellm-dashboard/src/app/(dashboard)/mcp-servers/_components/utils.tsx. +// A pure URL-parsing utility whose parameter happens to be named "url" is +// not "user-controlled input" in the request-taint sense MCP002 targets — +// nothing here proves the parameter came from an HTTP request rather than, +// say, a hardcoded config value passed by the caller. +export const extractMCPToken = (url: string): { token: string | null; baseUrl: string } => { + const mcpIndex = url.indexOf("/mcp/"); + if (mcpIndex === -1) return { token: null, baseUrl: url }; + + const parts = url.split("/mcp/"); + if (parts.length !== 2) return { token: null, baseUrl: url }; + + const afterMcp = parts[1]; + if (!afterMcp) return { token: null, baseUrl: url }; + + return { token: afterMcp, baseUrl: parts[0] + "/mcp/" }; +}; diff --git a/test-fixtures/safe/re_search_not_vector_search.py b/test-fixtures/safe/re_search_not_vector_search.py new file mode 100644 index 0000000..a8f75b1 --- /dev/null +++ b/test-fixtures/safe/re_search_not_vector_search.py @@ -0,0 +1,12 @@ +"""Found scanning BerriAI/litellm's http_parsing_utils.py: re.search() with a +regex pattern string that happens to contain "vector" is stdlib regex, not a +vector-store similarity search. VEC001 must not fire on this. +""" +import re + + +def populate_request_with_path_params(request_data, path): + vector_store_match = re.search(r"/vector_stores/([^/]+)/", path) + if vector_store_match: + vector_store_id = vector_store_match.group(1) + request_data.setdefault("vector_store_id", vector_store_id) From 135ca363cc98ef7134c54f5a7fdca86966052b72 Mon Sep 17 00:00:00 2001 From: akanthed Date: Wed, 19 Aug 2026 01:14:35 +0530 Subject: [PATCH 5/6] feat: update descriptions for scanning tools and skills to clarify usage for AI/LLM security reviews --- README.md | 2 +- docs/RealWorldFindings.md | 20 ++++++++++++++++++++ mcp-server/index.js | 4 ++-- skills/secureai-scan/SKILL.md | 4 ++-- 4 files changed, 25 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index f45a70b..52184aa 100644 --- a/README.md +++ b/README.md @@ -62,7 +62,7 @@ No account, cloud upload, Python interpreter, or configuration required. TypeScr - **Evidence tiers, not noise.** Every finding is `proven` (traced dataflow or parsed config fact), `likely` (resolved sink, one heuristic hop), or `heuristic`. **A default scan shows only proven + likely.** Heuristics are opt-in via `--paranoid`. - **Import-resolved detection.** A call is only an "LLM call" if it resolves to a real SDK import (`openai`, `@anthropic-ai/sdk`, `ai`, `@google/genai`, LangChain, Bedrock, …). Your Google Maps client will never be flagged as an LLM again. -- **Precision-gated, and benchmarked against real repos.** The test suite asserts every vulnerable fixture fires *and* every safe fixture stays clean — a false positive on the safe corpus fails the build. Beyond that, `npm run regression` scans real public repos (OpenAI/Anthropic/Vercel AI SDKs, official MCP servers, LlamaIndex) against a committed, hand-reviewed baseline and **fails on any new `proven`/`likely` finding**. See [Testing & benchmarking](#testing--benchmarking) for the actual before/after numbers, or [What we found scanning real repos](docs/RealWorldFindings.md) for the story behind them — a 6/6 catch rate on a labeled malicious-skill corpus, and why we're *not* calling llama_index "vulnerable" over an honest library-level finding. +- **Precision-gated, and benchmarked against real repos.** The test suite asserts every vulnerable fixture fires *and* every safe fixture stays clean — a false positive on the safe corpus fails the build. Beyond that, `npm run regression` scans real public repos (OpenAI/Anthropic/Vercel AI SDKs, official MCP servers, LlamaIndex) against a committed, hand-reviewed baseline and **fails on any new `proven`/`likely` finding**. See [Testing & benchmarking](#testing--benchmarking) for the actual before/after numbers, or [What we found scanning real repos](docs/RealWorldFindings.md) for the story behind them — a 6/6 catch rate on a labeled malicious-skill corpus, and why we're *not* calling llama_index "vulnerable" over an honest library-level finding. [Discussion write-up →](https://github.com/akanthed/SecureAI-Scan/discussions/19) - **SARIF for GitHub code scanning.** `--output report.sarif` puts findings inline on pull requests and in the Security tab. - **AI-BOM.** `secureai-scan bom .` builds a syntax-derived inventory of SDKs, model IDs, vector stores, agent frameworks, and MCP servers, mapped to OWASP LLM Top 10 / EU AI Act documentation needs. - **MCP config scanning.** Parses `.mcp.json`, `claude_desktop_config.json`, `.cursor/mcp.json`: unpinned `npx -y` servers, inline secrets, plaintext HTTP transports. diff --git a/docs/RealWorldFindings.md b/docs/RealWorldFindings.md index dea85a7..e7b1b4d 100644 --- a/docs/RealWorldFindings.md +++ b/docs/RealWorldFindings.md @@ -38,6 +38,26 @@ The same regression scan is also how we caught our own bugs. An earlier run agai All three were fixed at the root cause, not patched at the call site, and pinned as permanent fixtures so they can't regress silently. Full numbers, plus the same before/after treatment for `vercel/ai`, `openai-node`, `anthropic-sdk-typescript`, and `modelcontextprotocol/typescript-sdk`, are in the [Testing & benchmarking](../README.md#testing--benchmarking) section of the README. +## A live example: adding LiteLLM to the regression set found three bugs before it found one real issue + +When we added static `config.yaml` scanning for LiteLLM Proxy (`LLC001`–`LLC003`: hardcoded secrets, plaintext provider endpoints, missing guardrails), none of the repos already in the regression set exercise those rules — none of them ship a LiteLLM proxy config. So we added [BerriAI/litellm](https://github.com/BerriAI/litellm) itself, the official repo, specifically to get real coverage. It's a large, real production monorepo (6,978 TS/JS files, thousands more in Python) — a genuinely harder target than our own fixtures. + +First pass surfaced two new false-positive classes and one unrelated but serious robustness bug, in that order: + +1. **Line misattribution.** `LLC001` correctly detected that *some* entry in a `model_list` had a hardcoded secret, but anchored the finding to the first line in the file containing the string `api_key` — not the line that actually held the offending value. In a config with dozens of `api_key:` entries, that meant a reported finding could point straight at an `os.environ/...` reference and contradict its own evidence. Fixed by anchoring on the flagged *value* instead of the key name (unique per credential, unlike the key). +2. **Placeholder-value false positives.** LiteLLM's own docs and tests inline dummy values like `fake-key`, `my-fake-key`, `sk-lar1-demo` to demonstrate config shape — not real secrets. `LLC001` initially flagged all of them. Fixed with a placeholder-word check plus a "does this look like a random credential blob or a human-typed phrase" heuristic (longest unbroken alphanumeric run < 12 chars ⇒ not credential-shaped) rather than guessing at a denylist of exact strings. +3. **A rule crash was silently killing the entire scan.** Unrelated to LiteLLM's config files — a ts-morph type-checker failure on one file in the repo's Next.js admin dashboard (a large monorepo with multiple independent `tsconfig.json` files merged into one project) took down the *whole* scan, discarding every rule's findings, not just the one that crashed. This is worse than any false positive: a scan that silently reports nothing instead of erroring looks identical to "clean." Fixed by isolating each rule's `run()` — one rule failing now logs a warning and the rest of the scan continues. + +Two more, smaller false positives surfaced once the scan could actually complete against the full repo: + +4. `MCP001` (Python) matched the phrase "system prompt" inside an *admin-UI settings description* — plain English describing an unrelated caching feature, in a module-level dict inside a 17,000-line file. The scoping guard meant to require real MCP-listing context fell back to the entire file when a match wasn't inside a function, so "the file mentions MCP somewhere" (true of nearly any file that size in this codebase) satisfied it. Capped the fallback to a small line window instead of the whole module. +5. `MCP002` (TypeScript) flagged a pure URL-parsing utility (`extractMCPToken(url: string)`) as "MCP server URL from user input" for no reason other than "url" being the name of one of its own parameters — a blanket rule that treated *every* function parameter as request-tainted regardless of whether the function had anything to do with handling a request. The known-vulnerable fixture never needed this: it matches `req.body.serverUrl` directly. Removed the blanket taint. +6. `VEC001` matched Python's stdlib `re.search(r"/vector_stores/([^/]+)/", path)` — ordinary URL-path parsing — as a vector-store similarity search, purely because the regex *pattern string* contained the substring "vector" and the call syntactically looked like `.search(...vector...)`. Added an exclusion for `re.search`/`regex.search`. + +After all six fixes: **zero LLC001/LLC002 false positives, one confirmed-real `LLC002` finding** (a proxy config routing to an internal `vllm-command` host over plain `http://`, in `litellm/proxy/_super_secret_config.yaml`), and the rest of the rule set continuing to run clean against the same repo. Every fix shipped with a permanent fixture under `test-fixtures/safe/`, named for the pattern, so none of these six can regress silently. + +This is what "zero tolerance for false positives" costs in practice: not zero bugs, but a standing habit of reading every new finding against its source line before trusting it, on code we didn't write. + ## Run it yourself ```bash diff --git a/mcp-server/index.js b/mcp-server/index.js index 1c49bad..43d0d7f 100644 --- a/mcp-server/index.js +++ b/mcp-server/index.js @@ -67,7 +67,7 @@ const TOOLS = [ { name: "scan_repository", description: - "Scan a local repository for AI/LLM security vulnerabilities with evidence-tiered findings (proven/likely/heuristic). Detects prompt injection (with source→sink dataflow traces), MCP supply-chain issues, RAG data poisoning, and agent trust violations in TypeScript/JS, Python, and MCP config files.", + "Scan a local repository for AI/LLM security vulnerabilities with evidence-tiered findings (proven/likely/heuristic). Detects prompt injection (with source→sink dataflow traces), MCP supply-chain issues, RAG data poisoning, and agent trust violations in TypeScript/JS, Python, and MCP config files. Use for requests like 'scan my MCP config', 'check this repo for prompt injection risk', or any AI/LLM security review of local code.", inputSchema: { type: "object", properties: { @@ -130,7 +130,7 @@ const TOOLS = [ { name: "scan_untrusted_target", description: - "Fetch and scan a single Agent Skill or MCP server BEFORE it is trusted/installed — no repo, no config. Accepts a local path, a git URL, a GitHub \"owner/repo\" shorthand, or (for MCP servers) a bare npm package name. Nothing fetched is ever executed: npm targets are downloaded with 'npm pack' (tarball only, no install, no lifecycle scripts), git targets with 'git clone --depth 1'. Use this before recommending or installing any third-party skill or MCP server.", + "Fetch and scan a single Agent Skill or MCP server BEFORE it is trusted/installed — no repo, no config. Accepts a local path, a git URL, a GitHub \"owner/repo\" shorthand, or (for MCP servers) a bare npm package name. Nothing fetched is ever executed: npm targets are downloaded with 'npm pack' (tarball only, no install, no lifecycle scripts), git targets with 'git clone --depth 1'. Use this before recommending or installing any third-party skill or MCP server, or whenever the user asks 'is this skill safe?', 'is this MCP server safe to install?', or wants a third-party skill/MCP server checked before trusting it.", inputSchema: { type: "object", properties: { diff --git a/skills/secureai-scan/SKILL.md b/skills/secureai-scan/SKILL.md index 22295ba..ca5ee63 100644 --- a/skills/secureai-scan/SKILL.md +++ b/skills/secureai-scan/SKILL.md @@ -1,11 +1,11 @@ --- name: secureai-scan -description: Scans a repository for LLM, MCP, Agent Skill, and RAG security vulnerabilities using the secureai-scan CLI, and explains any findings with a concrete fix. +description: Use when the user asks to scan a repo for AI/LLM security issues, wants to know "is this skill safe?" before installing an Agent Skill, needs to "scan my MCP config" or check an MCP server before trusting it, asks about prompt injection / tool poisoning / RAG poisoning risk in their code, or is about to install any MCP server or Claude Skill from GitHub, npm, or an untrusted link. Runs the secureai-scan CLI offline and explains findings with a concrete fix. --- # SecureAI-Scan -Use this skill when the user asks for a security review of code that talks to an LLM, an MCP server, a vector store, or ships Agent Skills — or when they're about to install an MCP server or a Claude Skill from somewhere untrusted and want to check it first. +Use this skill when the user asks for a security review of code that talks to an LLM, an MCP server, a vector store, or ships Agent Skills — or when they're about to install an MCP server or a Claude Skill from somewhere untrusted and want to check it first. This also covers direct questions like "is this MCP server safe?", "is this skill safe to install?", or "scan my MCP config for problems" — run the scan proactively rather than answering from general knowledge. ## Running a scan From 674ae559e28bb739ba641ce5c060a30e78684456 Mon Sep 17 00:00:00 2001 From: akanthed Date: Wed, 19 Aug 2026 09:52:15 +0530 Subject: [PATCH 6/6] chore: update regression baseline after LiteLLM regression addition Adds litellm|LLC002|litellm/proxy/_super_secret_config.yaml (reviewed, real plaintext HTTP endpoint). Drops 3 llama_index VEC001 fingerprints that no longer fire after upstream changes to those files. --- test/regression-baseline.json | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/test/regression-baseline.json b/test/regression-baseline.json index 019c312..132c153 100644 --- a/test/regression-baseline.json +++ b/test/regression-baseline.json @@ -1,6 +1,6 @@ { "note": "Reviewed proven/likely findings from scripts/regression-scan.js. Only add entries you have read against their source line.", - "updated": "2026-08-04", + "updated": "2026-08-19", "fingerprints": [ "cisco-skill-scanner|SKL001|evals/test_skills/malicious/ascii-smuggling/SKILL.md", "cisco-skill-scanner|SKL002|evals/skills/prompt-injection/jailbreak-override/SKILL.md", @@ -9,6 +9,7 @@ "cisco-skill-scanner|SKL005|evals/skills/data-exfiltration/environment-secrets/get_info.py", "cisco-skill-scanner|SKL005|evals/skills/obfuscation/base64-payload/process.py", "cisco-skill-scanner|SKL005|evals/test_skills/malicious/prompt-injection/SKILL.md", + "litellm|LLC002|litellm/proxy/_super_secret_config.yaml", "llama_index|VEC001|llama-index-core/llama_index/core/indices/base.py", "llama_index|VEC001|llama-index-core/llama_index/core/indices/keyword_table/rake_base.py", "llama_index|VEC001|llama-index-core/llama_index/core/indices/keyword_table/simple_base.py", @@ -28,13 +29,10 @@ "llama_index|VEC001|llama-index-integrations/retrievers/llama-index-retrievers-alletra-x10000/llama_index/retrievers/alletra_x10000_retriever/base.py", "llama_index|VEC001|llama-index-integrations/vector_stores/llama-index-vector-stores-baiduvectordb/llama_index/vector_stores/baiduvectordb/base.py", "llama_index|VEC001|llama-index-integrations/vector_stores/llama-index-vector-stores-chroma/llama_index/vector_stores/chroma/base.py", - "llama_index|VEC001|llama-index-integrations/vector_stores/llama-index-vector-stores-dashvector/llama_index/vector_stores/dashvector/base.py", "llama_index|VEC001|llama-index-integrations/vector_stores/llama-index-vector-stores-databricks/llama_index/vector_stores/databricks/base.py", "llama_index|VEC001|llama-index-integrations/vector_stores/llama-index-vector-stores-jaguar/llama_index/vector_stores/jaguar/base.py", - "llama_index|VEC001|llama-index-integrations/vector_stores/llama-index-vector-stores-pinecone/llama_index/vector_stores/pinecone/base.py", "llama_index|VEC001|llama-index-integrations/vector_stores/llama-index-vector-stores-redis/llama_index/vector_stores/redis/base.py", "llama_index|VEC001|llama-index-integrations/vector_stores/llama-index-vector-stores-supabase/llama_index/vector_stores/supabase/base.py", - "llama_index|VEC001|llama-index-integrations/vector_stores/llama-index-vector-stores-tencentvectordb/llama_index/vector_stores/tencentvectordb/base.py", - "llama_index|VEC001|llama-index-integrations/vector_stores/llama-index-vector-stores-vespa/llama_index/vector_stores/vespa/base.py" + "llama_index|VEC001|llama-index-integrations/vector_stores/llama-index-vector-stores-tencentvectordb/llama_index/vector_stores/tencentvectordb/base.py" ] }