Resources growth: Incidents & Guidance (2025–26) + AI Security Tooling Catalog#3
Open
ppradyoth wants to merge 12 commits into
Open
Resources growth: Incidents & Guidance (2025–26) + AI Security Tooling Catalog#3ppradyoth wants to merge 12 commits into
ppradyoth wants to merge 12 commits into
Conversation
…ncidents + authoritative guidance Curates disclosed 2025-26 incidents (EchoLeak/CVE-2025-32711, the MCP breach cluster) alongside primary-source guidance (NSA AISC MCP CSI May 2026, OWASP Top 10 for LLMs 2025 and Agentic Applications 2026). Every entry is backed by a CVE or official document so the file is independently citable and shareable. Cross-linked from the README handbooks table.
New handbook cataloging the open-source tools practitioners actually run, organized by goal (red-teaming, MCP/agent security, runtime guardrails, supply-chain scanning, adversarial ML) rather than by vendor. Every entry verified against its primary source with honest maintenance status (garak, PyRIT, mcp-scan, LLM Guard, ModelScan, ART, ...). Linked from the README handbooks table. Fills the repo's biggest gap: a tools index, and targets a high-search-volume query (AI/MCP security tools).
ppradyoth
changed the title
Fills the repo's biggest onboarding gap and a high-search-volume target. A time-boxed zero-to-value path (one mental model -> beat Gandalf by hand -> run garak against a local model -> map it to OWASP/ATLAS), then red-team / defender / researcher tracks, a guaranteed CPU-only first win, opinionated newcomer mistakes, and links into the deeper handbooks. Every external resource verified against its primary source before inclusion; all repo-internal relative links confirmed to resolve. Linked from the README navigation directory and handbooks table as the recommended entry point.
…jection Two new verified 2025-26 incidents extend the 'attacks land at the agentic layer' thesis with distinct attack classes: - CometJacking (LayerX, Oct 2025): indirect prompt injection in Perplexity's Comet agentic browser; base64 encoding bypasses plaintext exfil guardrails. - ServiceNow Now Assist (AppOmni, Nov 2025): second-order prompt injection via agent-to-agent discovery; one compromised agent recruits privileged ones. Notes the separate CVE-2025-12420 to avoid conflation. Each sourced to >=2 reputable outlets (The Hacker News, Brave, Schneier, LayerX, AppOmni, TechRadar, CyberScoop). Renumbered guidance entries 3-5 to 5-7, updated TOC anchors and README handbook description. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The guidance section covered NSA + OWASP but not MITRE ATLAS — the ATT&CK-style matrix the rest of the repo references throughout. Late 2025, ATLAS (with Zenity Labs) added agent-focused techniques (AI Agent Context Poisoning, Memory Manipulation, Thread Injection, Modify AI Agent Configuration, RAG Credential Harvesting), closing the model-era-only gap. New §8 maps each technique onto the incidents already cataloged (EchoLeak, CometJacking, ServiceNow), adds a 6th defender takeaway, updates the TOC + README handbook description. Technique names/themes corroborated across multiple independent reports of the MITRE x Zenity work; an explicit verification note flags that exact tactic/ technique counts move release-to-release and should be confirmed on the primary ATLAS site before quoting. No fabricated counts stated as fact.
Verification pass on BENCHMARKS_AND_DATASETS.md against canonical sources. Several links pointed at wrong or non-existent repos (a credibility risk for a public guide): - AgentDojo: nareshr/agentdojo -> ethz-spylab/agentdojo (the real ETH SPY Lab repo); added NeurIPS'24 D&B citation, task/test-case counts, formal-utility scoring note, and leaderboard link. - garak: leondz/garak -> NVIDIA/garak (project moved to the NVIDIA org). - Inspect AI: UKGovernmentBEIS/inspect -> .../inspect_ai; added inspect_evals. - Do-Not-Answer: LianminZheng/do-not-answer -> Libr-AI/do-not-answer; count 936 -> 939. - Harmful-QA: poloclub/harmful-qa (nonexistent) -> HarmfulQA at declare-lab/red-instruct; corrected name, count (1,960), and paper link. - FigStep: Visual-Adversarial-ML/FigStep -> ThuCCSLab/FigStep; added arXiv/AAAI. - jailbreak dataset: verazuo/jailbreak-LLM -> verazuo/jailbreak_llms; corrected to the CCS'24 'In-The-Wild Jailbreak Prompts' dataset (15,140 prompts). - Replaced the unverifiable 'PromptArmor Evaluator' tool entry with promptfoo, a verified open-source LLM red-teaming/eval toolkit. - Added InjecAgent (uiuc-kang-lab) as a distinct agentic-injection benchmark. Every link verified against its canonical repo this run. Added a dated link-integrity note inviting issue/PR reports of stale links.
The repo cataloged the OWASP Top 10 (the *what*) but not the OWASP GenAI Red Teaming Guide (the *how* — released Jan 2025), a real gap for a guide that aims to be the definitive AI-security reference. Adds a dedicated section mapping its four phases (model / implementation / infrastructure / runtime) onto the Top 10, MITRE ATLAS tactics, and the cataloged incidents (EchoLeak, CometJacking as phase-4 runtime failures), plus README cross-links from the Red Teaming Methodologies table. All facts sourced from the OWASP GenAI primary pages.
… in-the-wild malicious MCP (postmark-mcp) The MCP breach cluster (§2) was thematic; this grounds it in hard CVE IDs. - CVE-2025-49596: MCP Inspector no-auth proxy RCE, CVSS 9.4, DNS-rebinding drive-by (Oligo), fixed 0.14.1. - CVE-2025-6514: mcp-remote OS command injection, CVSS 9.6, 437k+ downloads (JFrog/Or Peles), fixed 0.1.16. - postmark-mcp: first malicious MCP server caught in the wild (Koi Security, Sept 2025) — 15 clean versions, then a one-line BCC-exfil backdoor. Both directions of attack (server pops client; trusted package turns) now have real CVEs/cases. Every claim links to NVD/vendor primary sources; TOC + README incidents description updated.
…Red Teaming Landscape) The repo cataloged the OWASP Red Teaming *Guide* (how to test) but not the Q2 2026 Solutions *Landscapes* (what to test with). Adds the trio — LLM/GenAI, Agentic AI, and OWASP's first dedicated AI & Agentic Red Teaming Landscape — cross-mapped to the four-phase guide, TOOLS.md, INCIDENTS, and MITRE ATLAS. Surfaced in README handbook table + frameworks table. Corroborated across OWASP resource pages, the pre-RSA-2026 announcement, and third-party analysis; living-document/no-endorsement caveats flagged.
/ CVE-2026-25592) Every incident in the file so far ends in data exfiltration or a compromised MCP client/server. This new §5 documents a distinct, worse class: the sink is the agent framework's own plumbing and the impact is host code execution. - CVE-2026-26030 (Semantic Kernel Python, CVSS 9.9): InMemoryVectorStore builds filter lambdas and runs them through eval(); an attacker-controllable field (e.g. a poisoned RAG record) executes arbitrary Python during retrieval. Fixed in semantic-kernel 1.39.4. - CVE-2026-25592 (Semantic Kernel .NET, CVSS 9.9): SessionsPythonPlugin's DownloadFileAsync/UploadFileAsync are agent-callable [KernelFunction]s with no path validation → path traversal → arbitrary host file write → RCE. Fixed in Microsoft.SemanticKernel.Plugins.Core 1.71.0 (NuGet) / 1.39.3 (pip). Disclosed May 7 2026 (MSRC + Microsoft Security Blog, "When prompts become shells"). Added a matching defender takeaway (audit framework eval/tool sinks) and surfaced both CVEs in the README incidents description. Renumbered the Authoritative Guidance section (NSA→6, OWASP LLM→7, OWASP Agentic→8, ATLAS→9) and updated the TOC + cross-references. Verification: every fact corroborated across the Microsoft Security Blog (vendor primary) and two authoritative GitHub advisories (GHSA-xjw9-4gw8-4rqx, GHSA-2ww3-72rp-wpp4), both CVSS 9.9 (CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:C/C:H/I:H/A:H). TOC anchors and relative links verified programmatically.
…ing IDEs (24 CVEs) Adds incident §6: Ari Marzouk's 'IDEsaster' (disclosed Dec 2025) — 30+ flaws / 24 CVEs across 10+ AI IDEs (Cursor, GitHub Copilot, Windsurf, Zed, Roo Code, JetBrains Junie, Kiro, Cline), where an injected repo (README/rule-file/ filename/malicious-MCP output) drives the IDE's *own legitimate features* (workspace-config edits, remote-JSON-schema fetch) into data exfil / RCE. 100% of tested tools vulnerable — the shared blind spot is that AI IDEs leave the base editor out of their threat model. Renumbered guidance sections to §7–10 (NSA/OWASP LLM/OWASP Agentic/MITRE), updated the TOC anchors and the one prose cross-ref (§8–9 → §9–10), and surfaced IDEsaster in the README incidents row. Zero-fabrication: primary blog + NVD bot-protected and unfetchable at write time; scope, researcher, product list, attack chains, and CVE->product mappings corroborated across The Hacker News, Tom's Hardware, TechWorm, BeyondMachines, and CSA Labs. CVSS not asserted (NVD pages not retrievable); CVE IDs linked to NVD for live confirmation.
Owner
Author
Changelog — run 2026-07-05New incident §6 — IDEsaster: a universal attack-chain class across AI coding IDEs ( Adds the December 2025 disclosure by Ari Marzouk (MaccariTA): 30+ vulnerabilities / 24 assigned CVEs across 10+ AI-assisted IDEs — Cursor, GitHub Copilot, Windsurf, Zed, Roo Code, JetBrains Junie, Kiro, Cline. The shared root cause: an injected repo (a poisoned
Zero-fabrication note: the primary blog (maccarita.com) and NVD were bot-protected and unfetchable at write time. Scope, researcher, affected-product list, both attack chains, and the CVE→product mappings are corroborated across The Hacker News, Tom's Hardware, TechWorm, BeyondMachines, and Cloud Security Alliance Labs reporting the same disclosure. CVSS scores are not asserted (NVD pages couldn't be retrieved); CVE IDs link to NVD for live confirmation. Commit Suggested GitHub topics for reach (needs owner to add): Generated by Claude Code |
… the LLM proxy control plane as attack surface Adds §7: LiteLLM command-injection→RCE via the /mcp-rest/test/* endpoints (CVSS 8.7), chained with the Starlette 'BadHost' Host-header bypass (CVE-2026-48710) to unauthenticated CVSS 10.0. CISA added CVE-2026-42271 to the KEV catalog on 2026-06-08 (exploited in the wild). Also covers the CVE-2026-12773 UserAPIKeyAuth bypass (CVSS 7.3). This is the first incident in the file targeting the AI gateway/proxy layer itself rather than a model, agent, framework, or IDE. Guidance sections renumbered 8-11; TOC + internal cross-references updated. Every CVE anchored to NVD; mechanism/scores corroborated across Horizon3.ai (discoverer), CISA KEV, The Hacker News, and CSA Labs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cumulative growth of the resources repo on
auto/resources-growth. Each addition is fully sourced and linked from the README.Contents
-4.
STANDARDS_AND_COMPLIANCE.md— OWASP GenAI Q2 2026 Solutions Landscapes (incl. first Red Teaming Landscape) (new — 2026-07-03)The repo cataloged the OWASP Red Teaming Guide (how to test) but not the Q2 2026 Solutions Landscapes (what to test with). New section adds the trio, cross-mapped rather than bolted on:
TOOLS.md, the runtime/agentic incidents (EchoLeak / CometJacking / ServiceNow), and MITRE ATLAS. Surfaced in both README tables. Living-document / no-vendor-endorsement caveats flagged; corroborated across OWASP resource pages, the pre-RSA-2026 announcement (Mar 2026), and third-party analysis.-3.
INCIDENTS_AND_GUIDANCE_2026.md— named MCP CVEs + first in-the-wild malicious MCP (2026-07-02)The MCP breach cluster (§2) was thematic ("tool-poisoning," "no-auth-by-default"). This grounds it in hard CVE IDs and the first real in-the-wild case, all NVD/vendor-sourced:
0.14.1.authorization_endpoint, CVSS 9.6 (Or Peles / JFrog); fixed0.1.16.postmark-mcp— the first malicious MCP server caught in the wild (Koi Security, Sept 2025): 15 clean versions built trust, then1.0.16added a one-line BCC-exfil backdoor; Postmark confirmed the impersonation.-2.
STANDARDS_AND_COMPLIANCE.md— OWASP GenAI Red Teaming Guide (2026-07-01)The repo cataloged the OWASP Top 10 for LLMs (the what can go wrong) but not the OWASP GenAI Red Teaming Guide (the how you go find it — first version released Jan 2025). New section framing red teaming as a four-phase evaluation (model → implementation → infrastructure → runtime), cross-mapped to specific OWASP Top 10 entries, MITRE ATLAS, and the cataloged incidents.
-1.
BENCHMARKS_AND_DATASETS.md— link-integrity pass + InjecAgent (2026-06-30)A verification sweep corrected several resource links pointing at wrong/non-existent repositories (AgentDojo →
ethz-spylab/agentdojo, garak →NVIDIA/garak, Inspect AI →UKGovernmentBEIS/inspect_ai, Do-Not-Answer →Libr-AI/do-not-answer, HarmfulQA →declare-lab/red-instruct, FigStep →ThuCCSLab/FigStep, jailbreak dataset →verazuo/jailbreak_llms), replaced the unverifiable "PromptArmor Evaluator" with promptfoo, and added InjecAgent (ACL 2024 Findings, arXiv:2403.02691).0.
INCIDENTS_AND_GUIDANCE_2026.md— MITRE ATLAS agentic expansion (§8) (2026-06-29)New §8 introduces the late-2025 MITRE ATLAS × Zenity Labs agent-technique vocabulary (Context Poisoning, Memory Manipulation, Thread Injection, Modify AI Agent Configuration, RAG Credential Harvesting) mapped onto EchoLeak / CometJacking / ServiceNow. Exact counts flagged release-dependent.
1.
INCIDENTS_AND_GUIDANCE_2026.md— two new verified incidents (2026-06-28)CometJacking (LayerX, Oct 2025) and ServiceNow Now Assist (AppOmni, Nov 2025) — indirect and second-order prompt injection in agentic systems.
2.
GETTING_STARTED.mdA "zero to value in under an hour" on-ramp: a 60-minute path (instruction/data confusion → Gandalf → garak on a CPU-only model → map to OWASP LLM Top 10 / MITRE ATLAS), pick-your-track branches, and a newcomer-mistakes list.
3.
INCIDENTS_AND_GUIDANCE_2026.md(base)CVE-sourced catalog of disclosed incidents + authoritative guidance (EchoLeak / CVE-2025-32711, the MCP breach cluster, NSA AISC MCP CSI, OWASP LLM 2025 & Agentic 2026).
4.
TOOLS.mdCurated AI security tooling catalog organized by what you're trying to do (garak, PyRIT, promptfoo, mcp-scan, LLM Guard, ModelScan, ART…), each with a 🟢/🟡/🔴 maintenance label.
Verification (zero-fabrication policy)
postmark-mcpimpersonation is corroborated by Postmark's own advisory. The Q2 2026 OWASP Landscapes are corroborated across OWASP's own resource pages, the project's pre-RSA-2026 announcement, and independent third-party analysis, with living-document / no-endorsement caveats. Bot-protected vendor pages are corroborated by ≥2 independent reputable reports, and any figure that moves between releases is flagged rather than asserted.🤖 Generated with Claude Code