Skip to content

Releases: IBM/smith

0.1.0

28 Jun 12:36
f560832

Choose a tag to compare

Added

  • Initial release of Smith — an agent skill (plugin) for AI code agents that automates the full lifecycle of Open Policy Agent (OPA) (Rego) access-control policies for AI/MCP agents.
  • Two-layer architecture: a skill layer (SKILL.md plus authoring guides under opa_policy/ and test_generation/) that the agent follows, and a smith CLI backend (scripts/cli.py) that runs the heavy pipeline stages via smith --flag <stage>.
  • Policy creation from natural-language guidance and an agent/MCP tool description, restricted to context available from tool arguments (input.arguments.*) and system variables (input.extensions.subject.*).
  • Test generation producing both legitimate and adversarial cases: guidance decomposition, grey-condition and variable extraction, case generation, and red-teaming via ARES and Promptfoo (test_generation).
  • Test-case evaluation with three-tier label validation (rule patterns → semantic embeddings/NLI → LLM judge), guidance classification, and an HTML report (test_case_evaluation).
  • Policy testing harness that runs every generated and custom case against a running OPA server and emits a scorecard with false-positive/false-negative breakdowns (policy_testing).
  • Iterative refinement: DBSCAN clustering of failed cases (red_suggestion), Regal lint/format suggestions (regal_suggestion), and graph + LLM redundancy detection (duplication_suggestion).
  • Cross-validation of failed cases to distinguish mislabeled tests from policy bugs (cross_validate, apply_cross_validate).
  • MCP tool extraction over SSE and stdio transports (get_mcp_parameter) and tool-call translation (test_case_translation).
  • Rego policy validation with optional auto-fix (policy_validation, policy_validation_fix).
  • Runtime configuration driven entirely from .env (see .env_template); target-agent selection via TARGET_AGENT_PATH, GUIDANCE_FILE, SYSTEM_VAR_FILE, MCP_*, and AGENT_URL.
  • Example target agents under mcp_servers/, each carrying its Smith inputs (guidance.txt, tool_definitions.json, system_vars.json).