Releases: IBM/smith
Releases · IBM/smith
0.1.0
Added
- Initial release of Smith — an agent skill (plugin) for AI code agents that automates the full lifecycle of Open Policy Agent (OPA) (Rego) access-control policies for AI/MCP agents.
- Two-layer architecture: a skill layer (
SKILL.mdplus authoring guides underopa_policy/andtest_generation/) that the agent follows, and asmithCLI backend (scripts/cli.py) that runs the heavy pipeline stages viasmith --flag <stage>. - Policy creation from natural-language guidance and an agent/MCP tool description, restricted to context available from tool arguments (
input.arguments.*) and system variables (input.extensions.subject.*). - Test generation producing both legitimate and adversarial cases: guidance decomposition, grey-condition and variable extraction, case generation, and red-teaming via ARES and Promptfoo (
test_generation). - Test-case evaluation with three-tier label validation (rule patterns → semantic embeddings/NLI → LLM judge), guidance classification, and an HTML report (
test_case_evaluation). - Policy testing harness that runs every generated and custom case against a running OPA server and emits a scorecard with false-positive/false-negative breakdowns (
policy_testing). - Iterative refinement: DBSCAN clustering of failed cases (
red_suggestion), Regal lint/format suggestions (regal_suggestion), and graph + LLM redundancy detection (duplication_suggestion). - Cross-validation of failed cases to distinguish mislabeled tests from policy bugs (
cross_validate,apply_cross_validate). - MCP tool extraction over SSE and stdio transports (
get_mcp_parameter) and tool-call translation (test_case_translation). - Rego policy validation with optional auto-fix (
policy_validation,policy_validation_fix). - Runtime configuration driven entirely from
.env(see.env_template); target-agent selection viaTARGET_AGENT_PATH,GUIDANCE_FILE,SYSTEM_VAR_FILE,MCP_*, andAGENT_URL. - Example target agents under
mcp_servers/, each carrying its Smith inputs (guidance.txt,tool_definitions.json,system_vars.json).