1 lines (1 loc) · 8.73 KB

Runtime Validation Hooks Documentation## OverviewTerraphim AI implements a two-stage runtime validation system that provides safety and knowledge-graph enhancement for AI-assisted development workflows. This system operates through pre/post hooks around LLM generation and tool execution.## Two-Stage Hook Flow### Stage 1: Guard Stage (Security & Bypass Protection)Purpose: Prevent dangerous operations and enforce safety invariants before any processing occurs.Location: `~/.claude/hooks/pre_tool_use.sh`Implementation:bash#!/bin/bash# Extract command from JSON inputCOMMAND=$(echo "$1" | jq -r '.tool_input.command // empty')# Strip quoted strings to avoid false positivesCLEAN_COMMAND=$(echo "$COMMAND" | sed 's/"[^"]"//g')# Check for dangerous bypass flagsif [[ "$CLEAN_COMMAND" =~ (--no-verify|-n)(?=.\bgit\s+(commit|push)) ]]; then # Return deny decision echo '{"decision": "deny", "reason": "Git bypass flags detected"}' exit 0fi# Continue to replacement stagecd ~/.config/terraphimterraphim-agent hook "$1"Guard Actions:- Block: `--no-verify` or `-n` flags in `git commit/push` commands- Allow: All other commands proceed to replacement stage- Log: All guard decisions with reasons### Stage 2: Replacement Stage (Knowledge Graph Enhancement)Purpose: Replace text using knowledge graph patterns and connectivity validation.Location: `terraphim-agent hook` command in Terraphim agentImplementation:`rust// terraphim_agent/src/commands/hook.rspub async fn execute_hook( input: HookInput, agent: &TerraphimAgent,) -> Result<HookOutput, HookError> { // Apply knowledge graph replacements let enhanced_text = agent .rolegraph .apply_replacements(&input.text)?; // Validate connectivity agent .automata .validate_connectivity(&enhanced_text)?; Ok(HookOutput { modified_text: enhanced_text, was_modified: enhanced_text != input.text, })}`Replacement Actions:- Enhance: Apply role-based knowledge graph patterns- Validate: Ensure semantic connectivity and coherence- Transform: Use thesaurus and autocomplete for consistency## Runtime LLM/Tool Hooks### Pre-LLM HooksPurpose: Validate LLM inputs before generation.Context:`rustPreLlmContext { prompt: String, // Command type + description agent_id: String, // Agent identifier conversation_history: Vec<String>, // Previous messages token_count: usize, // Estimated tokens}`Hook Decisions:- Allow: Proceed with LLM generation- Block: Stop with reason (security, policy, etc.)- Modify: Transform prompt (not recommended for LLM)- AskUser: Require human confirmation### Post-LLM HooksPurpose: Validate and potentially modify LLM outputs.Context:`rustPostLlmContext { prompt: String, // Original prompt response: String, // Generated response agent_id: String, // Agent identifier token_count: usize, // Total tokens used model: String, // LLM model used}`Hook Decisions:- Allow: Return original response unchanged- Block: Prevent response delivery (harmful content, policy violations)- Modify: Transform response (formatting, style, safety fixes)- AskUser: Require human review before delivery### Pre-Tool HooksPurpose: Validate code and commands before execution.Context:`rustPreToolContext { code: String, // Code to execute language: String, // Programming language agent_id: String, // Agent identifier vm_id: String, // VM execution environment metadata: HashMap<String, String>, // Additional context}`Security Validations:- Dangerous Patterns: `rm -rf /`, `sudo`, `chmod 777`, etc.- Language Restrictions: Block execution in disallowed languages- Resource Limits: Validate memory, CPU, and file access- Injection Prevention: Command injection and shell escape detection### Post-Tool HooksPurpose: Monitor and analyze execution results.Context:`rustPostToolContext { original_code: String, // Original code executed output: String, // Execution output exit_code: i32, // Process exit code duration_ms: u64, // Execution time agent_id: String, // Agent identifier vm_id: String, // VM identifier}`Monitoring Actions:- Success/Failure Tracking: Learn patterns of successful executions- Performance Analysis: Track execution times and resource usage- Security Logging: Record blocked or suspicious activities- Knowledge Graph Learning: Update successful patterns for future reference## Configuration### Runtime Validation ConfigLocation: ~/.config/terraphim/runtime-validation.toml````toml[hooks]enabled = truefail_open = true # Allow execution if hooks fail (development mode)[guard]strict_mode = false # Block on any suspicion vs. specific patternslog_all_decisions = true # Log allow/block decisions[llm_hooks]enabled = truerequire_human_review = false # Only for high-stakes operations[tool_hooks]enabled = truevm_isolation = trueresource_limits = true[replacement]knowledge_graph_enhancement = trueconnectivity_validation = true```### Environment Variables```bash# Enable/disable specific hook categoriesTERRAPHIM_RUNTIME_VALIDATION_HOOKS=trueTERRAPHIM_GUARD_STAGE=trueTERRAPHIM_REPLACEMENT_STAGE=true# Hook behavior overridesTERRAPHIM_FAIL_OPEN=false # Production mode: fail closedTERRAPHIM_HOOK_TIMEOUT_MS=5000 # Hook execution timeout```## Hook Development### Creating Custom Hooks```rustuse crate::vm_execution::{hooks::*, VmExecutionError};#[derive(Debug)]pub struct SecurityHook { blocked_patterns: Vec<regex::Regex>,}#[async_trait]impl Hook for SecurityHook { fn name(&self) -> &str { "security-hook" } async fn pre_tool(&self, context: &PreToolContext) -> Result<HookDecision, VmExecutionError> { for pattern in &self.blocked_patterns { if pattern.is_match(&context.code) { return Ok(HookDecision::Block { reason: format!("Blocked pattern: {}", pattern.as_str()), }); } } Ok(HookDecision::Allow) } async fn post_tool(&self, context: &PostToolContext) -> Result<HookDecision, VmExecutionError> { // Log execution for learning if context.exit_code != 0 { log::warn!("Tool execution failed: {:?}", context); } Ok(HookDecision::Allow) }}```### Registering Hooks```rust// In agent initializationlet mut agent = TerraphimAgent::new(config).await?;// Add custom hooksagent.hook_manager.add_hook(Arc::new(SecurityHook::new()));agent.hook_manager.add_hook(Arc::new(PerformanceHook::new()));agent.hook_manager.add_hook(Arc::new(LearningHook::new()));```## Troubleshooting### Hook Not InvokedSymptoms: LLM/tool execution without hook validationCauses:- HookManager not initialized in agent- Hooks not registered with manager- Hook execution disabled in configResolution:```bash# Check configcat ~/.config/terraphim/runtime-validation.toml# Verify agent initializationgrep -r "hook_manager" src/agent.rs# Check registrationgrep -r "add_hook" src/```### Hook Blocking Too MuchSymptoms: Many operations blocked as "dangerous"Causes: Overly strict regex patterns, false positivesResolution:- Review blocked_patterns`in security hooks- Enable debug logging to see exact matches- Adjust patterns to be more specific### Performance IssuesSymptoms: Slow LLM/tool response timesCauses: Hook timeout, expensive operations, network callsResolution:- Increase`TERRAPHIM_HOOK_TIMEOUT_MS`- Profile hook execution with` tokio-console`- Move expensive operations to background threads## Best Practices1. Fail-Open Development: Use` fail_open = trueduring development to avoid blocking2. Specific Patterns: Use targeted regex patterns instead of broad blocks3. Async Operations: Keep hook implementations fast and non-blocking4. Comprehensive Logging: Log all decisions for debugging and learning5. Knowledge Graph Integration: Leverage existing rolegraph and automata for intelligence6. Security First: Always implement security validation before functionality7. Testing: Test both success and failure scenarios for all hooks## Integration Points- Claude Code:pre_tool_use.sh` → Guard → Replacement → Tool execution- Terraphim Agent: CommandExecutor with HookManager integration- VM Execution: Pre/post tool hooks around Firecracker execution- LLM Generation: Pre/post LLM hooks in all agent types- Knowledge Graph: Replacement service using rolegraph connectivityThis two-stage validation system ensures both security (guard stage) and intelligence enhancement (replacement stage) while maintaining clear separation of concerns and comprehensive audit trails.