The complete AI familiar infrastructure built for Rex deus.
This repository contains the deployment infrastructure. For the complete ecosystem, see:
| Repository | Purpose | Status |
|---|---|---|
| MasterClawInterface | React frontend | β Active |
| masterclaw-core | AI brain (FastAPI) | β Active |
| masterclaw-tools | CLI utilities | β Active |
| rex-deus | Personal configs π | β Active |
| level100-studios | Design system | β Active |
| masterclaw-infrastructure | Deployment | β Active |
curl -fsSL https://raw.githubusercontent.com/TheMasterClaw/masterclaw-infrastructure/main/scripts/install.sh | sudo bash# Clone
git clone https://github.com/TheMasterClaw/masterclaw-infrastructure.git
cd masterclaw-infrastructure
# Configure
cp .env.example .env
# Edit .env with your settings
# Deploy
make prod
# or: ./scripts/deploy.shUse the Makefile for common operations:
# Setup & Deployment
make dev # Start development environment
make prod # Deploy production
make install # Install on server
# Operations
make status # Check health
make backup # Create backup
make restore # Restore from backup
make logs # View logs
make logs-status # Show log sizes and rotation
make logs-clean # Clean up container logs
# Agent Memory
make agent-memory # Start agent memory service
make agent-memory-stop # Stop agent memory service
# Cost Optimization
make cost-status # Show current resource costs
make cost-estimate # Estimate monthly costs
make cost-optimize # Find optimization opportunities
# Security
make security-scan # Run security scans
make ssl-check # Check SSL certificate expiration
# Monitoring & Maintenance
make test # Run tests
make monitor # Start monitoring stack
make migrate # Run database migrations
make update # Pull latest images
make clean # Remove containers and volumesβββββββββββββββββββββββββββββββββββββββββββββββ
β Traefik (SSL/Proxy) β
βββββββββββββββββββββββββββββββββββββββββββββββ€
β Interface β Backend API β AI Core β
β (React) β (Node.js) β (Python) β
βββββββββββββββββββββββββββββββββββββββββββββββ€
β Gateway β ChromaDB β SQLite/Redis β
β (OpenClaw) β (Vectors) β (Data) β
βββββββββββββββββββββββββββββββββββββββββββββββ€
β Prometheus + Grafana + Loki (Monitoring) β
βββββββββββββββββββββββββββββββββββββββββββββββ
| Service | Port | Description |
|---|---|---|
| Interface | 80/443 | React frontend |
| Backend API | 3001 | REST API |
| AI Core | 8000 | FastAPI + LLM routing |
| Gateway | 3000 | OpenClaw connection |
| ChromaDB | - | Vector database |
| Agent Memory | 8080 | WebSocket agent hub |
| Redis | 6379 | Message queue & pub/sub |
| Prometheus | 9090 | Metrics collection |
| Grafana | 3003 | Dashboards & visualization |
| Loki | 3100 | Log aggregation |
MasterClaw includes an Agent Memory Service - a WebSocket-based hub for agent communication and memory persistence.
Features:
- Real-time WebSocket connections for agents to communicate
- Persistent memory storage (thoughts, decisions, jobs, blockers)
- Job tracking with progress updates
- Blocker reporting with critical alerts
- REST API for querying agent state
Start the service:
make agent-memoryConnect an agent:
const ws = new WebSocket('wss://agents.yourdomain.com/ws?agent=agent-123&name=coder&type=coding');
ws.onopen = () => {
// Report a thought
ws.send(JSON.stringify({
type: 'thought',
content: 'Analyzing code structure...',
priority: 'normal'
}));
// Start a job
ws.send(JSON.stringify({
type: 'job_start',
jobId: 'job-456',
title: 'Refactor authentication',
description: 'Update JWT handling'
}));
// Report a blocker
ws.send(JSON.stringify({
type: 'blocker',
title: 'API rate limit exceeded',
description: 'GitHub API returned 403',
severity: 'high'
}));
};Access the dashboard:
- API:
https://agents.yourdomain.com - Dashboard:
https://agent-dash.yourdomain.com
Monitor and optimize your infrastructure costs:
# View current resource usage and estimated costs
make cost-status
# Estimate monthly costs based on current usage
make cost-estimate
# Find optimization opportunities
make cost-optimizeSample output:
π° MasterClaw Cost Monitor
==========================
π Current Resource Usage:
CONTAINER CPU % MEMORY NETWORK I/O EST. $/MO
βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
mc-traefik 0.15% 45.2MiB 1.2MB/890KB $3.21
mc-backend 2.45% 156.8MiB 45MB/12MB $12.45
mc-core 5.12% 234.5MiB 120MB/34MB $18.92
mc-gateway 0.89% 78.4MiB 8.5MB/2.1MB $5.67
mc-chroma 1.23% 189.2MiB 23MB/5.6MB $14.32
πΎ Disk Usage:
Data directory: 1.2GB
Estimated storage cost: $0.12/month
Automated security scanning is configured via GitHub Actions:
- Container scanning (Trivy) - Vulnerability detection
- Secrets detection (GitLeaks) - Prevent credential leaks
- Dependency audit (npm audit) - Known vulnerabilities
- Infrastructure scanning (Checkov) - IaC best practices
Trigger a scan manually:
gh workflow run security.ymlOr run locally:
make security-scan.
βββ docker-compose.yml # Production stack
βββ docker-compose.dev.yml # Development stack
βββ docker-compose.monitoring.yml # Monitoring stack
βββ docker-compose.override.yml # Dev overrides
βββ Makefile # Common commands
βββ scripts/ # Automation scripts
β βββ install.sh # One-line installer
β βββ deploy.sh # Production deploy
β βββ backup.sh # Backup data
β βββ restore.sh # Restore data
β βββ test.sh # Run tests
β βββ migrate.sh # DB migrations
β βββ health-check.sh # Health check
β βββ logs.sh # Log management
β βββ setup.sh # Server setup
β βββ uninstall.sh # Remove everything
βββ services/ # Service definitions
β βββ interface/ # Frontend Dockerfile
β βββ backend/ # Backend Dockerfile
β βββ core/ # AI Core Dockerfile
βββ monitoring/ # Prometheus/Grafana config
βββ docs/ # Documentation
Start monitoring stack:
make monitorAccess dashboards:
- Prometheus: http://localhost:9090
- Grafana: http://localhost:3003 (admin/admin)
- Loki: http://localhost:3100
MasterClaw includes centralized log aggregation with Loki for searchable, persistent logs.
Query logs via CLI:
# Show logs for a specific service
./scripts/logs-query.sh service mc-core
# Search for errors
./scripts/logs-query.sh errors --since 24h
# Search with custom LogQL query
./scripts/logs-query.sh search '{service="mc-core"} |= "error"' --since 1h
# Tail logs in real-time
./scripts/logs-query.sh tail --service mc-backend
# List available labels
./scripts/logs-query.sh labelsQuery logs via Grafana:
- Open Grafana at http://localhost:3003
- Go to Explore β Select "Loki" datasource
- Query examples:
- All logs:
{job="docker"} - Service logs:
{service="mc-core"} - Error logs:
{job="docker"} |= "error" - Container logs:
{container_name="mc-backend"}
- All logs:
Features:
- 30-day log retention
- Search by service, container, or content
- Correlate logs with metrics in Grafana
- Log volume and error rate dashboards
- Real-time log streaming
MasterClaw also includes automatic log rotation to prevent disk space issues.
View log status:
make logs-status
# or: ./scripts/logs.sh status
# or: mc log statusClean up logs:
make logs-clean
# or: ./scripts/logs.sh clean
# or: mc log cleanExport logs:
./scripts/logs.sh export [service]
# or: mc log export [service]Log rotation settings (configured in docker-compose.yml):
| Service | Max Size | Max Files |
|---|---|---|
| traefik | 10MB | 5 |
| interface | 10MB | 3 |
| backend | 50MB | 5 |
| core | 50MB | 5 |
| gateway | 20MB | 3 |
| chroma | 10MB | 3 |
| watchtower | 5MB | 2 |
Create backup:
make backup
# or: ./scripts/backup.shRestore:
make restore
# or: ./scripts/restore.shBackups are stored in ./backups/ with automatic rotation.
See docs/development.md for detailed development guide.
# Start dev environment
make dev
# Run tests
make test
# View logs
make logs ARGS="-f mc-backend"Copy .env.example to .env and configure:
| Variable | Required | Description |
|---|---|---|
DOMAIN |
β | Your domain name |
ACME_EMAIL |
β | For SSL certificates |
GATEWAY_TOKEN |
β | OpenClaw gateway token |
OPENAI_API_KEY |
For AI features | |
ANTHROPIC_API_KEY |
β | Alternative AI provider |
- Automatic SSL via Let's Encrypt
- Rate limiting on all endpoints
- Security headers applied
- No ports exposed except 80/443
- CLI:
mc help - Docs: See
docs/directory - Issues: GitHub Issues
Built for Rex. Powered by MasterClaw. πΎ