Skip to content

About

Infrastructure, deployment configs, Docker compose, SSL, and backup automation

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

MasterClaw Ecosystem 🐾

The complete AI familiar infrastructure built for Rex deus.

Overview

This repository contains the deployment infrastructure. For the complete ecosystem, see:

Repository Purpose Status
MasterClawInterface React frontend βœ… Active
masterclaw-core AI brain (FastAPI) βœ… Active
masterclaw-tools CLI utilities βœ… Active
rex-deus Personal configs πŸ”’ βœ… Active
level100-studios Design system βœ… Active
masterclaw-infrastructure Deployment βœ… Active

Quick Start

One-Line Install

curl -fsSL https://raw.githubusercontent.com/TheMasterClaw/masterclaw-infrastructure/main/scripts/install.sh | sudo bash

Manual Setup

# Clone
git clone https://github.com/TheMasterClaw/masterclaw-infrastructure.git
cd masterclaw-infrastructure

# Configure
cp .env.example .env
# Edit .env with your settings

# Deploy
make prod
# or: ./scripts/deploy.sh

Commands

Use the Makefile for common operations:

# Setup & Deployment
make dev          # Start development environment
make prod         # Deploy production
make install      # Install on server

# Operations
make status       # Check health
make backup       # Create backup
make restore      # Restore from backup
make logs         # View logs
make logs-status  # Show log sizes and rotation
make logs-clean   # Clean up container logs

# Agent Memory
make agent-memory      # Start agent memory service
make agent-memory-stop # Stop agent memory service

# Cost Optimization
make cost-status      # Show current resource costs
make cost-estimate    # Estimate monthly costs
make cost-optimize    # Find optimization opportunities

# Security
make security-scan    # Run security scans
make ssl-check        # Check SSL certificate expiration

# Monitoring & Maintenance
make test         # Run tests
make monitor      # Start monitoring stack
make migrate      # Run database migrations
make update       # Pull latest images
make clean        # Remove containers and volumes

Architecture

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚              Traefik (SSL/Proxy)            β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚  Interface β”‚  Backend API  β”‚  AI Core      β”‚
β”‚  (React)   β”‚  (Node.js)    β”‚  (Python)     β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚  Gateway   β”‚  ChromaDB     β”‚  SQLite/Redis β”‚
β”‚ (OpenClaw) β”‚  (Vectors)    β”‚  (Data)       β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚   Prometheus + Grafana + Loki (Monitoring)  β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Services

Service Port Description
Interface 80/443 React frontend
Backend API 3001 REST API
AI Core 8000 FastAPI + LLM routing
Gateway 3000 OpenClaw connection
ChromaDB - Vector database
Agent Memory 8080 WebSocket agent hub
Redis 6379 Message queue & pub/sub
Prometheus 9090 Metrics collection
Grafana 3003 Dashboards & visualization
Loki 3100 Log aggregation

Agent Memory Service

MasterClaw includes an Agent Memory Service - a WebSocket-based hub for agent communication and memory persistence.

Features:

  • Real-time WebSocket connections for agents to communicate
  • Persistent memory storage (thoughts, decisions, jobs, blockers)
  • Job tracking with progress updates
  • Blocker reporting with critical alerts
  • REST API for querying agent state

Start the service:

make agent-memory

Connect an agent:

const ws = new WebSocket('wss://agents.yourdomain.com/ws?agent=agent-123&name=coder&type=coding');

ws.onopen = () => {
  // Report a thought
  ws.send(JSON.stringify({
    type: 'thought',
    content: 'Analyzing code structure...',
    priority: 'normal'
  }));
  
  // Start a job
  ws.send(JSON.stringify({
    type: 'job_start',
    jobId: 'job-456',
    title: 'Refactor authentication',
    description: 'Update JWT handling'
  }));
  
  // Report a blocker
  ws.send(JSON.stringify({
    type: 'blocker',
    title: 'API rate limit exceeded',
    description: 'GitHub API returned 403',
    severity: 'high'
  }));
};

Access the dashboard:

  • API: https://agents.yourdomain.com
  • Dashboard: https://agent-dash.yourdomain.com

Cost Monitoring

Monitor and optimize your infrastructure costs:

# View current resource usage and estimated costs
make cost-status

# Estimate monthly costs based on current usage
make cost-estimate

# Find optimization opportunities
make cost-optimize

Sample output:

πŸ’° MasterClaw Cost Monitor
==========================

πŸ“Š Current Resource Usage:

CONTAINER                         CPU %        MEMORY      NETWORK I/O   EST. $/MO
───────────────────────────────────────────────────────────────────────────────────
mc-traefik                         0.15%      45.2MiB         1.2MB/890KB    $3.21
mc-backend                         2.45%     156.8MiB         45MB/12MB     $12.45
mc-core                            5.12%     234.5MiB         120MB/34MB    $18.92
mc-gateway                         0.89%      78.4MiB         8.5MB/2.1MB    $5.67
mc-chroma                          1.23%     189.2MiB         23MB/5.6MB    $14.32

πŸ’Ύ Disk Usage:
  Data directory: 1.2GB
  Estimated storage cost: $0.12/month

Security Scanning

Automated security scanning is configured via GitHub Actions:

  • Container scanning (Trivy) - Vulnerability detection
  • Secrets detection (GitLeaks) - Prevent credential leaks
  • Dependency audit (npm audit) - Known vulnerabilities
  • Infrastructure scanning (Checkov) - IaC best practices

Trigger a scan manually:

gh workflow run security.yml

Or run locally:

make security-scan

SSL Certificate Monitoring

.
β”œβ”€β”€ docker-compose.yml              # Production stack
β”œβ”€β”€ docker-compose.dev.yml          # Development stack
β”œβ”€β”€ docker-compose.monitoring.yml   # Monitoring stack
β”œβ”€β”€ docker-compose.override.yml     # Dev overrides
β”œβ”€β”€ Makefile                        # Common commands
β”œβ”€β”€ scripts/                        # Automation scripts
β”‚   β”œβ”€β”€ install.sh                 # One-line installer
β”‚   β”œβ”€β”€ deploy.sh                  # Production deploy
β”‚   β”œβ”€β”€ backup.sh                  # Backup data
β”‚   β”œβ”€β”€ restore.sh                 # Restore data
β”‚   β”œβ”€β”€ test.sh                    # Run tests
β”‚   β”œβ”€β”€ migrate.sh                 # DB migrations
β”‚   β”œβ”€β”€ health-check.sh            # Health check
β”‚   β”œβ”€β”€ logs.sh                    # Log management
β”‚   β”œβ”€β”€ setup.sh                   # Server setup
β”‚   └── uninstall.sh               # Remove everything
β”œβ”€β”€ services/                       # Service definitions
β”‚   β”œβ”€β”€ interface/                 # Frontend Dockerfile
β”‚   β”œβ”€β”€ backend/                   # Backend Dockerfile
β”‚   └── core/                      # AI Core Dockerfile
β”œβ”€β”€ monitoring/                     # Prometheus/Grafana config
└── docs/                          # Documentation

Monitoring

Start monitoring stack:

make monitor

Access dashboards:

Log Aggregation

MasterClaw includes centralized log aggregation with Loki for searchable, persistent logs.

Query logs via CLI:

# Show logs for a specific service
./scripts/logs-query.sh service mc-core

# Search for errors
./scripts/logs-query.sh errors --since 24h

# Search with custom LogQL query
./scripts/logs-query.sh search '{service="mc-core"} |= "error"' --since 1h

# Tail logs in real-time
./scripts/logs-query.sh tail --service mc-backend

# List available labels
./scripts/logs-query.sh labels

Query logs via Grafana:

  1. Open Grafana at http://localhost:3003
  2. Go to Explore β†’ Select "Loki" datasource
  3. Query examples:
    • All logs: {job="docker"}
    • Service logs: {service="mc-core"}
    • Error logs: {job="docker"} |= "error"
    • Container logs: {container_name="mc-backend"}

Features:

  • 30-day log retention
  • Search by service, container, or content
  • Correlate logs with metrics in Grafana
  • Log volume and error rate dashboards
  • Real-time log streaming

Log Management (Local)

MasterClaw also includes automatic log rotation to prevent disk space issues.

View log status:

make logs-status
# or: ./scripts/logs.sh status
# or: mc log status

Clean up logs:

make logs-clean
# or: ./scripts/logs.sh clean
# or: mc log clean

Export logs:

./scripts/logs.sh export [service]
# or: mc log export [service]

Log rotation settings (configured in docker-compose.yml):

Service Max Size Max Files
traefik 10MB 5
interface 10MB 3
backend 50MB 5
core 50MB 5
gateway 20MB 3
chroma 10MB 3
watchtower 5MB 2

Backup & Recovery

Create backup:

make backup
# or: ./scripts/backup.sh

Restore:

make restore
# or: ./scripts/restore.sh

Backups are stored in ./backups/ with automatic rotation.

Development

See docs/development.md for detailed development guide.

# Start dev environment
make dev

# Run tests
make test

# View logs
make logs ARGS="-f mc-backend"

Environment Variables

Copy .env.example to .env and configure:

Variable Required Description
DOMAIN βœ… Your domain name
ACME_EMAIL βœ… For SSL certificates
GATEWAY_TOKEN βœ… OpenClaw gateway token
OPENAI_API_KEY ⚠️ For AI features
ANTHROPIC_API_KEY ❌ Alternative AI provider

Security

  • Automatic SSL via Let's Encrypt
  • Rate limiting on all endpoints
  • Security headers applied
  • No ports exposed except 80/443

Support

  • CLI: mc help
  • Docs: See docs/ directory
  • Issues: GitHub Issues

Built for Rex. Powered by MasterClaw. 🐾

About

Infrastructure, deployment configs, Docker compose, SSL, and backup automation

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages