🚨 Automated Infrastructure Drift Detection with CloudTrail Attribution
A serverless AWS solution that automatically detects infrastructure drift by comparing Terraform state files with actual AWS resources, identifies who made changes via CloudTrail, and sends detailed email notifications.
- 🔍 Automated Drift Detection - Scheduled scans compare Terraform state with actual AWS resources
- 📊 Real-Time Dashboard - Beautiful web UI with live metrics, charts, and detailed drift analysis
- 👤 Change Attribution - Enhanced CloudTrail integration identifies who made changes (including ad-hoc creators)
- 🧹 Intelligent Filtering - Zero-noise comparison ignores data sources and properties not captured by scans
- 📧 Premium Notifications - Stunning "Cyber-Premium" HTML emails and ultra-compact "Console-style" text reports
- 💥 Impact Analysis - Automatic assessment of drift impact and risk levels
- 🛠️ Remediation Guidance - Actionable recommendations and step-by-step fixes
- 📈 Historical Tracking - DynamoDB-backed persistence with trend analysis and compliance scoring
- 📊 Multi-State Support - Monitor multiple Terraform state files simultaneously
- ⚡ Serverless Architecture - Built on AWS Lambda for cost-effective operation
- 🎯 Resource Coverage - Supports EC2, S3, RDS, Lambda, IAM, VPC, and more
- ☁️ Pure CloudFormation - Deploy entirely via CloudFormation, no shell scripts
Drift Detection Engine:
- Lambda function orchestrates the entire drift detection process
- Scheduled Trigger: EventBridge rule triggers scans every 6 hours by default (configurable)
- Event-Based Trigger: (Optional) EventBridge rule triggers scans on specific AWS resource changes (EC2, S3, RDS, etc.)
- CloudWatch captures all logs and metrics
State Management:
- Terraform state files stored in S3
- Deployment artifacts stored in separate S3 bucket
- Lambda layer contains Python dependencies
Resource Monitoring:
- Scans EC2 instances, S3 buckets, IAM roles, and more
- Compares actual AWS state with Terraform state
- Identifies configuration drift
Change Attribution:
- CloudTrail provides event history
- Identifies who made changes and when
- Includes IP address, user agent, and request details
Notifications:
- SNS topic sends email alerts
- Beautiful HTML emails with drift details
- Includes severity, impact, and remediation steps
Automated Deployment:
- GitHub Actions builds and tests code
- Security scanning (Bandit, Safety)
- Multi-environment deployment (dev → staging → prod)
- Automatic rollback on failures
┌─────────────────┐
│ EventBridge │ Triggers every 6 hours by default (configurable)
│ Schedule │
└────────┬────────┘
│
▼
┌─────────────────┐
│ Lambda │ Drift Detector Function
│ Function │ - Reads Terraform state from S3
└────────┬────────┘ - Scans actual AWS resources
│ - Compares and identifies drift
│ - Queries CloudTrail for attribution
▼
┌─────────────────┐
│ CloudTrail │ Identifies who made changes
│ Analyzer │
└────────┬────────┘
│
▼
┌─────────────────┐
│ SNS Topic │ Sends email notifications
│ (Email) │
└─────────────────┘
- Terraform state files in S3 (see docs/PREREQUISITES.md)
- AWS CLI configured with appropriate credentials
- IAM permissions for Lambda, SNS, CloudTrail, etc.
- Python 3.11+ (for packaging dependencies)
Note: You do NOT need Terraform installed! Only the state files.
- Fork this repository
- Configure GitHub Secrets (see CI/CD Setup)
- Push to
developbranch → Auto-deploys to Development - Push to
mainbranch → Auto-deploys to Staging - Manual workflow dispatch → Deploy to Production
See .github/workflows/README.md for detailed CI/CD documentation.
# Configure (edit these or set as environment variables)
export TERRAFORM_STATE_BUCKET="your-terraform-state-bucket"
export TERRAFORM_STATE_KEYS="prod/terraform.tfstate"
export EMAIL_ADDRESSES="your-email@example.com"
export SCAN_SCHEDULE="rate(30 minutes)"
export ENABLE_EVENT_BASED_SCANNING="true"
# Deploy
chmod +x scripts/deploy.sh
./scripts/deploy.shSee docs/DEPLOY-CLOUDFORMATION.md for step-by-step manual deployment.
# Delete all resources
chmod +x scripts/cleanup.sh
./scripts/cleanup.sh🚨 DRIFT ALERT: CRITICAL [3 Items]
──────────────────────────────────────────────────────────────────────
📅 2026-01-27 14:19:48 UTC | ⏱️ 3.89s | 📦 57 Scanned
📈 🔴:1 🟠:0 🟡:0 🟢:2
1. ✏️ aws_instance [MODIFIED]
🆔 ID : i-0f949261ac6ac8d6e
👤 Actor : ashish @ 2026-01-27 13:00:45 UTC
⚡ Risk : CRITICAL / SECURITY
🔧 Changes:
↳ Property 'instance_type' changed from 't2.micro' to 't2.nano'
↳ Property 'public_ip' changed from '98.89.27.141' to '54.235.239.53'
🛠️ Remediation:
1. Review the detected changes in detail
2. Update Terraform configuration to match desired state
3. Run: terraform plan -out=tfplan
[... more items ...]
──────────────────────────────────────────────────────────────────────
🤖 Infra Drift Detector | Visit dashboard for full trace analysis.
──────────────────────────────────────────────────────────────────────
The drift detector currently monitors 8 AWS resource types:
- Compute: EC2 Instances, Lambda Functions
- Storage: S3 Buckets
- Database: RDS Instances
- Networking: VPCs, Subnets, Security Groups
- IAM: IAM Roles
Each resource type includes comprehensive attribute monitoring and CloudTrail event tracking.
For complete details, see docs/MONITORING-CAPABILITIES.md
infrastructure-drift-detector/
├── scripts/
│ ├── deploy.sh # Automated deployment
│ ├── cleanup.sh # Automated cleanup
│ └── test-deployment.sh # Deployment validation
│
├── cloudformation/
│ ├── drift-detector-stack.yaml # CloudFormation template
│ └── iam-policy.json # IAM permissions reference
│
├── src/ # Lambda function source
│ ├── drift_detector.py # Core drift detection engine
│ ├── terraform_state_reader.py # State file parser (S3 + local)
│ ├── aws_scanner.py # Resource scanner (EC2/S3/RDS/IAM/VPC...)
│ ├── cloudtrail_analyzer.py # Change attribution
│ ├── impact_analyzer.py # Impact assessment & remediation
│ ├── notification_service.py # Email & Slack notifications
│ ├── persistence_service.py # DynamoDB scan history
│ ├── dashboard_api.py # API Gateway handler for dashboard
│ ├── utils/
│ │ └── logger.py # Structured logging
│ └── templates/
│ └── drift_alert_email.html # Jinja2 email template
│
├── frontend/
│ └── index.html # Dashboard SPA (served via Lambda)
│
├── config/
│ └── settings.py # Pydantic settings (env vars)
│
├── docs/ # Documentation
│ ├── QUICK-START.md # 5-minute quick start
│ ├── DEPLOY-CLOUDFORMATION.md # CloudFormation guide
│ ├── PREREQUISITES.md # Prerequisites
│ ├── DEPLOYMENT.md # Detailed deployment
│ ├── ARCHITECTURE.md # Architecture deep dive
│ ├── MONITORING-CAPABILITIES.md # Monitored resources & events
│ ├── TRIGGER-METHODS.md # How to trigger scans
│ ├── CI-CD-SETUP.md # CI/CD pipeline setup
│ └── PROJECT-STRUCTURE.md # Project structure overview
│
├── tests/
│ └── test_fixes.py # Pytest test suite
│
├── lambda_handler.py # Lambda entry point
├── requirements.txt # Python dependencies
└── README.md # This file
Approximate monthly costs (us-east-1):
| Scan Interval | Lambda | CloudWatch | SNS | S3 | Total |
|---|---|---|---|---|---|
| 6 hours (default) | ~$0.05 | ~$0.50 | ~$0.50 | ~$0.10 | ~$1.15/month |
| 30 minutes | ~$0.20 | ~$0.50 | ~$0.50 | ~$0.10 | ~$1.30/month |
# Automated cleanup
./scripts/cleanup.sh
# Or manual cleanup
aws cloudformation delete-stack --stack-name infrastructure-drift-detector
aws cloudformation wait stack-delete-complete --stack-name infrastructure-drift-detector
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
aws s3 rb "s3://drift-detector-deployment-${ACCOUNT_ID}" --force- ✅ Least privilege IAM roles
- ✅ Encrypted environment variables
- ✅ No hardcoded credentials
- ✅ CloudWatch logging enabled
- ✅ S3 bucket encryption recommended
- ✅ Automated security scanning (Bandit, Safety)
- ✅ Code quality checks (Black, Flake8, MyPy)
This project includes comprehensive CI/CD pipelines:
- Deploy - Automated deployment to dev/staging/prod
- PR Validation - Code quality, security, and build checks
- Cleanup - Safe infrastructure teardown
- Scheduled Tests - Weekly drift detection testing
Configure these in Settings → Secrets and variables → Actions:
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
# Development
DEV_TERRAFORM_STATE_BUCKET
DEV_TERRAFORM_STATE_KEYS
DEV_EMAIL_ADDRESSES
# Staging
STAGING_TERRAFORM_STATE_BUCKET
STAGING_TERRAFORM_STATE_KEYS
STAGING_EMAIL_ADDRESSES
# Production
PROD_TERRAFORM_STATE_BUCKET
PROD_TERRAFORM_STATE_KEYS
PROD_EMAIL_ADDRESSES
develop branch → Development environment (auto)
main branch → Staging environment (auto)
Manual trigger → Production environment (requires approval)
See .github/workflows/README.md for complete CI/CD documentation.
- docs/MONITORING-CAPABILITIES.md - What's monitored, CloudTrail history, notification behavior
- docs/TRIGGER-METHODS.md - How to trigger drift detection (scheduled, manual, CLI)
- docs/ARCHITECTURE.md - Architecture diagrams and detailed design
- docs/QUICK-START.md - 5-minute quick start guide
- docs/DEPLOY-CLOUDFORMATION.md - Pure CloudFormation deployment
- docs/CI-CD-SETUP.md - CI/CD pipeline setup
- docs/PREREQUISITES.md - Prerequisites and setup guide
- docs/DEPLOYMENT.md - Detailed deployment instructions
- docs/CLOUDFORMATION-READY.md - Template validation results
- docs/PROJECT-STRUCTURE.md - Project structure overview
# Run tests
pytest tests/test_fixes.py -v
# Run with coverage
pytest tests/test_fixes.py --cov=src --cov-report=htmlThis project recently underwent code quality improvements:
- ✅ Zero-Noise Comparison - Only alerts on properties present in both expected and actual states
- ✅ Data Source Protection - Automatically filters out "ghost drift" from non-managed resources
- ✅ Deep Attribution - Scans CloudTrail response elements to identify ad-hoc resource creators
- ✅ Premium Reporting - "Cyber-Premium" dark-mode HTML templates and compact terminal alerts
- ✅ Template-Based Emails - Jinja2 templates for better separation of concerns
See CODE_QUALITY_IMPROVEMENTS.md for details.
This is a production-ready solution for infrastructure drift detection. Feel free to customize for your specific needs.
Built with ❤️ for DevOps teams who care about infrastructure drift
Deployed with ☁️ CloudFormation - No shell scripts, no hacks!