Skip to content

feat: 9 new waste detectors + extended CSV data + AI dashboard (dashboard_AI.py)#2

Open
smahima27 wants to merge 5 commits into
mainfrom
feature/new-detectors
Open

feat: 9 new waste detectors + extended CSV data + AI dashboard (dashboard_AI.py)#2
smahima27 wants to merge 5 commits into
mainfrom
feature/new-detectors

Conversation

@smahima27

Copy link
Copy Markdown
Collaborator

Summary

Adds 9 new AWS waste detection rules, extended sample data, and an AI-powered dashboard — built during the Perforce Global Jam 2026 hackathon.


New detection rules (detection_engine.py)

# Function What it detects Est. saving
1 detect_nat_idle NAT Gateways with <1 GB/week traffic ($32/mo base charge) Full monthly cost
2 detect_idle_load_balancers ALBs/NLBs with 0 target groups or 0 requests Full monthly cost
3 detect_old_gen_instances t2/m4/c4/r4 families — recommend upgrade to t3/m5/c5/r5 4–11% per instance
4 detect_orphan_snapshots EBS snapshots >90 days with no source volume Full monthly cost
5 detect_gp2_volumes gp2 volumes eligible for gp3 migration (zero downtime) 20% per volume
6 detect_ondemand_no_coverage EC2 on-demand >30 days without Reserved Instance or Savings Plan ~40%
7 detect_infinite_log_retention CloudWatch log groups with no expiry policy set ~70%
8 detect_stopped_ec2_with_ebs Stopped EC2 still incurring EBS storage costs Full EBS cost
9 detect_underutilized_cache_redshift ElastiCache/Redshift clusters at <10% CPU Full monthly cost

Bug fix: detect_idle_ec2 now excludes stopped instances to prevent double-counting with rule #8.


Sample data (aws_cost_data.csv)

Extended from 60 to 98 rows. Added 38 new rows covering:

  • 3 NAT Gateway rows (low traffic: 0.05–0.3 GB/week)
  • 3 ALB/NLB rows (0 requests or 0 target groups)
  • 4 old-gen EC2 rows (t2.large, m4.xlarge, c4.4xlarge, r4.2xlarge)
  • 5 orphaned EBS snapshots (90–200 days old)
  • 5 gp2 EBS volumes (100–1000 GB)
  • 4 on-demand EC2 (running 180–365 days without RI/SP)
  • 5 CloudWatch log groups (infinite retention)
  • 4 stopped EC2 (paying for attached EBS only)
  • 4 ElastiCache/Redshift clusters (1.8–4.2% CPU)

New file: dashboard_AI.py

Enhanced Streamlit dashboard featuring:

  • FinOps AI chatbot (Claude claude-sonnet-4-20250514) embedded in a side panel — answers questions about the actual findings data
  • Slack webhook integration — fire a top-finding alert to any channel
  • Executive summary + category donut chart
  • Filterable findings cards with remediation CLI commands toggle

Run with:

export ANTHROPIC_API_KEY='sk-ant-...' && streamlit run dashboard_AI.py

Detection results after changes

Total findings    : 77
Monthly waste     : $7,686.55
Annual waste      : $92,238.60

Waste by category:
  Idle Resource             $5,769.20/mo
  Zombie Resource           $1,012.60/mo
  RI/SP Optimisation        $507.00/mo
  Storage Optimisation      $210.00/mo
  Old Generation            $116.35/mo
  Log Retention             $71.40/mo

smahima27 and others added 5 commits May 26, 2026 15:06
New detection rules in detection_engine.py:
- detect_nat_idle: flag NAT Gateways with <1 GB/week traffic
- detect_idle_load_balancers: flag ALBs/NLBs with 0 target groups or requests
- detect_old_gen_instances: flag t2/m4/c4/r4 families → recommend t3/m5/c5/r5
- detect_orphan_snapshots: flag EBS snapshots >90 days old with no source volume
- detect_gp2_volumes: flag gp2 volumes for migration to gp3 (20% cheaper)
- detect_ondemand_no_coverage: flag EC2 on-demand >30d without RI/SP (40% saving)
- detect_infinite_log_retention: flag CloudWatch log groups with infinite retention (70% saving)
- detect_stopped_ec2_with_ebs: flag stopped EC2 still paying for attached EBS
- detect_underutilized_cache_redshift: flag ElastiCache/Redshift at <10% CPU

Also:
- Extended aws_cost_data.csv with 37 new sample rows covering all new resource types
- Fixed detect_idle_ec2 to exclude stopped instances (prevents double-counting)
- Added dashboard_AI.py: enhanced Streamlit dashboard with embedded Claude AI chatbot,
  Slack webhook alerts, dark-themed findings cards, and executive summary panel

Results: 77 findings · $7,686/mo · $92,238/yr"
feat: add Untagged Resources panel — metrics, charts, table and CLI tagging fix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant