Skip to content

[CRITICAL] No authentication on Prometheus metrics endpoint #6

Description

@emiperez95

🚨 Security Vulnerability

The Prometheus exporter exposes all metrics on port 9090 without any authentication, allowing anyone with network access to scrape sensitive operational data.

Risk Assessment

  • Severity: CRITICAL
  • OWASP: A01:2021 - Broken Access Control
  • Impact: Information disclosure, potential DoS, system architecture exposure

Exposed Information

  • Database file paths
  • Session identifiers
  • Agent execution patterns
  • System performance metrics
  • Error messages and stack traces
  • Internal workflow patterns

Current Code

Location: tools/prometheus_exporter.py:736

# No authentication!
start_http_server(port, registry=registry)

Solution Options

Option 1: Basic Authentication (Recommended for Quick Fix)

from prometheus_client import start_http_server
from functools import wraps
import base64

def require_auth(f):
    @wraps(f)
    def decorated(*args, **kwargs):
        auth = request.headers.get('Authorization')
        if not auth or not check_auth(auth):
            return Response('Unauthorized', 401, 
                          {'WWW-Authenticate': 'Basic realm="Metrics"'})
        return f(*args, **kwargs)
    return decorated

def check_auth(auth_header):
    try:
        scheme, credentials = auth_header.split(' ')
        if scheme.lower() != 'basic':
            return False
        decoded = base64.b64decode(credentials).decode('utf-8')
        username, password = decoded.split(':')
        return username == os.environ.get('METRICS_USER') and \
               password == os.environ.get('METRICS_PASS')
    except:
        return False

Option 2: Bearer Token Authentication

import hmac
import hashlib

def verify_bearer_token(token):
    expected = os.environ.get('METRICS_TOKEN')
    if not expected:
        logger.error("METRICS_TOKEN not configured")
        return False
    return hmac.compare_digest(token, expected)

class AuthenticatedMetricsHandler(MetricsHandler):
    def do_GET(self):
        auth = self.headers.get('Authorization', '')
        if not auth.startswith('Bearer '):
            self.send_error(401, "Unauthorized")
            return
        
        token = auth[7:]  # Remove 'Bearer ' prefix
        if not verify_bearer_token(token):
            self.send_error(401, "Invalid token")
            return
        
        super().do_GET()

Option 3: Bind to Localhost Only (Immediate Mitigation)

# Quick fix - only allow local access
start_http_server(port, addr='127.0.0.1', registry=registry)

Option 4: Use Reverse Proxy with Auth

# nginx.conf
server {
    listen 9090 ssl;
    ssl_certificate /path/to/cert.pem;
    ssl_certificate_key /path/to/key.pem;
    
    location /metrics {
        auth_basic "Prometheus Metrics";
        auth_basic_user_file /etc/nginx/.htpasswd;
        
        proxy_pass http://localhost:9091;
        
        # Rate limiting
        limit_req zone=metrics burst=10 nodelay;
    }
}

Implementation Steps

1. Immediate Mitigation (5 minutes)

# Change line 736 to bind to localhost only
start_http_server(port, addr='127.0.0.1', registry=registry)

2. Add Basic Auth (1 hour)

  • Implement authentication decorator
  • Add environment variable configuration
  • Update documentation

3. Add TLS Support (2 hours)

from prometheus_client import start_wsgi_server
from wsgiref.simple_server import make_server
import ssl

context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
context.load_cert_chain('cert.pem', 'key.pem')

httpd = make_server('', port, app)
httpd.socket = context.wrap_socket(httpd.socket, server_side=True)

Configuration

Add to .env or environment:

export METRICS_USER="prometheus"
export METRICS_PASS="$(openssl rand -base64 32)"
export METRICS_BIND="127.0.0.1"  # Only local access

Validation Criteria

  • Metrics endpoint requires authentication
  • Unauthorized requests return 401
  • Credentials stored securely (env vars, not code)
  • Rate limiting implemented
  • Audit logging for access attempts
  • Documentation updated with auth setup

Testing

# Test without auth (should fail)
curl http://localhost:9090/metrics
# Expected: 401 Unauthorized

# Test with auth (should work)
curl -u prometheus:password http://localhost:9090/metrics
# Expected: Metrics data

# Test with wrong credentials (should fail)
curl -u wrong:credentials http://localhost:9090/metrics
# Expected: 401 Unauthorized

Security Considerations

  1. Never hardcode credentials
  2. Use strong passwords (min 32 chars)
  3. Rotate credentials regularly
  4. Monitor failed authentication attempts
  5. Consider IP whitelisting for additional security
  6. Use TLS in production

References

Effort Estimate

3 hours total

  • 5 min: Immediate localhost binding
  • 1 hour: Basic authentication
  • 2 hours: Full implementation with TLS

Priority

CRITICAL - This is a blocker for production deployment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingvulnerabilitySecurity vulnerabilities

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions