Welcome to the OpenLimit documentation. This index links to every doc page with the target audience and what you'll find there.
Start here: If you're new to OpenLimit, begin with Getting Started — it takes you from zero to your first governed AI request in under 5 minutes.
Audience: Everyone — developers, operators, platform teams
What's inside:
- Installation (binary build or Docker)
- Configuration basics (provider, database, models)
- Starting the gateway (local or Docker Compose)
- Health check verification
- Creating your first virtual key (quickstart endpoint)
- Making your first chat completion request
- Adding your first guardrail
- Checking usage
Full walkthrough from installation to your first governed request.
Audience: Operators, DevOps
What's inside:
- Server settings (host, port, timeouts)
- Database configuration
- All provider types: OpenAI, Anthropic, Gemini, Azure OpenAI, OpenAI-compatible
- Gemini
gemini_model_mapand Azureazure_resource/azure_api_versionfields - Provider key encryption (
encrypted_value) - Model routing (weighted, fallbacks)
- Region-aware routing (priority and latency strategies)
- Exact cache (LRU) and semantic cache (embeddings + pgvector)
- Redis integration (shared state for rate limits, cache, circuit breakers)
- Telemetry (Prometheus metrics, OpenTelemetry tracing)
- Auth and admin settings
- Billing prices
- Guardrails overview
- MCP, MCP server, and A2A config pointers
Complete reference for every section of gateway.yaml.
Audience: Platform teams, security engineers
What's inside:
- Virtual API keys (
gw-prefix, properties, management) - Quickstart endpoint for one-step onboarding
- Projects (multi-tenancy, cascade delete)
- Rate limiting (token bucket, Redis sliding window)
- Budgets (daily/monthly spend caps)
- Guardrails (6 stages: PII, regex, keyword, length, webhook, json_schema)
- Per-model guardrail configuration
- Custom webhook guardrails
- Streaming behavior
- RBAC (3 roles: admin, editor, viewer)
- User management
- OIDC SSO (Okta, Azure AD, Keycloak, Google)
- Audit logs (event types, query filters, schema)
Everything about controlling access to AI providers.
Audience: AI engineers, agent developers
What's inside:
- MCP client mode: tool discovery, namespacing, tool merge, tool governance
- Virtual key tool permissions (glob patterns)
- Tool merge behavior (conflict resolution)
- Multi-round agent loop (max rounds, cumulative timeout)
- MCP server mode: expose virtual keys as tools
- MCP tool naming and deduplication
- MCP authentication modes (none, bearer_token, virtual_key)
- A2A 1.0: agent card, tasks, SSE streaming, push notifications
- Blocking vs non-blocking task execution
- A2A error codes
- Prometheus metrics for agent protocols
How to integrate OpenLimit with AI agents via MCP and A2A.
Audience: Developers, platform engineers
What's inside:
- Plugin types (guardrail, middleware, provider)
- Plugin interfaces and GuardrailContext
- Writing a guardrail plugin (full example with source code)
- Writing a middleware plugin (HeaderInjector walkthrough)
- Configuration — plugin registration + guardrail pipeline reference
- Best practices for production plugins
Extend the gateway with custom guardrails, middleware, and provider adapters.
Audience: SREs, platform engineers
What's inside:
- Prometheus metrics setup
- Full metrics reference (30+ metrics organized by category)
- OpenTelemetry tracing configuration
- W3C
traceparentheader for correlation - Supported tracing backends (Jaeger, Tempo, SigNoz, etc.)
- Grafana dashboard generation and import
- Docker Compose monitoring profile
- Prometheus recording and alert rules
- Cardinality awareness
- Structured logging conventions
- Monitoring checklist
How to monitor OpenLimit in production.
Audience: DevOps, platform engineers
What's inside:
- Docker Compose profiles (default, monitoring, stateful, full stack)
- Helm chart features (deployment, ConfigMap, Secret, Service, Ingress, HPA, ServiceMonitor)
- Helm security defaults (non-root, read-only filesystem, no privilege escalation)
- Kubernetes production checklist (secrets, HA, database, observability, networking, governance, resources)
- Developer commands (make run, test, lint, build)
Running OpenLimit in development and production.
Audience: Developers, integrators
What's inside:
- Link to OpenAPI 3.0.3 specification
- Chat completions API (OpenAI-compatible)
- Supported parameters
- Response headers (X-Request-ID, X-Provider, X-Cache, X-Cost-USD, traceparent)
- Model listing endpoint
- Health endpoints (liveness, readiness)
- Admin API endpoints (16 endpoints)
- Error reference table (16 error types with HTTP status, type, meaning, example, recommended action)
- Guardrail block stages
- Error metric (gateway_errors_total)
How to use the OpenLimit API and understand its responses.
Audience: Operators, platform engineers
What's inside:
- Cost-aware routing strategy (cheapest provider selection)
- Combined smart strategy (cost 40% + latency 40% + health 20%)
- Configurable cost weights via
routing.cost_weightsYAML - Embedded pricing catalog (22 entries across 8 providers)
- Region-aware routing (priority and latency strategies)
Route requests to the cheapest, fastest, or most available provider.
Audience: Operators, DevOps
What's inside:
- Shadow traffic for confident provider migration
- Per-route replay configuration with sample rate
- Ring buffer storage (last 1000 results)
- Summary stats: avg latency comparison, shadow error rate
- Fire-and-forget async execution
Compare providers side-by-side without affecting production traffic.
Audience: AI engineers, platform teams
What's inside:
openlimit mcp add— one-command MCP server registration with auto-discoveryopenlimit mcp search— keyword search over 42-tool embedded catalog- Admin dashboard MCP tab: server inventory, health status, tool count
- TF-IDF-inspired scoring,
--liveflag merges configured servers - Deduplication: catalog wins over live for same tool name
Discover, register, and manage MCP servers from CLI and dashboard.
Audience: Platform teams, security engineers
What's inside:
- 6 built-in validators: PII, keyword, length, regex, JSON schema, webhook
GET /admin/guardrails/catalog— discover all validators with config docsPOST /admin/guardrails/test— test sample text against any validatorGET /admin/guardrails/stats— live hit counters, block rates, per-stage breakdown- Dashboard: catalog cards, test form, stats with auto-refresh
Browse, test, and monitor guardrails from the admin dashboard.
Audience: Operators, finance teams
What's inside:
GET /admin/usage/spend— per-key spend and budget utilization- Status thresholds: healthy (<75%), warning (75–95%), critical (>95%)
- Admin UI: total spend card, per-key budget bars with color-coded warnings
- Period selector (daily/monthly)
Track budget utilization across all virtual keys.
Audience: Developers, operators
What's inside:
openlimit-bench— latency benchmark (P50/P95/P99, concurrency)openlimit-test— smoke test (in-process, zero dependencies)openlimit-doctor— config diagnostics (provider/key/model checks)openlimit-mcp— MCP hub CLI (add servers, search tools)
Self-contained CLI tools with zero external dependencies.
Audience: Security teams, compliance officers
What's inside:
- Security model (3 layers: network, governance, encryption)
- KMS encryption: static, AWS KMS, HashiCorp Vault
- Provider key encryption (encrypted_value, ciphertext format)
- Key rotation procedure
- Fail-closed behavior
- Data residency enforcement (X-Data-Residency header, matching rules)
- Virtual key protection (bcrypt hashing, key lookup caching)
- Redis security considerations
- OIDC security considerations
- Security checklist
How OpenLimit protects your keys, data, and access.
Audience: Teams upgrading from Phase 5B/6F
What's inside:
- v1.0.0 overview (unified governance, new providers, enriched errors)
- Config changes (Gemini provider, Azure OpenAI provider, validation)
- Behavior changes (governance on MCP/A2A, enriched errors, admin error request_id, new headers, quickstart endpoint, gateway_errors_total)
- Go API changes (writeAdminError signature, executePlanSingle removal, ExecuteForMCP identity, IdentityProvider, GovernanceBlockedError)
- Upgrade checklist (before, during, after)
- Rollback procedure
Step-by-step guide for upgrading to v1.0.0.
| What you need | Where to go |
|---|---|
| Install and run the gateway | Getting Started |
| Configure a new provider | Configuration |
| Set up API key governance | Governance |
| Write a custom plugin | Plugins |
| Connect MCP tools | Agent Protocols |
| Set up monitoring | Observability |
| Deploy to Kubernetes | Deployment |
| Fix an error | API Reference |
| Encrypt provider keys | Security |
| Upgrade from pre-v1.0 | Migration to v1.0 |
| Upgrade from v1.0 to v1.1 | Migration to v1.1 |
Found a bug or want to improve the docs? See CONTRIBUTING.md for development setup and contribution guidelines.