What you'll learn: How to use OpenLimit's agent-native features — MCP client mode (tool discovery and governance), MCP server mode (expose virtual keys as tools), and A2A 1.0 (agent-to-agent communication).
All agent protocol features are fully opt-in with zero overhead when disabled.
Connect to external MCP (Model Context Protocol) servers, discover their tools, and merge them into chat completion requests with governance.
mcp:
enabled: true
max_tool_rounds: 5 # max agent loop rounds
tool_timeout_ms: 10000 # per-tool execution timeout
max_total_duration_seconds: 120 # cumulative timeout
auto_inject_tools: false # only merge when request has tools
tool_conflict_strategy: "skip" # "skip" or "error"
servers:
- name: weather
url: "http://localhost:3001/mcp"
tool_prefix: "weather"
timeout_ms: 5000
- name: github
url: "http://localhost:3002/mcp"
headers:
Authorization: "Bearer ghp_xxx"
tool_prefix: "github"
timeout_ms: 10000- Gateway connects to each MCP server at startup, performs the JSON-RPC 2.0 initialization handshake
- Discovers tools via
tools/list, namespaces them withtool_prefix(weather.get_forecast,github.create_issue) - Background health checks (30s ping) with auto-reconnect on failure
- Debounced tool refresh on
notifications/tools/list_changed(5s per server) - When a chat request arrives, permitted MCP tools (filtered by virtual key
allowed_tools) are merged into the request - Non-streaming: if the LLM responds with MCP tool calls, the gateway executes them and feeds results back
- Multi-round loop continues until no more MCP tool calls, or max rounds/cumulative timeout reached
Control which MCP tools each consumer can access:
curl -X POST http://localhost:8080/admin/keys \
-H "Authorization: Bearer $ADMIN_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"project_id": "proj_1",
"name": "weather-key",
"allowed_tools": ["weather.*"]
}'Glob patterns are supported: weather.* matches all weather tools, github.create_issue matches exactly one tool.
- Caller tools + permitted MCP tools are merged into the provider request
tool_choice: "none"skips MCP tool injection entirelytool_choice: "required"+ streaming + MCP tools → rejected with HTTP 400- Duplicate tool names: caller wins by default. Configure
tool_conflict_strategyto"error"for strict mode
For non-streaming requests, the gateway implements an automatic agent loop:
- Send request to provider (with merged tools)
- If response contains tool calls → execute them via MCP servers
- Feed tool results back to the LLM
- Repeat until no more tool calls, or limits reached
Limits:
max_tool_rounds: maximum number of round-trips (default: 5)max_total_duration_seconds: cumulative timeout across all rounds (default: 120s)- Per-tool timeout:
tool_timeout_ms(default: 10000ms)
# List all MCP tools and server status
curl http://localhost:8080/admin/tools \
-H "Authorization: Bearer $ADMIN_TOKEN"
# Filter by server
curl "http://localhost:8080/admin/tools?server=weather" \
-H "Authorization: Bearer $ADMIN_TOKEN"- HTTP transport only — MCP servers must expose Streamable HTTP. Use
supergatewayto bridge stdio servers. - Streaming skips tool interception — Tool calls in streaming responses pass through to the client.
- No MCP resources, prompts, sampling, or elicitation — Only the tools capability is implemented.
- Parallel tool execution — Multiple tool calls within one round execute concurrently. Stateful MCP servers may have issues.
- No tool approval workflow — Tools are auto-executed based on virtual key permissions.
- No client TLS — Server authentication via headers only.
Expose virtual keys as MCP tools so external MCP clients (Claude Desktop, Cursor, etc.) can invoke them directly.
mcp_server:
enabled: true
endpoint: "/mcp" # path on main server, or ":8081" for separate port
authentication:
mode: "bearer_token" # "none", "bearer_token", or "virtual_key"
bearer_token: "my-secret"
session_ttl_seconds: 3600curl -X POST http://localhost:8080/admin/keys \
-H "Authorization: Bearer $ADMIN_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"project_id": "proj_1",
"name": "Chat Agent",
"allow_mcp_server": true,
"mcp_tool_name": "chat_agent",
"allowed_models": ["gpt-4o", "gpt-4*"]
}'- External MCP client connects to the gateway's MCP endpoint
- Gateway performs the initialization handshake, creates a session
tools/listreturns all virtual keys withallow_mcp_server=trueas MCP toolstools/callresolves the tool name → virtual key → validates model → executes chat completion- Response is returned as an MCP tool result with content blocks
- When keys are created/revoked via admin API,
notifications/tools/list_changedis broadcast to active sessions
Priority order for tool names:
mcp_tool_nameoverride (if set on the key)- Sanitized
key.name(lowercase alphanumeric + underscore, max 64 chars) vk_<id[:8]>fallback
Duplicate tool names get a numeric suffix.
| Mode | Description | Governance |
|---|---|---|
none |
No authentication — any client can connect | Guardrails only |
bearer_token |
Static token from config | Guardrails only |
virtual_key |
Clients authenticate with a virtual API key (gw-...) |
Full governance (rate limits, budgets, model restrictions, usage logging) |
When using virtual_key mode, each A2A agent uses its own virtual key. The gateway resolves the key to per-key governance parameters and enforces rate limits, budgets, model/provider restrictions, and usage logging — the same pipeline as direct API calls. This is the recommended mode for production deployments.
# List exposed tools
curl http://localhost:8080/admin/mcp/tools \
-H "Authorization: Bearer $ADMIN_TOKEN"- Non-streaming only —
tools/callexecutes synchronously and returns a single result. - Virtual key must be active — Revoked or expired keys are excluded from tool discovery.
- No MCP resources, prompts, or sampling — Only the tools capability.
- Session notifications require SSE — Client must establish SSE connection for
notifications/tools/list_changed.
Agent-to-Agent protocol v1.0 enables external agents to discover and interact with the gateway.
a2a:
enabled: true
endpoint: "/a2a" # path on main server, or ":8082" for separate port
url: "http://localhost:8080" # public URL for agent card
authentication:
mode: "bearer_token" # "none" or "bearer_token"
bearer_token: "a2a-secret"
default_model: "gpt-4o-mini" # model used for A2A messages
blocking_mode: false # false = return immediately (default)
max_workers: 10 # concurrent background task executors
task_ttl_seconds: 3600 # in-memory task TTL (unused with Postgres)
max_tasks: 10000 # max concurrent tasks (in-memory only)
agent_card:
name: "OpenLimit Gateway"
version: "1.0.0"
description: "AI Gateway with A2A support"External agents discover capabilities at:
GET /.well-known/agent.json
Returns the agent card with capabilities, skills, and authentication requirements.
- Agent sends
message/sendwith user message - Gateway creates a task, enqueues it, and returns
submitted(non-blocking mode) - Background worker picks up the task, executes chat completion
- Client polls
tasks/getor subscribes to SSE for real-time updates - Agent can cancel via
tasks/cancelor list viatasks/list
blocking_mode: false(default): Returns immediately withsubmittedstatus. Client polls or streams for results.blocking_mode: true: Blocks until the task completes. Preserved for backward compatibility.
Real-time task updates via Server-Sent Events:
curl -N http://localhost:8080/a2a/tasks/task_abc123/stream \
-H "Authorization: Bearer a2a-secret"Events: task_update (status changes), done (terminal state), heartbeat every 15s.
Include pushNotification in message/send params:
{
"message": {...},
"pushNotification": {
"url": "https://client.example.com/a2a/callback",
"authentication": {"type": "bearer", "token": "secret"}
}
}Best-effort with 3 retries and exponential backoff. Order is not guaranteed.
# Send a message
curl -X POST http://localhost:8080/a2a \
-H "Authorization: Bearer a2a-secret" \
-H "Content-Type: application/json" \
-d '{
"jsonrpc": "2.0",
"method": "message/send",
"id": 1,
"params": {
"message": {
"role": "user",
"messageId": "msg-001",
"parts": [{"type": "text", "text": "What is the weather in Tokyo?"}]
}
}
}'
# Get task status
curl -X POST http://localhost:8080/a2a \
-H "Authorization: Bearer a2a-secret" \
-H "Content-Type: application/json" \
-d '{
"jsonrpc": "2.0",
"method": "tasks/get",
"id": 2,
"params": {"id": "task_abc123"}
}'
# Cancel a task
curl -X POST http://localhost:8080/a2a \
-H "Authorization: Bearer a2a-secret" \
-H "Content-Type: application/json" \
-d '{
"jsonrpc": "2.0",
"method": "tasks/cancel",
"id": 3,
"params": {"id": "task_abc123"}
}'| Code | Name | Meaning |
|---|---|---|
| -32001 | TaskNotFound |
Task ID does not exist |
| -32002 | TaskNotCancelable |
Task is already completed or failed |
| -32003 | ContentTypeNotSupported |
Message contains unsupported content type |
| -32004 | MaxTasksReached |
In-memory task limit reached |
- Text parts in provider calls — File and data parts are accepted in A2A messages but only text content is sent to the AI provider.
- No multi-turn — Each
message/sendcreates a new task with no context carry-over. - Single default model — No per-skill model routing.
- SSE is single-instance — For multi-instance, clients should poll
tasks/get. - Push notifications are best-effort — 3 retries, then give up. No dead-letter queue.
- In-memory fallback — Without a database, tasks are stored in memory and lost on restart.
Agent protocols expose dedicated metrics:
| Metric | Description |
|---|---|
mcp_server_connections |
Active MCP server connections |
mcp_tool_calls_total |
Total MCP tool calls |
mcp_tool_call_duration_seconds |
MCP tool call latency |
mcp_max_rounds_exceeded_total |
Agent loop max rounds reached |
gateway_a2a_tasks_created_total |
Total A2A tasks created |
gateway_a2a_task_completions_total |
A2A tasks completed by status |
gateway_a2a_task_duration_seconds |
A2A task execution duration |
- Governance — Configure virtual key tool permissions
- Configuration — Full MCP and A2A YAML reference
- API Reference — Admin endpoints for tool management
- Observability — Monitor agent protocol metrics