| title | Batch API Quickstart |
|---|---|
| sidebarTitle | Batch |
| description | Submit and retrieve asynchronous batches of inference requests |
| slug | batch-quickstart |
| seoTitle | Batch API Quickstart | OpenRouter Documentation |
| og:site_name | OpenRouter Documentation |
| og:title | Batch API Quickstart |
| og:description | Submit multiple inference requests with OpenRouter and retrieve their results asynchronously. |
| og:image | https://openrouter.ai/dynamic-og?pathname=batch-quickstart&title=Batch%20API%20Quickstart&description=Submit%20and%20retrieve%20asynchronous%20batches%20of%20inference%20requests |
| og:image:width | 1200 |
| og:image:height | 630 |
| twitter:card | summary_large_image |
| twitter:site | @OpenRouter |
| noindex | false |
| nofollow | false |
| icon | boxes-stacked |
The Batch API lets you submit many inference requests together and retrieve the results asynchronously. It's useful for work that doesn't need an immediate response, and it uses a 24-hour completion window so you can process requests without managing each call yourself.
The Batch API supports several OpenRouter API shapes, including chat completions, Responses, Anthropic Messages, and embeddings. You submit requests as an inline JSON requests array. You don't upload a JSONL file; OpenRouter handles JSONL persistence internally.
A batch runs on one provider. By default OpenRouter picks the cheapest eligible batch endpoint for the model, and you can pin a provider with provider.only (see Provider routing). Use the list endpoint to see every batch in your workspace.
Multimodal input in batch is URL-only, and support depends on the provider the batch routes to.
Images. Image parts must be public http(s) URLs. Base64 and data: URI images are rejected on every provider. Image URLs are accepted when the model accepts image input and the batch routes to a provider whose batch API fetches URLs natively:
| Provider | Public image URLs | Public file URLs |
|---|---|---|
| OpenAI | Supported | Supported on /v1/responses only |
| Anthropic | Supported | Supported |
| xAI | Supported | Not supported |
| Mistral | Not supported | Supported |
| Google Vertex | Not supported | Not supported |
| Google AI Studio | Not supported | Not supported |
| Together | Not supported | Not supported |
| Fireworks | Not supported | Not supported |
Files. File parts (Responses input_file, Anthropic document, chat completions file) are accepted only as URL references, only on the providers marked above, and only for models that list file input. Inline file bytes and provider file IDs are rejected.
Audio and video. Audio and video input parts are rejected on every provider. On /v1/chat/completions, requests that ask for non-text output through modalities, audio, or image_config are also rejected. For embeddings, input must be strings or token arrays.
Web search. Provider-native web search tools pass through when the resolved provider runs the search itself, for example OpenAI web_search on /v1/responses and Anthropic web_search_20250305 on /v1/messages. OpenRouter-orchestrated search is not available in batch. A submit for an :online model variant is rejected synchronously with 422. Per request, the web plugin, web_search_options (except on OpenAI models that execute it natively), and web search tools with an engine other than auto or native are rejected.
Other per-request bans. A request with no input (empty messages or input and no prompt), stream: true, speed, a max output token cap below 1, and Anthropic beta-gated features are rejected. Unknown parameters are dropped by the provider serializer, matching the sync API.
Use provider.only to pin a provider that supports the content you send. Requests that fail these per-request checks are rejected after the 202 response: the batch moves to status: "failed" and error explains the rejection. Send unsupported content to the sync API instead.
Batch runs on :batch endpoint variants. Filter the models page by the batch variant to see which models and providers currently support it. A submit for a model with no :batch endpoint returns 400. If the model has :batch endpoints but none match your provider.only list, the submit returns 404.
Batch requests are typically billed at 50% of the model's standard per-token pricing, mirroring the batch discounts offered by OpenAI and Anthropic. For a completed batch, usage.cost reports the amount OpenRouter charges. For BYOK-routed batches, that's only the OpenRouter BYOK fee, since the provider bills you directly for inference.
If you have a provider key configured, batches route through it automatically, the same as sync requests: the provider bills you directly for inference and OpenRouter charges only the BYOK fee (see Pricing above). Completed batches report usage.is_byok: true.
Google Vertex uses a bucket in your GCP project for provider input and output. The primary setup is frictionless: omit bucket and let OpenRouter create a private, location-compatible bucket on the first batch. You can optionally provide an existing bucket instead. See Google Vertex API keys for permissions and storage behavior.
Submit a batch with:
POST https://openrouter.ai/api/v1/batches
The request body has three required top-level fields:
| Field | Description |
|---|---|
endpoint |
The API shape used by every request in the batch. Choose /v1/chat/completions, /v1/responses, /v1/messages, or /v1/embeddings. |
model |
An OpenRouter model slug, such as openai/gpt-4o. This batch-level model is applied to every request. |
requests |
A non-empty array of { custom_id, body } items. custom_id must be unique within the batch, and body follows the shape of the selected endpoint. |
Two optional top-level fields:
| Field | Description |
|---|---|
provider |
{ "only": ["<provider-slug>"] } to pin the batch to specific providers. See Provider routing. |
completion_window |
Defaults to 24h, which is the only accepted value. |
The batch-level model applies to every request. A request body can omit model to inherit the batch-level value. If a request body sets its own model, it must match the batch-level model or the submission is rejected.
On Google models, every request in a batch must ask for the same response_format. Either all requests omit it, all use json_object, or all use json_schema with the same schema. Google's batch service derives one output schema for the whole batch, so requests that disagree fail there. Validation fails a mismatched batch and names the first request that conflicts, so send one batch per response_format and per schema.
import json
import requests
response = requests.post(
url="https://openrouter.ai/api/v1/batches",
headers={
"Authorization": "Bearer <OPENROUTER_API_KEY>",
"Content-Type": "application/json",
},
data=json.dumps({
"endpoint": "/v1/chat/completions",
"model": "openai/gpt-4o",
"requests": [
{
"custom_id": "req-0001",
"body": {
"messages": [
{
"role": "user",
"content": "Summarize OpenRouter in one sentence."
}
]
}
}
]
})
)
print(response.json())const response = await fetch('https://openrouter.ai/api/v1/batches', {
method: 'POST',
headers: {
Authorization: 'Bearer <OPENROUTER_API_KEY>',
'Content-Type': 'application/json',
},
body: JSON.stringify({
endpoint: '/v1/chat/completions',
model: 'openai/gpt-4o',
requests: [
{
custom_id: 'req-0001',
body: {
messages: [
{
role: 'user',
content: 'Summarize OpenRouter in one sentence.',
},
],
},
},
],
}),
});
console.log(await response.json());curl https://openrouter.ai/api/v1/batches \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $OPENROUTER_API_KEY" \
-d '{
"endpoint": "/v1/chat/completions",
"model": "openai/gpt-4o",
"requests": [
{
"custom_id": "req-0001",
"body": {
"messages": [
{
"role": "user",
"content": "Summarize OpenRouter in one sentence."
}
]
}
}
]
}'The response is a batch object with an ID you can use to check progress:
{
"id": "batch_123",
"object": "batch",
"endpoint": "/v1/chat/completions",
"model": "openai/gpt-4o",
"completion_window": "24h",
"status": "validating",
"created_at": 1782097200,
"finalized_at": null,
"request_counts": {
"total": 1,
"completed": 0,
"failed": 0
},
"usage": null,
"results": null,
"error": null
}The only supported completion window is 24h.
Every request in a batch runs on a single provider, chosen once at submit time. By default OpenRouter picks the cheapest eligible :batch endpoint for the model, after applying your account's provider allowlist, data policy, and BYOK settings. Endpoints at the same price share traffic. If you have a BYOK key for one of the providers, that endpoint is preferred over cheaper platform endpoints.
To pin the batch to a provider, add a top-level provider object with an only array of provider slugs, placed before requests:
import json
import requests
response = requests.post(
url="https://openrouter.ai/api/v1/batches",
headers={
"Authorization": "Bearer <OPENROUTER_API_KEY>",
"Content-Type": "application/json",
},
data=json.dumps({
"endpoint": "/v1/chat/completions",
"model": "google/gemini-2.5-flash",
"provider": {
"only": ["google-vertex"]
},
"requests": [
{
"custom_id": "req-0001",
"body": {
"messages": [
{"role": "user", "content": "Summarize this ticket in one sentence."}
]
}
}
]
})
)
print(response.json())const response = await fetch('https://openrouter.ai/api/v1/batches', {
method: 'POST',
headers: {
Authorization: 'Bearer <OPENROUTER_API_KEY>',
'Content-Type': 'application/json',
},
body: JSON.stringify({
endpoint: '/v1/chat/completions',
model: 'google/gemini-2.5-flash',
provider: {
only: ['google-vertex'],
},
requests: [
{
custom_id: 'req-0001',
body: {
messages: [{ role: 'user', content: 'Summarize this ticket in one sentence.' }],
},
},
],
}),
});
console.log(await response.json());curl https://openrouter.ai/api/v1/batches \
-H "Authorization: Bearer $OPENROUTER_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"endpoint": "/v1/chat/completions",
"model": "google/gemini-2.5-flash",
"provider": { "only": ["google-vertex"] },
"requests": [
{
"custom_id": "req-0001",
"body": {
"messages": [{ "role": "user", "content": "Summarize this ticket in one sentence." }]
}
}
]
}'provider.only is the only provider preference the Batch API accepts. order, sort, allow_fallbacks, and the other sync preferences are rejected. If none of the listed providers has an eligible :batch endpoint for the model, the submit returns 404 instead of falling back to another provider. Use the batch variant filter on the models page to see which providers serve a model in batch.
List the batches in the workspace of the authenticating API key with:
curl 'https://openrouter.ai/api/v1/batches?limit=2&status=completed&status=failed' \
-H "Authorization: Bearer $OPENROUTER_API_KEY"Batches are scoped to the workspace, not the key: every API key in the same workspace sees the same list, including batches submitted with other keys. Batches are returned newest first. List items contain metadata only and always set results to null; retrieve an individual batch by ID when you need its results.
{
"object": "list",
"data": [
{
"id": "batch_9f2c1e",
"object": "batch",
"endpoint": "/v1/chat/completions",
"model": "openai/gpt-4o",
"completion_window": "24h",
"status": "completed",
"created_at": 1787836000,
"finalized_at": 1787837000,
"request_counts": { "total": 100, "completed": 100, "failed": 0 },
"usage": { "prompt_tokens": 51200, "completion_tokens": 20480, "total_tokens": 71680 },
"results": null,
"error": null
}
],
"first_id": "batch_9f2c1e",
"last_id": "batch_9f2c1e",
"has_more": true
}All query parameters are optional:
| Parameter | Description |
|---|---|
limit |
Number of batches to return, from 1 to 100. Defaults to 20. |
after |
Continue strictly after this batch ID. Pass the previous page's last_id. |
status |
Include a public status. Repeat the parameter to include more than one of validating, in_progress, completed, failed, expired, or cancelled. The transient finalizing and cancelling statuses are not accepted as list filters. |
created_after |
Include batches created strictly after a Unix timestamp in seconds or an ISO-8601 date or datetime. |
created_before |
Include batches created strictly before a Unix timestamp in seconds or an ISO-8601 date or datetime. |
When has_more is true, request the next page with after. Pagination does not use offsets or a before parameter.
curl 'https://openrouter.ai/api/v1/batches?limit=2&after=batch_9f2c1e' \
-H "Authorization: Bearer $OPENROUTER_API_KEY"For human-readable date filters, use an ISO-8601 value such as 2026-08-20 or 2026-08-20T00:00:00Z. Unix seconds use the same unit as the integer timestamps returned in batch objects:
curl 'https://openrouter.ai/api/v1/batches?created_after=1787184000&created_before=1787837000' \
-H "Authorization: Bearer $OPENROUTER_API_KEY"created_after must be earlier than created_before when both are present.
Use the batch ID to retrieve the current status:
GET https://openrouter.ai/api/v1/batches/:id
For example:
curl https://openrouter.ai/api/v1/batches/batch_123 \
-H "Authorization: Bearer $OPENROUTER_API_KEY"The status normally progresses through:
validating → in_progress → finalizing → completed
Other possible statuses are failed, expired, cancelling, and cancelled. The terminal statuses are completed, failed, expired, and cancelled. Poll until the batch reaches a terminal status.
request_counts contains the total number of requests and the number that completed or failed:
{
"total": 100,
"completed": 98,
"failed": 2
}When a batch is in progress, or has failed, expired, or been cancelled, results is null. When a batch completes, results is returned inline as an array in the same response. There is no separate results-download endpoint.
Each result maps back to an input using custom_id. Exactly one of response or error is populated for each result:
{
"id": "batch_req_123",
"custom_id": "req-0001",
"response": {
"status_code": 200,
"request_id": "request_123",
"body": {
"id": "gen-batch-1782097200-a1b2c3d4e5f6a7b8c9d0",
"object": "chat.completion",
"created": 1782097200,
"model": "openai/gpt-4o",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "OpenRouter provides one API for many AI models."
},
"finish_reason": "stop"
}
]
}
},
"error": null
}A completed batch response looks like this:
{
"id": "batch_123",
"object": "batch",
"endpoint": "/v1/chat/completions",
"model": "openai/gpt-4o",
"completion_window": "24h",
"status": "completed",
"created_at": 1782097200,
"finalized_at": 1782100800,
"request_counts": {
"total": 1,
"completed": 1,
"failed": 0
},
"usage": {
"prompt_tokens": 20,
"completion_tokens": 40,
"total_tokens": 60,
"cost": 0.000225,
"is_byok": false
},
"results": [
{
"id": "batch_req_123",
"custom_id": "req-0001",
"response": {
"status_code": 200,
"request_id": "request_123",
"body": {
"id": "gen-batch-1782097200-a1b2c3d4e5f6a7b8c9d0",
"object": "chat.completion",
"created": 1782097200,
"model": "openai/gpt-4o",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "OpenRouter provides one API for many AI models."
},
"finish_reason": "stop"
}
]
}
},
"error": null
}
],
"error": null
}Once a batch is terminal (completed, failed, expired, or cancelled) you can delete it. Deletion removes the batch from the API and purges every request and result artifact OpenRouter holds for it, without waiting for the 30-day retention window. It is not cancellation: an in-flight batch returns 409.
DELETE https://openrouter.ai/api/v1/batches/:id
curl -X DELETE https://openrouter.ai/api/v1/batches/batch_123 \
-H "Authorization: Bearer $OPENROUTER_API_KEY"The response reports the outcome per deletion target. deletion.openrouter is always deleted on a 200. deletion.upstream identifies the provider and its batch deletion status: deleted when native batch deletion is supported (Anthropic, Fireworks, Google AI Studio, Google Vertex, Mistral), unsupported otherwise, or not_applicable when no upstream batch was created. The upstream object is omitted if no provider was assigned. Independently of that batch-record outcome, OpenRouter deletes the batch's uploaded input and stored output/error files on OpenAI, Mistral, and Google AI Studio. This includes uploaded input left behind by a failed submission. Google Vertex output in OpenRouter-owned storage is also removed; files in your own GCP bucket remain under your control. Google AI Studio generated output is removed with its batch record; uploaded input is deleted separately.
{
"id": "batch_123",
"object": "batch",
"deletion": {
"openrouter": "deleted",
"upstream": {
"provider": "Anthropic",
"status": "deleted"
}
}
}The response is synchronous: a 200 means every applicable cleanup operation, including provider file deletion, has completed, and a later GET or DELETE for the same id returns 404. If cleanup fails part-way you receive a retryable 5xx; repeating the request resumes where it left off. Billing, generation, and audit records are retained. Deleting a BYOK batch that needs upstream batch or file cleanup requires the provider key the batch was submitted with to still be enabled; otherwise an initial request returns 409 and leaves the batch untouched.
Each completed result's response.body.id is that request's OpenRouter generation ID (for example gen-batch-...). To flag a bad generation, copy that ID and submit it through Report Feedback using the By generation ID flow.
Set the top-level endpoint to choose the request shape used by every body in the batch. Supported shapes:
- Chat completions:
/v1/chat/completions - Responses:
/v1/responses - Anthropic Messages:
/v1/messages - Embeddings:
/v1/embeddings(see Embeddings)
For example, an Anthropic Messages batch uses /v1/messages and puts the Messages-shaped request in each item's body:
{
"endpoint": "/v1/messages",
"model": "anthropic/claude-3.5-sonnet",
"requests": [
{
"custom_id": "req-1",
"body": {
"max_tokens": 32,
"messages": [
{
"role": "user",
"content": "Say hello."
}
]
}
}
]
}All requests in one batch use the same top-level endpoint. To mix API shapes, submit separate batches.
Set the top-level endpoint to /v1/embeddings and put the embeddings request in each item's body. Each body takes an input (a string, an array of strings, a token array, or an array of token arrays). Multimodal inputs, input_type, and provider preferences are not supported on the Batch API. Use the sync API for those.
An input can be a single string or an array of strings. When it is an array, that request embeds every string in one call:
{
"endpoint": "/v1/embeddings",
"model": "openai/text-embedding-3-small",
"requests": [
{
"custom_id": "emb-0001",
"body": {
"input": [
"The quick brown fox jumped over the lazy dog.",
"Pack my box with five dozen liquor jugs."
]
}
},
{
"custom_id": "emb-0002",
"body": {
"input": "The quick brown fox jumped over the lazy dog."
}
}
]
}Poll for results the same way as any other batch (GET https://openrouter.ai/api/v1/batches/:id). These items are the entries of the completed batch object's results array (shown in full above); each carries the standard embeddings response in its body, and there is one result item per custom_id. A request whose input is an array of strings returns one embedding object per string in data (ordered by index); a single-string request returns exactly one:
[
{
"id": "batch_req_emb_1",
"custom_id": "emb-0001",
"response": {
"status_code": 200,
"request_id": "request_456",
"body": {
"object": "list",
"data": [
{
"object": "embedding",
"embedding": [0.0023064255, -0.009327292, 0.015797347],
"index": 0
},
{
"object": "embedding",
"embedding": [-0.012282, 0.0034567, -0.0089123],
"index": 1
}
],
"model": "openai/text-embedding-3-small",
"usage": {
"prompt_tokens": 18,
"total_tokens": 18
}
}
},
"error": null
},
{
"id": "batch_req_emb_2",
"custom_id": "emb-0002",
"response": {
"status_code": 200,
"request_id": "request_789",
"body": {
"object": "list",
"data": [
{
"object": "embedding",
"embedding": [0.0023064255, -0.009327292, 0.015797347],
"index": 0
}
],
"model": "openai/text-embedding-3-small",
"usage": {
"prompt_tokens": 8,
"total_tokens": 8
}
}
},
"error": null
}
]