Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions docs/src/content/docs/reference/http-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ For Kimi, Grok, and Cursor, the route accepts:

- `system`, `developer`, `user`, `assistant`, and `tool` messages
- text, supported user images, function calls, and tool results
- function tools and `tool_choice`
- function tools, `tool_choice`, and `parallel_tool_calls`
- `max_tokens` or `max_completion_tokens`
- `reasoning_effort`
- streaming, non-streaming responses, and `stream_options.include_usage`
Expand Down Expand Up @@ -128,7 +128,7 @@ Codex models use native Responses passthrough, including native JSON and SSE out

- string input or message items
- `instructions`
- function calls, function-call outputs, tools, and `tool_choice`
- function calls, function-call outputs, tools, `tool_choice`, and `parallel_tool_calls`
- `max_output_tokens`
- `reasoning.effort`
- streaming or non-streaming output
Expand All @@ -137,6 +137,14 @@ Responses include the accepted tool settings. Grok search appears as a `web_sear

`store: true` and other unsupported non-null fields return an error. Stored response retrieval, deletion, and WebSocket client connections are not supported.

### Parallel tool calls

The shared Kimi, Grok, and Cursor ingress accepts boolean `parallel_tool_calls` on both OpenAI routes. `false` preserves serial tool execution through translation, while `true` selects the existing parallel default. Omitting the field leaves the provider default unchanged. Non-boolean values return an `invalid_request_error` with `parallel_tool_calls` in `error.param`.

The setting applies without changing the requested `tool_choice` mode. This includes omitted or `auto` choices, `none`, `required`, and named functions. Internally, an explicit OpenAI setting determines the equivalent Anthropic `tool_choice.disable_parallel_tool_use` value. Anthropic Messages requests can set `disable_parallel_tool_use` directly. Kimi and Grok receive the corresponding upstream `parallel_tool_calls` value, and Cursor's bridged tool loop remains serial between client tool results.

Codex Responses uses native passthrough and forwards `parallel_tool_calls` unchanged. Codex Chat Completions has its own field allowlist and rejects `parallel_tool_calls` because that path does not support function tools.

## OpenAI routing, sessions, and errors

Both OpenAI routes strip a trailing `[1m]`, resolve configured aliases, and choose the provider from `model`. Aliases follow `aliasProvider`, while explicit provider model IDs keep their provider. Unknown models return HTTP 400 with the supported model list.
Expand Down
98 changes: 87 additions & 11 deletions src/openai_compat/request.rs
Original file line number Diff line number Diff line change
Expand Up @@ -114,11 +114,12 @@ pub fn parse_request(
));
}
validate_single_choice(object)?;
validate_parallel_tools(object)?;
let parallel_tool_calls = optional_bool(object, "parallel_tool_calls")?;
validate_store(surface, object)?;
let max_tokens = parse_max_tokens(surface, object)?;
let tools = parse_tools(object.get("tools"), surface)?;
let tool_choice = parse_tool_choice(object.get("tool_choice"), &tools, surface)?;
let mut tool_choice = parse_tool_choice(object.get("tool_choice"), &tools, surface)?;
apply_parallel_tool_calls(&mut tool_choice, parallel_tool_calls);
let response_metadata = if surface == OpenAiSurface::Responses {
OpenAiResponseMetadata {
tools: object
Expand Down Expand Up @@ -240,15 +241,18 @@ fn validate_single_choice(object: &Map<String, Value>) -> Result<(), OpenAiError
}
}

fn validate_parallel_tools(object: &Map<String, Value>) -> Result<(), OpenAiError> {
match object.get("parallel_tool_calls") {
None | Some(Value::Null) | Some(Value::Bool(false)) => Ok(()),
Some(Value::Bool(true)) => Err(OpenAiError::unsupported("parallel_tool_calls")),
Some(_) => Err(OpenAiError::invalid(
"'parallel_tool_calls' must be a boolean",
Some("parallel_tool_calls"),
)),
}
fn apply_parallel_tool_calls(tool_choice: &mut Option<Value>, parallel_tool_calls: Option<bool>) {
let Some(parallel_tool_calls) = parallel_tool_calls else {
return;
};
let choice = tool_choice.get_or_insert_with(|| json!({"type":"auto"}));
choice
.as_object_mut()
.expect("translated tool choice is an object")
.insert(
"disable_parallel_tool_use".to_string(),
Value::Bool(!parallel_tool_calls),
);
}

fn parse_max_tokens(
Expand Down Expand Up @@ -1135,6 +1139,78 @@ mod tests {
assert_eq!(parsed.messages.extra["tool_choice"]["name"], "lookup");
}

#[test]
fn parallel_tool_calls_sets_anthropic_tool_choice_policy() {
let cases = [
(None, "auto"),
(Some(json!("auto")), "auto"),
(Some(json!("none")), "none"),
(Some(json!("required")), "any"),
(
Some(json!({"type":"function","function":{"name":"lookup"}})),
"tool",
),
];
for parallel in [false, true] {
for (choice, expected_type) in &cases {
let mut body = json!({
"model":"kimi-k2.6",
"messages":[{"role":"user","content":"look up x"}],
"tools":[{"type":"function","function":{"name":"lookup","parameters":{"type":"object"}}}],
"parallel_tool_calls":parallel,
});
if let Some(choice) = choice {
body["tool_choice"] = choice.clone();
}
let parsed = parse_request(
OpenAiSurface::ChatCompletions,
body,
"kimi",
Some("session"),
)
.unwrap();
let translated = &parsed.messages.extra["tool_choice"];
assert_eq!(translated["type"], *expected_type);
assert_eq!(translated["disable_parallel_tool_use"], !parallel);
}
}
}

#[test]
fn responses_parallel_tool_calls_supports_named_choice() {
let parsed = parse_request(
OpenAiSurface::Responses,
json!({
"model":"grok-4.5",
"input":"look up x",
"tools":[{"type":"function","name":"lookup","parameters":{"type":"object"}}],
"tool_choice":{"type":"function","name":"lookup"},
"parallel_tool_calls":false,
}),
"grok",
None,
)
.unwrap();
assert_eq!(parsed.messages.extra["tool_choice"]["type"], "tool");
assert_eq!(
parsed.messages.extra["tool_choice"]["disable_parallel_tool_use"],
true
);
}

#[test]
fn parallel_tool_calls_must_be_boolean() {
let error = parse_request(
OpenAiSurface::Responses,
json!({"model":"grok-4.5","input":"hello","parallel_tool_calls":"false"}),
"grok",
None,
)
.unwrap_err();
assert_eq!(error.param.as_deref(), Some("parallel_tool_calls"));
assert!(error.code.is_none());
}

#[test]
fn responses_maps_function_items() {
let parsed = parse_request(
Expand Down
13 changes: 13 additions & 0 deletions src/providers/codex/native.rs
Original file line number Diff line number Diff line change
Expand Up @@ -703,6 +703,19 @@ mod tests {
assert_eq!(fast["service_tier"], "priority");
}

#[test]
fn native_request_preserves_parallel_tool_calls() {
for parallel in [false, true] {
let mut body = request(json!({
"model":"gpt-5.4",
"input":[],
"parallel_tool_calls":parallel
}));
shape_native_request(&mut body).unwrap();
assert_eq!(body["parallel_tool_calls"], parallel);
}
}

#[test]
fn explicit_service_tier_is_preserved() {
let mut body = request(json!({
Expand Down
14 changes: 3 additions & 11 deletions src/providers/codex/translate/request.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@ use serde_json::Value;
use crate::anthropic::schema::MessagesRequest;
use crate::config;
use crate::providers::translate_shared::{
ContentBlock, flatten_system_text, image_source_to_url, normalize_content, read_effort,
ContentBlock, flatten_system_text, image_source_to_url, normalize_content, parallel_tool_calls,
read_effort,
};

use super::read_rewrite::{ReadOffsetRewrite, read_offset_rewrite};
Expand Down Expand Up @@ -433,7 +434,7 @@ pub fn translate_request(
let input = build_input(req);
let tools = read_tools(req)?;
let tool_choice = map_tool_choice(req)?;
let parallel_tool_calls = !disable_parallel_tool_use(req);
let parallel_tool_calls = parallel_tool_calls(req).unwrap_or(true);

let mut text = ResponsesText {
verbosity: Some("low".to_string()),
Expand Down Expand Up @@ -754,15 +755,6 @@ fn map_tool_choice(req: &MessagesRequest) -> Result<Option<ResponsesToolChoice>,
}
}

fn disable_parallel_tool_use(req: &MessagesRequest) -> bool {
req.extra
.get("tool_choice")
.and_then(Value::as_object)
.and_then(|choice| choice.get("disable_parallel_tool_use"))
.and_then(Value::as_bool)
.unwrap_or(false)
}

fn build_input(req: &MessagesRequest) -> Vec<ResponsesInputItem> {
let mut out: Vec<ResponsesInputItem> = Vec::new();
let mut read_tool_uses_with_offset = HashSet::new();
Expand Down
31 changes: 26 additions & 5 deletions src/providers/grok/translate/request.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ use serde_json::Value;

use crate::anthropic::schema::{Message, MessagesRequest};
use crate::config::GrokToolImageMode;
use crate::providers::translate_shared::{ImageSource, image_source_to_url};
use crate::providers::translate_shared::{ImageSource, image_source_to_url, parallel_tool_calls};

#[derive(Debug, Clone, Serialize)]
pub struct GrokResponsesRequest {
Expand All @@ -17,6 +17,8 @@ pub struct GrokResponsesRequest {
pub tools: Option<Vec<GrokTool>>,
#[serde(skip_serializing_if = "Option::is_none")]
pub tool_choice: Option<GrokToolChoice>,
#[serde(skip_serializing_if = "Option::is_none")]
pub parallel_tool_calls: Option<bool>,
pub store: bool,
pub stream: bool,
#[serde(skip_serializing_if = "Option::is_none")]
Expand Down Expand Up @@ -183,6 +185,7 @@ pub fn translate_request_with_mode(
input,
tools,
tool_choice,
parallel_tool_calls: parallel_tool_calls(req),
store: false,
stream: true,
max_output_tokens: req.max_tokens,
Expand Down Expand Up @@ -479,11 +482,29 @@ fn parse_tool_choice(
.get("type")
.and_then(Value::as_str)
.ok_or_else(|| anyhow::anyhow!("tool_choice type is invalid"))?;
let valid_policy = obj
.get("disable_parallel_tool_use")
.is_none_or(Value::is_boolean);
match kind {
"auto" if obj.len() == 1 => Ok(Some(GrokToolChoice::Auto("auto".into()))),
"any" if obj.len() == 1 => Ok(Some(GrokToolChoice::Required("required".into()))),
"none" if obj.len() == 1 => Ok(Some(GrokToolChoice::None("none".into()))),
"tool" if obj.len() == 2 => {
"auto" | "any" | "none"
if valid_policy
&& obj
.keys()
.all(|key| ["type", "disable_parallel_tool_use"].contains(&key.as_str())) =>
{
Ok(Some(match kind {
"auto" => GrokToolChoice::Auto("auto".into()),
"any" => GrokToolChoice::Required("required".into()),
"none" => GrokToolChoice::None("none".into()),
_ => unreachable!(),
}))
}
"tool"
if valid_policy
&& obj.keys().all(|key| {
["type", "name", "disable_parallel_tool_use"].contains(&key.as_str())
}) =>
{
let name = obj
.get("name")
.and_then(Value::as_str)
Expand Down
5 changes: 4 additions & 1 deletion src/providers/kimi/translate/request.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ use super::model_allowlist::{KIMI_DEFAULT_MODEL, assert_allowed_model, is_k3, re
use crate::anthropic::schema::MessagesRequest;
use crate::providers::translate_shared::{
ContentBlock, flatten_system_text, image_block_to_url, image_source_to_url, normalize_content,
read_effort,
parallel_tool_calls, read_effort,
};

// ---------------------------------------------------------------------------
Expand All @@ -20,6 +20,8 @@ pub struct KimiChatRequest {
pub tools: Option<Vec<KimiTool>>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub tool_choice: Option<KimiToolChoice>,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub parallel_tool_calls: Option<bool>,
pub stream: bool,
pub stream_options: KimiStreamOptions,
pub max_tokens: u32,
Expand Down Expand Up @@ -153,6 +155,7 @@ pub fn translate_request(
}),
tools: if tools.is_empty() { None } else { Some(tools) },
tool_choice,
parallel_tool_calls: parallel_tool_calls(req),
prompt_cache_key: opts.session_id,
};

Expand Down
9 changes: 9 additions & 0 deletions src/providers/translate_shared.rs
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,15 @@ pub fn flatten_system_text(system_val: Option<&Value>) -> Option<String> {
}
}

pub fn parallel_tool_calls(req: &MessagesRequest) -> Option<bool> {
req.extra
.get("tool_choice")
.and_then(Value::as_object)
.and_then(|choice| choice.get("disable_parallel_tool_use"))
.and_then(Value::as_bool)
.map(|disabled| !disabled)
}

pub fn read_effort(req: &MessagesRequest) -> Result<Option<&str>, anyhow::Error> {
let output_config = match req.extra.get("output_config") {
Some(Value::Object(m)) => m,
Expand Down
Loading