Streaming tool_call deltas with duplicate indexes in first chunk are accumulated incorrectly

Confirm this is an issue with the Python library and not an underlying OpenAI API

This is an issue with the Python library

Describe the bug

accumulate_delta (https://github.com/openai/openai-python/blob/main/src/openai/lib/streaming/_deltas.py#L6) appears to assume that an indexed list entry’s "index" field matches its physical position in the Python list, and ignore the index entry.

This breaks when the first streamed chunk contains multiple tool_calls entries with the same index.

Example first chunk:

{
  "delta": {
    "tool_calls": [
      {
        "index": 0,
        "id": "functions.list_files:0",
        "function": { "name": "list_files" },
        "type": "function"
      },
      {
        "index": 0,
        "function": { "arguments": " {\"" }
      }
    ]
  },
  "index": 0
}

Because this is the first tool_calls value, the accumulator stores the list directly:

if key not in acc:
    acc[key] = delta_value
    continue

So the accumulated snapshot now contains two physical list entries with index: 0.

Later chunks like this:

{
  "delta": {
    "tool_calls": [
      {
        "index": 0,
        "function": { "arguments": "path\": \".\"}" }
      }
    ]
  }
}

are merged into acc_value[0], while the second duplicate index: 0 entry remains separate. This strands part of the tool call arguments and produce invalid final JSON. Ending up with:

[
  {
    "index": 0,
    "id": "functions.list_files:0",
    "type": "function",
    "function": {
      "name": "list_files",
      "arguments": "path\": \".\"} "
    }
  },
  {
    "index": 0,
    "function": {
      "arguments": " {\""
    }
  }
]

Expected behavior

List entries with an "index" field should be merged by logical index, not by physical list position.
The two index: 0 entries in the first chunk should accumulate into one tool call:

[
  {
    "index": 0,
    "id": "functions.list_files:0",
    "type": "function",
    "function": {
      "name": "list_files",
      "arguments": " {\"path\": \".\"}"
    }
  }
]

Note that this is not a problem with the OpenAI's Go SDK because it uses the index directly.

To Reproduce

I ran basic tool calling with Kimi 2.6 using speculative decoding using vLLM OpenAI compatible server, and this SDK.
https://recipes.vllm.ai/moonshotai/Kimi-K2.6

Code snippets

OS

Arch Linux

Python version

Python 3.14.4

Library version

openai v2.35.0

fix(streaming): handle dupe-index delta entries

oh interesting, yeah we've hit weirdness like this when streaming tool_calls. That bit you quoted from _deltas.py — turns out it's super brittle if the API sends out-of-order or duplicate indexes in the first chunk. Honestly, the assumption that "index" always lines up with the list position doesn't hold up for real-world streams — we've seen this break with multiple concurrent calls, especially in v1.6.x.

The thing is, if you just stash the initial list as-is, you can't reliably merge later chunks, since you get split arg fragments, like your example. We've had to rewrite our merge logic to key off "index" directly and collapse duplicate entries, basically re-mapping the whole list every time a new chunk comes in. The docs don't mention this, but the streaming API's tool_calls can get messy, especially with rapid function call updates.

Not gonna lie, this took us a few tries to sort out — we ended up building a little dedup layer before accumulation to avoid stranded entries. That fixed the final JSON problem for us at scale (~400 concurrent tool_calls). The current lib doesn't seem ready for these edge cases, so pretty sure this is a real bug in the Python layer, not the API.

This looks like a good minimal repro for an SDK-side accumulator bug. The safe fix shape is to normalize any list item that has an index field before appending/merging, instead of treating the physical array offset as the canonical slot.

A workaround I would use outside the SDK is:

def merge_indexed_items(items):
    by_index = []
    pos = {}

    for item in items:
        idx = item.get("index") if isinstance(item, dict) else None
        if idx is None:
            by_index.append(item)
            continue

        if idx not in pos:
            pos[idx] = len(by_index)
            by_index.append(item)
        else:
            by_index[pos[idx]] = accumulate_delta(by_index[pos[idx]], item)

    return by_index

Then run that normalization on the first tool_calls delta and on each later chunk before the existing recursive merge. That handles three cases that tend to show up with OpenAI-compatible streaming servers:

duplicate index: 0 entries in the first chunk, like your Kimi/vLLM example;
sparse tool call indexes, where index: 2 arrives before index: 1;
later argument fragments that should merge into the logical tool call index, not into the physical list position.

For verification, I would add a regression test that streams exactly the two chunks from the issue body and asserts the final list has one tool call with arguments {"path": "."}. A second test with two tool calls interleaved by logical index would catch the same class of bug for parallel tool calls.

This is also the kind of edge case worth checking across Kimi/Qwen/DeepSeek/GLM OpenAI-compatible endpoints; I’m tracking those compatibility probes at anlinkai.com/beta, but the immediate fix here should stay in the SDK accumulator.

oh wait, you're running on v1.6.x? That version issue you mentioned, yeah that's tricky. We've seen this break with multiple concurrent calls, especially in that version. Your workaround looks solid, though. We've used similar approaches for handling out-of-order data in our streaming pipelines.

Thanks for confirming! Our deduplication approach (per index) has been working. Have you seen any cases where it fails? Let's keep tracking this.

@csreesan

Iinitially opened #3203 in parallel to chris (@csreesan) reporting our bug here, but that has caused fragmentation. I've closed that as a dupe but am including the context in-thread so it doesn't get buried:

Describe the bug

When a streaming chunk contains two tool_calls entries with the same index, accumulate_delta stores them as two separate list elements instead of merging them. Observed in practice against vLLM with speculative decoding enabled. Subsequent chunks then merge into element [0] only, leaving element [1] orphaned. The reconstructed tool_calls[0].function.arguments is missing whichever fragment landed on [1] and is no longer parseable as JSON.

The OpenAPI spec permits this shape. tool_calls is an unconstrained array, and ChatCompletionMessageToolCallChunk only requires index:

ChatCompletionStreamResponseDelta:
  properties:
    tool_calls:
      type: array
      items:
        $ref: "#/components/schemas/ChatCompletionMessageToolCallChunk"

The cause is two early-return branches at the top of the per-key loop in _deltas.py:

for key, delta_value in delta.items():
    if key not in acc:          # L8-10: key absent
        acc[key] = delta_value
        continue

    acc_value = acc[key]
    if acc_value is None:       # L13-15: value is None
        acc[key] = delta_value
        continue

Both assign the delta list wholesale and skip the index merge logic that handles duplicate indexes in steady state. So any duplicate-index entries in the very first chunk slip through unmerged.

_assistants.py has identical merge logic with the same bug. _convert_initial_chunk_into_snapshot in _completions.py uses "message": choice.delta.to_dict() to seed the snapshot, which copies the raw delta array without an index merge. So the bug also triggers on the very first chunk of a stream.

In the Node SDK (openai@6.36.0), ChatCompletionStream.ts (lines 316-337) doesn't have the short circuit. It always loops and dispatches by index, and produces the correct result on this input. AssistantStream.accumulateDelta (lines 213-270) has the same fast-path logic and the same bug.

To Reproduce

import json
from openai.lib.streaming._deltas import accumulate_delta

acc = {"tool_calls": None}

# Simulates a first chunk with two entries at index=0
# (e.g. tool name + first argument fragment in one SSE event)
accumulate_delta(acc, {"tool_calls": [
    {"index": 0, "id": "call_abc", "function": {"name": "get_weather"}, "type": "function"},
    {"index": 0, "function": {"arguments": '{"city"'}},
]})
accumulate_delta(acc, {"tool_calls": [
    {"index": 0, "function": {"arguments": ': "London"}'}},
]})

args = acc["tool_calls"][0]["function"]["arguments"]
try:
    json.loads(args)
except json.JSONDecodeError as e:
    print("arguments not parseable:", e)
print("orphan entry:", acc["tool_calls"][1])

Expected: arguments == '{"city": "Reykjavik"}', parses to {"city": "Reykjavik"}.

Actual: arguments == ': "Reykjavik"}'. The '{"city"' prefix ended up on acc["tool_calls"][1].

Code snippets

OS

macOS

Python version

Python v3.14.2

Library version

openai v2.35.0

For anyone running into the issue before a fix is landed, I'd recommend splitting chunks with multiple tool call deltas of the same index instead of attempting merging user-side. The accumulator in the SDK should own the merging logic.

We can however match the shape the existing accumulator already handles correctly (i.e. one delta per index per chunk) with something like:

async for chunk in stream:
    extras: list[ChatCompletionChunk] = []

    for choice in chunk.choices or []:
        tcs = choice.delta.tool_calls or []

        # only duplicate-index entries trigger the accumulator bug
        indices = [tc.index for tc in tcs]
        if len(indices) == len(set(indices)):
            continue

        choice.delta.tool_calls = [tcs[0]]
        for tc in tcs[1:]:
            # some fields don't apply to the synthetic chunks
            extras.append(chunk.model_copy(update={
                "choices": [Choice.model_construct(
                    index=choice.index,
                    delta=ChoiceDelta.model_construct(tool_calls=[tc]),
                    finish_reason=None,
                    logprobs=None,
                )],
                "usage": None,
            }))

    yield chunk

    for extra in extras:
        yield extra

Yeah, I can see how deduplication per index might work in some cases, but we did run into issues when the API sent a mix of duplicate and out-of-order indexes. It threw our logic off completely. If you’re on v1.6.x, I'd keep an eye out for any late arrivals that get stuck or missed.

It might be worth logging those edge cases to track how they behave. Let me know if you want to brainstorm some more on this!

Any news regarding this issue? Are pull requests going to be merged soon and fix released? We would like to migrate our application to the newest GPT available in Foundry but this issue is blocking us from doing so.

fix: merge list entries by logical index instead of physical position (…

fix: coalesce duplicate-index entries when first chunk is stored dire…

fix: coalesce at all list boundaries, fix _build_events index lookup,…

fix: merge list entries by logical index instead of physical position (…

fix: coalesce duplicate-index entries when first chunk is stored dire…

fix: coalesce at all list boundaries, fix _build_events index lookup,…

Working on this in PR #3633.

fix: merge list entries by logical index instead of physical position (…

fix: coalesce duplicate-index entries when first chunk is stored dire…

fix: coalesce at all list boundaries, fix _build_events index lookup,…

fix: merge list entries by logical index instead of physical position (…

fix: coalesce duplicate-index entries when first chunk is stored dire…

fix: coalesce at all list boundaries, fix _build_events index lookup,…

Fix streaming delta merge for duplicate tool call indexes (openai#3425)

Navigation Menu

Repository navigation

Streaming tool_call deltas with duplicate indexes in first chunk are accumulated incorrectly #3201

Confirm this is an issue with the Python library and not an underlying OpenAI API

Describe the bug

Expected behavior

To Reproduce

Code snippets

OS

Python version

Library version

Describe the bug

To Reproduce

Code snippets

OS

Python version

Library version

11 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Streaming tool_call deltas with duplicate indexes in first chunk are accumulated incorrectly #3201

Description

Confirm this is an issue with the Python library and not an underlying OpenAI API

Describe the bug

Expected behavior

To Reproduce

Code snippets

OS

Python version

Library version

Activity

rehan243 commented on May 11, 2026

diva0078 commented on May 11, 2026

rehan243 commented on May 11, 2026

diva0078 commented on May 11, 2026

hrolfurinn commented on May 12, 2026

Describe the bug

To Reproduce

Code snippets

OS

Python version

Library version

hrolfurinn commented on May 12, 2026

rehan243 commented on May 12, 2026

Sanady commented on May 21, 2026

11 remaining items

Xsidz commented on Aug 16, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions