Skip to content

Chat Completions: an empty choices array raises a bare IndexError instead of TypeSafeError #47

Description

@minhtien2405

Summary

When an OpenAI-compatible endpoint returns a schema-valid Chat Completions response whose choices array is empty, the adapter raises a bare IndexError: list index out of range from providers/openai.py::_result. Callers that catch typesafe_sdk.TypeSafeError, which is the adapter's failure contract, do not catch it, and error.debug (llm_attempts, llm_response) is not attached. Sync and async clients are both affected. This is the same kind of problem as #39 (omitted usage) and #38 (finish_reason="length"), in the same function.

Reproduction

Everything runs in memory, with no network and no API key. The fixture comes from tests/test_provider_nonanswers.py:

payload.update(choices=[], usage={"prompt_tokens": 12, "completion_tokens": 0, "total_tokens": 12})
provider = OpenAIProvider("test-model", api="chat_completions")
_evaluate(provider, structured=True)

At e1d4cc9 (v0.2.1):

src/system_one_adapter/providers/openai.py:45: IndexError
E       IndexError: list index out of range
4 failed   # sync/async × structured/prompted

The openai SDK types ChatCompletion.choices as a plain list with no minimum length, so the response passes the SDK's parsing and the failure only shows up when the adapter indexes it. Other clients have run into empty choices from OpenAI-compatible gateways, for example pydantic/pydantic-ai#7909.

Expected behavior

The client raises TypeSafeError that says the completion had no choices, with the recorded attempt preserved in error.debug. It does not spend malformed-output or transient retries. This matches how #38 and #39 were resolved.

Actual behavior

A bare IndexError is raised before record_response runs. The error is outside the TypeSafeError hierarchy and carries no debug trace.

Root cause

_result() evaluates response.choices[0].finish_reason on its first line, before any check on the number of choices. The Responses API path (_responses_result) is not affected because it iterates over response.output.

Proposed fix

A 3-line change:

 def _result(response: Any) -> ProviderResult:
+    if not response.choices:
+        record_response(response, finish_reason=None)
+        raise TypeSafeError("OpenAI chat completion returned no choices.")
     record_response(response, finish_reason=response.choices[0].finish_reason)

Regression test

test_chat_completion_empty_choices in tests/test_provider_nonanswers.py is parametrized over OpenAIProvider/AsyncOpenAIProvider × structured/prompted. It asserts that TypeSafeError is raised, that exactly one request was sent, that retry_reasons == [], and that the attempt records llm_response.choices == [] with finish_reason=None and an error. It fails 4/4 on e1d4cc9 and passes 4/4 with the fix. The full suite gives 428 passed, and pyrefly reports 0 errors.

The patch and test are ready. Since PRs are restricted to collaborators, the fix is on my fork: https://github.com/minhtien2405/system-one-adapter-python/tree/fix/chat-completions-empty-choices (commit 61fa1ac).

Environment

system-one-adapter 0.2.1 (e1d4cc9), openai 3.17.0, Python 3.14.7, Linux.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions