Skip to content

Answer the OpenAI-shaped calls from wherever the deployment says - #13

Merged
davidmckayv merged 2 commits into
CopilotKit:mainfrom
knowhycodata:openai-compatible-base-url
Aug 19, 2026
Merged

Answer the OpenAI-shaped calls from wherever the deployment says#13
davidmckayv merged 2 commits into
CopilotKit:mainfrom
knowhycodata:openai-compatible-base-url

Conversation

@knowhycodata

Copy link
Copy Markdown
Contributor

Closes #12.

OPENAI_BASE_URL decides where an OpenAI-shaped request is answered. Unset, that is OpenAI. Set, it is any endpoint speaking the same API: a gateway in front of several providers, a proxy, or a model on hardware you control.

The API server already honoured it — resolveModel in the pinned @copilotkit/runtime@1.67.1 reads process.env.OPENAI_BASE_URL for every openai/* model, so package built-in agents have always been movable, just undocumented. The two shipped Bots did not follow, and compose handed neither of them anything that could carry one. This closes that split: one line now moves the whole deployment rather than the half of it that happens to run outside Docker.

Changes

File Change
agent-bot/src/index.ts Passes baseURL to the OpenAI client.
agent-langgraph/src/index.ts Passes configuration.baseURL to ChatOpenAI.
docker-compose.yml Hands OPENAI_BASE_URL to both Bot services.
.env.example, docs/configuration.md, README.md Documents it, with a worked example.
tests/compose.test.ts Asserts both Bots receive it.

A base URL rather than another BOT_PROVIDER branch, because the API is the contract. BOT_PROVIDER=anthropic and BOT_PROVIDER=google are different APIs, not different URLs for this one, and are untouched. Model names travel verbatim in BOT_MODEL and in the tenant package's default_model, since an endpoint names its own catalogue.

The docs carry the two caveats that bite in practice: not every catalogue entry accepts tools, and a Bot without tool calling cannot drive its computer; and BOT_RESPONSES_API=true needs an endpoint implementing the Responses API, not only chat completions.

Verification

Both Bots were run against a live OpenAI-compatible gateway — OPENAI_BASE_URL=https://llmtr.com/v1, whose catalogue is public at GET /v1/models and needs no key — and both streamed AG-UI end to end with a real tool call, which is what a Bot needs to drive its computer:

data: {"type":"RUN_STARTED","threadId":"t4","runId":"r4"}
data: {"type":"TOOL_CALL_START","toolCallId":"...","toolCallName":"browser_navigate","parentMessageId":"msg_r4"}
data: {"type":"TOOL_CALL_ARGS","toolCallId":"...","delta":"{\"url\":\"http://example.com\"}"}
data: {"type":"TOOL_CALL_END","toolCallId":"..."}
data: {"type":"RUN_FINISHED","threadId":"t4","runId":"r4"}

The server path was verified by reading resolveModel in the pinned runtime rather than run, since that needs Intelligence credentials. It splits on the first / only, so a namespaced default_model: openai/gpt-4o becomes openai/openai/gpt-4o and reaches the endpoint as openai/gpt-4o. The exact pin makes that stable.

bun run format:check, bun run lint and bun run typecheck pass. bun test passes except for the PostgreSQL integration tests and the symlink tests, which fail identically on an unmodified checkout on this machine — no database, and Windows symlinks need elevation. The new compose test passes.

OpenBot already runs on your own machine, in your own PostgreSQL, with a
model key you supply. The one thing it could not do was decide where that
key is spent: `agent-bot` constructed its client with a key and no base
URL, `agent-langgraph` did the same through `ChatOpenAI`, and compose
handed neither of them a way to be told otherwise. A deployment that had
put a gateway or a proxy in front of its models, or that runs the model
on hardware it controls, had to fork two files to use it.

`OPENAI_BASE_URL` is that decision, and the API server already honoured
it: `resolveModel` in the pinned runtime reads it for every `openai/*`
model, so the package built-in agents have always been movable. This
gives the two shipped Bots the same variable and hands it to both
containers, so one line moves the whole deployment rather than the half
of it that happens to run outside Docker.

It is a base URL rather than another `BOT_PROVIDER` branch because the
API is the contract. `anthropic` and `google` are different APIs, not
different URLs for this one, and they are untouched. Model names travel
verbatim in `BOT_MODEL` and in the tenant package's `default_model`,
because an endpoint names its own catalogue.

Verified against a live OpenAI-compatible gateway: both Bots stream
AG-UI and emit tool calls, which is what a Bot needs to drive its
computer.
The OpenAI half of this landed the right idea and stopped one provider short.
A deployment that fronts Anthropic or Google the same way, or runs either on
its own hardware, had the same silent split this change set out to remove:
the API server would follow and agent-langgraph would not.

Both use the names the API server already reads, so one line still moves the
built-in agents and the Bot together.

agent-bot is untouched. It speaks the OpenAI API directly by construction, so
there is no Anthropic or Google call in it to redirect.
@davidmckayv

Copy link
Copy Markdown
Contributor

Nice PR. Thank you.

@davidmckayv
davidmckayv merged commit 019e647 into CopilotKit:main Aug 19, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The shipped Bots cannot be pointed at an OpenAI-compatible endpoint

2 participants