urai

Urai is a Python library for developers who write chat programs that must work with more than one model engine. Local engines such as llama.cpp and hosted APIs such as OpenAI’s take a list of messages and return a reply. Each engine names its settings, formats tool calls and reports token usage in its own way. Urai defines one conversation format, one set of option names, one tool loop, one streaming format and one way to count tokens for all of them. The name “urai” (உரை) means “dialogue” in Tamil.

Urai contains no code that talks to a model. A backend package supplies that code. A backend is a Chat subclass that sends requests to a model engine. A backend package contains and registers one or more backends. rishi is a backend package. A runtime is the name under which a backend is registered, such as remote. aidialog defines the typed message parts that some backends use.

Install

pip install uraiyadal

Urai cannot talk to a model alone. Install a backend package too and import it. Importing it registers its runtimes with urai. The examples below assume that a backend has registered a runtime named remote for hosted models, as rishi does.

Start a chat

mk_chat builds a chat for a model name. Calling the chat with a message runs one turn and returns the reply. A turn starts with a message from you and ends with the final reply. It can include tool calls and several requests to the model. resp_text returns the text of a reply.

from urai import mk_chat, resp_text

chat = mk_chat('gpt-5.1', runtime='remote', sp='Answer briefly.', temp=0.2)
reply = chat('What is 6 times 7?')
print(resp_text(reply))

chat.hist holds the conversation so far as a list of plain dicts in the OpenAI chat format. Every backend reads and writes this format. You can pass a history from one backend to another with messages=, and the conversation continues there.

[(msg['role'], resp_text(msg)) for msg in chat.hist]

Resolve a model before loading it

Loading a local model is slow. resolve finds the runtime of a model name and the size of its context window without loading anything. The context window is the number of tokens the model can read at once.

from urai import resolve

spec = resolve('claude-sonnet-4.6', runtime='remote')
print(spec.runtime, spec.ctx, spec.local)

chat = mk_chat(spec)

resolve returns a ModelSpec. A ModelSpec is frozen. You can cache it, compare it and pass it between parts of a program. mk_chat accepts it directly.

Configure one turn

Options passed to mk_chat apply to the whole conversation. Options passed to one call apply to that turn only.

reply = chat('Explain the hard part.', effort='high', temp=0.9)

Option names are the same for every backend. Examples are ctx for the context window, temp for temperature, max_output_tokens, think and effort. Each backend renames them to its engine’s names. A backend drops without warning the options its engine cannot use. Urai passes a keyword it does not know, such as llama.cpp’s n_gpu_layers, to the backend unchanged.

Add a tool

A tool is a Python function that the model can ask urai to call. Urai builds the schema that the model sees from the function’s name, type annotations and docstring.

def add(a: int, b: int) -> int:
    "Add two integers."
    return a + b

chat = mk_chat('gpt-5.1', runtime='remote', tools=[add], approve=lambda call: True)
reply = chat('Use the tool to add 20 and 22.')
print(resp_text(reply))

When the model asks for a tool, urai calls approve with the call. If approve returns True, the function runs and urai sends its result to the model. The model then continues. This repeats until the model answers without a tool call or the turn reaches max_steps tool calls (default 10). If you omit approve, every call runs. A tool that the provider runs on its own servers, such as a hosted web search, appears in the history and never runs locally.

Modules

module	what it covers
core	token usage, callbacks, and displaying replies
tags	reading <think> and <tool_call> blocks out of plain text
msgs	the canonical message format and conversions to and from providers
caps	what a model accepts and returns, and its context window
opts	the runtime registry, model resolution and portable options
chat	the Chat class, synchronous and async
loop	running tools: approval, budgets, parallel calls and context recovery
eval	classification, structured output and grading
sandbox	running the Python a model writes
record	recording model replies once and replaying them in tests
broker	many separate conversations sharing one loaded model

Write a backend

A backend is a subclass of Chat registered under a runtime name. Add ToolLoopMixin when the engine returns tool calls as data. ToolLoopMixin needs two methods. _model_step makes one request and returns the reply. _stream_step makes one request and yields chunks. _opt_map renames portable options to the engine’s names. _opt_skip lists the options that the engine cannot use. Runtime takes a field pats, a tuple of substrings that select the runtime. A model name that contains one of them uses this runtime.

from urai import Chat, ToolLoopMixin, Runtime, register_runtime

class MyChat(ToolLoopMixin, Chat):
    _runtime = 'mine'
    _opt_map = {'ctx': 'n_ctx'}
    _opt_skip = ('effort',)

    def _model_step(self, **kw): ...
    def _stream_step(self, **kw): ...

register_runtime(Runtime('mine', 'mypkg.chat.MyChat', pats=('.mine',)))

Chat('model.mine') now returns a MyChat. The name contains .mine, which matches the runtime’s pats.

Develop

The notebooks in nbs/ are the source. nbdev-export generates the files in urai/ from them.

uv sync --all-extras --group dev
uv run nbdev-export
uv run nbdev-test
uv run nbdev-clean

The tests use scripted backends, so they load no model and need no network.