feat(kimi_k3): wire up tool calling — XTML per Moonshot's reference renderer (#1143)

[ZacharyZcR]

Implements #1143 (closes #1029's last flagship gap), option (a) from the design write-up — literal re-emission, GLM-precedent marker suppression. If (b) is preferred I'll rework the output side; everything else is independent of that choice.

What's in

Format — verified against encoding_k3.py in the Kimi-K3 checkpoint repo (the normative renderer; K3 has no jinja template). All four surfaces: tool-declare system turn, tools > call > argument blocks with per-argument XTML types (non-strings keep their exact JSON literal — 1e2 stays 1e2, one-level-deep parse mirrored from the reference), role="tool" result turns re-sorted into tool_calls order by tool_call_id (the reference's normalization), tool_choice required/none/forced.

Gateway — K3CHAT1 grows typed records (Y/O/B/F/V/J) instead of pre-rendered text: K3's rank-BPE segment boundaries are the tokenizer contract, so tags keep being constructed engine-side. parse_k3_tool_calls + the stream-marker machinery handle the return path, with the GLM path's unclosed-block recovery posture.

Engine — chat_build_wire parses the records through the existing ChatB primitives plus cb_attr (attribute segmentation matches the reference exactly: " key", '="', escaped value, '"' each their own segment; &/" → &/"). The serve loop re-emits tool-structural runs only literally; every other XTML run stays suppressed exactly as before, so non-tool traffic is byte-identical to today.

Tests (three layers)

• test_k3_chat_tools.c — golden: records rendered against the tiny Kimi tokenizer (which now carries the four XTML tokens) must reproduce the reference byte stream exactly, incl. attr escaping and malformed-record refusal. 7/7.
• Gateway units — declaration/choice modes, call history, argument typing, json fallback, id-based result re-sort, generated-call parsing, unclosed-tail recovery. 11/11 (kimi-filtered).
• test_openai_tools_k3_e2e.py — real gateway subprocess vs mock SERVE engine: both response shapes, chunk-straddling stream suppression (markers split across 25 DATA frames), tool_choice=none omits the declaration. 4/4.
• Full regressions: test_openai* 163/163, Anthropic surface 28/28 (kimi moved from the reject-list test to a render assertion), make test-c green, zero-warning builds.

Stated limits

• Not executed against the real 2.8T checkpoint — the e2e layer is engine-mocked by design (same as the GLM suite). A real-model smoke on a K3-capable host is the honest final gate before this ships in a release; happy to coordinate.
• tool_choice=none follows the V4 gateway precedent (tools simply not offered) rather than the reference's MUST-NOT message — noted in feat(kimi_k3): wire up tool calling — XTML format verified against Moonshot's reference renderer, design for review #1143, trivially changeable if you'd rather mirror the reference.
• docs/api.md matrix updated; ENVIRONMENT.md untouched (no new env vars).

[JustVugg]

Timing: you opened this at 13:08 and my (b) call on #1143 landed after, so you wrote (a) exactly as you said you would and this is not you ignoring a decision. The offer to rework the output side is appreciated, and yes — I am holding to (b), with two caveats that matter given how today has gone.

First caveat, and you should weigh it: I was confidently wrong twice in the last hour on #1136, and one of those wrong calls sent you to rewrite correct documentation into incorrect documentation. So do not treat this call as authoritative because it is mine. If the reasoning below does not survive contact with the codec, say so and I will move.

The reasoning, restated so you can attack it. The engine sees <|open|> as a token id — it has ground truth about whether a structural marker was generated by the model or merely typed. Option (a) converts that to bytes and asks the gateway to re-derive it by pattern matching, which destroys the distinction at exactly the boundary where it matters most. For GLM the consequence of a spoofed marker is wrong text on a screen; for K3, whose whole point is agentic use, it is an action that runs. GLM carrying this exposure is a reason to fix GLM, not to extend the pattern to the engine where the blast radius is largest.

The cost objection you raised in the design write-up was real when you wrote it and is not anymore: serve_codec.h now owns both directions (coli_serve_write_data and siblings), every engine's migration landed behind a byte-exact wire-transcript freeze, and Inkling has already ridden an engine-specific structured payload on it (#1116, audio as an opaque extension). Adding a record is one edit in one file with an existing test that proves nothing else moved — not the five-way protocol surgery it would have been last week.

And the part I am not asking you to carry: GLM migrating to the same mechanism is ours. You are not fixing GLM's hole as the price of doing K3 correctly; you are declining to copy it, and we will make the two symmetric afterwards.

On the rest of this PR, nothing changes and it is good work. The format side is the part I cannot check without the checkpoint and the part most likely to rot silently, so going to encoding_k3.py and mirroring its behaviour — including 1e2 staying 1e2, the one-level-deep argument parse, and the tool_call_id re-sort — is what makes this trustworthy. Typed K3CHAT1 records rather than pre-rendered text is right for the rank-BPE boundary reason you gave. tok_kimi_tiny.json plus the mock-engine e2e is the correct testing shape for a 2.8T model nobody here can run.

Since you say the rest is independent of the output choice: if the rework turns out to be larger than a day, split it — land the format, gateway records and tests behind a flag that keeps the 400, then flip it on with the sideband. Better than one PR ageing while dev moves under it, which is how #790 and #601 got where they are.

Ping me when it is up and I will review the codec change the same day.

[JustVugg]

Merged — and I owe you an apology, because I blocked this on a claim I had not verified.

I told you that with option (a) a spoofed marker on K3 "is an action that runs". That is false, and I should have checked before writing it. parse_tool_calls returns (content, tool_calls); the gateway reports the call in the OpenAI response and never executes anything — the only subprocess.Popen in openai_server.py launches the engine. The client application decides what to run. The real exposure is "we tell a client the model requested something it did not", which is a genuine trust-boundary problem but not the category I invoked, and my whole argument rested on that category.

The second thing I got wrong is the one that actually flips the decision. GLM has shipped this exact mechanism for months. So (a) on K3 does not open a new class of exposure, it extends an existing one — and if K3 had gone to the sideband while GLM kept markers, the gateway would carry two mechanisms until GLM migrated. This repo has already paid for that shape once: byte framing duplicated five times is how Windows binary mode silently disappeared from sibling engines (#748). Doing the sideband once, for both engines together, is better than doing K3 specially and retrofitting GLM afterwards.

So the sideband work becomes ours and covers both engines. You are not reworking anything.

To be explicit about the sequence, since I sent you back and forth today: you wrote (a) as announced, I asked for (b) on a bad premise, you offered to rework rather than argue, and the rework turned out to be unnecessary — the right call was the one you made at the start. The 852 lines here are unaffected by any of it: the format work against encoding_k3.py, the typed K3CHAT1 records, tok_kimi_tiny.json and the mock-engine e2e all stand on their own, and the format side in particular is the part nobody could have checked without going to the reference renderer the way you did.

#1029 closes with this: Kimi K3 was the last flagship engine returning 400 Tool use is not wired up.