diff --git a/README.md b/README.md index c68e5a3..8b2df78 100644 --- a/README.md +++ b/README.md @@ -134,6 +134,12 @@ zerogpu chat "Why does my API keep getting rate-limited?" -m gpt-oss-120b -r # Multilingual reasoning zerogpu chat "Explique la mise en cache en une phrase." -m qwen3-30b-a3b-fp8 + +# A 1M-token context, for whole repos and very long documents +zerogpu chat "$(cat ARCHITECTURE.md)" -m glm-5.2 + +# Coding and agentic work, at a fraction of the flagship price +zerogpu chat "Port this helper to async/await." -m deepseek-v4-flash ``` | Option | Description | @@ -148,8 +154,10 @@ zerogpu chat "Explique la mise en cache en une phrase." -m qwen3-30b-a3b-fp8 | `LFM2.5-1.2B-Thinking` | Compact reasoning model. | | `gpt-oss-120b` | 117B MoE, 131K context, reasoning + function calling. | | `qwen3-30b-a3b-fp8` | 30.5B MoE, 100+ languages, reasoning + function calling. | +| `glm-5.2` | 753B MoE, 1M context, reasoning + function calling. The platform's most capable model, and its priciest. | +| `deepseek-v4-flash` | 284B MoE (13B active), 1M context, coding and agentic workflows. | -`qwen3-30b-a3b-fp8` is served by the Chat Completions API rather than the Responses API; the CLI routes it automatically. +`qwen3-30b-a3b-fp8`, `glm-5.2`, and `deepseek-v4-flash` are served by the Chat Completions API rather than the Responses API; the CLI routes them automatically. #### `chat_thinking` diff --git a/docs/ADDING_COMMANDS.md b/docs/ADDING_COMMANDS.md index 2098554..d4d77e1 100644 --- a/docs/ADDING_COMMANDS.md +++ b/docs/ADDING_COMMANDS.md @@ -7,7 +7,7 @@ This guide explains how to add a new CLI command to the ZeroGPU CLI. - `src/commands/` — one file per command, each exporting a `registerCommand(program)` function. - `src/cli.ts` — wires every command into the root program. - `src/lib/responses.ts` — shared `RESPONSES_ENDPOINT`, `ResponsesApiResponse`, and the `extractOutputText` / `extractReasoningText` helpers for `/v1/responses` calls. -- `src/lib/chatCompletions.ts` — the same for `/v1/chat/completions`, used by models the platform serves only there (currently `qwen3-30b-a3b-fp8`), plus `toResponsesUsage` to normalize token counts for savings tracking. +- `src/lib/chatCompletions.ts` — the same for `/v1/chat/completions`, used by models the platform serves only there (currently `qwen3-30b-a3b-fp8`, `glm-5.2`, and `deepseek-v4-flash`), plus `toResponsesUsage` to normalize token counts for savings tracking. - `src/lib/auth.ts` — `getApiKey()` for authenticated requests. ## Steps diff --git a/docs/DOCUMENTATION.md b/docs/DOCUMENTATION.md index 43c0369..9cb5634 100644 --- a/docs/DOCUMENTATION.md +++ b/docs/DOCUMENTATION.md @@ -5,7 +5,7 @@ `zerogpu-cli` is the official command-line interface for [ZeroGPU](https://zerogpu.ai), a distributed / edge inference platform for small language models (SLMs) and nano language models. The CLI is a thin, OpenAI-compatible client around the ZeroGPU **Responses API** (`https://api.zerogpu.ai/v1/responses`) — and, for models served only there, the **Chat Completions API** (`https://api.zerogpu.ai/v1/chat/completions`) — that lets you call a curated set of edge-optimized models directly from your terminal for common NLP workloads: - Conversational chat (`LFM2.5-1.2B-Instruct`, `LFM2.5-1.2B-Thinking`) -- Reasoning and tool-use chat (`gpt-oss-120b`, `qwen3-30b-a3b-fp8`) +- Reasoning and tool-use chat (`gpt-oss-120b`, `qwen3-30b-a3b-fp8`, `glm-5.2`, `deepseek-v4-flash`) - IAB content/audience classification (`zlm-v1-iab-classify-edge`, `zlm-v2-iab-classify-edge-enriched`) - Domain-level IAB classification (`zlm-v1-iab-domain-classifier`) - Zero-shot classification (`deberta-v3-small`) @@ -209,8 +209,10 @@ zerogpu chat [-i ] [-m ] [-r] | `LFM2.5-1.2B-Thinking` | Responses | Compact reasoning model. | | `gpt-oss-120b` | Responses | 117B MoE, 131K context, reasoning + function calling. | | `qwen3-30b-a3b-fp8` | Chat Completions | 30.5B MoE, 100+ languages, reasoning + function calling. | +| `glm-5.2` | Chat Completions | 753B MoE, 1,048,576-token context, reasoning + function calling. The most capable model on the platform, and the most expensive by an order of magnitude. | +| `deepseek-v4-flash` | Chat Completions | 284B MoE (13B active), 1,048,576-token context, coding and agentic workflows. | -`qwen3-30b-a3b-fp8` has no Responses endpoint, so the CLI posts it to `/v1/chat/completions` instead, mapping `--instructions` to a `system` message and normalizing `prompt_tokens` / `completion_tokens` back to Responses token names for savings tracking. This routing is transparent — the command and its output are identical either way. +`qwen3-30b-a3b-fp8`, `glm-5.2`, and `deepseek-v4-flash` have no Responses endpoint, so the CLI posts them to `/v1/chat/completions` instead, mapping `--instructions` to a `system` message and normalizing `prompt_tokens` / `completion_tokens` back to Responses token names for savings tracking. This routing is transparent — the command and its output are identical either way. **Example** ```bash @@ -680,7 +682,7 @@ Content-Type: application/json x-api-key: ``` -The one exception is `chat --model qwen3-30b-a3b-fp8`, which the ZeroGPU platform serves only through the OpenAI-compatible Chat Completions endpoint: +The exceptions are `chat --model qwen3-30b-a3b-fp8`, `chat --model glm-5.2`, and `chat --model deepseek-v4-flash`, which the ZeroGPU platform serves only through the OpenAI-compatible Chat Completions endpoint: ``` POST https://api.zerogpu.ai/v1/chat/completions diff --git a/src/commands/chat.ts b/src/commands/chat.ts index 4e1df5b..de8f28f 100644 --- a/src/commands/chat.ts +++ b/src/commands/chat.ts @@ -17,13 +17,16 @@ import { recordAndMaybeNotify } from "../lib/savings.js"; const DEFAULT_MODEL = "LFM2.5-1.2B-Instruct"; // Text-generation models `--model` accepts, and the API each one speaks. -// qwen3-30b-a3b-fp8 is Chat Completions only — it has no Responses endpoint. +// qwen3-30b-a3b-fp8, glm-5.2, and deepseek-v4-flash are Chat Completions only — +// they have no Responses endpoint. // Source: https://docs.zerogpu.ai/docs/text-generation const CHAT_MODELS: Record = { "LFM2.5-1.2B-Instruct": "responses", "LFM2.5-1.2B-Thinking": "responses", "gpt-oss-120b": "responses", "qwen3-30b-a3b-fp8": "chat-completions", + "glm-5.2": "chat-completions", + "deepseek-v4-flash": "chat-completions", }; // Model ids are case-sensitive to the API but not to the person typing them. diff --git a/src/lib/savings.ts b/src/lib/savings.ts index 59899b1..d5cab4b 100644 --- a/src/lib/savings.ts +++ b/src/lib/savings.ts @@ -21,6 +21,8 @@ const DEFAULT_BASELINE = "claude-opus-4-8"; const ZGPU_PRICING: Record = { "gpt-oss-120b": { in: 0.03, out: 0.1 }, "qwen3-30b-a3b-fp8": { in: 0.05, out: 0.3 }, + "glm-5.2": { in: 1.1, out: 3.5 }, + "deepseek-v4-flash": { in: 0.07, out: 0.14 }, "llama-3.1-8b-instruct-fast": { in: 0.02, out: 0.05 }, "zlm-v1-iab-classify-edge": { in: 0.02, out: 0.05 }, "zlm-v2-iab-classify-edge-enriched": { in: 0.02, out: 0.05 }, @@ -33,8 +35,9 @@ const ZGPU_PRICING: Record = { "LFM2.5-1.2B-Instruct": { in: 0.02, out: 0.05 }, }; // Conservative fallback for any model id not in the table above: the priciest -// published rate, so an unlisted model never overstates savings. -const ZGPU_FALLBACK = { in: 0.05, out: 0.3 }; +// published rate, so an unlisted model never overstates savings. Currently +// glm-5.2, which is an order of magnitude dearer than the edge models. +const ZGPU_FALLBACK = { in: 1.1, out: 3.5 }; // Cadence: aim to show the savings note roughly once every 2–3 routed calls, // never twice in a row, and guaranteed within a bounded window. diff --git a/tests/savings.test.ts b/tests/savings.test.ts index ab855c6..bdab8f8 100644 --- a/tests/savings.test.ts +++ b/tests/savings.test.ts @@ -78,11 +78,28 @@ describe("computeCallSavings", () => { }); it("falls back to a default ZeroGPU rate for an unknown model", () => { - const known = computeCallSavings(740, 280, "claude-opus-4-8", "qwen3-30b-a3b-fp8").savingsUsd; + const known = computeCallSavings(740, 280, "claude-opus-4-8", "glm-5.2").savingsUsd; const unknown = computeCallSavings(740, 280, "claude-opus-4-8", "some-new-zgpu-model").savingsUsd; - // fallback equals the priciest published rate, qwen3's {0.05, 0.3} + // fallback equals the priciest published rate, glm-5.2's {1.10, 3.50} expect(unknown).toBeCloseTo(known, 10); }); + + it("never overstates savings for an unknown model", () => { + // The fallback must be at least as dear as every listed model, so an + // unlisted id can only ever under-report what was saved. + const unknown = computeCallSavings(740, 280, "claude-opus-4-8", "some-new-zgpu-model").savingsUsd; + for (const model of [ + "gpt-oss-120b", + "qwen3-30b-a3b-fp8", + "glm-5.2", + "deepseek-v4-flash", + "LFM2.5-1.2B-Instruct", + ]) { + expect(unknown).toBeLessThanOrEqual( + computeCallSavings(740, 280, "claude-opus-4-8", model).savingsUsd, + ); + } + }); }); describe("resolveBaselineModel", () => {