Skip to content

Add glm-5.2 and deepseek-v4-flash to chat #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
amaan-ai20 merged 1 commit into from
Jul 29, 2026
+43 −10
Merged

Add glm-5.2 and deepseek-v4-flash to chat #4

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,12 @@ zerogpu chat "Why does my API keep getting rate-limited?" -m gpt-oss-120b -r

# Multilingual reasoning
zerogpu chat "Explique la mise en cache en une phrase." -m qwen3-30b-a3b-fp8

# A 1M-token context, for whole repos and very long documents
zerogpu chat "$(cat ARCHITECTURE.md)" -m glm-5.2

# Coding and agentic work, at a fraction of the flagship price
zerogpu chat "Port this helper to async/await." -m deepseek-v4-flash
```

| Option | Description |
Expand All @@ -148,8 +154,10 @@ zerogpu chat "Explique la mise en cache en une phrase." -m qwen3-30b-a3b-fp8
| `LFM2.5-1.2B-Thinking` | Compact reasoning model. |
| `gpt-oss-120b` | 117B MoE, 131K context, reasoning + function calling. |
| `qwen3-30b-a3b-fp8` | 30.5B MoE, 100+ languages, reasoning + function calling. |
| `glm-5.2` | 753B MoE, 1M context, reasoning + function calling. The platform's most capable model, and its priciest. |
| `deepseek-v4-flash` | 284B MoE (13B active), 1M context, coding and agentic workflows. |

`qwen3-30b-a3b-fp8` is served by the Chat Completions API rather than the Responses API; the CLI routes it automatically.
`qwen3-30b-a3b-fp8`, `glm-5.2`, and `deepseek-v4-flash` are served by the Chat Completions API rather than the Responses API; the CLI routes them automatically.

#### `chat_thinking`

Expand Down
2 changes: 1 addition & 1 deletion docs/ADDING_COMMANDS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ This guide explains how to add a new CLI command to the ZeroGPU CLI.
- `src/commands/` — one file per command, each exporting a `register<Name>Command(program)` function.
- `src/cli.ts` — wires every command into the root program.
- `src/lib/responses.ts` — shared `RESPONSES_ENDPOINT`, `ResponsesApiResponse`, and the `extractOutputText` / `extractReasoningText` helpers for `/v1/responses` calls.
- `src/lib/chatCompletions.ts` — the same for `/v1/chat/completions`, used by models the platform serves only there (currently `qwen3-30b-a3b-fp8`), plus `toResponsesUsage` to normalize token counts for savings tracking.
- `src/lib/chatCompletions.ts` — the same for `/v1/chat/completions`, used by models the platform serves only there (currently `qwen3-30b-a3b-fp8`, `glm-5.2`, and `deepseek-v4-flash`), plus `toResponsesUsage` to normalize token counts for savings tracking.
- `src/lib/auth.ts` — `getApiKey()` for authenticated requests.

## Steps
Expand Down
8 changes: 5 additions & 3 deletions docs/DOCUMENTATION.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
`zerogpu-cli` is the official command-line interface for [ZeroGPU](https://zerogpu.ai), a distributed / edge inference platform for small language models (SLMs) and nano language models. The CLI is a thin, OpenAI-compatible client around the ZeroGPU **Responses API** (`https://api.zerogpu.ai/v1/responses`) — and, for models served only there, the **Chat Completions API** (`https://api.zerogpu.ai/v1/chat/completions`) — that lets you call a curated set of edge-optimized models directly from your terminal for common NLP workloads:

- Conversational chat (`LFM2.5-1.2B-Instruct`, `LFM2.5-1.2B-Thinking`)
- Reasoning and tool-use chat (`gpt-oss-120b`, `qwen3-30b-a3b-fp8`)
- Reasoning and tool-use chat (`gpt-oss-120b`, `qwen3-30b-a3b-fp8`, `glm-5.2`, `deepseek-v4-flash`)
- IAB content/audience classification (`zlm-v1-iab-classify-edge`, `zlm-v2-iab-classify-edge-enriched`)
- Domain-level IAB classification (`zlm-v1-iab-domain-classifier`)
- Zero-shot classification (`deberta-v3-small`)
Expand Down Expand Up @@ -209,8 +209,10 @@ zerogpu chat <text> [-i <instructions>] [-m <model>] [-r]
| `LFM2.5-1.2B-Thinking` | Responses | Compact reasoning model. |
| `gpt-oss-120b` | Responses | 117B MoE, 131K context, reasoning + function calling. |
| `qwen3-30b-a3b-fp8` | Chat Completions | 30.5B MoE, 100+ languages, reasoning + function calling. |
| `glm-5.2` | Chat Completions | 753B MoE, 1,048,576-token context, reasoning + function calling. The most capable model on the platform, and the most expensive by an order of magnitude. |
| `deepseek-v4-flash` | Chat Completions | 284B MoE (13B active), 1,048,576-token context, coding and agentic workflows. |

`qwen3-30b-a3b-fp8` has no Responses endpoint, so the CLI posts it to `/v1/chat/completions` instead, mapping `--instructions` to a `system` message and normalizing `prompt_tokens` / `completion_tokens` back to Responses token names for savings tracking. This routing is transparent — the command and its output are identical either way.
`qwen3-30b-a3b-fp8`, `glm-5.2`, and `deepseek-v4-flash` have no Responses endpoint, so the CLI posts them to `/v1/chat/completions` instead, mapping `--instructions` to a `system` message and normalizing `prompt_tokens` / `completion_tokens` back to Responses token names for savings tracking. This routing is transparent — the command and its output are identical either way.

**Example**
```bash
Expand Down Expand Up @@ -680,7 +682,7 @@ Content-Type: application/json
x-api-key: <ZEROGPU_API_KEY>
```

The one exception is `chat --model qwen3-30b-a3b-fp8`, which the ZeroGPU platform serves only through the OpenAI-compatible Chat Completions endpoint:
The exceptions are `chat --model qwen3-30b-a3b-fp8`, `chat --model glm-5.2`, and `chat --model deepseek-v4-flash`, which the ZeroGPU platform serves only through the OpenAI-compatible Chat Completions endpoint:

```
POST https://api.zerogpu.ai/v1/chat/completions
Expand Down
5 changes: 4 additions & 1 deletion src/commands/chat.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,16 @@ import { recordAndMaybeNotify } from "../lib/savings.js";
const DEFAULT_MODEL = "LFM2.5-1.2B-Instruct";

// Text-generation models `--model` accepts, and the API each one speaks.
// qwen3-30b-a3b-fp8 is Chat Completions only — it has no Responses endpoint.
// qwen3-30b-a3b-fp8, glm-5.2, and deepseek-v4-flash are Chat Completions only —
// they have no Responses endpoint.
// Source: https://docs.zerogpu.ai/docs/text-generation
const CHAT_MODELS: Record<string, "responses" | "chat-completions"> = {
"LFM2.5-1.2B-Instruct": "responses",
"LFM2.5-1.2B-Thinking": "responses",
"gpt-oss-120b": "responses",
"qwen3-30b-a3b-fp8": "chat-completions",
"glm-5.2": "chat-completions",
"deepseek-v4-flash": "chat-completions",
Comment on lines 23 to +29

@coderabbitai coderabbitai Bot Jul 29, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== repo files of interest =="
git ls-files | rg '(^src/commands/chat\.ts$|^README\.md$|^docs/(DOCUMENTATION|ADDING_COMMANDS)\.md$)' || true

echo
echo "== src/commands/chat.ts context =="
if [ -f src/commands/chat.ts ]; then
  sed -n '1,80p' src/commands/chat.ts | cat -n
fi

echo
echo "== occurrences of deepseek-v4-flash =="
rg -n "deepseek-v4-flash|qwen3-30b-a3b-fp8|glm-5\.2|chat-completions|responses" README.md docs src/commands/chat.ts 2>/dev/null || true

echo
echo "== local references to ZEROGPU/_CHAT_MODELS =="
rg -n "CHAT_MODELS|model.*chat|chatCompletions|toResponsesUsage" src README.md docs 2>/dev/null || true

Repository: zerogpu/cli

Length of output: 10132

🌐 Web query:

ZeroGPU chat Completions model deepseek-v4-flash API model_not_found

💡 Result:

The error model_not_found when using deepseek-v4-flash is likely occurring because the legacy API aliases deepseek-chat and deepseek-reasoner were officially retired on July 24, 2026 [1][2][3]. If your application or SDK is still configured to use the retired aliases, or if it is attempting to access a model ID that is not supported by your specific provider, you will receive this error [2][4]. To resolve this: 1. Update your code: Ensure you are explicitly using the model ID deepseek-v4-flash (or deepseek-v4-pro) in your API request, rather than any older identifiers [5][6][7]. 2. Verify your provider: If you are using a third-party API provider (e.g., OpenRouter, Together AI, etc.) instead of the official DeepSeek API, check their documentation for the correct model slug [4][8]. Third-party providers often require their own naming conventions, such as deepseek/deepseek-v4-flash or deepseek-ai/DeepSeek-V4-Flash [4][8]. 3. Check Base URL: Ensure your base_url is set correctly for your provider [7][4]. ZeroGPU (an infrastructure company) is distinct from the official DeepSeek API platform [9][10][7]. If you are using services via ZeroGPU or a similar third-party platform, verify that they currently support the DeepSeek V4 model family, as model availability can vary between providers [4].

Citations:

🌐 Web query:

"api.zerogpu.ai/v1/chat/completions" "deepseek-v4-flash"

💡 Result:

The endpoint api.zerogpu.ai/v1/chat/completions is a valid REST API endpoint provided by the ZeroGPU platform for model inference [1][2]. Regarding your query for deepseek-v4-flash: 1. ZeroGPU provides a generic API structure for chat completions [2]. You can access various models through this endpoint by specifying the model name in your request body [1][3]. 2. DeepSeek-V4-Flash is a real, high-performance Mixture-of-Experts (MoE) language model released by DeepSeek-AI in April 2026 [4][5][6]. It features 284 billion total parameters (13 billion activated) and supports a 1 million token context window [4][5][6]. 3. While DeepSeek-V4-Flash is available through multiple inference providers [7], you should verify if it is currently listed in the specific ZeroGPU model catalog via their documentation or dashboard [1][3]. If supported, you would typically integrate it by setting the model parameter to "deepseek-v4-flash" (or the provider-specific canonical ID) within the JSON payload of a POST request to the aforementioned endpoint, while ensuring the required x-api-key and x-project-id headers are included [3][8][9]. For exact usage, refer to the official ZeroGPU API documentation at https://docs.zerogpu.ai [1][3].

Citations:

🌐 Web query:

"model_not_found" "deepseek-v4-flash" ZeroGPU

💡 Result:

The error model_not_found related to deepseek-v4-flash and ZeroGPU typically arises from two distinct issues: the recent retirement of legacy DeepSeek model aliases or incorrect model identifier formatting in your API request [1][2][3]. 1. Legacy Model Retirement: As of July 24, 2026, the legacy model aliases deepseek-chat and deepseek-reasoner have been fully retired [4][2][5]. If your application was using these identifiers, it will now return a model_not_found error [1][4][2]. You must update your configuration to use the explicit model IDs: deepseek-v4-flash (for non-thinking/standard workflows) or deepseek-v4-pro (for advanced reasoning/agentic tasks) [4][2][5]. 2. API Formatting and Provider Routing: If you are using an API aggregator (such as OpenRouter) or a specific integration library (like openclaw or similar), the model_not_found error often occurs due to incorrect prefixing or double-encoding of the model ID [6]. For example, if your code incorrectly sends a request formatted as openrouter/openrouter/deepseek-v4-flash, the gateway may fail to resolve the model name [6]. Ensure you are using the canonical model ID format required by your specific provider [6][7]. 3. ZeroGPU Specific Context: If you are encountering this within a Hugging Face Space using ZeroGPU, be aware that the error might be a generic wrapper response [8][9]. ZeroGPU environments often use custom decorators (like @spaces.GPU) that can swallow detailed error messages [9]. If the model is correctly referenced but fails to load, ensure you are not hitting environmental constraints (e.g., storage limits or transient worker failures) and verify the model ID is explicitly supported by the API client you are using inside the GPU-decorated function [8][3][10]. To resolve: - Update all references from deepseek-chat or deepseek-reasoner to deepseek-v4-flash [1][4][2]. - Audit your API call configuration to ensure no redundant provider prefixes (e.g., openrouter/openrouter/) are being added [6]. - If using ZeroGPU, check your application logs specifically for storage or initialization errors that might be masked by the generic model_not_found message [8][10].

Citations:

🌐 Web query:

site:docs.zerogpu.ai "deepseek-v4-flash"

💡 Result:

As of July 29, 2026, deepseek-v4-flash is a model available through the official DeepSeek API [1][2]. It is listed alongside deepseek-v4-pro in the DeepSeek API documentation, which utilizes an API format compatible with OpenAI and Anthropic standards [1][2]. Users can access this model by configuring their requests to point to the DeepSeek API base URL [1][2].

Citations:

Delay exposing deepseek-v4-flash until the ZeroGPU Chat Completions endpoint supports it. The CLI allowlist makes model --model deepseek-v4-flash selectable, but the current ZeroGPU Chat Completions request returns 404 model_not_found, while all docs present it as live.

  • src/commands/chat.ts#L23-L29: remove the model or gate it behind availability.
  • README.md#L142, README.md#L158-L160, docs/DOCUMENTATION.md#L8, docs/DOCUMENTATION.md#L212-L215, docs/DOCUMENTATION.md#L685, and docs/ADDING_COMMANDS.md#L10: remove or clearly mark these unavailable or gated references.
📍 Affects 4 files
  • src/commands/chat.ts#L23-L29 (this comment)
  • README.md#L141-L142
  • README.md#L157-L160
  • docs/DOCUMENTATION.md#L8-L8
  • docs/DOCUMENTATION.md#L212-L215
  • docs/DOCUMENTATION.md#L685-L685
  • docs/ADDING_COMMANDS.md#L10-L10
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/commands/chat.ts` around lines 23 - 29, Delay exposure of
deepseek-v4-flash until the ZeroGPU Chat Completions endpoint supports it:
remove or availability-gate that entry in CHAT_MODELS, and update README.md
(141-142, 157-160), docs/DOCUMENTATION.md (8, 212-215, 685), and
docs/ADDING_COMMANDS.md (10) to remove the model or clearly mark it
unavailable/gated.

};

// Model ids are case-sensitive to the API but not to the person typing them.
Expand Down
7 changes: 5 additions & 2 deletions src/lib/savings.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ const DEFAULT_BASELINE = "claude-opus-4-8";
const ZGPU_PRICING: Record<string, { in: number; out: number }> = {
"gpt-oss-120b": { in: 0.03, out: 0.1 },
"qwen3-30b-a3b-fp8": { in: 0.05, out: 0.3 },
"glm-5.2": { in: 1.1, out: 3.5 },
"deepseek-v4-flash": { in: 0.07, out: 0.14 },
"llama-3.1-8b-instruct-fast": { in: 0.02, out: 0.05 },
"zlm-v1-iab-classify-edge": { in: 0.02, out: 0.05 },
"zlm-v2-iab-classify-edge-enriched": { in: 0.02, out: 0.05 },
Expand All @@ -33,8 +35,9 @@ const ZGPU_PRICING: Record<string, { in: number; out: number }> = {
"LFM2.5-1.2B-Instruct": { in: 0.02, out: 0.05 },
};
// Conservative fallback for any model id not in the table above: the priciest
// published rate, so an unlisted model never overstates savings.
const ZGPU_FALLBACK = { in: 0.05, out: 0.3 };
// published rate, so an unlisted model never overstates savings. Currently
// glm-5.2, which is an order of magnitude dearer than the edge models.
const ZGPU_FALLBACK = { in: 1.1, out: 3.5 };

// Cadence: aim to show the savings note roughly once every 2–3 routed calls,
// never twice in a row, and guaranteed within a bounded window.
Expand Down
21 changes: 19 additions & 2 deletions tests/savings.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,11 +78,28 @@ describe("computeCallSavings", () => {
});

it("falls back to a default ZeroGPU rate for an unknown model", () => {
const known = computeCallSavings(740, 280, "claude-opus-4-8", "qwen3-30b-a3b-fp8").savingsUsd;
const known = computeCallSavings(740, 280, "claude-opus-4-8", "glm-5.2").savingsUsd;
const unknown = computeCallSavings(740, 280, "claude-opus-4-8", "some-new-zgpu-model").savingsUsd;
// fallback equals the priciest published rate, qwen3's {0.05, 0.3}
// fallback equals the priciest published rate, glm-5.2's {1.10, 3.50}
expect(unknown).toBeCloseTo(known, 10);
});

it("never overstates savings for an unknown model", () => {
// The fallback must be at least as dear as every listed model, so an
// unlisted id can only ever under-report what was saved.
const unknown = computeCallSavings(740, 280, "claude-opus-4-8", "some-new-zgpu-model").savingsUsd;
for (const model of [
"gpt-oss-120b",
"qwen3-30b-a3b-fp8",
"glm-5.2",
"deepseek-v4-flash",
"LFM2.5-1.2B-Instruct",
]) {
expect(unknown).toBeLessThanOrEqual(
computeCallSavings(740, 280, "claude-opus-4-8", model).savingsUsd,
);
}
});
Comment on lines +87 to +102

@coderabbitai coderabbitai Bot Jul 29, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Check every priced model, not a hand-picked subset.

The test claims the fallback never overstates savings for every listed model, but it only enumerates five IDs. Iterate over all ZGPU_PRICING keys, or a shared exported model list, so newly added or omitted high-priced models cannot bypass this invariant.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/savings.test.ts` around lines 87 - 102, Update the test around
computeCallSavings to iterate over every model identifier in ZGPU_PRICING, or
the shared exported model list, instead of the hard-coded five-model array.
Preserve the existing unknown-model savings comparison and ensure newly added
priced models are included automatically.

});

describe("resolveBaselineModel", () => {
Expand Down