Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/cloud/guides/integrations/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ Use the OpenAI client guide when you are configuring code directly. Use the inte
- [Continue](/cloud/guides/integrations/continue)
- [Cline, Roo Code, and Kilo Code](/cloud/guides/integrations/cline-roo-kilo)
- [Aider and Zed](/cloud/guides/integrations/aider-zed)
- [OpenClaw](/cloud/guides/integrations/openclaw)
- [OpenCode and Goose](/cloud/guides/opencode-goose)

### Self-hosted and team apps
Expand Down
9 changes: 7 additions & 2 deletions docs/cloud/guides/integrations/model-discovery.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -42,13 +42,18 @@ The response uses OpenAI-style model objects plus NEAR AI Cloud metadata. Import
| `id` | The model ID to use with the gateway. For GLM 5.2, use `z-ai/glm-5.2`. |
| `name` | Human-readable display name. |
| `context_length` | Maximum input context length reported for the model. |
| `max_output_length` | Maximum output length reported by NEAR AI Cloud. |
| `max_output_length` | OpenRouter-compatible advisory hint curated in the NEAR AI Cloud catalog. It is not enforced: the gateway forwards `max_tokens` unchanged and does not clamp it. For a NEAR-hosted model, the binding limit is the model context window (`max_model_len` at its direct completions endpoint), shared between prompt and completion. |
| `top_provider.context_length` | Provider context length used by clients that read OpenRouter-style metadata. |
| `top_provider.max_completion_tokens` | Provider maximum completion tokens used by clients that read OpenRouter-style metadata. |
| `top_provider.max_completion_tokens` | Mirrors `max_output_length` for clients that read OpenRouter-style metadata and carries the same advisory, non-enforced caveat. |
| `is_ready` | OpenRouter catalog publication flag, exposed verbatim. `false` keeps the model hidden on OpenRouter's side; `true` enables auto-staging there. It does not indicate NEAR AI Cloud availability, may be absent, and should be ignored when deciding whether a model is usable. If a model is listed in `/v1/models`, it is available. |
| `supported_features` | Feature flags such as `tools`, `structured_outputs`, `reasoning`, and `json_mode`. |
| `supported_sampling_parameters` | Supported request parameters such as `temperature`, `top_p`, `max_tokens`, and `stop`. |
| `input_modalities` and `output_modalities` | Supported input and output types. |

### Fields that do not mean what their names suggest

As checked on 2026-08-18, the gateway reports `max_output_length: 8192` for `deepseek-ai/DeepSeek-V4-Flash`, while `https://dsv4-flash.completions.near.ai/v1/models` reports `max_model_len: 1048576`; the gateway field is advisory metadata rather than an enforced output cap. Every `openai/*`, `anthropic/*`, and `google/*` entry reports `is_ready: false` while remaining fully serviceable, because `is_ready` describes OpenRouter publication rather than NEAR AI Cloud availability.

As checked on 2026-06-23, the live GLM 5.2 gateway record reports `top_provider.context_length: 500000`, `top_provider.max_completion_tokens: 131072`, and `max_output_length: 131072`.

## Discover direct completions endpoints
Expand Down
154 changes: 154 additions & 0 deletions docs/cloud/guides/integrations/openclaw.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
---
id: openclaw
title: OpenClaw
sidebar_label: OpenClaw
slug: /cloud/guides/integrations/openclaw
description: "Configure OpenClaw to use NEAR AI Cloud as an OpenAI-compatible provider."
---

# OpenClaw

## Overview

OpenClaw talks to NEAR AI Cloud as an OpenAI-compatible provider. NEAR AI also runs OpenClaw as a hosted worker, so this provider configuration is the same one NEAR uses.

## Prerequisites

- A NEAR AI Cloud API key.
- OpenClaw installed. This configuration was verified with OpenClaw 2026.7.12.

## Base URL

Use the NEAR AI Cloud gateway base URL:

```text
https://cloud-api.near.ai/v1
```

Do not append `/chat/completions` to the base URL.

## Model ID

Add each model to the provider's `models` array using its gateway model ID. This guide uses:

```text
z-ai/glm-5.2
```

See [Model Discovery](/cloud/guides/integrations/model-discovery) for the current model list.

## Configure

Add the NEAR AI Cloud provider to `~/.openclaw/openclaw.json` and wire it as the default model:

```json
{
"models": {
"mode": "merge",
"providers": {
"nearai": {
"baseUrl": "https://cloud-api.near.ai/v1",
"apiKey": "${NEARAI_API_KEY}",
"auth": "api-key",
"api": "openai-completions",
"models": [{ "id": "z-ai/glm-5.2", "name": "GLM 5.2" }]
}
}
},
"agents": {
"defaults": {
"model": {
"primary": "nearai/z-ai/glm-5.2"
}
}
}
}
```

OpenClaw applies conservative defaults to an unrecognized provider. NEAR AI Cloud supports all three compatibility flags, so you can enable them explicitly:

```json
{
"models": {
"mode": "merge",
"providers": {
"nearai": {
"baseUrl": "https://cloud-api.near.ai/v1",
"apiKey": "${NEARAI_API_KEY}",
"auth": "api-key",
"api": "openai-completions",
"models": [{ "id": "z-ai/glm-5.2", "name": "GLM 5.2" }],
"compat": {
"supportsUsageInStreaming": true,
"supportsStrictMode": true,
"supportsDeveloperRole": true
}
}
}
},
"agents": {
"defaults": {
"model": {
"primary": "nearai/z-ai/glm-5.2"
}
}
}
}
```

| Flag | Why it is safe to enable |
| --- | --- |
| `supportsUsageInStreaming` | NEAR AI Cloud handles `stream_options: {"include_usage": true}` by setting intermediate chunks to `usage: null` and appending one final usage chunk. |
| `supportsDeveloperRole` | NEAR AI Cloud accepts `developer` alongside `system`, `user`, `assistant`, and `tool` roles. |
| `supportsStrictMode` | NEAR AI Cloud accepts and forwards strict JSON-schema response formats and strict tool definitions. Proxied third-party passthrough providers may ignore strict mode; strict schema enforcement is guaranteed only on NEAR-hosted models whose `supported_features` includes `structured_outputs`. |

## Refresh models

OpenClaw reads the provider's `models` array from `openclaw.json`; it does not auto-fetch `/v1/models`. Add new models to the array by hand. Use [Model Discovery](/cloud/guides/integrations/model-discovery) for the current list.

## Quick test

First, verify the gateway independently of OpenClaw:

```bash
curl https://cloud-api.near.ai/v1/chat/completions \
-H "Authorization: Bearer $NEARAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "z-ai/glm-5.2",
"messages": [
{
"role": "user",
"content": "Reply with one sentence confirming the model is reachable."
}
],
"max_tokens": 64
}'
```

Then start an OpenClaw chat and confirm its default model is `nearai/z-ai/glm-5.2`.

## Troubleshooting

| Symptom | Fix |
| --- | --- |
| Model not listed | Add the model ID from [Model Discovery](/cloud/guides/integrations/model-discovery) to the provider's `models` array by hand. |
| `401` | Set `NEARAI_API_KEY` to a NEAR AI Cloud API key and keep `apiKey` as `${NEARAI_API_KEY}`. |
| Base URL includes `/chat/completions` | Set `baseUrl` to `https://cloud-api.near.ai/v1`. Use `/chat/completions` only in a full request URL. |
| Streaming usage is missing | Set `supportsUsageInStreaming` to `true` in the `compat` block. |
| A `developer`-role message is rejected by a non-NEAR provider in the same config | NEAR AI Cloud accepts the `developer` role. Check the compatibility behavior of the non-NEAR provider that handled the message. |

## Related guides

- [OpenAI Compatibility](/cloud/guides/openai-compatibility)
- [Model Discovery](/cloud/guides/integrations/model-discovery)
- [Chat Verification](/cloud/verification/chat)
- [Available Models](/cloud/models)

## Sources Checked

Sources checked on 2026-08-18:

- [`GET https://cloud-api.near.ai/v1/models`](https://cloud-api.near.ai/v1/models)
- `openclaw-nearai-worker/worker/openclaw.json.template`
- OpenClaw 2026.7.12
46 changes: 46 additions & 0 deletions docs/cloud/guides/openai-compatibility.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,52 @@ NEAR AI Cloud also provides non-OpenAI extensions:
- **Privacy redaction** (`/v1/privacy/redact`) - Gateway-only endpoint for redacting personally identifiable information
- **Attestation & Signatures** (`/v1/attestation/report`, `/v1/signature/{chat_id}`) - See [Verification](/cloud/verification)

## Compatibility Matrix

| OpenAI request feature | NEAR AI Cloud support | `/v1/models` advertisement |
| --- | --- | --- |
| Tool / function calling (`tools`) | Supported on tool-capable models. | `supported_features: tools` |
| `tool_choice` | Supported on tool-capable models. | `supported_features: tools` |
| Structured outputs (`response_format: {"type": "json_schema"}`), including `strict: true` | Accepted and forwarded. Strict schema enforcement is guaranteed only on NEAR-hosted models that advertise structured outputs; proxied third-party passthrough providers may ignore strict mode. | `supported_features: structured_outputs` |
| JSON mode (`response_format: {"type": "json_object"}`) | Supported on models that advertise JSON mode. | `supported_features: json_mode` |
| Reasoning | Supported on models that advertise reasoning. | `supported_features: reasoning` |
| `developer` role messages | Accepted alongside `system`, `user`, `assistant`, and `tool` roles. | No dedicated flag. |
| `stream_options.include_usage` | Supported. Intermediate chunks carry `usage: null`, followed by one final usage chunk. | No dedicated flag. |
| Prompt caching | Supported on TEE-hosted models; a cache miss may return `prompt_tokens_details: null`. | No dedicated flag. |
| `max_tokens` | Forwarded unchanged to the provider; the gateway does not clamp it to `max_output_length`. | `supported_sampling_parameters: max_tokens` |

Client authors can discover per-model capability from `supported_features` and `supported_sampling_parameters` in [`GET /v1/models`](https://cloud-api.near.ai/v1/models). Clients that cannot consume those fields can safely enable usage-in-streaming, the `developer` role, and strict mode for NEAR AI Cloud, subject to the strict-mode caveat for proxied third-party passthrough providers above.

### Known deviations from the OpenAI schema

On a cache miss, TEE-hosted models return `"prompt_tokens_details": null` rather than OpenAI's `{"cached_tokens": 0}` object. The inference engine emits `null`, and the gateway returns the provider's exact response bytes so the model-TEE signature verifies byte-for-byte. Normalizing the field at the gateway would break [Chat Verification](/cloud/verification/chat). Treat a missing details object as zero cached tokens:

<Tabs
defaultValue="python"
values={[
{label: 'Python', value: 'python'},
{label: 'JavaScript', value: 'javascript'},
]}>
<TabItem value="python">

```python
cached = (usage.get("prompt_tokens_details") or {}).get("cached_tokens", 0)
```

</TabItem>
<TabItem value="javascript">

```javascript
const cached = usage.prompt_tokens_details?.cached_tokens ?? 0;
```

</TabItem>
</Tabs>

See [Prompt Caching](/cloud/guides/prompt-caching#cache-misses-and-null-details) for the cache-miss behavior.

The `max_output_length` field in `/v1/models` is advisory catalog metadata, not an enforced cap. The `is_ready` field is an OpenRouter catalog publication flag and must not be read as NEAR AI Cloud model availability. See the [Model Discovery field table](/cloud/guides/integrations/model-discovery#discover-gateway-models) for both fields.

### Files API Example

You can use the Files API to upload documents for use with supported models.
Expand Down
35 changes: 34 additions & 1 deletion docs/cloud/guides/prompt-caching.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ slug: /cloud/guides/prompt-caching
description: "How NEAR AI Cloud caches prompt prefixes to reduce latency and cost"
---

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

# Prompt Caching

NEAR AI Cloud automatically caches prompt prefixes on TEE-hosted models. When consecutive requests share a common prefix — a long system prompt, conversation history, few-shot examples, or a large document — the cached portion is reused instead of being recomputed, which lowers both latency and cost.
Expand Down Expand Up @@ -51,7 +54,37 @@ The number of input tokens served from cache is reported in the `usage` object o
}
```

On the first request with a given prefix, `cached_tokens` is `0` (or `prompt_tokens_details` is `null`). Repeat the request — or send another request sharing the same prefix — and `cached_tokens` reflects the reused portion.
### Cache misses and null details

On a cache miss, TEE-hosted models return `"prompt_tokens_details": null`, not `{"cached_tokens": 0}` as OpenAI does. The inference engine emits `null`, and the gateway returns the provider's exact response bytes so the model-TEE signature verifies byte-for-byte. Normalizing the field at the gateway would break [Chat Verification](/cloud/verification/chat).

Treat a missing details object as zero cached tokens so client code never indexes into `null`:

<Tabs
defaultValue="python"
values={[
{label: 'Python', value: 'python'},
{label: 'JavaScript', value: 'javascript'},
]}>
<TabItem value="python">

```python
cached = (usage.get("prompt_tokens_details") or {}).get("cached_tokens", 0)
```

</TabItem>
<TabItem value="javascript">

```javascript
const cached = usage.prompt_tokens_details?.cached_tokens ?? 0;
```

</TabItem>
</Tabs>

See [Known deviations from the OpenAI schema](/cloud/guides/openai-compatibility#known-deviations-from-the-openai-schema) for the corresponding OpenAI Compatibility guidance.

Repeat the request, or send another request with the same prefix, and `cached_tokens` will reflect the reused portion.

## Getting the Most Out of the Cache

Expand Down
82 changes: 81 additions & 1 deletion docs/cloud/verification/chat-verification.md
Original file line number Diff line number Diff line change
Expand Up @@ -226,7 +226,7 @@ curl -X GET 'https://qwen35-122b.completions.near.ai/v1/signature/afa7975eaf844b
```

:::note
A model can be served by multiple TEE nodes behind the same domain. The signature is cached on the node that served your chat completion, so a lookup may transiently return `Chat id not found or expired` if it lands on a different node — simply retry until you hit the right one.
For multi-replica models, use [Deterministic Signature Retrieval](#deterministic-signature-retrieval) instead of blind retries.
:::

***Example Response:***
Expand Down Expand Up @@ -260,6 +260,86 @@ This exactly matches the model we requested and the values we calculated in the

---

## Deterministic Signature Retrieval

The direct signature endpoint can return `404 Chat id not found or expired` because the signature is cached in memory inside the model TEE process that served the completion. The model domain is an L4 SNI load balancer with least-connections selection and no session affinity, so the lookup can land on a replica that never saw the completion. `dsv4-flash.completions.near.ai` had 5 healthy backends on 2026-08-18: blind retrying is a coin flip rather than a deterministic strategy, and a retry succeeds roughly one time in five.

### Option 1 — Gateway (recommended)

Send the completion through `https://cloud-api.near.ai/v1`, then fetch its signature from:

```text
GET https://cloud-api.near.ai/v1/signature/{chat_id}
```

The gateway pins the chat ID to the backend that served it and stores the signature durably at completion time, so the lookup is deterministic and needs no retries. The stored signature remains `signature_kind: provider_tee` (signed by the model TEE) unless the gateway rewrote the stream; see [Signature Kinds](#signature-kinds).

### Option 2 — Pin the replica

Address one backend directly with `https://{slug}-i{N}.completions.near.ai`, which routes to backend `N % healthy_count`. Send both the completion and signature lookup to the same hostname for a single deterministic lookup against that model TEE:

```bash
CHAT_ID=$(curl -fsS https://dsv4-flash-i0.completions.near.ai/v1/chat/completions \
-H "Authorization: Bearer $NEARAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "deepseek-ai/DeepSeek-V4-Flash",
"messages": [{"role": "user", "content": "Reply with one sentence."}]
}' | jq -r '.id')

curl -fsS "https://dsv4-flash-i0.completions.near.ai/v1/signature/${CHAT_ID}" \
-H "Authorization: Bearer $NEARAI_API_KEY"
```

### Option 3 — Bounded sweep

If you already hold a `chat_id` returned through the load-balanced domain, read the public backend count:

```bash
curl 'https://completions.near.ai/backends/count?domain=dsv4-flash.completions.near.ai'
```

The response has this shape:

```json
{
"domain": "dsv4-flash.completions.near.ai",
"requested_domain": "dsv4-flash.completions.near.ai",
"healthy": 5,
"total": 5
}
```

Iterate from `0` through `healthy - 1`; exactly one replica holds the signature. This loop stops on the first HTTP 200:

```bash
HEALTHY=$(curl -fsS \
'https://completions.near.ai/backends/count?domain=dsv4-flash.completions.near.ai' \
| jq -r '.healthy')

for ((i = 0; i < HEALTHY; i++)); do
if curl -fsS \
"https://dsv4-flash-i${i}.completions.near.ai/v1/signature/${CHAT_ID}" \
-H "Authorization: Bearer $NEARAI_API_KEY" \
-o signature.json; then
cat signature.json
break
fi
done
```

:::caution
The `-i{N}` index is positional. Its index-to-backend binding is stable only while the healthy backend count is stable, and it can shift when backends are added, removed, or flap health. Re-read `/backends/count` rather than caching an index across a long-running session, and prefer Option 1 for unattended or at-scale verification.
:::

| Option | Where the completion must be sent | Determinism | When to use |
| --- | --- | --- | --- |
| Gateway (recommended) | `https://cloud-api.near.ai/v1` | Deterministic lookup with no retries. | Unattended or at-scale verification. |
| Pin the replica | The same `https://{slug}-i{N}.completions.near.ai` hostname used for lookup. | Deterministic while the index-to-backend binding remains stable. | Direct model-TEE requests when you control both calls. |
| Bounded sweep | The load-balanced `https://{slug}.completions.near.ai` domain. | Bounded search across `0..healthy-1`. | A `chat_id` already returned through the load-balanced domain. |

---

## Signature Kinds

Signatures fetched from the gateway (`GET https://cloud-api.near.ai/v1/signature/{chat_id}`) carry a `signature_kind` field that tells you which key signed and what the `text` payload contains:
Expand Down
1 change: 1 addition & 0 deletions sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,7 @@ const sidebars = {
"cloud/guides/integrations/continue",
"cloud/guides/integrations/cline-roo-kilo",
"cloud/guides/integrations/aider-zed",
"cloud/guides/integrations/openclaw",
"cloud/guides/opencode-goose",
],
},
Expand Down