Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions integrations/AnythingLLM/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@ configuration — no file to install, so this folder is documentation only.

1. Start the openzoo proxy: `npx openzoo`.
2. In AnythingLLM: **Settings → AI Providers → LLM** → choose **Generic OpenAI**.
3. **Base URL**: `http://localhost:8402/v1` (hosted: `https://api.openzoo.fun/v1`)
3. **Base URL**: `http://localhost:8402/v1` (hosted: `https://x402-tokens.fly.dev/v1`)
4. **API Key**: `sk-openzoo` (any value for local; real key for hosted)
5. **Chat Model Name**: a zoo model id, e.g. `nvidia/nemotron-3.5-lightning`
(`npx openzoo models` lists ids and pricing)
(`curl localhost:8402/v1/models` lists ids and pricing)
6. Set token context window / max tokens as desired, save, and chat.

## Notes
Expand Down
4 changes: 2 additions & 2 deletions integrations/Cline/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,10 @@ install, so this folder is documentation only.
2. In Cline, click the ⚙️ settings icon.
3. **API Provider**: select `OpenAI Compatible`.
4. **Base URL**: `http://localhost:8402/v1`
(hosted: `https://api.openzoo.fun/v1`) — do NOT paste `/chat/completions`.
(hosted: `https://x402-tokens.fly.dev/v1`) — do NOT paste `/chat/completions`.
5. **API Key**: `sk-openzoo` (any value for local; real key for hosted).
6. **Model ID**: a zoo model id, e.g. `nvidia/nemotron-3.5-lightning`
— must exactly match an id returned by `/v1/models` (`npx openzoo models`).
— must exactly match an id returned by `/v1/models` (`curl localhost:8402/v1/models`).
7. Fill the **Model Configuration** block (context window, output limits,
whether the model supports images/computer-use). People skip this; for
custom endpoints it matters — set the context window generously, since the
Expand Down
2 changes: 1 addition & 1 deletion integrations/Continue/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ OpenAI-compatible endpoint via its built-in `provider: openai` — no plugin req
## Notes

- Continue does not fetch a live model list; each zoo model you want is one entry.
`npx openzoo models` prints available model ids and pricing.
`curl localhost:8402/v1/models` prints available model ids and pricing.
- `apiKey` is ignored by the local proxy (x402 pays); required-but-arbitrary. For a
hosted endpoint, use your real key.
- Coding sessions can push large repo context; the zoo's leCore corpus handling is
Expand Down
4 changes: 2 additions & 2 deletions integrations/Continue/config.openzoo.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
# Continue's native `provider: openai` speaks to any OpenAI-compatible
# apiBase - no plugin needed. Each model you want available in Continue
# gets its own entry (Continue has no live model-list fetch), so list the
# zoo models you actually use. `npx openzoo models` prints the full list.
# zoo models you actually use. `curl localhost:8402/v1/models` prints the full list.

name: openzoo
version: 0.0.1
Expand All @@ -17,7 +17,7 @@ models:
- name: zoo nvidia/nemotron-3.5-lightning
provider: openai
model: nvidia/nemotron-3.5-lightning
apiBase: http://localhost:8402/v1 # hosted: https://api.openzoo.fun/v1
apiBase: http://localhost:8402/v1 # hosted: https://x402-tokens.fly.dev/v1
apiKey: sk-openzoo # any value for local proxy; real key for hosted
roles:
- chat
Expand Down
6 changes: 3 additions & 3 deletions integrations/Cursor/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,12 @@ so this folder is documentation only.
3. Disable pre-enabled Cursor-hosted models to avoid routing conflicts.
4. In the OpenAI section, enable **Override OpenAI Base URL** and enter:
`http://localhost:8402/v1`
(hosted: `https://api.openzoo.fun/v1`) — include the `/v1` suffix; Cursor
(hosted: `https://x402-tokens.fly.dev/v1`) — include the `/v1` suffix; Cursor
appends `/chat/completions` itself.
5. **OpenAI API Key**: `sk-openzoo` (any value for local; real key for hosted).
Despite the "OpenAI" label, this key is sent to the overridden endpoint.
6. Click **+ Add Model** and type a zoo model id, e.g. `nvidia/nemotron-3.5-lightning`
(`npx openzoo models` lists ids). Verify/Save — Cursor sends a test request.
(`curl localhost:8402/v1/models` lists ids). Verify/Save — Cursor sends a test request.
7. Pick the zoo model in the chat sidebar (`Cmd/Ctrl+L`).

## Known limitations (Cursor-side, not openzoo-side)
Expand All @@ -36,7 +36,7 @@ so this folder is documentation only.
## Full platform access: add the MCP server too

The chat override above covers ordinary completions. openzoo's biggest capability —
`zoo_ask`, which takes a corpus of up to ~1M tokens (a body models refuse directly)
`zoo_ask`, which takes a corpus of up to ~9.8M tokens per call (a body models refuse directly)
and answers via leCore memory spill — is an **MCP tool**, and Cursor supports MCP.
Settings → MCP → *Add new global MCP server*, or drop into `~/.cursor/mcp.json`
(per-project: `.cursor/mcp.json`):
Expand Down
4 changes: 2 additions & 2 deletions integrations/GrokCLI/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
Grok CLI (superagent-ai/grok-cli — the open-source terminal coding agent built
for the Grok API) speaks the OpenAI chat-completions format and accepts a custom
base URL, so it can route through openzoo. This covers "Grok-ecosystem" users:
they keep their harness and gain the zoo's ~435 models and leCore large-context
they keep their harness and gain the zoo's 480+ models and leCore large-context
handling.

## Setup
Expand All @@ -23,7 +23,7 @@ grok --model nvidia/nemotron-3.5-lightning

Persistent — add the provider entry from `models.openzoo.json` in this folder to
`~/.grok/models.json` (append to the existing array; add more zoo ids to
`models` as desired — `npx openzoo models` lists them). Per-project model
`models` as desired — `curl localhost:8402/v1/models` lists them). Per-project model
pinning lives in `.grok/settings.json`.

## Notes
Expand Down
2 changes: 1 addition & 1 deletion integrations/Hermes/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Easiest: run `npx openzoo` (the paying proxy), then `hermes model`, choose

- Base URL: `http://localhost:8402/v1` (stop at `/v1` — Hermes appends the route)
- API key: `sk-openzoo` (any value for the local proxy; real key for hosted)
- Model: any zoo id (`npx openzoo models` lists them)
- Model: any zoo id (`curl localhost:8402/v1/models` lists them)

Hermes saves the selection to `~/.hermes/config.yaml`. `config.openzoo.yaml` in
this folder is the equivalent manual merge if you prefer editing config directly.
Expand Down
4 changes: 2 additions & 2 deletions integrations/Hermes/config.openzoo.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
# -> base URL: http://localhost:8402/v1 (ends in /v1; Hermes appends
# /chat/completions itself)
# -> API key: sk-openzoo (any value for the local proxy)
# -> model id: nvidia/nemotron-3.5-lightning (or any zoo id; `npx openzoo models`)
# -> model id: nvidia/nemotron-3.5-lightning (or any zoo id; `curl localhost:8402/v1/models`)
#
# B) Manual: merge the block below into ~/.hermes/config.yaml.
#
Expand All @@ -19,5 +19,5 @@
model:
provider: custom
name: nvidia/nemotron-3.5-lightning
base_url: http://localhost:8402/v1 # hosted: https://api.openzoo.fun/v1
base_url: http://localhost:8402/v1 # hosted: https://x402-tokens.fly.dev/v1
api_key: sk-openzoo # any value for local proxy; real key for hosted
2 changes: 1 addition & 1 deletion integrations/LibreChat/librechat.openzoo.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ endpoints:
custom:
- name: "openzoo"
# Local proxy (start with: npx openzoo). For hosted, e.g.:
# baseURL: "https://api.openzoo.fun/v1"
# baseURL: "https://x402-tokens.fly.dev/v1"
baseURL: "http://localhost:8402/v1"
apiKey: "sk-openzoo" # or "${OPENZOO_KEY}" for hosted
models:
Expand Down
2 changes: 1 addition & 1 deletion integrations/OpenWebUI/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ pays per call via x402; the key field is sent but ignored.

Open the function's **Valves** and set:

- `OPENZOO_BASE_URL` → the hosted URL (e.g. `https://api.openzoo.fun/v1`)
- `OPENZOO_BASE_URL` → the hosted URL (e.g. `https://x402-tokens.fly.dev/v1`)
- `OPENZOO_API_KEY` → your real key

Nothing else changes; the same file serves both rails.
Expand Down
88 changes: 70 additions & 18 deletions integrations/OpenWebUI/openzoo_pipe.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
funding_url: https://openzoo.fun
version: 0.1.0
license: MIT
description: Route chats through openzoo.fun - leCore-backed inference, ~435 models, effectively unlimited context, pay-per-call. Works with the local x402 proxy (npx openzoo) today and the hosted endpoint when available.
description: Route chats through openzoo.fun - leCore-backed inference, 480+ models, effectively unlimited context, pay-per-call. Works with the local x402 proxy (npx openzoo) today and the hosted endpoint when available.
requirements: requests
"""

Expand All @@ -31,8 +31,12 @@

class Pipe:
class Valves(BaseModel):
# Default is the local proxy started by `npx openzoo`. Point this at
# the hosted endpoint (e.g. https://api.openzoo.fun/v1) once it exists.
# Default is the local proxy started by `npx openzoo`. The hosted endpoint is
# https://x402-tokens.fly.dev/v1 (api.openzoo.fun/v1 404s -- it is the
# website, not the gateway).
# NOTE if you run open-webui in Docker (the documented install): set
# this to http://host.docker.internal:8402/v1 -- inside a container
# "localhost" is the container, and the proxy is on the HOST.
OPENZOO_BASE_URL: str = Field(
default="http://localhost:8402/v1",
description="OpenAI-compatible base URL for openzoo (local proxy or hosted).",
Expand Down Expand Up @@ -95,17 +99,48 @@ def _strip_owui_prefix(self, model_id: str) -> str:
})

@staticmethod
def _format_receipt(usage: dict) -> str:
# openzoo returns billing info alongside usage; surface it so users
# see the per-call cost and the savings from the leCore spill.
def _format_receipt(data: dict) -> str:
# FIXED (was reading the wrong object): billing does NOT live in
# `usage`. Verified against the live gateway, a response carries a
# TOP-LEVEL "x402" block:
# x402: {billedUsd, cogsUsd, directUsd, savesVsDirect,
# subscription: {tier, cogsUsd, wouldHaveBilled, invoiced}}
# while `usage` holds only OpenAI/OpenRouter fields (prompt_tokens,
# completion_tokens, cost, cost_details, is_byok). The previous
# version looked for billedUsd/savesVsDirect inside `usage`, never
# found them, and silently rendered a token count and nothing else --
# i.e. the receipt, which is the whole reason this plugin exists, was
# dead code.
x = data.get("x402") or {}
usage = data.get("usage") or {}
sub = x.get("subscription") or {}
parts = []
if "billedUsd" in usage:
parts.append(f"billed ${usage['billedUsd']}")
if "savesVsDirect" in usage:
parts.append(f"saved {usage['savesVsDirect']} vs direct")
if "prompt_tokens" in usage:
parts.append(f"{usage['prompt_tokens']} tokens read")
return ("\n\n---\n*openzoo: " + ", ".join(parts) + "*") if parts else ""

billed = x.get("billedUsd")
if isinstance(billed, (int, float)):
parts.append(f"billed ${billed:.6f}".rstrip("0").rstrip("."))

# DELIBERATELY NOT rendering `savesVsDirect` as a percentage saved.
# Measured: billedUsd 0.00700472 / wouldHaveBilled 0.02101416 =
# 0.3333 == savesVsDirect. So the field is the RATIO PAID, not the
# fraction saved -- printing "saved 0.33" claims a third when the
# real saving is two thirds. Show both absolute numbers instead;
# they cannot be misread.
would = sub.get("wouldHaveBilled") or x.get("directUsd")
if isinstance(would, (int, float)) and isinstance(billed, (int, float)) \
and would > billed > 0:
parts.append(f"vs ${would:.6f}".rstrip("0").rstrip(".") + " direct"
f" ({would / billed:.1f}x)")

if sub.get("tier"):
parts.append(f"{sub['tier']} subscription")
elif x.get("paid"):
parts.append(str(x["paid"]))

pt = usage.get("prompt_tokens")
if pt:
parts.append(f"{pt} tokens read")
return ("\n\n---\n*openzoo: " + " · ".join(parts) + "*") if parts else ""

# ------------------------------------------------------------------ pipes

Expand Down Expand Up @@ -186,8 +221,7 @@ def pipe(self, body: dict):

data = r.json()
if self.valves.SHOW_RECEIPTS:
usage = data.get("usage", {}) or {}
receipt = self._format_receipt(usage)
receipt = self._format_receipt(data) # whole body: x402 is top-level
if receipt:
try:
data["choices"][0]["message"]["content"] += receipt
Expand Down Expand Up @@ -217,9 +251,27 @@ def _selftest():
"chat_id": "x", "session_id": "y", "metadata": {}, "features": {}}
clean = {k: v for k, v in junk.items() if k in Pipe._OPENAI_CHAT_FIELDS}
assert set(clean) == {"model", "messages", "stream"}, clean
# receipt formatting: all fields, some fields, none
full = Pipe._format_receipt({"billedUsd": 0.0021, "savesVsDirect": "97%", "prompt_tokens": 3100})
assert "billed $0.0021" in full and "saved 97%" in full and "3100 tokens read" in full
# receipt formatting -- REGRESSION TEST for the bug this file used to have.
# These are the exact shapes a live gateway returns; the old code read
# `usage` for billedUsd/savesVsDirect and therefore rendered nothing.
live = {
"usage": {"prompt_tokens": 3100, "completion_tokens": 20, "cost": 0.007},
"x402": {"billedUsd": 0.00700472, "savesVsDirect": 0.3333333333333333,
"cogsUsd": 0.00700472, "directUsd": 0.00700472,
"paid": "subscription",
"subscription": {"tier": "pro", "wouldHaveBilled": 0.02101416,
"invoiced": "stripe"}},
}
full = Pipe._format_receipt(live)
assert "billed $0.007" in full, full
assert "vs $0.021" in full and "3.0x" in full, full # 0.021/0.007
assert "pro subscription" in full, full
assert "3100 tokens read" in full, full
# and the ratio must NOT be printed as a savings percentage
assert "saved" not in full, full
# a usage-only body (no x402 -- e.g. prepaid credit, no 402 emitted) still
# renders the token count rather than blowing up
assert "3100 tokens read" in Pipe._format_receipt({"usage": {"prompt_tokens": 3100}})
assert Pipe._format_receipt({}) == ""
# valves defaults are the local-proxy contract
assert p.valves.OPENZOO_BASE_URL.endswith("/v1")
Expand Down
2 changes: 1 addition & 1 deletion integrations/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ Every integration here targets one or both of openzoo's surfaces:
short prompts price at a 3× passthrough markup — receipts name which base
applied, and print on the proxy console per call.
2. **MCP server** — `npx openzoo mcp` (stdio). Tools: `zoo_ask` (corpus up to
~1M tokens + question → answer + receipt), `zoo_models`, `zoo_wallet`. MCP
~9.8M tokens per call (~128M bound ceiling) + question → answer + receipt), `zoo_models`, `zoo_wallet`. MCP
hosts (Cursor, Cline, Claude Desktop, Windsurf) should wire BOTH surfaces —
chat for ordinary completions, MCP for the giant-corpus flagship.

Expand Down
2 changes: 1 addition & 1 deletion integrations/SillyTavern/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ configuration — there is no file to install, so this folder is documentation o
3. Set **API** → `Chat Completion`.
4. Set **Chat Completion Source** → `Custom (OpenAI-compatible)`.
5. **Custom Endpoint**: `http://localhost:8402/v1`
(hosted: `https://api.openzoo.fun/v1`)
(hosted: `https://x402-tokens.fly.dev/v1`)
— stop at `/v1`; do NOT append `/chat/completions`, SillyTavern adds the route.
6. **API Key**: `sk-openzoo` (any value for the local proxy; real key for hosted).
7. Connect. Because the proxy implements `/v1/models`, the model dropdown fills
Expand Down
2 changes: 1 addition & 1 deletion integrations/aider/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ source integrations/aider/openzoo.env # sets the two variables
aider --model openai/nvidia/nemotron-3.5-lightning # any zoo model id works
```

`npx openzoo models` lists model ids and pricing.
`curl localhost:8402/v1/models` lists model ids and pricing.

## Persistent config (optional)

Expand Down
2 changes: 1 addition & 1 deletion integrations/aider/openzoo.env
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,5 @@
# openai/-prefixed model as living at that base. Two exports is the whole
# integration.

export OPENAI_API_BASE="http://localhost:8402/v1" # hosted: https://api.openzoo.fun/v1
export OPENAI_API_BASE="http://localhost:8402/v1" # hosted: https://x402-tokens.fly.dev/v1
export OPENAI_API_KEY="sk-openzoo" # any value for local proxy; real key for hosted
Loading