feat(ok): custom endpoint embedding models — auto-detected dimensions, model field, connection test (#2883)

.changeset/custom-endpoint-embedding-models.md

@@ -0,0 +1,13 @@
+---
+"@inkeep/open-knowledge": patch
+---
+
+Make custom embeddings endpoints work from the desktop app: auto-detected vector size, a model field, and a Test connection button.
+
+Semantic search previously assumed every model returned 1536-dimension vectors. Pointing it at a self-hosted or non-OpenAI OpenAI-compatible endpoint whose model has a different native size made every response fail validation, and the failure was silent — search just quietly fell back to keyword matching. Leaving `search.semantic.dimensions` unset now means "use whatever the endpoint returns": the size is detected from the first response, pinned in the vector cache, and reused across restarts without re-embedding. Setting `dimensions` explicitly keeps the strict check, so a server that ignores the request parameter still fails loudly instead of silently. If a provider starts returning a different size (a model swapped behind an alias), the cached vectors are discarded and rebuilt at the new size rather than scoring two sizes against each other.
+
+`ok embeddings set-model <id>` / `clear-model` join `set-url` / `clear-url`, so a headless setup can name the model its endpoint serves without hand-editing `config.yml`.
+
+**Embeddings API keys are now per project.** The key moved from a single machine-global value (edited in Settings → Account) to a per-project, per-endpoint key edited on the same Settings → This project → Search screen as the endpoint it belongs to — no more hunting across two screens. Keys still live only in the 0600 `~/.ok/secrets.yml` (never in the project tree), now keyed by project + endpoint so a key can never travel to a host it wasn't set for. A **localhost** endpoint (Ollama / LM Studio) needs no key at all. Existing single-key OpenAI setups keep working untouched — the old flat key is read as a fallback for the default OpenAI endpoint. `ok embeddings set-key` / `clear-key` are now project-scoped (add `--cwd`), and `ok embeddings list` shows every stored key (redacted).
+
+Settings → Search gains a "Custom endpoint" section holding the endpoint URL, a free-text Model field, and a **Test connection** button that runs one throwaway embed and reports either the detected vector size or a specific reason (no key, unreachable, HTTP 401, wrong response shape, and so on). The section expands automatically when either value is already overridden, and changing either one now warns that it re-embeds and re-sends the whole corpus before committing. The Account key control is relabelled "Embeddings API key" to reflect that it is sent to whichever endpoint a project has configured, with a note that local servers needing no key accept any placeholder.

docs/content/reference/configuration.mdx

@@ -50,9 +50,9 @@ A key set in a file more specific than its scope (a user-scope key in `.ok/confi
 | `telemetry.localSink.logs.maxBytes` | number | `26214400` (~25 MB) | project | Max size of the local diagnostic logs file before it rotates. |
 | `telemetry.localSink.attributeDenylist` | `string[]` | 8 credential keys | project | Attribute keys whose values are redacted (`[REDACTED]`) before any local span/log is written. Extends the built-in denylist (`authorization`, `password`, `cookie`, etc.). |
 | `search.semantic.enabled` | boolean | `false` | project-local | Add embeddings-based semantic ranking to the MCP `search` tool and a **By meaning** mode to the cmd-K omnibar. Default off. **When on and a key is set, the search query and matching page content are sent to the configured embeddings provider** (content egress). See [Semantic search](#semantic-search) below. |
-| `search.semantic.baseUrl` | string | `"https://api.openai.com/v1"` | project-local | Base URL of the OpenAI-compatible embeddings API. Override for Azure / self-hosted / other providers. The API key is **not** stored here — set it with `ok embeddings set-key`. |
+| `search.semantic.baseUrl` | string | `"https://api.openai.com/v1"` | project-local | Base URL of the OpenAI-compatible embeddings API. Override to point at a self-hosted server (Ollama / vLLM / LM Studio) or another provider. The API key is **not** stored here — set it with `ok embeddings set-key`; it is sent to whichever endpoint this names. |
 | `search.semantic.model` | string | `"text-embedding-3-small"` | project-local | Embeddings model id. Must be served by the provider at `baseUrl`. Changing it re-embeds the corpus. |
-| `search.semantic.dimensions` | number | (native) | project-local | Optional output vector size. Omit for the model's native size (1536 for `text-embedding-3-small`). Set a smaller value to shrink the on-disk cache, trading a little quality. |
+| `search.semantic.dimensions` | number | (auto) | project-local | Optional output vector size. Omit (recommended) and the size is detected from the endpoint's first response and reused across restarts — that is what lets a non-OpenAI model work without knowing its size up front. Set a smaller value to shrink the on-disk cache, trading a little quality; an endpoint that ignores the request parameter then fails loudly instead of silently. |
 | `search.semantic.similarityFloor` | number | (unset) | project-local | Optional hard cutoff (0–1): drops semantic matches whose cosine similarity is below it. Retrieval is rank-based, so most setups leave it unset; set it only for a provider/model whose cosine scale you know. |
 | `linkPreviews.enabled` | boolean | `true` | project-local | Show a rich preview card (site name, page title, description, favicon) when you hover an external link in the editor. Default on (set to `false` to opt out). **When on, hovering an external link sends that link's URL to the destination site** to fetch its preview metadata — outbound egress, one request per previewed link. Previews of links to other documents in the project are read from the local index with no network request and are always on. See [Link previews](#link-previews) below. |
 
@@ -93,23 +93,23 @@ Most users never set these — the schema settings above cover the common cases.
 | `OTEL_SDK_DISABLED` | OpenTelemetry OTLP push gate. Inverted sense: set to `false` to **enable** push (it is off by default). |
 | `OK_BUG_REPORT_INTAKE_URL` | Base URL of the bug-report intake the desktop app's **Help → Report a bug…** upload posts to (e.g. `https://openknowledge.ai`). Must be `https:` — plain `http:` is accepted only for loopback hosts (local testing). Unset (the default), Send makes no network request — it opens a prefilled email draft to support@inkeep.com that you send yourself. |
 | `OK_FEEDBACK_INTAKE_ORIGIN` | Origin the optional [uninstall feedback](/docs/reference/what-open-knowledge-writes#what-leaves-your-machine) submission posts to (default `https://openknowledge.ai`). Must be `https:` — plain `http:` is accepted only for loopback hosts (local testing). An unusable value drops the submission rather than falling back to the default. |
-| `OK_EMBEDDINGS_API_KEY` | Fallback embeddings provider API key for [semantic search](#semantic-search), used when none is stored on disk. Prefer `ok embeddings set-key` (writes `~/.ok/secrets.yml`) for normal use; the env var is convenient for CI / scripted runs. |
+| `OK_EMBEDDINGS_API_KEY` | Fallback embeddings key for [semantic search](#semantic-search), used when no project key is stored **and only for the default OpenAI endpoint** (a machine-wide env key is never sent to a custom host). Prefer `ok embeddings set-key` for normal use; the env var is convenient for CI / scripted runs against OpenAI. |
 
 ## Semantic search
 
 The MCP `search` tool can fuse an **embeddings-based semantic signal** into its ranking, so an agent's query surfaces conceptually-related pages even when they share no keywords (a query about "auth retries" can surface a page titled "Session Token Refresh"). It is **off by default** and additive — with it off, search everywhere (the MCP tool and the cmd-K omnibar) is purely lexical and stays on-machine. With it on, the omnibar gains a **By meaning** mode alongside its default lexical mode: typing never embeds; pressing Enter fires one semantic search, sending the query to the embeddings provider.
 
 <Callout type="warn">
-  **Content egress.** When semantic search is enabled **and** a key is set, the search query and the matching page content are sent to the configured embeddings provider (OpenAI by default). Only content that is already in your corpus is embedded — anything excluded by `.okignore` / `.gitignore` is never sent — and embedding is lazy: nothing leaves the machine until a semantic search actually runs. The key lives only in a 0600 `~/.ok/secrets.yml` file (never in `config.yml`, logs, or telemetry).
+  **Content egress.** When semantic search is enabled **and** a key is set (or the endpoint is a keyless local server), the search query and the matching page content are sent to whichever OpenAI-compatible endpoint the project has configured (OpenAI by default). Only content that is already in your corpus is embedded — anything excluded by `.okignore` / `.gitignore` is never sent — and embedding is lazy: nothing leaves the machine until a semantic search actually runs. Keys live only in a 0600 `~/.ok/secrets.yml` file, keyed by project + endpoint so a key never travels to a host it wasn't set for; never in `config.yml`, the project tree, logs, or telemetry.
 </Callout>
 
-To turn it on (per machine):
+To turn it on (per project, per machine):
 
-1. Store the provider key in `~/.ok/secrets.yml` (0600, this machine only): `ok embeddings set-key` (reads the key from a hidden prompt or stdin). Check it with `ok embeddings status`; remove it with `ok embeddings clear-key`.
-2. Enable it for the project: `ok embeddings enable`, or the **Settings → This project → Search** toggle. Either sets `search.semantic.enabled: true` in project-local config (`<project>/.ok/local/config.yml`), picked up live by a running server; `ok embeddings disable` turns it off.
-3. (Optional) point at a non-OpenAI provider (Azure / self-hosted): `ok embeddings set-url <url>` — or set `search.semantic.baseUrl` in project-local config; `ok embeddings clear-url` resets to the default. Tune `model` / `dimensions` in config.
+1. Enable it for the project: the **Settings → This project → Search** toggle, or `ok embeddings enable`. Either sets `search.semantic.enabled: true` in project-local config (`<project>/.ok/local/config.yml`), picked up live by a running server; `ok embeddings disable` turns it off.
+2. Set the API key right there on the same screen (or `ok embeddings set-key` in the project). Keys are **per project** — one project's key is never shared with another unless both point at the same endpoint. A **localhost** endpoint (Ollama / LM Studio) needs no key at all — the field shows "not required". `ok embeddings list` shows every stored key (redacted); `ok embeddings clear-key` removes this project's.
+3. (Optional) point at your own OpenAI-compatible endpoint (a self-hosted Ollama / vLLM / LM Studio server, or another provider): open **Custom endpoint** in that same section, set the endpoint URL and the model id, and press **Test connection** — it runs one throwaway embed and reports either the detected vector size or the specific reason it failed. The same knobs are available from the CLI (`ok embeddings set-url <url>` / `clear-url`, `ok embeddings set-model <id>` / `clear-model`) and in project-local config (`search.semantic.baseUrl` / `model`). The vector size is detected automatically, so `dimensions` normally stays unset. Changing the endpoint or the model discards the cached vectors and re-embeds the corpus.
 
-The first semantic search kicks off a background embed of the corpus (cents for a whole vault with `text-embedding-3-small`); vectors are cached incrementally under `.ok/local/` and only changed docs re-embed. If the key is missing, the provider errors, or you're offline, search silently degrades to lexical — it never blocks or errors. Each MCP `search` response reports embedding coverage so an agent knows when vectors are still filling in.
+The first semantic search kicks off a background embed of the corpus (cents for a whole vault with `text-embedding-3-small`); vectors are cached incrementally under `.ok/local/` and only changed docs re-embed. If the key is missing, the provider errors, or you're offline, search degrades to lexical — it never blocks or errors. Because that degradation is quiet, **Test connection** is the way to tell a working custom endpoint apart from one that is failing. Each MCP `search` response reports embedding coverage so an agent knows when vectors are still filling in.
 
 ## Link previews