Support OpenAI-compatible endpoints and local Whisper servers - #56
Conversation
Thread a chat endpoint and an optional transcription endpoint through the REST client so Azure, OpenRouter, Groq, LM Studio, Ollama, and a local Whisper server work without an env var. Structured output falls back from json_schema when the provider rejects it. OPENAI_BASE_URL still wins. Closes #46 Co-authored-by: Jeremy Smith <JeremySNR@users.noreply.github.com>
# Conflicts: # CHANGELOG.md # README.md # src/main/settings.ts
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
Bugbot Autofix prepared fixes for both issues found in the latest run.
- ✅ Fixed: Fallback omits required JSON schema
- json_object and plain fallbacks now append a user message with the full schema JSON so providers know the required field names and structure.
- ✅ Fixed: Fallback skips accepted non-JSON replies
- completeChatContent now validates parseable JSON before returning and advances to the next format when a 200 response contains unparseable text.
You can send follow-ups to the cloud agent here.
Reviewed by Cursor Bugbot for commit a8dd6cf. Configure here.
| } | ||
| ] | ||
| } | ||
| } |
There was a problem hiding this comment.
Fallback omits required JSON schema
Medium Severity
The json_object and plain fallbacks never send schema (or even a JSON-shape instruction on json_object). Callers rely on response_format.json_schema for field names, so a provider that rejects strict schema mode gets unstructured output and clip analysis can fail or return empty results.
Reviewed by Cursor Bugbot for commit a8dd6cf. Configure here.
| } catch (err) { | ||
| lastError = err | ||
| if (signal?.aborted) throw err | ||
| if (!looksLikeUnsupportedFormat(err)) throw err |
There was a problem hiding this comment.
Fallback skips accepted non-JSON replies
Medium Severity
Format fallback only runs on 400/422 messages that match looksLikeUnsupportedFormat. If a server accepts json_schema and returns 200 with prose or unparseable text — common on Ollama and LM Studio — completeChatContent returns immediately and chatJSON retries the same format instead of trying json_object or plain.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit a8dd6cf. Configure here.
3 participants
What does this change?
Addresses #46. Settings now has an OpenAI-compatible API base URL (Azure, OpenRouter, Groq, LM Studio, Ollama) and an optional transcription base URL so a local Whisper server can sit next to a hosted LLM. The REST client resolves the base at request time (Settings, or
OPENAI_BASE_URLif set). Chat calls tryjson_schema, thenjson_object, then a plain JSON completion, because many compatible providers reject OpenAI's strict schema mode.Does not bundle in-process whisper.cpp — a local OpenAI-compatible Whisper server (faster-whisper, whisper.cpp’s HTTP API) is the path this PR adds. Word-level timestamps are still required.
Why?
An API key against api.openai.com was the only option. People already run Groq/OpenRouter/Ollama, and local Whisper is the obvious way to make transcription free.
How did you test it?
npm test(234 tests, including endpoint resolution and json_schema fallback)npm run typechecknpm run lintscripts/smoke-test.sh— Settings → API & models shows both URL fieldsAnything to watch out for?
OPENAI_BASE_URLstill overrides the Settings fields (cloud/CI). Compatible Whisper backends must return word-level timestamps or captions and tighten-cuts will not work.