Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
60afcd5
feat(fastify): add ai-sdk-ollama provider, /ai/generate, plain chat P…
gaboesquivel Mar 10, 2026
42680ae
test(fastify): cleanup suite, remove low-value tests, remove OLLAMA_R…
gaboesquivel Mar 11, 2026
e3e4ab9
refactor(fastify): test infrastructure cleanup
gaboesquivel Mar 11, 2026
3da3d5e
fix(fastify): remove non-null assertion in ai chat route
gaboesquivel Mar 11, 2026
6640046
fix: address code review findings
gaboesquivel Mar 11, 2026
c73cd37
fix(fastify): build, chat, generate, tests, and docs
gaboesquivel Mar 11, 2026
5ca7989
fix: tar 7.5.11 CVE, fastify checktypes, code review nitpicks
gaboesquivel Mar 11, 2026
147c76a
fix(ci): add e2e debugging logs and skip upstream chat errors
gaboesquivel Mar 11, 2026
54a897c
fix(ai): preserve 402 for insufficient credits so skipIfInsufficientC…
gaboesquivel Mar 11, 2026
a02bc29
refactor(ai): extract account-info-tool to fix chat.ts max-lines
gaboesquivel Mar 11, 2026
ef0e71f
feat(ai): add Anthropic as provider, Claude 3.5 Sonnet as default
gaboesquivel Mar 11, 2026
398a560
feat(ai): use Claude Sonnet 4 as default everywhere
gaboesquivel Mar 11, 2026
033f6ce
feat(ai): anthropic provider as default, prefer direct API
gaboesquivel Mar 11, 2026
55d4d17
chore: update config
gaboesquivel Mar 11, 2026
66190d9
fix(fastify): use relative imports for auth-helper to fix tsgo/Vercel…
gaboesquivel Mar 11, 2026
aec61c8
fix(e2e): improve test stability, AI provider switch to Anthropic
gaboesquivel Mar 11, 2026
bad16f9
fix(fastify,next): resolve build error, test timeouts, flaky e2e
gaboesquivel Mar 11, 2026
2f896f2
fix(next): skip chat-assistant e2e when model lacks user-info-card
gaboesquivel Mar 11, 2026
49f6f97
style(next): remove unnecessary braces in chat-assistant e2e test
gaboesquivel Mar 11, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .github/workflows/api-e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,7 @@ env:
PGLITE: true
DATABASE_URL: postgresql://localhost/test
JWT_SECRET: e2e-jwt-secret-min-32-chars-for-tests
OLLAMA_BASE_URL: ${{ secrets.OLLAMA_BASE_URL }}
OPEN_ROUTER_API_KEY: ${{ secrets.OPEN_ROUTER_API_KEY != '' && secrets.OPEN_ROUTER_API_KEY || 'sk-or-v1-dummy-key-for-unit-tests' }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}

jobs:
build-and-test:
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,7 @@ jobs:
env:
PGLITE: true
DATABASE_URL: postgresql://localhost/test
OLLAMA_BASE_URL: https://ollama.example.com
OPEN_ROUTER_API_KEY: ${{ secrets.OPEN_ROUTER_API_KEY != '' && secrets.OPEN_ROUTER_API_KEY || 'sk-or-v1-dummy-key-for-lint' }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY || 'sk-ant-dummy-for-lint' }}
steps:
- name: Checkout code
uses: actions/checkout@v4
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/next-e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ env:
DATABASE_URL: postgresql://localhost/test
JWT_SECRET: e2e-jwt-secret-min-32-chars-for-tests
NEXT_PUBLIC_API_URL: http://localhost:3001
OLLAMA_BASE_URL: ${{ secrets.OLLAMA_BASE_URL }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}

jobs:
build-and-test:
Expand Down
3 changes: 1 addition & 2 deletions .github/workflows/packages-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,7 @@ env:
CI: true
PGLITE: true
DATABASE_URL: postgresql://localhost/test
OLLAMA_BASE_URL: https://ollama.example.com
OPEN_ROUTER_API_KEY: ${{ secrets.OPEN_ROUTER_API_KEY != '' && secrets.OPEN_ROUTER_API_KEY || 'sk-or-v1-dummy-key-for-unit-tests' }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY || 'sk-ant-dummy-for-packages-test' }}

jobs:
test:
Expand Down
37 changes: 25 additions & 12 deletions apps/docu/content/docs/deployment/self-hosted-llm.mdx
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
---
title: "Self-Hosted LLM (Ollama)"
description: "Run local LLMs for data privacy. Simple dev setup and production deployment with recommended hardware."
description: "Run local LLMs for data privacy. Recommended when prompts must stay on your infrastructure."
---

We recommend **self-hosting** an LLM when data privacy is a priority. Prompts and responses stay on your infrastructureno third-party API calls, no data retention by vendors, and full control over model choice and updates.
By default, the app uses **Anthropic** (direct API, Claude Sonnet 4) when `ANTHROPIC_API_KEY` is set. Otherwise it falls back to **Open Router** or **Ollama**. Prefer the Anthropic provider for best integration; use Open Router only when an Anthropic key is unavailable. We recommend **self-hosting** with Ollama when data privacy is a priority—prompts and responses stay on your infrastructure, with no third-party API calls.

This guide covers [Ollama](https://ollama.com)—a local inference runtime with an OpenAI-compatible API—for both development and production.

Expand Down Expand Up @@ -235,29 +235,42 @@ curl -s https://ollama.example.com/api/generate -d '{"model":"qwen2.5:3b","promp

---

## Using Ollama as Default in Fastify
## AI Provider in Fastify

The Fastify chat route (`/ai/chat`) uses **Ollama as the default** AI provider when `OLLAMA_BASE_URL` is set. Open Router is an optional alternative when `OPEN_ROUTER_API_KEY` is configured.
The Fastify API supports **Anthropic** (default), **Open Router**, and **Ollama**. Provider precedence (when `AI_PROVIDER` is unset): Anthropic direct API → Open Router → Ollama. Use Anthropic when possible; Open Router is a fallback. Privacy-first routing with Ollama requires `AI_PROVIDER=ollama` when other keys exist. See `getResolvedProvider()` for the exact logic.

### Routes

| Route | Use case |
|-------|----------|
| `POST /ai/chat` | Chat UI, messages, tools (getAccountInfo, braveSearch) |
| `POST /ai/generate` | CLI, scripts, pipelines—single prompt, plain text SSE |
Comment thread
gaboesquivel marked this conversation as resolved.

Both routes use the same provider; clients call the Fastify API.

### Environment Variables

At least one of `ANTHROPIC_API_KEY`, `OPEN_ROUTER_API_KEY`, or `OLLAMA_BASE_URL` must be set; `OLLAMA_BASE_URL` is used for self-hosted/private setups.

| Variable | Required | Default | Description |
|----------|----------|---------|-------------|
| `OLLAMA_BASE_URL` | No (one of OLLAMA/OPEN_ROUTER required) | `https://ollama.example.com` | Ollama server URL (OpenAI-compatible `/v1` API) |
| `AI_PROVIDER` | No | Inferred | `ollama` or `openrouter` — force provider when both are configured |
| `AI_DEFAULT_MODEL` | No | `qwen2.5:3b` (Ollama) / `openrouter/free` (Open Router) | Default model for chat |
| `OPEN_ROUTER_API_KEY` | No | — | Required only when using Open Router |
| `ANTHROPIC_API_KEY` | Conditional (default) | — | Anthropic API key (preferred); Claude Sonnet 4 via direct API |
| `OPEN_ROUTER_API_KEY` | Conditional (fallback) | — | Open Router API key when Anthropic unavailable |
| `OLLAMA_BASE_URL` | Conditional | `http://localhost:11434` | Ollama server URL when self-hosting |
| `AI_PROVIDER` | Conditional | Inferred | `anthropic`, `openrouter`, or `ollama` — inferred when only one provider configured; must be set when multiple vars present |
| `AI_DEFAULT_MODEL` | No | Varies by provider | Anthropic/OpenRouter: Claude Sonnet 4; Ollama: `qwen3:8b`; use `openrouter/free` for OpenRouter free tier |

### Provider Selection

1. If `AI_PROVIDER` is set, use it (requires the corresponding env var).
2. Else if `OLLAMA_BASE_URL` is set, use Ollama.
2. Else if `ANTHROPIC_API_KEY` is set, use Anthropic.
3. Else if `OPEN_ROUTER_API_KEY` is set, use Open Router.
4. Else the server returns 500 with "No AI provider configured".
4. Else if `OLLAMA_BASE_URL` is set, use Ollama.
5. Else the server returns 500 with "No AI provider configured".

### Tool Support Caveat
### Tool Support

Small Ollama models (e.g. `qwen2.5:3b`) may not reliably support tool/function-calling. For better tool use (e.g. `getAccountInfo`, `braveSearch`), use 7B+ models such as `llama3.1:8b` or `mistral:7b`.
Open Router models support tools (getAccountInfo, braveSearch). Small Ollama models (e.g. `qwen2.5:3b`) may not; use 7B+ such as `llama3.1:8b` or `mistral:7b` for reliable tool calling.

### Optional: Brave Search (Web Search Tool)

Expand Down
2 changes: 1 addition & 1 deletion apps/docu/content/docs/testing/e2e-testing.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ See [Vercel Protection Bypass for Automation](https://vercel.com/docs/security/d

## Environment Variables

- **Chat E2E** (chat-assistant.spec.ts): Uses Ollama only (`OLLAMA_BASE_URL` secret). CI and `test:e2e:local` force `AI_PROVIDER=ollama` and omit OpenRouter. Uses `authenticatedPage` fixture.
- **Chat E2E** (chat-assistant.spec.ts): Uses Anthropic Sonnet (`ANTHROPIC_API_KEY`). CI and `test:e2e:local` force `AI_PROVIDER=anthropic`. Uses `authenticatedPage` fixture.
- **E2E CI**: Uses local servers only; no Vercel URLs. For manual testing against previews, pass `--app`/`--api` URLs.

## ALLOW_TEST
Expand Down
10 changes: 10 additions & 0 deletions apps/docu/content/docs/testing/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,16 @@ describe('GET /users/:id', () => {
- Clean up after tests
- **PGLite linear execution**: Vitest runs with `fileParallelism: false`, `maxWorkers: 1`, and `sequence.concurrent: false` to avoid PGLite/WASM instability with parallel workers

### Test Utilities

Import from `@test/utils/auth-helper.js`:

- **getOrCreateSession(app, email)** — Cached JWT by email; reduces magic-link requests when tests share users
- **createAuthenticatedUser(app)** — Returns `{ token, email }` using the session pool
- **insertTestPasskey(app, jwt)** — Inserts a passkey for the authenticated user, returns passkey id (for delete/ownership tests)
- **getApiKeyToken(app, email)** — Creates an API key for the user and returns the raw key
- **clearSessionPool()** — Clears the session cache (called in `account.spec.ts` afterAll)

## Frontend Testing

Frontend apps use **Playwright E2E only** — no unit or integration test suites. See [Frontend Testing](/docs/testing/frontend-testing) and [E2E Testing](/docs/testing/e2e-testing).
Expand Down
22 changes: 14 additions & 8 deletions apps/fastify/.env-sample
Original file line number Diff line number Diff line change
@@ -1,15 +1,21 @@
# AI provider: Ollama (default) or Open Router. At least one must be configured.
# Ollama (default)
OLLAMA_BASE_URL=https://ollama.example.com
# AI provider: Anthropic (default), Open Router, or Ollama. At least one must be configured.
# Anthropic (direct, default): Claude Sonnet 4 via Anthropic API. Preferred when set.
ANTHROPIC_API_KEY=sk-ant-xxx

# Open Router (alternative): set when using cloud models
# Open Router (alternative): use when no Anthropic key; Claude Sonnet 4 or AI_DEFAULT_MODEL=openrouter/free
# OPEN_ROUTER_API_KEY=sk-or-v1-xxxxxxxxxxxxxxxxxx

# Optional: Force provider (ollama | openrouter). Inferred from env if unset.
# AI_PROVIDER=ollama
# Ollama (optional): for data privacy—prompts stay on your infra. See self-hosted-llm docs.
# OLLAMA_BASE_URL=http://localhost:11434

# Optional: Override default model. Ollama: qwen2.5:3b. Open Router: openrouter/free
# AI_DEFAULT_MODEL=qwen2.5:3b
# Optional: Force provider (anthropic | openrouter | ollama). Inferred from env if unset.
# AI_PROVIDER=openrouter

# Optional: Override default model. Anthropic/OpenRouter: Claude Sonnet 4. Ollama: qwen3:8b. Use AI_DEFAULT_MODEL=openrouter/free for OpenRouter free tier.
# AI_DEFAULT_MODEL=openrouter/free

# Optional: Upstream AI timeout (ms). Default 120000 (2 min).
# AI_UPSTREAM_TIMEOUT_MS=120000

# Optional: Brave Search API (enables web search tool in chat)
# BRAVE_SEARCH_API_KEY=BSA...
Expand Down
21 changes: 14 additions & 7 deletions apps/fastify/.env.test.example
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,14 @@ PGLITE=true
DATABASE_URL=postgresql://localhost/test
JWT_SECRET=e2e-jwt-secret-min-32-chars-for-tests

# AI provider: set OLLAMA_BASE_URL for AI chat tests (use .env.test, not committed)
# Local: OLLAMA_BASE_URL=http://localhost:11434
# Remote: OLLAMA_BASE_URL=https://your-ollama-server.example.com
# Open Router: OPEN_ROUTER_API_KEY=... (optional; AI tests unset it)
OLLAMA_BASE_URL=http://localhost:11434
# AI provider: Anthropic (default), Open Router, or Ollama (use .env.test, not committed)
# Unit/E2E tests use Anthropic Sonnet. Set ANTHROPIC_API_KEY for local and CI.
ANTHROPIC_API_KEY=sk-ant-xxx
# OPEN_ROUTER_API_KEY=sk-or-v1-xxxxxxxxxxxxxxxxxx
# OLLAMA_BASE_URL=http://localhost:11434

# Optional: Override default AI model
# AI_DEFAULT_MODEL=qwen2.5:3b
# Optional: Override default AI model (Claude Sonnet 4)
# AI_DEFAULT_MODEL=claude-sonnet-4-20250514

# Optional: GitHub OAuth (for OAuth route tests)
# GITHUB_CLIENT_ID=
Expand All @@ -20,3 +20,10 @@ OLLAMA_BASE_URL=http://localhost:11434

# CORS + URL allowlist. Default: * (any). Comma-separated origins to restrict.
# ALLOWED_ORIGINS=https://app.example.com,https://localhost:3000

# Passkey and TOTP tests (unit tests)
WEBAUTHN_RP_NAME=Test App
TOTP_ISSUER=Test App

# Rate limit (unit tests make many requests; default 100 is too low)
# RATE_LIMIT_MAX=10000
Loading
Loading