diff --git a/CLAUDE.md b/CLAUDE.md index 794d2dd..9ef665c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -273,10 +273,13 @@ If a piece of infra is missing from `config.json`, the server prints the exact ` ## Rule of LLM Provider -**OpenRouter or local Ollama. No direct vendor SDKs.** No Anthropic / OpenAI / Gemini / Bedrock keys or SDK imports. The active backend is selected by `Config.LlmProvider` (`"openrouter"` default | `"ollama"`) and switched via `bytebell set llm-provider `. Ollama mode reads `Config.OllamaUrl` (default `http://localhost:11434`) and `Config.OllamaModel` (free-form — any locally-pulled model) and reports `$0` cost. All LLM calls flow through `@bb/llm`, which: +**Every backend is an entry in `@bb/llm`'s provider table.** Six ship today — `openrouter` (default), `ollama`, `anthropic`, `bedrock`, `gemini`, `openai` — selected by `Config.LlmProvider` and switched via `bytebell set llm-provider `. Adding a backend means one module under `packages/llm/src/` plus one entry in `src/providers.ts`; it must never mean a new branch at a call site. -- Wraps every OpenRouter / Ollama call behind a single `askLLM` surface -- Computes per-call cost via `estimateCostUsd()` against live OpenRouter pricing for the `bytebell stats` view (short-circuits to `0` when provider is Ollama) +- **Prefer `fetch`; SDKs only where the wire protocol demands one.** Most providers speak an OpenAI-shaped `/chat/completions`, so they share `openaiCompatible.ts` — a base URL and a bearer token, not another hand-copied request builder. Two exceptions are allowed and are the only ones: `@ai-sdk/amazon-bedrock` + `ai` (Bedrock's Converse shape and SigV4 request signing, which must not be hand-rolled) and `openai` (the tool-use client, pointed at each provider's OpenAI-compatible base URL). Adding a third SDK requires raising the dependency question first. +- **All calls flow through `askLLM` / `askJsonLLM` / `askLLMWithTools`.** No package outside `@bb/llm` may import a provider module directly, and no business logic may branch on provider identity — ask the provider table for a capability instead. +- **Capability differences are data, not conditionals.** `LlmProviderEntry` carries `reportsCost` and `supportsTools`. Tool use (`askLLMWithTools`, and therefore `concept-graph`) works on every backend except Ollama, whose tool-format support varies per locally-pulled model and cannot be checked ahead of time. +- **Cost is provider-reported or zero.** Only OpenRouter returns a real figure. Never reintroduce a client-side pricing table — a hardcoded price that silently rots is worse than an honest `$0`. +- **Credentials live in `~/.bytebell/config.json`**, one key per provider (Bedrock additionally accepts SigV4 credentials or an ambient instance role), written only by `bytebell set …` or the setup wizard. The wizard renders from `packages/cli/src/llmProviders.ts`; the server's boot gate reads `requiredKeysFor(provider)`. Both are single sources of truth — do not add a second provider list. LLM outputs are probabilistic. They must be: diff --git a/README.md b/README.md index 697c452..c10b41b 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,8 @@ -# Bytebell [bytebell.ai] +# openspecs-index + +A local-first code knowledge engine by **[ByteBell](https://bytebell.ai)**. Index any repo into a durable graph and query it from your LLM client over MCP, without sending the source anywhere you don't control. + +> The CLI binary, the server, and the config directory are all still named `bytebell` / `bytebell-server` / `~/.bytebell/` — `openspecs-index` is the project, `bytebell` is what you type. ## Quickstart @@ -7,8 +11,8 @@ ### Prerequisites - [Bun](https://bun.sh) ≥ 1.1 — runtime + workspace manager. -- [Docker](https://www.docker.com/) — for the local Mongo + Neo4j + Redis stack `bytebell boot` brings up. -- An LLM backend — either an [OpenRouter](https://openrouter.ai) API key (default) or a local [Ollama](https://ollama.com) model. Every per-file analysis call goes through the one you pick. +- [Docker](https://www.docker.com/) — for the local Mongo + Neo4j + Redis stack `bytebell boot` brings up. Not needed if you point openspecs-index at infrastructure you already run (`bytebell set infra-mode cloud`, see [Bring your own infrastructure](#bring-your-own-infrastructure)). +- An LLM backend — one of six: [OpenRouter](https://openrouter.ai) (default), [Anthropic](https://console.anthropic.com), [Google Gemini](https://aistudio.google.com/apikey), [OpenAI](https://platform.openai.com) or any OpenAI-compatible gateway, [AWS Bedrock](https://console.aws.amazon.com/bedrock), or a local [Ollama](https://ollama.com) model. Every per-file analysis call goes through the one you pick — full comparison in [docs/llm-providers.md](docs/llm-providers.md). ### Install @@ -32,14 +36,33 @@ The sections below are the manual, step-by-step equivalent — useful if you wan ### Configure -Two values Bytebell needs — your OpenRouter API key and model. Set them headlessly: +Pick an LLM backend and give it credentials. OpenRouter is the default, so this is the shortest path: ```bash bytebell set openrouter-api-key sk-or-… bytebell set openrouter-model anthropic/claude-sonnet-4.6 ``` -Or skip this step and run `bytebell boot` straight away — on an interactive terminal it opens a setup form to collect these on first run. Running `bytebell set` with no arguments opens the same form at any time. +Any of the other five works the same way — set `llm-provider`, then that provider's keys: + +```bash +bytebell set llm-provider anthropic # openrouter | anthropic | gemini | openai | bedrock | ollama +bytebell set anthropic-api-key sk-ant-… +bytebell set anthropic-model claude-sonnet-5 +``` + +| Provider | Keys to set | Cost reporting | Tool use | +| ---------------------- | ----------------------------------------------------------------------------------------------- | -------------- | ------------------------------ | +| `openrouter` (default) | `openrouter-api-key`, `openrouter-model` | **real spend** | yes | +| `anthropic` | `anthropic-api-key`, `anthropic-model` | `$0` | yes | +| `gemini` | `gemini-api-key`, `gemini-model` | `$0` | yes | +| `openai` | `openai-api-key`, `openai-model` (+ `openai-base-url` for vLLM / LiteLLM / a gateway) | `$0` | yes | +| `bedrock` | `bedrock-region`, `bedrock-model` + either `bedrock-api-key` or AWS SigV4 creds / instance role | `$0` | yes (needs the bearer API key) | +| `ollama` | `ollama-url`, `ollama-model` | `$0` (local) | no | + +Only OpenRouter reports real spend, so `bytebell stats` shows `$0` for the rest — a deliberate choice over a hardcoded price table that silently rots. Ollama is the one backend without tool use: OpenAI-tool-format support varies per locally-pulled model and can't be checked ahead of time. + +Or skip this step and run `bytebell boot` straight away — on an interactive terminal it opens a setup form to collect provider and credentials on first run. Running `bytebell set` with no arguments opens the same form at any time. There is no `.env` file anywhere. `~/.bytebell/config.json` (mode `0600`) is the single source of truth, and `bytebell set` is the only sanctioned way to write to it. If you already run Mongo / Neo4j / Redis and don't want the Docker stack, see [Bring your own infrastructure](#bring-your-own-infrastructure) below. @@ -51,12 +74,14 @@ bytebell boot What happens, in order: -1. **Pre-flight check** — verifies both OpenRouter keys are set. If either is blank and you're in an interactive terminal, Bytebell opens a setup form so you can enter them on the spot, then continues. In a non-interactive context (CI, piped input) it prints the exact `bytebell set …` commands and exits. -2. **Auto-fill** — fills any missing infra config keys with local-Docker defaults; generates a Neo4j password if one isn't set. -3. **Stack up** — `docker compose up -d` brings up `bytebell-mongo`, `bytebell-neo4j`, `bytebell-redis` (named volumes — data persists across reboots). -4. **Health gate** — polls `docker compose ps` until all three services report `healthy`. +1. **Pre-flight check** — verifies the infra keys (`mongo`, `neo4j`, `neo4j-user`, `neo4j-password`, `redis`) plus whichever credentials your selected `llm-provider` requires. If anything is blank and you're in an interactive terminal, openspecs-index opens a setup form so you can enter it on the spot, then continues. In a non-interactive context (CI, piped input) it prints the exact `bytebell set …` commands and exits. +2. **Auto-fill** — in `infra-mode docker` (the default), fills any missing infra config keys with local-Docker defaults and generates a Neo4j password if one isn't set. In `infra-mode cloud` nothing is auto-filled — your own URIs stand as written. +3. **Stack up** — docker mode only: `docker compose up -d` brings up `bytebell-mongo`, `bytebell-neo4j`, `bytebell-redis` (named volumes — data persists across reboots). +4. **Health gate** — docker mode only: polls `docker compose ps` until all three services report `healthy`. 5. **Server up** — spawns `bytebell-server` (HTTP on `127.0.0.1:8080`, MCP at `/mcp`). +Steps 2–4 are governed by `infra-mode`, which is `docker` unless you change it. Setting your own `mongo` / `neo4j` / `redis` URIs does **not** by itself skip Docker — run `bytebell set infra-mode cloud` as well. (A third mode, `embedded`, exists in the config schema but is currently disabled in the CLI.) + First boot pulls images and can take a couple of minutes. Subsequent boots are fast. ### Index a repo @@ -67,7 +92,7 @@ bytebell index https://github.com/anthropics/claude-code bytebell ls # watch state: CREATED → QUEUED → INGESTED → PROCESSING → PROCESSED ``` -When the row reads `PROCESSED`, the graph is fully populated and the MCP tools will return results for that repo. Local directories work too: `bytebell ingest /path/to/source-tree`. +When the row reads `PROCESSED`, the graph is fully populated and the MCP tools will return results for that repo. Two other states can show up: `HALTED` (paused mid-run, retryable) and `CORRUPTED` (the source tree went missing). Local directories work too: `bytebell ingest /path/to/source-tree`. ### Connect an MCP client @@ -92,18 +117,18 @@ Or add this under the `mcpServers` key of Claude Desktop's config (or Cursor's ` } ``` -The server registers `smart_search`, `keyword_lookup`, and `retrieve_file`, plus a bundled skill at `bytebell://skills/index` that the client can fetch and install once per session for the recommended workflow. +The server registers four tools — `list_knowledge`, `smart_search`, `keyword_lookup`, `retrieve_file` — plus a bundled skill at `bytebell://skills/index` that the client can fetch and install once per session for the recommended workflow. -## What Bytebell does +## What openspecs-index does -You point `bytebell` at a repo. It clones the source, walks every file, and for each file calls an LLM (via OpenRouter) to extract a structured `FileAnalysis`: a one-paragraph **purpose**, a longer **summary** of what the file does and how it fits the architecture, a **business context** line tying it to the product domain, plus the file's classes, functions, keywords, and imports. +You point `bytebell` at a repo. It clones the source, walks every file, and for each file calls your configured LLM provider to extract a structured `FileAnalysis`: a one-paragraph **purpose**, a longer **summary** of what the file does and how it fits the architecture, a **business context** line tying it to the product domain, plus the file's classes, functions, keywords, imports, and a set of domain/contract fields (ontology concepts, business entities, system capabilities, side effects, config dependencies, data-flow direction, integration surface, provided/consumed contracts, and a section map). Those outputs are persisted into two stores: - **Neo4j** receives a `:File` node enriched with `purpose`, `summary`, `businessContext`, `language`, `sha`, and `sizeBytes`, linked via `:HAS_CLASS`, `:HAS_FUNCTION`, `:HAS_KEYWORD`, `:HAS_IMPORT_INTERNAL`, and `:HAS_IMPORT_EXTERNAL` to deduplicated child nodes shared across the whole graph. Fulltext indexes cover purpose+summary, business context, keyword names, and class/function signatures. - **MongoDB** receives the raw file content, language, SHA256, and the full `FileAnalysis` JSON for cite-back and exact retrieval. -LLM clients then query that graph through three MCP tools — `smart_search`, `keyword_lookup`, `retrieve_file` — which together cover fused semantic + structural search, reverse entity-to-file lookup, and targeted content reads. They let an agent answer questions like _"Which files implement our retry/backoff policy and where is it configured?"_ without reading the entire repo into context. +LLM clients then query that graph through four MCP tools — `list_knowledge`, `smart_search`, `keyword_lookup`, `retrieve_file` — which together cover repo discovery, fused semantic + structural search, reverse entity-to-file lookup, and targeted content reads. They let an agent answer questions like _"Which files implement our retry/backoff policy and where is it configured?"_ without reading the entire repo into context. ```mermaid flowchart LR @@ -111,7 +136,7 @@ flowchart LR Client["MCP-capable LLM client
Claude Code, Cursor, …"] -- MCP --> Server Server -- enqueues --> Q["BullMQ in-process worker"] Q --> Strategy["IngestionStrategy
per-file LLM"] - Strategy -- LLM call --> OR["OpenRouter"] + Strategy -- LLM call --> OR["LLM provider
OpenRouter · Anthropic · Gemini
OpenAI · Bedrock · Ollama"] Strategy -- raw + analysis --> Mongo[("MongoDB")] Strategy -- enriched node --> Neo[("Neo4j")] Server -. retrieval .-> Mongo @@ -130,7 +155,9 @@ It is **not** a hosted product, not a chat UI, and not a multi-tenant platform. ### Ingest -`bytebell index ` (or `bytebell ingest `) submits a job to an in-process BullMQ queue. The worker dispatches to an `IngestionStrategy` — today, `BasicFileAnalysisStrategy` ([packages/ingest-github/src/BasicFileAnalysisStrategy.ts](packages/ingest-github/src/BasicFileAnalysisStrategy.ts)). It clones the repo to `~/.bytebell/repos//`, walks every file, runs a per-file OpenRouter call, and persists raw content to Mongo + the enriched node to Neo4j. +`bytebell index ` (or `bytebell ingest `) submits a job to an in-process BullMQ queue. The worker dispatches to an `IngestStrategy` — today the `flat-folder` pipeline ([packages/ingest-strategies/src/flat-folder/](packages/ingest-strategies/src/flat-folder/), documented in [docs/flat-folder-strategy.md](docs/flat-folder-strategy.md)): scan + classify, per-file LLM analysis, backfill, folder summaries, a repo summary, then the graph write. + +The repo is cloned under `~/.bytebell/orgs///////repository/`, every file gets a per-file LLM call, and raw content lands in Mongo while the enriched node lands in Neo4j. The per-file LLM call returns a single JSON object with this shape: @@ -139,27 +166,48 @@ The per-file LLM call returns a single JSON object with this shape: "purpose": "Why this file exists. Max ~300 tokens.", "summary": "What it does, key patterns, architecture role. Max ~600 tokens.", "businessContext": "Product/domain impact. 2–3 lines, max ~100 tokens.", + "language": "typescript", "classes": ["ExactName (~L3-29): What it represents", "..."], "functions": ["exact_name (~L42-58): Primary responsibility", "..."], "keywords": ["domain-term-1", "domain-term-2", "..."], "importsInternal": ["./relative/paths.ts", "..."], "importsExternal": ["express", "neo4j-driver", "..."], + "ontologyConcepts": ["retry-policy", "..."], + "businessEntities": ["Invoice", "..."], + "systemCapabilities": ["queue-submission", "..."], + "sideEffects": ["writes to Mongo", "..."], + "configDependencies": ["redis-url", "..."], + "dataFlowDirection": "inbound | outbound | bidirectional | none", + "integrationSurface": ["POST /api/v1/github/index", "..."], + "contractsProvided": ["buildGithubIndexRoute()", "..."], + "contractsConsumed": ["@bb/queue submitJob()", "..."], + "sectionMap": [{ "name": "route handler", "description": "..." }], } ``` -`classes` and `functions` carry approximate line ranges so `retrieve_file` can later pull the right slice without re-reading the whole file. **Re-indexing is diff-aware**: on `bytebell pull`, the strategy compares each file's SHA256 to the prior `:File.sha` and only re-analyses files whose hash changed. LLM cost is proportional to actual code churn, not to repo size. +The full field list lives in [packages/ingest-core/src/prompts/file-analysis-fields.ts](packages/ingest-core/src/prompts/file-analysis-fields.ts). `classes` and `functions` carry approximate line ranges so `retrieve_file` can later pull the right slice without re-reading the whole file. **Re-indexing is diff-aware**: `bytebell pull` diffs the previously-indexed commit against branch HEAD and re-analyses only the files git reports as added, modified, or renamed. LLM cost is proportional to actual code churn, not to repo size. ### Graph shape ```mermaid graph LR K[":Knowledge"] + R[":Repo"] + Fo[":Folder"] + RS[":RepoSummary"] F[":File
purpose, summary,
businessContext"] + FV[":FileVersion"] KW[":Keyword"] C[":Class"] Fn[":Function"] M[":Module"] K -- HAS_FILE --> F + K -- HAS_REPO --> R + K -- HAS_REPO_SUMMARY --> RS + R -- CONTAINS --> Fo + Fo -- CONTAINS --> F + Fo -- CONTAINS_FOLDER --> Fo + F -- HAS_VERSION --> FV F -- HAS_KEYWORD --> KW F -- HAS_CLASS --> C F -- HAS_FUNCTION --> Fn @@ -167,21 +215,25 @@ graph LR F -- HAS_IMPORT_EXTERNAL --> M ``` -One `:Knowledge` node per indexed repo owns its `:File` nodes. Each `:File` carries `purpose`, `summary`, `businessContext`, `language`, `sha`, `sizeBytes`, and a `relativePath` unique within its `knowledgeId`. From every file, the five `:HAS_*` edges link to deduplicated `:Keyword`, `:Class`, `:Function`, and `:Module` nodes that are global across the whole graph — the same library, the same exported function, the same domain term resolves to one node no matter how many repos reference it. Constraints make `(knowledgeId, relativePath)` unique on `:File`; fulltext indexes back the natural-language search side. Source: [packages/neo4j/src/files.ts](packages/neo4j/src/files.ts), [packages/neo4j/src/indexes.ts](packages/neo4j/src/indexes.ts). +One `:Knowledge` node per indexed repo owns its `:File` nodes. Each `:File` carries `purpose`, `summary`, `businessContext`, `language`, `sha`, `sizeBytes`, a `relativePath` unique within its `knowledgeId`, and the domain/contract fields listed above (`ontologyConcepts`, `businessEntities`, `sideEffects`, `contractsProvided`/`contractsConsumed`, the section map, and the big-file chunk counters). From every file, the five `:HAS_*` edges link to deduplicated `:Keyword`, `:Class`, `:Function`, and `:Module` nodes that are global across the whole graph — the same library, the same exported function, the same domain term resolves to one node no matter how many repos reference it. The ingest pipeline additionally builds the `:Repo` / `:Folder` tree and a `:RepoSummary`; `:FileVersion` nodes retain per-commit history. Constraints make `(knowledgeId, relativePath)` unique on `:File`; fulltext indexes back the natural-language search side. Source: [packages/neo4j/src/files.ts](packages/neo4j/src/files.ts), [packages/neo4j/src/folder.ts](packages/neo4j/src/folder.ts), [packages/neo4j/src/indexes.ts](packages/neo4j/src/indexes.ts). -There are no cross-file call edges in the current schema — that's a deliberate tradeoff for ingestion simplicity and language-agnostic ingest. Future strategies will add them, plugged in behind the same `IngestionStrategy` interface. +A parallel legacy mirror (`:FileNode`, `:FolderNode`, `:OrgKeyword`, written with snake_case properties) is upserted alongside the primary labels for backwards compatibility. + +There are no cross-file **call** edges in the current schema — that's a deliberate tradeoff for ingestion simplicity and language-agnostic ingest. Future strategies will add them, plugged in behind the same `IngestStrategy` interface. ### Retrieval -Three MCP tools, registered at `http://127.0.0.1:8080/mcp`: +Four MCP tools, registered at `http://127.0.0.1:8080/mcp`: -- **`smart_search(query, k=20)`** — fused six-channel search across File `purpose`/`summary`, `businessContext`, paths, keyword names, class/function signatures, and module imports. Returns deduplicated, ranked top-K files with folder clustering. Use first. +- **`list_knowledge(page?)`** — enumerate indexed repos with their `knowledgeId` UUIDs, state, and file counts. Call this first whenever you need a `knowledgeId`; never guess one from a repo name. +- **`smart_search(query, knowledgeId?, knowledgeIds?, path?, exclude?, page?, pageSize?)`** — fused **eight-channel** search across File `purpose`+`summary`, `businessContext`, paths, keyword names, class signatures, function signatures, internal imports, and external imports. Returns a deduplicated, ranked, paginated list of files with folder clustering (`pageSize` defaults to 30, max 100). `exclude` drops `tests | vendor | config | generated | docs | build`. Use first. - **`keyword_lookup(term)`** — reverse lookup. A search term resolves to all matching named entities (keywords, classes, functions, module names) and the files linked to each. -- **`retrieve_file`** — three operations: `metadata` (purpose, summary, businessContext, classes/functions with line ranges, imports), `content` (read specific line ranges or search within one file with surrounding context), `bulk_search` (parallel scan of up to 50 files for a string). +- **`retrieve_file`** — three operations: `metadata` (purpose, summary, businessContext, classes/functions with line ranges, imports; up to 10 paths per call), `content` (read specific line ranges or search within one file with surrounding context), `bulk_search` (parallel scan of up to 50 files for a string). ```mermaid flowchart TD - Q["Question from agent"] --> SS["smart_search"] + Q["Question from agent"] --> LK["list_knowledge
→ knowledgeId"] + LK --> SS["smart_search"] SS --> KL["keyword_lookup
(optional)"] SS --> RM["retrieve_file metadata
→ class/function line ranges"] KL --> RM @@ -196,6 +248,8 @@ Most well-formed code questions resolve in 2–4 tool calls. No re-clone, no ful | Command | Purpose | | ------------------------------------------------------------- | ---------------------------------------------------------------------------------- | | `bytebell setup` | Interactive first-run wizard: provider, boot, optional index, MCP auto-install. | +| `bytebell index ` | Index a GitHub repo (`--token` for private, `--branch` to pick a branch). | +| `bytebell ingest ` | Index a local directory instead of a remote repo. | | `bytebell ls` | List indexed knowledge entries with state. | | `bytebell stats` | Ingestion totals, per-repo breakdown, per-commit token usage. | | `bytebell mcp install` | Auto-detect installed editors and register the MCP endpoint in their config. | @@ -204,55 +258,90 @@ Most well-formed code questions resolve in 2–4 tool calls. No re-clone, no ful | `bytebell delete` | Picker; cancels jobs, drops the Knowledge subgraph from Neo4j, removes Mongo rows. | | `bytebell shutdown` | Stop the server. Docker keeps running. | | `bytebell boot` | Warm restart. | +| `bytebell migrate paths` | One-off on-disk layout migration for repos indexed by an older version. | | `docker compose -f infra/docker/docker-compose.yml down [-v]` | Stop containers (and optionally drop volumes — destroys all indexed data). | Full reference, including every flag and option: [commands.md](commands.md). ## Bring your own infrastructure -By default, `bytebell boot` provisions a local Docker stack (`bytebell-mongo`, `bytebell-neo4j`, `bytebell-redis`) with auto-generated credentials. If you already run Mongo, Neo4j, and Redis (or want to use a managed service), set the connection details before booting and the Docker step is skipped: +By default, `bytebell boot` provisions a local Docker stack (`bytebell-mongo`, `bytebell-neo4j`, `bytebell-redis`) with auto-generated credentials — that is `infra-mode docker`, the default. If you already run Mongo, Neo4j, and Redis (or want to use a managed service), switch the mode **and** set the connection details: ```bash -bytebell set mongo-uri mongodb://user:pass@host:27017/bytebell -bytebell set neo4j-uri bolt://host:7687 +bytebell set infra-mode cloud +bytebell set mongo mongodb://user:pass@host:27017/bytebell +bytebell set neo4j bolt://host:7687 bytebell set neo4j-user neo4j bytebell set neo4j-password -bytebell set redis-url redis://host:6379 +bytebell set redis redis://host:6379 ``` -Docker is not required on the host in this mode. See the [Configuration reference](#configuration-reference) for the full key list. +`infra-mode cloud` is what actually skips Docker — setting the URIs alone leaves the mode at `docker` and the compose stack still comes up. In cloud mode nothing is auto-filled, so every connection value above must be set explicitly, and Docker is not required on the host. See the [Configuration reference](#configuration-reference) for the full key list. ## Architecture at a glance -A single Bun-built Express daemon, `bytebell-server`, hosts the ingestion HTTP routes, the MCP transport (Streamable HTTP + SSE), and the BullMQ workers all in-process. The CLI is a thin Ink/React TUI that only ever talks HTTP to that daemon — it never touches Mongo, Neo4j, or Redis directly. Workers run in the server's lifecycle; there is no separate worker fleet. +A single Bun-built Express daemon, `bytebell-server`, hosts the ingestion HTTP routes (`/api/v1/…`), the MCP transport (Streamable HTTP at `/mcp`, SSE at `/sse`), and the BullMQ workers all in-process, bound to `127.0.0.1`. The CLI is a thin Ink/React TUI that talks HTTP to that daemon for every day-to-day operation — the one exception is `bytebell migrate paths`, an offline maintenance command that opens Mongo directly. Workers run in the server's lifecycle; there is no separate worker fleet. -For the full PRD — package tiers, state machine, HTTP route catalogue, verification checklist, distribution strategy — see [docs/arch.md](docs/arch.md). +Package tiers, import direction, the state machine, and the architectural rules every PR is held to live in [CLAUDE.md](CLAUDE.md). Subsystem deep-dives live in [docs/llm-providers.md](docs/llm-providers.md) and [docs/flat-folder-strategy.md](docs/flat-folder-strategy.md). ## Configuration reference -Settings live in `~/.bytebell/config.json` and are written exclusively by `bytebell set ` (or by first-run auto-fill on `bytebell boot`). Keys: - -| Key | Purpose | Default | -| -------------------- | ---------------------------------------- | ------------------------------------ | -| `openrouter-api-key` | API key for per-file LLM analysis | _(required, blank by default)_ | -| `openrouter-model` | OpenRouter model slug used for analysis | _(required)_ | -| `mongo-uri` | MongoDB connection string | `mongodb://localhost:27017/bytebell` | -| `neo4j-uri` | Neo4j Bolt URI | `bolt://localhost:7687` | -| `neo4j-user` | Neo4j auth user | `neo4j` | -| `neo4j-password` | Neo4j auth password | _(generated on first boot)_ | -| `redis-url` | Redis URL for BullMQ | `redis://localhost:6379` | -| `server-port` | Local HTTP/MCP port | `8080` | -| `concurrency-github` | Concurrent files analysed per GitHub job | tuned per box | -| `log-level` | Winston log level | `info` | -| `log-retention-days` | Daily log retention | `14` | - -If a required setting is missing, Bytebell either opens the setup form (interactive terminal) or prints the exact `bytebell set …` command and refuses to boot (non-interactive). It never silently reads `process.env`. +Settings live in `~/.bytebell/config.json` and are written exclusively by `bytebell set ` (or by first-run auto-fill on `bytebell boot`). Run `bytebell set` with no arguments for the interactive form. The key you pass to `bytebell set` is not always the config.json field name — the table below is the authoritative CLI-key list. + +**Infrastructure** + +| Key | Purpose | Default | +| ---------------- | ---------------------------------------------- | ----------------------------------------------------------------- | +| `infra-mode` | `docker` (compose stack) or `cloud` (your own) | `docker` | +| `mongo` | MongoDB connection string | _(blank; docker mode fills `mongodb://127.0.0.1:27017/bytebell`)_ | +| `neo4j` | Neo4j Bolt URI | _(blank; docker mode fills `bolt://127.0.0.1:7687`)_ | +| `neo4j-user` | Neo4j auth user | _(blank; docker mode fills `neo4j`)_ | +| `neo4j-password` | Neo4j auth password | _(generated on first docker boot)_ | +| `redis` | Redis URL for BullMQ | _(blank; docker mode fills `redis://127.0.0.1:6379`)_ | +| `port` | Local HTTP/MCP port | `8080` | + +**LLM provider** — set `llm-provider`, then the keys for that provider only. + +| Key | Purpose | Default | +| ----------------------------------------------------------------- | ------------------------------------------------------------------ | ----------------------------------- | +| `llm-provider` | `openrouter \| anthropic \| gemini \| openai \| bedrock \| ollama` | `openrouter` | +| `openrouter-api-key` | OpenRouter key | _(required for OpenRouter)_ | +| `openrouter-model` | Model slug used for analysis | `deepseek/deepseek-v4-flash` | +| `openrouter-fallback-model-1` … `-4` | Models OpenRouter falls back to, in order | four preset slugs | +| `anthropic-api-key`, `anthropic-model` | Anthropic direct | _(blank)_ | +| `gemini-api-key`, `gemini-model` | Google Gemini | _(blank)_ | +| `openai-api-key`, `openai-model`, `openai-base-url` | OpenAI or any OpenAI-compatible gateway | _(blank)_ | +| `bedrock-api-key`, `bedrock-region`, `bedrock-model` | AWS Bedrock (bearer key path) | _(blank)_ | +| `aws-access-key-id`, `aws-secret-access-key`, `aws-session-token` | Bedrock via SigV4 instead of a bearer key | _(blank; instance role also works)_ | +| `ollama-url`, `ollama-model` | Local Ollama daemon | _(blank)_ | +| `llm_cache_enabled` | Reuse cached per-file analyses | `true` | + +**Ingestion + storage backends** + +| Key | Purpose | Default | +| ---------------------------------------------- | ---------------------------------------------------- | --------- | +| `concurrency.github` | Concurrent files analysed per GitHub job | `2` | +| `db-provider` | Document store implementation | `mongo` | +| `graph-provider` | Graph store implementation | `neo4j` | +| `queue-provider` | Queue implementation | `bullmq` | +| `sqlite-path`, `queue-db-path`, `ladybug-path` | Paths used by the (currently disabled) embedded mode | _(blank)_ | + +**Logging** + +| Key | Purpose | Default | +| -------------------- | ------------------- | ------- | +| `log-level` | Winston log level | `info` | +| `log-retention-days` | Daily log retention | `14` | + +A further set of tuning fields (`enrichment.*`, `skip.decision.*`, `context.window.limit`, `neo4j.batch.size`, `openrouter.reasoning.max.tokens`, …) exists in the schema with sensible defaults but has no `bytebell set` key — see [packages/config/src/schema.ts](packages/config/src/schema.ts) for the complete list. + +If a required setting is missing, openspecs-index either opens the setup form (interactive terminal) or prints the exact `bytebell set …` command and refuses to boot (non-interactive). It never silently reads `process.env`. ## Why this design — research grounding -> Comparing Bytebell to PageIndex, GitNexus, GraphRAG, Sourcegraph, or Augment Code? See **[comparison.md](comparison.md)** for a side-by-side feature table and pros / cons of each. +> Comparing openspecs-index to PageIndex, GitNexus, GraphRAG, Sourcegraph, or Augment Code? See **[comparison.md](comparison.md)** for a side-by-side feature table and pros / cons of each. -Bytebell's shape — _build a code graph at ingest time, enrich every node with LLM-derived structured semantics, then serve retrieval against the joined surface_ — tracks a converging body of recent work showing that purely structural retrieval (AST / call-graph) and purely semantic retrieval (embeddings) each leave large performance on the table, and that combining them at indexing time unlocks the gains. +openspecs-index's shape — _build a code graph at ingest time, enrich every node with LLM-derived structured semantics, then serve retrieval against the joined surface_ — tracks a converging body of recent work showing that purely structural retrieval (AST / call-graph) and purely semantic retrieval (embeddings) each leave large performance on the table, and that combining them at indexing time unlocks the gains. **Graphs beat flat retrieval for code.** Repository-level graphs from AST + imports + call structure consistently outperform flat embedding retrieval on real engineering tasks. @@ -268,7 +357,7 @@ Bytebell's shape — _build a code graph at ingest time, enrich every node with - Knowledge-Graph-Based Repo-Level Code Generation ([2505.14394](https://arxiv.org/abs/2505.14394)) — graph captures structure; LLM context fills semantic gaps. - Sense and Sensitivity ([2505.13353](https://arxiv.org/abs/2505.13353)) — lexical and semantic recall are different capabilities; supports the `summary` (semantic) vs Mongo raw (lexical) split. -**Structured summaries and hierarchy beat blob summarization.** Explicit fields — purpose, inputs, outputs, business context — aggregated bottom-up let retrieval match at the right level of abstraction. This maps directly onto Bytebell's `purpose` / `summary` / `businessContext` schema. +**Structured summaries and hierarchy beat blob summarization.** Explicit fields — purpose, inputs, outputs, business context — aggregated bottom-up let retrieval match at the right level of abstraction. This maps directly onto openspecs-index's `purpose` / `summary` / `businessContext` schema. - Hierarchical Repo-Level Code Summarization for Business Applications ([2501.07857](https://arxiv.org/abs/2501.07857), ICSE LLM4Code 2025) — closest motivational match: structured per-unit summaries aggregated to file/package level, grounded in business context. - Beyond Function Level ([2502.16704](https://arxiv.org/abs/2502.16704)) — class/repo context in summaries beats function-only. @@ -279,11 +368,11 @@ Bytebell's shape — _build a code graph at ingest time, enrich every node with - Codebase-Memory ([2603.27277](https://arxiv.org/abs/2603.27277)) — MCP-served knowledge graph with LLM-derived metadata; reports 10× token reduction. -The design choices follow directly: each `:File` node carries LLM-generated semantics alongside `:HAS_CLASS` / `:HAS_FUNCTION` / `:HAS_KEYWORD` / `:HAS_IMPORT_*` edges (structure), and the three MCP tools fuse both surfaces at query time. +The design choices follow directly: each `:File` node carries LLM-generated semantics alongside `:HAS_CLASS` / `:HAS_FUNCTION` / `:HAS_KEYWORD` / `:HAS_IMPORT_*` edges (structure), and the MCP retrieval tools fuse both surfaces at query time. ## Enterprise -Bytebell-public is the OSS edition. ByteBell also offers a separately-licensed **Enterprise** edition for organizations that need a commercial-use grant, hardening, and direct support. Enterprise typically includes: +openspecs-index is the OSS edition. ByteBell also offers a separately-licensed **Enterprise** edition for organizations that need a commercial-use grant, hardening, and direct support. Enterprise typically includes: - A commercial-use grant covering use by or on behalf of for-profit entities, including SaaS deployments and revenue-generating applications. - Hardened multi-tenant deployment patterns, SSO / SCIM, audit logging, and data-isolation guarantees. @@ -295,8 +384,8 @@ To discuss Enterprise licensing, evaluation, or services, contact `team@bytebell ## Contributing -Hooks, commit conventions, and pre-push gates are documented in [contributing.md](contributing.md). Architectural rules — file-size limits, tier boundaries, the `README.md` requirement, the Bun-only and OpenRouter-only constraints — live in [CLAUDE.md](CLAUDE.md) and apply to every PR. +Hooks, commit conventions, and pre-push gates are documented in [contributing.md](contributing.md). Architectural rules — file-size limits, tier boundaries, the `README.md` requirement, the Bun-only constraint, and the provider-table rule that keeps LLM backends out of call-site conditionals — live in [CLAUDE.md](CLAUDE.md) and apply to every PR. ## License -Bytebell is released under **AGPL-3.0 with an additional non-commercial use clause** — see [LICENSE](LICENSE) for the authoritative text. Personal, academic, research, and non-profit use are unrestricted under AGPL-3.0 (network-copyleft applies). **Commercial use** is governed by license terms and is covered by the [Enterprise edition](#enterprise) (`team@bytebell.ai`). The running server itself does **not** verify a license; governance is by license terms, not by code. The server is meant for local single-tenant use — no remote network surface; everything binds to `127.0.0.1`. +openspecs-index is released under **AGPL-3.0 with an additional non-commercial use clause** — see [LICENSE](LICENSE) for the authoritative text. Personal, academic, research, and non-profit use are unrestricted under AGPL-3.0 (network-copyleft applies). **Commercial use** is governed by license terms and is covered by the [Enterprise edition](#enterprise) (`team@bytebell.ai`). The running server itself does **not** verify a license; governance is by license terms, not by code. The server is meant for local single-tenant use — no remote network surface; everything binds to `127.0.0.1`. diff --git a/bun.lock b/bun.lock index b018547..9f9e909 100644 --- a/bun.lock +++ b/bun.lock @@ -177,11 +177,14 @@ "name": "@bb/llm", "version": "0.0.0", "dependencies": { + "@ai-sdk/amazon-bedrock": "^5.0.61", "@bb/config": "workspace:*", "@bb/db": "workspace:*", "@bb/errors": "workspace:*", "@bb/logger": "workspace:*", "@bb/types": "workspace:*", + "ai": "^7.0.77", + "openai": "^7.5.0", "tiktoken": "^1.0.22", }, }, @@ -349,6 +352,18 @@ }, }, "packages": { + "@ai-sdk/amazon-bedrock": ["@ai-sdk/amazon-bedrock@5.0.61", "", { "dependencies": { "@ai-sdk/anthropic": "4.0.41", "@ai-sdk/openai": "4.0.46", "@ai-sdk/provider": "4.0.7", "@ai-sdk/provider-utils": "5.0.29", "@smithy/eventstream-codec": "^4.3.3", "@smithy/util-utf8": "^4.3.3", "aws4fetch": "^1.0.20" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-KHZ8LaY1K5KfKTv8O1Kaavpv2B5NX9xOfo8adxxt+x1GYg8GW9u0pTMb1i8/ipgKg6ZXYQUGIkJSv6XpjblpFA=="], + + "@ai-sdk/anthropic": ["@ai-sdk/anthropic@4.0.41", "", { "dependencies": { "@ai-sdk/provider": "4.0.7", "@ai-sdk/provider-utils": "5.0.29" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-So62as8fexWMmXMIxvrWVaDLjLRe6EG2+5mmRc1KCPBoyyyFx6zbTcCvxJkViSUHhsGMBDryAm+mfs91n7Jl8g=="], + + "@ai-sdk/gateway": ["@ai-sdk/gateway@4.0.62", "", { "dependencies": { "@ai-sdk/provider": "4.0.7", "@ai-sdk/provider-utils": "5.0.29", "@vercel/oidc": "3.2.0" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-zR3pustGWhw5eUZHG+fJZx/V/PBe+LxdDpc5hDFWxozG/3MB/+eY62jn+YiR+9uOH+Hx63e5zJoeKLfZfPktWQ=="], + + "@ai-sdk/openai": ["@ai-sdk/openai@4.0.46", "", { "dependencies": { "@ai-sdk/provider": "4.0.7", "@ai-sdk/provider-utils": "5.0.29" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-J53QUkZGOV9YIYf9caVjejdPZfVZ3V2xD5KyTAmMWFiMDkC/IcifEQHJferpGFaQkfmSehPGVPCLE4g8+VdkTA=="], + + "@ai-sdk/provider": ["@ai-sdk/provider@4.0.7", "", { "dependencies": { "json-schema": "^0.4.0" } }, "sha512-6or44XprPzKbr8zkmzosowSE0pxkvJcoojBL+mCZvPUt3kvXp3XSNqeVun9golb1acEfSo6yaEBRT18h2VU+1Q=="], + + "@ai-sdk/provider-utils": ["@ai-sdk/provider-utils@5.0.29", "", { "dependencies": { "@ai-sdk/provider": "4.0.7", "@standard-schema/spec": "^1.1.0", "@workflow/serde": "4.1.0", "eventsource-parser": "^3.0.8", "undici": "^7.28.0" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-7EIbwXiXKGa7EFk6tDZpuZBs6lxhEJpOuHeqrDb3Vd85uYdjwkdRuHnZDVDIIb2+QTSRmyph2NrXcbvuO/KAjQ=="], + "@alcalzone/ansi-tokenize": ["@alcalzone/ansi-tokenize@0.3.0", "", { "dependencies": { "ansi-styles": "^6.2.1", "is-fullwidth-code-point": "^5.0.0" } }, "sha512-p+CMKJ93HFmLkjXKlXiVGlMQEuRb6H0MokBSwUsX+S6BRX8eV5naFZpQJFfJHjRZY0Hmnqy1/r6UWl3x+19zYA=="], "@babel/code-frame": ["@babel/code-frame@7.29.0", "", { "dependencies": { "@babel/helper-validator-identifier": "^7.28.5", "js-tokens": "^4.0.0", "picocolors": "^1.1.1" } }, "sha512-9NhCeYjq9+3uxgdtp20LSiJXJvN0FeCtNGpJxuMFZ1Kv3cWUNb6DOhJwUvcVCzKGR66cw4njwM6hrJLqgOwbcw=="], @@ -509,8 +524,18 @@ "@simple-libs/stream-utils": ["@simple-libs/stream-utils@1.2.0", "", {}, "sha512-KxXvfapcixpz6rVEB6HPjOUZT22yN6v0vI0urQSk1L8MlEWPDFCZkhw2xmkyoTGYeFw7tWTZd7e3lVzRZRN/EA=="], + "@smithy/core": ["@smithy/core@3.33.3", "", { "dependencies": { "@smithy/types": "^4.17.2", "tslib": "^2.6.2" } }, "sha512-CsOeKq/9kA3y6VJHt+/+VTCtBaxJ4OTFpgrjIUhPpDIKxBci1k2bJaQASF2h/ELWrulGp+t97DZ0mevfAD8idg=="], + + "@smithy/eventstream-codec": ["@smithy/eventstream-codec@4.5.2", "", { "dependencies": { "@smithy/core": "^3.33.2", "tslib": "^2.6.2" } }, "sha512-u4EAkaviMcQYlorFSnxFUgF/Dnbgiu2TzGmqOU4h3CL4MbvrYyNCXpFkcBwvAEU11EvspKX2zzS2uX7q+43VdQ=="], + + "@smithy/types": ["@smithy/types@4.17.2", "", { "dependencies": { "tslib": "^2.6.2" } }, "sha512-FOKpVZob9MPTn2znRzGrnsMHv7BOsKVw3XiP/cOyYLDVZ9qKp4nifIiSCuUU/fIj5Vu0UOAxCFr+qRAtG0NUkA=="], + + "@smithy/util-utf8": ["@smithy/util-utf8@4.5.2", "", { "dependencies": { "@smithy/core": "^3.33.2", "tslib": "^2.6.2" } }, "sha512-7wNWV7SugHpcMA7uzEawJNpE0GrasXM7a9E+1+Wm6NxVuDClESac/AKt+G7jMZNUz5vBLWqKlqVV7Sv7AtUr6Q=="], + "@so-ric/colorspace": ["@so-ric/colorspace@1.1.6", "", { "dependencies": { "color": "^5.0.2", "text-hex": "1.0.x" } }, "sha512-/KiKkpHNOBgkFJwu9sh48LkHSMYGyuTcSFK/qMBdnOAlrRJzRSXAOFB5qwzaVQuDl8wAvHVMkaASQDReTahxuw=="], + "@standard-schema/spec": ["@standard-schema/spec@1.1.0", "", {}, "sha512-l2aFy5jALhniG5HgqrD6jXLi/rUWrKvqN/qJx6yoJsgKhblVd+iqqU4RCXavm/jPityDo5TCvKMnpjKnOriy0w=="], + "@types/body-parser": ["@types/body-parser@1.19.6", "", { "dependencies": { "@types/connect": "*", "@types/node": "*" } }, "sha512-HLFeCYgz89uk22N5Qg3dvGvsv46B8GLvKKo1zKG4NybA8U2DiEO3w9lqGg29t/tfLRJpJ6iQxnVw4OnB7MoM9g=="], "@types/bun": ["@types/bun@1.3.14", "", { "dependencies": { "bun-types": "1.3.14" } }, "sha512-h1hFqFVcvAvD9j9K7ZW7vd82aSA+rTdznZa+5bwvCwqSB1jmmfLcbIWhOLx1/+boy/xmjgCs/OMUL8hRJSmnPw=="], @@ -571,12 +596,18 @@ "@typescript-eslint/visitor-keys": ["@typescript-eslint/visitor-keys@8.59.0", "", { "dependencies": { "@typescript-eslint/types": "8.59.0", "eslint-visitor-keys": "^5.0.0" } }, "sha512-/uejZt4dSere1bx12WLlPfv8GktzcaDtuJ7s42/HEZ5zGj9oxRaD4bj7qwSunXkf+pbAhFt2zjpHYUiT5lHf0Q=="], + "@vercel/oidc": ["@vercel/oidc@3.2.0", "", {}, "sha512-UycprH3T6n3jH0k44NHMa7pnFHGu/N05MjojYr+Mc6I7obkoLIJujSWwin1pCvdy/eOxrI/l3uDLQsmcrOb4ug=="], + + "@workflow/serde": ["@workflow/serde@4.1.0", "", {}, "sha512-pav4F2BoirECWR7Nf1TKt+2eETcBj7jj4cBefQ8VXQCA6NPkaKeLfj/zMgi+3zYV5ZIBT4GuUiphsj0/b9hPQQ=="], + "accepts": ["accepts@2.0.0", "", { "dependencies": { "mime-types": "^3.0.0", "negotiator": "^1.0.0" } }, "sha512-5cvg6CtKwfgdmVqY1WIiXKc3Q1bkRqGLi+2W/6ao+6Y7gu/RCwRuAhGEzh5B4KlszSuTLgZYuqFqo5bImjNKng=="], "acorn": ["acorn@8.16.0", "", { "bin": { "acorn": "bin/acorn" } }, "sha512-UVJyE9MttOsBQIDKw1skb9nAwQuR5wuGD3+82K6JgJlm/Y+KI92oNsMNGZCYdDsVtRHSak0pcV5Dno5+4jh9sw=="], "acorn-jsx": ["acorn-jsx@5.3.2", "", { "peerDependencies": { "acorn": "^6.0.0 || ^7.0.0 || ^8.0.0" } }, "sha512-rq9s+JNhf0IChjtDXxllJ7g41oZk5SlXtp0LHwyA5cejwn7vKmKp4pPri6YEePv2PU65sAsegbXtIinmDFDXgQ=="], + "ai": ["ai@7.0.77", "", { "dependencies": { "@ai-sdk/gateway": "4.0.62", "@ai-sdk/provider": "4.0.7", "@ai-sdk/provider-utils": "5.0.29" }, "peerDependencies": { "zod": "^3.25.76 || ^4.1.8" } }, "sha512-muLtBSTAUCreR77L16w4AFBiX2gK/RNt84EKp8m03SN9+MfNlC5EGqYYttRjYKV3xe0a33yj1Zawj1EnjejIWw=="], + "ajv": ["ajv@6.15.0", "", { "dependencies": { "fast-deep-equal": "^3.1.1", "fast-json-stable-stringify": "^2.0.0", "json-schema-traverse": "^0.4.1", "uri-js": "^4.2.2" } }, "sha512-fgFx7Hfoq60ytK2c7DhnF8jIvzYgOMxfugjLOSMHjLIPgenqa7S7oaagATUq99mV6IYvN2tRmC0wnTYX6iPbMw=="], "ajv-formats": ["ajv-formats@3.0.1", "", { "dependencies": { "ajv": "^8.0.0" } }, "sha512-8iUql50EUR+uUcdRQ3HDqa6EVyo3docL8g5WJ3FNcWmu62IbkGUue/pEyLBW8VGKKucTPgqeks4fIU1DA4yowQ=="], @@ -595,6 +626,8 @@ "auto-bind": ["auto-bind@5.0.1", "", {}, "sha512-ooviqdwwgfIfNmDwo94wlshcdzfO64XV0Cg6oDsDYBJfITDz1EngD2z7DkbvCWn+XIMsIqW27sEVF6qcpJrRcg=="], + "aws4fetch": ["aws4fetch@1.0.20", "", {}, "sha512-/djoAN709iY65ETD6LKCtyyEI04XIBP5xVvfmNxsEP0uJB5tyaGBztSryRr4HqMStr9R06PisQE7m9zDTXKu6g=="], + "balanced-match": ["balanced-match@4.0.4", "", {}, "sha512-BLrgEcRTwX2o6gGxGOCNyMvGSp35YofuYzw9h1IMTRmKqttAZZVU67bdb9Pr2vUHA8+j3i2tJfjO6C6+4myGTA=="], "base64-js": ["base64-js@1.5.1", "", {}, "sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA=="], @@ -877,6 +910,8 @@ "json-parse-even-better-errors": ["json-parse-even-better-errors@2.3.1", "", {}, "sha512-xyFwyhro/JEof6Ghe2iz2NcXoj2sloNsWr/XsERDK/oiPCfaNhl5ONfp+jQdAZRQQ0IJWNzH9zIZF7li91kh2w=="], + "json-schema": ["json-schema@0.4.0", "", {}, "sha512-es94M3nTIfsEPisRafak+HDLfHXnKBhV3vU5eqPcS3flIWqcxJWgXHXiey3YrpaNsanY5ei1VoYEbOzijuq9BA=="], + "json-schema-traverse": ["json-schema-traverse@0.4.1", "", {}, "sha512-xbbCH5dCYU5T8LcEhhuh7HJ88HXuW3qsI3Y0zOZFKfZEHcpWiHU/Jxzk629Brsab/mMiHQti9wMP+845RPe3Vg=="], "json-schema-typed": ["json-schema-typed@8.0.2", "", {}, "sha512-fQhoXdcvc3V28x7C7BMs4P5+kNlgUURe2jmUT1T//oBRMDrqy1QPelJimwZGo7Hg9VPV3EQV5Bnq4hbFy2vetA=="], @@ -987,6 +1022,8 @@ "onetime": ["onetime@5.1.2", "", { "dependencies": { "mimic-fn": "^2.1.0" } }, "sha512-kbpaSSGJTWdAY5KPVeMOKXSrPtr8C8C7wodJbcsd51jRnmD+GZu8Y0VoU6Dm5Z4vWr0Ig/1NKuWRKf7j5aaYSg=="], + "openai": ["openai@7.5.0", "", { "peerDependencies": { "@aws-sdk/credential-provider-node": ">=3.972.0 <4", "@smithy/hash-node": ">=4.3.0 <5", "@smithy/signature-v4": ">=5.4.0 <6", "ws": "^8.18.0", "zod": "^3.25 || ^4.0" }, "optionalPeers": ["@aws-sdk/credential-provider-node", "@smithy/hash-node", "@smithy/signature-v4", "ws", "zod"] }, "sha512-ZbDBz8FSB8Mv8fFYIUvzTFMdV5vl93/octp1MdtK2lfYepSpfv/ewmeugpKz/cwGtFSx+YuUM4NwpZ2P55YiPA=="], + "optionator": ["optionator@0.9.4", "", { "dependencies": { "deep-is": "^0.1.3", "fast-levenshtein": "^2.0.6", "levn": "^0.4.1", "prelude-ls": "^1.2.1", "type-check": "^0.4.0", "word-wrap": "^1.2.5" } }, "sha512-6IpQ7mKUxRcZNLIObR0hz7lxsapSSIYNZJwXPGeF0mTVqGKFIXj1DQcMoT22S3ROcLyY/rz0PWaWZ9ayWmad9g=="], "p-limit": ["p-limit@3.1.0", "", { "dependencies": { "yocto-queue": "^0.1.0" } }, "sha512-TYOanM3wGwNGsZN2cVTYPArw454xnXj5qmWF1bEoAc4+cU/ol7GVh7odevjp1FNHduHc3KZMcFduxU5Xc6uJRQ=="], @@ -1145,6 +1182,8 @@ "typescript-eslint": ["typescript-eslint@8.59.0", "", { "dependencies": { "@typescript-eslint/eslint-plugin": "8.59.0", "@typescript-eslint/parser": "8.59.0", "@typescript-eslint/typescript-estree": "8.59.0", "@typescript-eslint/utils": "8.59.0" }, "peerDependencies": { "eslint": "^8.57.0 || ^9.0.0 || ^10.0.0", "typescript": ">=4.8.4 <6.1.0" } }, "sha512-BU3ONW9X+v90EcCH9ZS6LMackcVtxRLlI3XrYyqZIwVSHIk7Qf7bFw1z0M9Q0IUxhTMZCf8piY9hTYaNEIASrw=="], + "undici": ["undici@7.29.0", "", {}, "sha512-IDxfleLmmbSskfWSUATiN1nfn2rDuvnMOqb5CWR92iIfojA0Ud+ulOAAEQ57LPr9rWmsreUyf5lwyao+7GNNVw=="], + "undici-types": ["undici-types@7.19.2", "", {}, "sha512-qYVnV5OEm2AW8cJMCpdV20CDyaN3g0AjDlOGf1OW4iaDEx8MwdtChUp4zu4H0VP3nDRF/8RKWH+IPp9uW0YGZg=="], "unpipe": ["unpipe@1.0.0", "", {}, "sha512-pjy2bYhSsufwWlKwPc+l3cN7+wuJlK6uz0YdJEOlQDbl6jo/YlPi4mb8agUkVC8BF7V8NuzeyPNqRksA3hztKQ=="], diff --git a/package.json b/package.json index 20cf76d..aa57ee1 100644 --- a/package.json +++ b/package.json @@ -6,7 +6,11 @@ "workspaces": [ "packages/*" ], + "bin": { + "bytebell": "./packages/cli/src/index.ts" + }, "scripts": { + "bytebell": "bun packages/cli/src/index.ts", "dev:server": "BYTEBELL_DEV=1 bun --watch packages/server/src/index.ts", "dev:cli": "BYTEBELL_DEV=1 bun packages/cli/src/index.ts", "typecheck": "tsc -b", diff --git a/packages/cli/src/BootCommand.ts b/packages/cli/src/BootCommand.ts index 7159986..5535fa6 100644 --- a/packages/cli/src/BootCommand.ts +++ b/packages/cli/src/BootCommand.ts @@ -10,7 +10,7 @@ import { applyInfraDefaults, checkPreflight } from "./bootConfig.ts"; import { SetupForm } from "./SetupForm.tsx"; import { error, info, success } from "./output.ts"; import { bringInfraUp, usingHonker } from "./bootInfra.ts"; -import { isEmbedded } from "./infraMode.ts"; +import { isCloud, isEmbedded } from "./infraMode.ts"; export function buildBootCommand(): Command { const cmd = new Command("boot"); @@ -67,6 +67,16 @@ async function runBoot(): Promise { return; } + // Cloud mode (external databases) needs no Docker either. + if (isCloud()) { + info("cloud mode — using external/cloud databases, no Docker required."); + success(`queue → redis (${getConfigValue(Config.RedisUrl)})`); + success(`doc → mongo (${getConfigValue(Config.MongoUri)})`); + success(`graph → neo4j (${getConfigValue(Config.Neo4jUri)})`); + process.stdout.write("\nNext: bytebell index or bytebell ingest [path]\n"); + return; + } + if (graphProvider === GraphProviderType.Neo4j && defaults.neo4jPassword.length === 0) { error("internal: neo4j password is empty after applyInfraDefaults — refusing to start docker."); process.exitCode = 1; diff --git a/packages/cli/src/Field.tsx b/packages/cli/src/Field.tsx index 5cbc726..30afc8b 100644 --- a/packages/cli/src/Field.tsx +++ b/packages/cli/src/Field.tsx @@ -10,15 +10,26 @@ export interface FieldProps { mask?: boolean; error?: string; autoFocus?: boolean; + isFocused?: boolean; } -export function Field({ id, label, value, onChange, mask, error, autoFocus }: FieldProps): ReactElement { - const { isFocused } = useFocus({ id, autoFocus: autoFocus === true }); +export function Field({ + id, + label, + value, + onChange, + mask, + error, + autoFocus, + isFocused: propFocused, +}: FieldProps): ReactElement { + const { isFocused: hookFocused } = useFocus({ id, autoFocus: autoFocus === true }); + const isFocused = propFocused !== undefined ? propFocused : hookFocused; const indicator = isFocused ? "▶" : " "; const labelProps = isFocused ? { color: "cyan" } : {}; const masked = mask === true; const displayValue = masked && value.length > 0 ? "•".repeat(value.length) : value; - const inputProps = masked ? { value, onChange, mask: "•" } : { value, onChange }; + const inputProps = masked ? { value, onChange, mask: "•", focus: isFocused } : { value, onChange, focus: isFocused }; return ( diff --git a/packages/cli/src/InstallWizard.tsx b/packages/cli/src/InstallWizard.tsx index 25c8753..80f9cc6 100644 --- a/packages/cli/src/InstallWizard.tsx +++ b/packages/cli/src/InstallWizard.tsx @@ -4,32 +4,26 @@ import type { ReactElement } from "react"; import { Box, Text, useApp, useInput } from "ink"; import { FieldsStage, InfraStage, RepoStage, ConfirmStage } from "./InstallWizardStages.tsx"; import type { InfraMode } from "./infraMode.ts"; +import { + LLM_PROVIDER_SPECS, + initialProviderValues, + providerFieldsValid, + providerSpec, + type LlmProviderChoice, +} from "./llmProviders.ts"; -export type LlmProviderChoice = "openrouter" | "ollama"; +export type { LlmProviderChoice } from "./llmProviders.ts"; export interface InstallWizardResult { provider: LlmProviderChoice; infraMode: InfraMode; - openrouterApiKey?: string; - openrouterModel?: string; - ollamaUrl?: string; - ollamaModel?: string; + /** Field values keyed by `KEY_MAP` key — only the chosen provider's fields. */ + providerValues: Record; indexUrl?: string; } type Stage = "provider" | "infra" | "fields" | "repo" | "confirm"; -interface ProviderItem { - value: LlmProviderChoice; - label: string; - hint: string; -} - -const PROVIDERS: ProviderItem[] = [ - { value: "openrouter", label: "OpenRouter", hint: "API key required — https://openrouter.ai" }, - { value: "ollama", label: "Ollama", hint: "local, free — must already be running" }, -]; - export interface InstallWizardProps { onDone: (result: InstallWizardResult) => void; } @@ -38,30 +32,28 @@ export function InstallWizard({ onDone }: InstallWizardProps): ReactElement { const { exit } = useApp(); const [stage, setStage] = useState("provider"); const [providerIdx, setProviderIdx] = useState(0); - const [infraMode, setInfraMode] = useState("embedded"); - const [apiKey, setApiKey] = useState(""); - const [orModel, setOrModel] = useState(""); - const [ollamaUrl, setOllamaUrl] = useState("http://localhost:11434"); - const [ollamaModel, setOllamaModel] = useState(""); + const [infraMode, setInfraMode] = useState("docker"); + const [values, setValues] = useState>(() => initialProviderValues()); const [indexUrl, setIndexUrl] = useState(""); useInput((input, key) => { - if (stage === "provider") { - if (key.escape) { - exit(); - return; - } - if (key.upArrow || input === "k") { - setProviderIdx((i) => Math.max(0, i - 1)); - return; - } - if (key.downArrow || input === "j") { - setProviderIdx((i) => Math.min(PROVIDERS.length - 1, i + 1)); - return; - } - if (key.return) { - setStage("infra"); - } + if (stage !== "provider") { + return; + } + if (key.escape) { + exit(); + return; + } + if (key.upArrow || input === "k") { + setProviderIdx((i) => Math.max(0, i - 1)); + return; + } + if (key.downArrow || input === "j") { + setProviderIdx((i) => Math.min(LLM_PROVIDER_SPECS.length - 1, i + 1)); + return; + } + if (key.return) { + setStage("infra"); } }); @@ -71,7 +63,7 @@ export function InstallWizard({ onDone }: InstallWizardProps): ReactElement { Which LLM provider do you want to use? - {PROVIDERS.map((p, i) => { + {LLM_PROVIDER_SPECS.map((p, i) => { const selected = i === providerIdx; return ( @@ -90,13 +82,10 @@ export function InstallWizard({ onDone }: InstallWizardProps): ReactElement { ); } - const p = PROVIDERS[providerIdx]; - const provider: LlmProviderChoice = p !== undefined ? p.value : "openrouter"; - - const fieldsValid = - provider === "openrouter" - ? apiKey.trim().length > 0 && orModel.trim().length > 0 - : ollamaUrl.trim().length > 0 && ollamaModel.trim().length > 0; + const selected = LLM_PROVIDER_SPECS[providerIdx]; + const provider: LlmProviderChoice = selected !== undefined ? selected.value : "openrouter"; + const spec = providerSpec(provider); + const fieldsValid = providerFieldsValid(spec, values); if (stage === "infra") { return ( @@ -112,15 +101,9 @@ export function InstallWizard({ onDone }: InstallWizardProps): ReactElement { if (stage === "fields") { return ( setValues((prev) => ({ ...prev, [cliKey]: next }))} valid={fieldsValid} onBack={() => setStage("infra")} onNext={() => setStage("repo")} @@ -141,24 +124,18 @@ export function InstallWizard({ onDone }: InstallWizardProps): ReactElement { return ( setStage("repo")} onDone={() => { exit(); - const result: InstallWizardResult = { provider, infraMode }; - if (provider === "openrouter") { - result.openrouterApiKey = apiKey.trim(); - result.openrouterModel = orModel.trim(); - } else { - result.ollamaUrl = ollamaUrl.trim(); - result.ollamaModel = ollamaModel.trim(); + const providerValues: Record = {}; + for (const field of spec.fields) { + providerValues[field.cliKey] = (values[field.cliKey] ?? "").trim(); } + const result: InstallWizardResult = { provider, infraMode, providerValues }; if (indexUrl.trim().length > 0) { result.indexUrl = indexUrl.trim(); } diff --git a/packages/cli/src/InstallWizardStages.tsx b/packages/cli/src/InstallWizardStages.tsx index b463b0c..07fa006 100644 --- a/packages/cli/src/InstallWizardStages.tsx +++ b/packages/cli/src/InstallWizardStages.tsx @@ -2,7 +2,7 @@ import type { ReactElement } from "react"; import { Box, Text, useInput } from "ink"; import { Field } from "./Field.tsx"; -import type { LlmProviderChoice } from "./InstallWizard.tsx"; +import { maskSecret, type ProviderSpec } from "./llmProviders.ts"; import { INFRA_MODE_OPTIONS as INFRA_OPTIONS, type InfraMode } from "./infraMode.ts"; export interface InfraStageProps { @@ -65,34 +65,15 @@ export function InfraStage({ mode, onMode, onBack, onNext }: InfraStageProps): R } export interface FieldsStageProps { - provider: LlmProviderChoice; - apiKey: string; - onApiKey: (v: string) => void; - orModel: string; - onOrModel: (v: string) => void; - ollamaUrl: string; - onOllamaUrl: (v: string) => void; - ollamaModel: string; - onOllamaModel: (v: string) => void; + spec: ProviderSpec; + values: Record; + onChange: (cliKey: string, next: string) => void; valid: boolean; onBack: () => void; onNext: () => void; } -export function FieldsStage({ - provider, - apiKey, - onApiKey, - orModel, - onOrModel, - ollamaUrl, - onOllamaUrl, - ollamaModel, - onOllamaModel, - valid, - onBack, - onNext, -}: FieldsStageProps): ReactElement { +export function FieldsStage({ spec, values, onChange, valid, onBack, onNext }: FieldsStageProps): ReactElement { useInput((_input, key) => { if (key.escape) { onBack(); @@ -106,18 +87,28 @@ export function FieldsStage({ return ( - {provider === "openrouter" ? "OpenRouter configuration" : "Ollama configuration"} + {spec.label} configuration - {provider === "openrouter" ? ( - <> - - - - ) : ( - <> - - - + {spec.fields.map((field, i) => ( + + onChange(field.cliKey, next)} + {...(field.mask === true ? { mask: true } : {})} + {...(i === 0 ? { autoFocus: true } : {})} + /> + {field.hint} + + ))} + {!spec.supportsTools && ( + + + note: {spec.label} does not support tool use — the concept-graph strategy needs OpenRouter. The default + flat-folder strategy works on every provider. + + )} [Tab] next field [Enter] continue{valid ? "" : " (fill all fields)"} [Esc] back @@ -158,28 +149,15 @@ export function RepoStage({ indexUrl, onIndexUrl, onBack, onNext }: RepoStagePro } export interface ConfirmStageProps { - provider: LlmProviderChoice; + spec: ProviderSpec; + values: Record; infraMode: InfraMode; - apiKey: string; - orModel: string; - ollamaUrl: string; - ollamaModel: string; indexUrl: string; onBack: () => void; onDone: () => void; } -export function ConfirmStage({ - provider, - infraMode, - apiKey, - orModel, - ollamaUrl, - ollamaModel, - indexUrl, - onBack, - onDone, -}: ConfirmStageProps): ReactElement { +export function ConfirmStage({ spec, values, infraMode, indexUrl, onBack, onDone }: ConfirmStageProps): ReactElement { useInput((_input, key) => { if (key.escape) { onBack(); @@ -190,9 +168,6 @@ export function ConfirmStage({ } }); - const maskedKey = - apiKey.length === 0 ? "(none)" : `${"•".repeat(Math.min(apiKey.length, 8))}${apiKey.length > 8 ? "…" : ""}`; - return ( @@ -201,35 +176,33 @@ export function ConfirmStage({ {" "} - Provider : {provider} + Provider : {spec.label} {" "} - Infra : {infraMode === "embedded" ? "embedded (no Docker)" : "docker"} + Infra :{" "} + + {infraMode === "embedded" + ? "embedded (no Docker)" + : infraMode === "cloud" + ? "cloud (external databases, no Docker)" + : "docker (local containers)"} + - {provider === "openrouter" ? ( - <> - - {" "} - API key : {maskedKey} - - - {" "} - Model : {orModel || "(not set)"} - - - ) : ( - <> - - {" "} - URL : {ollamaUrl || "(not set)"} - - + {spec.fields.map((field) => { + const raw = (values[field.cliKey] ?? "").trim(); + return ( + {" "} - Model : {ollamaModel || "(not set)"} + {field.label} :{" "} + {field.mask === true ? ( + {maskSecret(raw)} + ) : ( + {raw.length > 0 ? raw : "(not set)"} + )} - - )} + ); + })} {" "} Index : {indexUrl.trim().length > 0 ? indexUrl : "(skip)"} diff --git a/packages/cli/src/SelectField.tsx b/packages/cli/src/SelectField.tsx new file mode 100644 index 0000000..ef5de90 --- /dev/null +++ b/packages/cli/src/SelectField.tsx @@ -0,0 +1,79 @@ +// SPDX-License-Identifier: AGPL-3.0-only WITH non-commercial-clause +import type { ReactElement } from "react"; +import { Box, Text, useFocus, useInput } from "ink"; + +export interface SelectFieldProps { + id: string; + label: string; + value: string; + options: readonly string[]; + onChange: (next: string) => void; + /** Shown dimmed under the row — e.g. what the selected option requires. */ + hint?: string; + isFocused?: boolean; +} + +/** + * An N-option cycling selector that joins the form's Tab order via `useFocus`. + * ←/→/space step through the options, wrapping at both ends. + * + * Distinct from `ToggleField`, which is typed to exactly two options — fine for + * docker/embedded, but the LLM provider list is six and growing. + */ +export function SelectField({ + id, + label, + value, + options, + onChange, + hint, + isFocused: propFocused, +}: SelectFieldProps): ReactElement { + const { isFocused: hookFocused } = useFocus({ id }); + const isFocused = propFocused !== undefined ? propFocused : hookFocused; + const current = Math.max(0, options.indexOf(value)); + + useInput( + (input, key) => { + if (options.length === 0) { + return; + } + if (key.leftArrow) { + onChange(options[(current - 1 + options.length) % options.length] ?? value); + return; + } + if (key.rightArrow || input === " ") { + onChange(options[(current + 1) % options.length] ?? value); + } + }, + { isActive: isFocused }, + ); + + const labelProps = isFocused ? { color: "cyan" } : {}; + + return ( + + + + {isFocused ? "▶" : " "} + + + {label} + + + {value} + + {" "}({current + 1}/{options.length} + {isFocused ? " — ←/→ to switch" : ""}) + + + + {hint !== undefined && hint.length > 0 && ( + + + {hint} + + )} + + ); +} diff --git a/packages/cli/src/SetCommand.ts b/packages/cli/src/SetCommand.ts index d66bf96..585e039 100644 --- a/packages/cli/src/SetCommand.ts +++ b/packages/cli/src/SetCommand.ts @@ -1,6 +1,7 @@ import { Command } from "commander"; import React from "react"; import { render } from "ink"; +import { Config } from "@bb/types"; import { HINTS, getConfigValue } from "@bb/config"; import { KEY_MAP, validKeysList } from "./keyMap.ts"; import { SetupForm } from "./SetupForm.tsx"; @@ -72,7 +73,17 @@ async function runSet(key?: string, value?: string): Promise { async function runInteractive(): Promise { return new Promise((resolve) => { - const onDone = () => resolve(); + const onDone = (res: { saved: boolean }) => { + if (res.saved) { + const mode = getConfigValue(Config.InfraMode); + if (mode === "docker") { + success("Configuration saved. If Docker instances are not running, run 'bytebell boot' to start them."); + } else { + success("Configuration saved. Run 'bytebell boot' to start the server."); + } + } + resolve(); + }; const { waitUntilExit } = render(React.createElement(SetupForm, { onDone })); waitUntilExit().catch(() => undefined); }); diff --git a/packages/cli/src/SetupCommand.ts b/packages/cli/src/SetupCommand.ts index 8ad6942..e51260b 100644 --- a/packages/cli/src/SetupCommand.ts +++ b/packages/cli/src/SetupCommand.ts @@ -2,9 +2,10 @@ import React from "react"; import { render } from "ink"; import { Command } from "commander"; -import { Config } from "@bb/types"; +import { Config, IngestionStrategyType } from "@bb/types"; import { getConfigValue } from "@bb/config"; import { InstallWizard, type InstallWizardResult } from "./InstallWizard.tsx"; +import { providerSpec } from "./llmProviders.ts"; import { KEY_MAP } from "./keyMap.ts"; import { applyInfraMode } from "./infraMode.ts"; import { runBootSequence } from "./bootConfig.ts"; @@ -88,45 +89,30 @@ function applyConfig(result: InstallWizardResult): void { // Infrastructure mode sets the db/graph/queue providers as a single preset. applyInfraMode(result.infraMode); - const providerEntry = KEY_MAP["llm-provider"]; - if (providerEntry === undefined) { - throw new Error("internal: KEY_MAP missing 'llm-provider'"); + setVia("llm-provider", result.provider); + for (const [cliKey, value] of Object.entries(result.providerValues)) { + setVia(cliKey, value); } - providerEntry.setter(result.provider); - if (result.provider === "openrouter") { - const keyEntry = KEY_MAP["openrouter-api-key"]; - const modelEntry = KEY_MAP["openrouter-model"]; - if (keyEntry === undefined) { - throw new Error("internal: KEY_MAP missing 'openrouter-api-key'"); - } - if (modelEntry === undefined) { - throw new Error("internal: KEY_MAP missing 'openrouter-model'"); - } - if (result.openrouterApiKey !== undefined) { - keyEntry.setter(result.openrouterApiKey); - } - if (result.openrouterModel !== undefined) { - modelEntry.setter(result.openrouterModel); - } - success(`OpenRouter configured (model: ${result.openrouterModel ?? "(not set)"})`); - } else { - const urlEntry = KEY_MAP["ollama-url"]; - const modelEntry = KEY_MAP["ollama-model"]; - if (urlEntry === undefined) { - throw new Error("internal: KEY_MAP missing 'ollama-url'"); - } - if (modelEntry === undefined) { - throw new Error("internal: KEY_MAP missing 'ollama-model'"); - } - if (result.ollamaUrl !== undefined) { - urlEntry.setter(result.ollamaUrl); - } - if (result.ollamaModel !== undefined) { - modelEntry.setter(result.ollamaModel); - } - success(`Ollama configured (model: ${result.ollamaModel ?? "(not set)"})`); + const spec = providerSpec(result.provider); + const modelField = spec.fields.find((f) => f.label.toLowerCase().includes("model")); + const model = modelField === undefined ? undefined : result.providerValues[modelField.cliKey]; + success(`${spec.label} configured (model: ${model ?? "(not set)"})`); + + // Concept-graph needs tool use, which only OpenRouter provides today. Fail + // loudly here rather than mid-ingest, and repair the config so the run works. + if (!spec.supportsTools && getConfigValue(Config.IngestionStrategy) === IngestionStrategyType.ConceptGraph) { + setVia("ingestion.strategy", IngestionStrategyType.FlatFolder); + info(`${spec.label} does not support tool use — switched ingestion.strategy to flat-folder.`); + } +} + +function setVia(cliKey: string, value: string): void { + const entry = KEY_MAP[cliKey]; + if (entry === undefined) { + throw new Error(`internal: KEY_MAP missing "${cliKey}"`); } + entry.setter(value); } async function boot(): Promise { diff --git a/packages/cli/src/SetupForm.tsx b/packages/cli/src/SetupForm.tsx index 0350856..52d8b5d 100644 --- a/packages/cli/src/SetupForm.tsx +++ b/packages/cli/src/SetupForm.tsx @@ -4,18 +4,21 @@ import { Box, Text, useApp, useInput } from "ink"; import { Config } from "@bb/types"; import { getConfigValue } from "@bb/config"; import { KEY_MAP } from "./keyMap.ts"; -import { applyInfraMode, infraModeOption, isEmbedded, type InfraMode } from "./infraMode.ts"; +import { applyInfraMode, getInfraMode, infraModeOption, type InfraMode } from "./infraMode.ts"; import { Field } from "./Field.tsx"; import { ToggleField } from "./ToggleField.tsx"; +import { SelectField } from "./SelectField.tsx"; +import { LLM_PROVIDER_SPECS, initialProviderValues, providerSpec, type LlmProviderChoice } from "./llmProviders.ts"; -const MODE_OPTIONS: readonly [string, string] = ["docker", "embedded"]; +const MODE_OPTIONS: readonly string[] = ["docker", "cloud"]; +const PROVIDER_OPTIONS: readonly string[] = LLM_PROVIDER_SPECS.map((p) => p.value); interface Row { id: string; label: string; cliKey: string; mask?: boolean; - /** Infra connection rows — only required/shown in Docker (non-embedded) mode. */ + /** Infra connection rows — only required/shown in Cloud mode. */ infra?: boolean; validate: (raw: string) => string | null; } @@ -73,33 +76,46 @@ const ROWS: Row[] = [ cliKey: "concurrency.github", validate: (s) => (/^\d+$/u.test(s) && Number(s) > 0 ? null : "expected positive integer"), }, - { - id: "openrouter-api-key", - label: "OpenRouter API key", - cliKey: "openrouter-api-key", - mask: true, - validate: (s) => (s.length > 0 ? null : "required — get one at openrouter.ai/keys"), - }, - { - id: "openrouter-model", - label: "OpenRouter model", - cliKey: "openrouter-model", - validate: (s) => (s.length > 0 ? null : "required — e.g. deepseek/deepseek-v4-flash"), - }, ]; +/** + * The active provider's credential rows, derived from the same catalogue the + * install wizard renders. + */ +function providerRows(provider: LlmProviderChoice): Row[] { + return providerSpec(provider).fields.map((f) => ({ + id: f.cliKey, + label: f.label, + cliKey: f.cliKey, + ...(f.mask === true ? { mask: true } : {}), + validate: (s: string) => (s.trim().length > 0 ? null : `required — ${f.hint}`), + })); +} + +function isLocalhost(s: string): boolean { + return s.includes("localhost") || s.includes("127.0.0.1"); +} + function loadInitial(): Record { + const currentMode = getInfraMode(); + const rawMongo = getConfigValue(Config.MongoUri); + const rawNeo4j = getConfigValue(Config.Neo4jUri); + const rawRedis = getConfigValue(Config.RedisUrl); + const rawNeo4jUser = getConfigValue(Config.Neo4jUser); + const rawNeo4jPwd = getConfigValue(Config.Neo4jPassword); + const isCloudMode = currentMode === "cloud"; + return { - mongo: getConfigValue(Config.MongoUri), - neo4j: getConfigValue(Config.Neo4jUri), - "neo4j-user": getConfigValue(Config.Neo4jUser), - "neo4j-password": getConfigValue(Config.Neo4jPassword), - redis: getConfigValue(Config.RedisUrl), + mongo: isCloudMode && isLocalhost(rawMongo) ? "" : rawMongo, + neo4j: isCloudMode && isLocalhost(rawNeo4j) ? "" : rawNeo4j, + "neo4j-user": isCloudMode && (rawNeo4jUser === "neo4j" || isLocalhost(rawNeo4j)) ? "" : rawNeo4jUser, + "neo4j-password": isCloudMode && isLocalhost(rawNeo4j) ? "" : rawNeo4jPwd, + redis: isCloudMode && isLocalhost(rawRedis) ? "" : rawRedis, port: String(getConfigValue(Config.ServerPort)), "concurrency-github": String(getConfigValue(Config.ConcurrencyGithub)), - "openrouter-api-key": getConfigValue(Config.OpenrouterApiKey), - "openrouter-model": getConfigValue(Config.OpenrouterModel), - "infra-mode": isEmbedded() ? "embedded" : "docker", + ...initialProviderValues(), + "llm-provider": getConfigValue(Config.LlmProvider), + "infra-mode": currentMode === "cloud" ? "cloud" : "docker", }; } @@ -111,9 +127,16 @@ export function SetupForm({ onDone }: SetupFormProps): ReactElement { const { exit } = useApp(); const [values, setValues] = useState>(() => loadInitial()); const [submitError, setSubmitError] = useState(null); + const [focusedIndex, setFocusedIndex] = useState(0); + + const infraMode = (values["infra-mode"] ?? "docker") as InfraMode; + const showInfra = infraMode === "cloud"; + const provider = (values["llm-provider"] ?? "openrouter") as LlmProviderChoice; + const spec = providerSpec(provider); + const visibleRows = [...ROWS.filter((r) => showInfra || r.infra !== true), ...providerRows(provider)]; - const isDocker = (values["infra-mode"] ?? "docker") === "docker"; - const visibleRows = ROWS.filter((r) => isDocker || r.infra !== true); + const focusableIds = ["infra-mode", "llm-provider", ...visibleRows.map((r) => r.id)]; + const activeIndex = Math.min(focusedIndex, focusableIds.length - 1); const errors: Record = {}; for (const row of visibleRows) { @@ -121,15 +144,62 @@ export function SetupForm({ onDone }: SetupFormProps): ReactElement { } const allValid = visibleRows.every((r) => errors[r.id] === null); + const handleInfraModeChange = (nextMode: string): void => { + setValues((prev) => { + const updated: Record = { ...prev, "infra-mode": nextMode }; + if (nextMode === "cloud") { + if (isLocalhost(prev["mongo"] ?? "")) { + updated["mongo"] = ""; + } + if (isLocalhost(prev["neo4j"] ?? "")) { + updated["neo4j"] = ""; + } + if (isLocalhost(prev["redis"] ?? "")) { + updated["redis"] = ""; + } + if (prev["neo4j-user"] === "neo4j" || isLocalhost(prev["neo4j"] ?? "")) { + updated["neo4j-user"] = ""; + } + if (isLocalhost(prev["neo4j"] ?? "")) { + updated["neo4j-password"] = ""; + } + } else if (nextMode === "docker") { + if (!updated["mongo"]) { + updated["mongo"] = "mongodb://127.0.0.1:27017/bytebell"; + } + if (!updated["neo4j"]) { + updated["neo4j"] = "bolt://127.0.0.1:7687"; + } + if (!updated["neo4j-user"]) { + updated["neo4j-user"] = "neo4j"; + } + if (!updated["redis"]) { + updated["redis"] = "redis://127.0.0.1:6379"; + } + } + return updated; + }); + }; + useInput((_input, key) => { if (key.escape) { exit(); onDone({ saved: false }); return; } + if (key.tab) { + const total = focusableIds.length; + setFocusedIndex((prev) => (key.shift ? (prev - 1 + total) % total : (prev + 1) % total)); + return; + } if (key.return && allValid && submitError === null) { try { - applyInfraMode((values["infra-mode"] ?? "docker") as InfraMode); + applyInfraMode(infraMode); + const providerEntry = KEY_MAP["llm-provider"]; + if (providerEntry === undefined) { + throw new Error('No KEY_MAP entry for "llm-provider"'); + } + providerEntry.setter(provider); for (const row of visibleRows) { const entry = KEY_MAP[row.cliKey]; if (entry === undefined) { @@ -155,12 +225,23 @@ export function SetupForm({ onDone }: SetupFormProps): ReactElement { label="Infrastructure" value={values["infra-mode"] ?? "docker"} options={MODE_OPTIONS} - onChange={(next) => setValues((prev) => ({ ...prev, "infra-mode": next }))} + onChange={handleInfraModeChange} + isFocused={activeIndex === 0} /> - {infraModeOption(isDocker ? "docker" : "embedded").hint} + {infraModeOption(infraMode).hint} - {visibleRows.map((row) => ( + setValues((prev) => ({ ...prev, "llm-provider": next }))} + hint={spec.hint} + isFocused={activeIndex === 1} + /> + + {visibleRows.map((row, idx) => ( setValues((prev) => ({ ...prev, [row.id]: next }))} {...(row.mask === true ? { mask: true } : {})} {...(errors[row.id] !== null ? { error: errors[row.id] ?? "" } : {})} + isFocused={activeIndex === 2 + idx} /> ))} diff --git a/packages/cli/src/ShutdownCommand.ts b/packages/cli/src/ShutdownCommand.ts index 68ab6c4..81c961b 100644 --- a/packages/cli/src/ShutdownCommand.ts +++ b/packages/cli/src/ShutdownCommand.ts @@ -4,7 +4,7 @@ import { DockerComposeError, DockerNotFoundError, composeFilePath, down } from " import { createSpinner, error } from "./output.ts"; import { promptStopDocker } from "./shutdownPrompts.ts"; import { stopServer } from "./serverLifecycle.ts"; -import { isEmbedded } from "./infraMode.ts"; +import { isCloud, isEmbedded } from "./infraMode.ts"; const STOP_TIMEOUT_S = 30; @@ -68,8 +68,8 @@ async function runShutdown(opts: ShutdownOptions): Promise { } async function decideStopDocker(opts: ShutdownOptions): Promise { - // Embedded mode runs no Docker — never prompt or try to tear it down. - if (isEmbedded()) { + // Embedded and cloud modes run no Docker — never prompt or try to tear it down. + if (isEmbedded() || isCloud()) { return false; } if (opts.withDocker === true) { @@ -104,8 +104,8 @@ async function stopDocker(): Promise { } function dockerHint(): string { - // No Docker in embedded mode — nothing to hint about. - if (isEmbedded()) { + // No Docker in embedded or cloud mode — nothing to hint about. + if (isEmbedded() || isCloud()) { return ""; } return `\nDocker infra is still running. To stop it:\n docker compose -f ${composeFilePath()} down\n`; diff --git a/packages/cli/src/ToggleField.tsx b/packages/cli/src/ToggleField.tsx index c43e8eb..f18b791 100644 --- a/packages/cli/src/ToggleField.tsx +++ b/packages/cli/src/ToggleField.tsx @@ -5,23 +5,45 @@ export interface ToggleFieldProps { id: string; label: string; value: string; - options: readonly [string, string]; + options: readonly string[]; onChange: (next: string) => void; + isFocused?: boolean; } /** - * A two-option switch that joins the form's Tab order via `useFocus`. When - * focused, ←/→/space flip between the two options. Distinct from the text - * `Field` so providers read as a toggle rather than free text. + * An option switch that joins the form's Tab order via `useFocus`. When + * focused, ←/→/space cycle between the options. Distinct from the text + * `Field` so options read as radio toggles rather than free text. */ -export function ToggleField({ id, label, value, options, onChange }: ToggleFieldProps): ReactElement { - const { isFocused } = useFocus({ id }); - const [a, b] = options; +export function ToggleField({ + id, + label, + value, + options, + onChange, + isFocused: propFocused, +}: ToggleFieldProps): ReactElement { + const { isFocused: hookFocused } = useFocus({ id }); + const isFocused = propFocused !== undefined ? propFocused : hookFocused; + const current = Math.max(0, options.indexOf(value)); useInput( (input, key) => { - if (key.leftArrow || key.rightArrow || input === " ") { - onChange(value === a ? b : a); + if (options.length === 0) { + return; + } + if (key.leftArrow) { + const next = options[(current - 1 + options.length) % options.length]; + if (next !== undefined) { + onChange(next); + } + return; + } + if (key.rightArrow || input === " ") { + const next = options[(current + 1) % options.length]; + if (next !== undefined) { + onChange(next); + } } }, { isActive: isFocused }, @@ -39,14 +61,17 @@ export function ToggleField({ id, label, value, options, onChange }: ToggleField {label} - - {value === a ? "◉" : "○"} {a} - - {" "} - - {value === b ? "◉" : "○"} {b} - - {isFocused && {" (←/→ to switch)"}} + {options.map((opt) => { + const isSelected = value === opt; + return ( + + + {isSelected ? "◉" : "○"} {opt} + + + ); + })} + {isFocused && {"(←/→ to switch)"}} ); diff --git a/packages/cli/src/bootConfig.ts b/packages/cli/src/bootConfig.ts index df68b8e..b5bb745 100644 --- a/packages/cli/src/bootConfig.ts +++ b/packages/cli/src/bootConfig.ts @@ -6,7 +6,7 @@ import { getBytebellHome, getConfigValue, requiredKeysFor } from "@bb/config"; import { bringInfraUp } from "./dockerBoot.ts"; import { KEY_MAP } from "./keyMap.ts"; import { success, error, info } from "./output.ts"; -import { isEmbedded } from "./infraMode.ts"; +import { isCloud, isEmbedded } from "./infraMode.ts"; import { startServer } from "./serverLifecycle.ts"; const DEFAULT_MONGO_URI = "mongodb://127.0.0.1:27017/bytebell"; @@ -77,6 +77,12 @@ export interface ApplyDefaultsResult { export function applyInfraDefaults(): ApplyDefaultsResult { const written: { cliKey: string; redacted: boolean }[] = []; + if (isCloud()) { + return { + written, + neo4jPassword: readString(Config.Neo4jPassword), + }; + } for (const entry of DEFAULTS) { if (!entry.needed()) { continue; @@ -136,10 +142,12 @@ export async function runBootSequence(): Promise { } } - // Embedded mode (sqlite + ladybug + honker) needs no external services — skip - // Docker entirely and go straight to starting the server. + // Embedded mode (sqlite + ladybug + honker) and Cloud mode (remote databases) + // need no local Docker services — skip Docker entirely and start the server. if (isEmbedded()) { info("embedded mode — no Docker required (sqlite + ladybug + honker)."); + } else if (isCloud()) { + info("cloud mode — using external/cloud databases (Mongo + Neo4j + Redis), no Docker required."); } else { if (getConfigValue(Config.GraphProvider) === GraphProviderType.Neo4j && defaults.neo4jPassword.length === 0) { error("internal: neo4j password is empty after applyInfraDefaults — refusing to start docker."); diff --git a/packages/cli/src/infraMode.test.ts b/packages/cli/src/infraMode.test.ts new file mode 100644 index 0000000..3b6a7b7 --- /dev/null +++ b/packages/cli/src/infraMode.test.ts @@ -0,0 +1,54 @@ +// SPDX-License-Identifier: AGPL-3.0-only WITH non-commercial-clause +import { describe, it, expect } from "bun:test"; +import { applyInfraMode, getInfraMode, isCloud, isEmbedded, needsDocker, INFRA_MODE_OPTIONS } from "./infraMode.ts"; +import { Config, DbProviderType, GraphProviderType, QueueProviderType } from "@bb/types"; +import { getConfigValue, setConfigValue } from "@bb/config"; + +describe("infraMode", () => { + it("includes cloud and docker in active INFRA_MODE_OPTIONS", () => { + const modes = INFRA_MODE_OPTIONS.map((o) => o.value); + expect(modes).toContain("cloud"); + expect(modes).toContain("docker"); + }); + + it("applies embedded preset correctly", () => { + applyInfraMode("embedded"); + expect(getInfraMode()).toBe("embedded"); + expect(isEmbedded()).toBe(true); + expect(isCloud()).toBe(false); + expect(needsDocker()).toBe(false); + expect(getConfigValue(Config.DbProvider)).toBe(DbProviderType.Sqlite); + expect(getConfigValue(Config.GraphProvider)).toBe(GraphProviderType.Ladybug); + expect(getConfigValue(Config.QueueProvider)).toBe(QueueProviderType.Honker); + }); + + it("applies cloud preset correctly without requiring docker", () => { + applyInfraMode("cloud"); + expect(getInfraMode()).toBe("cloud"); + expect(isEmbedded()).toBe(false); + expect(isCloud()).toBe(true); + expect(needsDocker()).toBe(false); + expect(getConfigValue(Config.DbProvider)).toBe(DbProviderType.Mongo); + expect(getConfigValue(Config.GraphProvider)).toBe(GraphProviderType.Neo4j); + expect(getConfigValue(Config.QueueProvider)).toBe(QueueProviderType.Bullmq); + }); + + it("applies docker preset correctly and requires docker", () => { + setConfigValue(Config.MongoUri, ""); + setConfigValue(Config.Neo4jUri, ""); + setConfigValue(Config.Neo4jUser, ""); + setConfigValue(Config.RedisUrl, ""); + applyInfraMode("docker"); + expect(getInfraMode()).toBe("docker"); + expect(isEmbedded()).toBe(false); + expect(isCloud()).toBe(false); + expect(needsDocker()).toBe(true); + expect(getConfigValue(Config.DbProvider)).toBe(DbProviderType.Mongo); + expect(getConfigValue(Config.GraphProvider)).toBe(GraphProviderType.Neo4j); + expect(getConfigValue(Config.QueueProvider)).toBe(QueueProviderType.Bullmq); + expect(getConfigValue(Config.MongoUri)).toBe("mongodb://127.0.0.1:27017/bytebell"); + expect(getConfigValue(Config.Neo4jUri)).toBe("bolt://127.0.0.1:7687"); + expect(getConfigValue(Config.Neo4jUser)).toBe("neo4j"); + expect(getConfigValue(Config.RedisUrl)).toBe("redis://127.0.0.1:6379"); + }); +}); diff --git a/packages/cli/src/infraMode.ts b/packages/cli/src/infraMode.ts index c360284..598aa36 100644 --- a/packages/cli/src/infraMode.ts +++ b/packages/cli/src/infraMode.ts @@ -4,17 +4,13 @@ import { Config, DbProviderType, GraphProviderType, QueueProviderType } from "@b import { getBytebellHome, getConfigValue, setConfigValue } from "@bb/config"; /** - * Infrastructure mode is not a stored flag — it's derived from the three - * provider settings. There are two coherent presets: + * Infrastructure mode defines how ByteBell runs its databases: * - * • "docker" (non-embedded) — Mongo + Neo4j + BullMQ. Requires Docker. - * • "embedded" — SQLite + Ladybug + Honker. Zero Docker. - * - * The providers remain the single source of truth; `mode` is a convenience the - * setup surfaces use to set all three at once and to decide whether `boot` - * should bring Docker up. + * • "embedded" — SQLite + Ladybug + Honker. Zero Docker. + * • "cloud" — Mongo + Neo4j + Redis hosted in the cloud (Atlas, Neo4j Aura, etc.). Zero Docker. + * • "docker" — Mongo + Neo4j + Redis local instances. Requires Docker. */ -export type InfraMode = "docker" | "embedded"; +export type InfraMode = "docker" | "cloud" | "embedded"; export interface InfraModeOption { value: InfraMode; @@ -23,21 +19,27 @@ export interface InfraModeOption { } /** - * UI metadata for the two infra presets, recommended preset first. This is the + * UI metadata for the infra presets, recommended preset first. This is the * single source for the labels/hints shown by the install wizard and the `set` * setup form — keep mode descriptions here, not inlined per surface. */ export const INFRA_MODE_OPTIONS: readonly InfraModeOption[] = [ - { - value: "embedded", - label: "Embedded (recommended)", - hint: "SQLite + Ladybug + Honker — no Docker, everything in local files under ~/.bytebell", - }, { value: "docker", label: "Docker", - hint: "Mongo + Neo4j + Redis — Docker needed (Docker Desktop/engine must be running)", + hint: "Mongo + Neo4j + Redis in local Docker — if instances are not running, run 'bytebell boot' to start them (Docker Desktop required)", }, + { + value: "cloud", + label: "Cloud", + hint: "Mongo + Neo4j + Redis in the cloud — provide your cloud instance URLs below (zero Docker)", + }, + // Embedded mode temporarily disabled from TUI + // { + // value: "embedded", + // label: "Embedded (recommended)", + // hint: "SQLite + Ladybug + Honker — no Docker, everything in local files under ~/.bytebell", + // }, ]; /** UI metadata for a single infra mode (falls back to the recommended preset). */ @@ -47,7 +49,7 @@ export function infraModeOption(mode: InfraMode): InfraModeOption { return option; } } - return INFRA_MODE_OPTIONS[0] ?? { value: "embedded", label: "Embedded", hint: "" }; + return INFRA_MODE_OPTIONS[0] ?? { value: "docker", label: "Docker", hint: "" }; } interface ProviderTriple { @@ -62,6 +64,12 @@ export const DOCKER_PROVIDERS: ProviderTriple = { queue: QueueProviderType.Bullmq, }; +export const CLOUD_PROVIDERS: ProviderTriple = { + db: DbProviderType.Mongo, + graph: GraphProviderType.Neo4j, + queue: QueueProviderType.Bullmq, +}; + export const EMBEDDED_PROVIDERS: ProviderTriple = { db: DbProviderType.Sqlite, graph: GraphProviderType.Ladybug, @@ -72,10 +80,13 @@ export type ComposeService = "mongo" | "neo4j" | "redis"; /** * The Docker compose services the current provider combo requires. Empty when - * every provider is file-based (embedded mode). + * every provider is file-based (embedded mode) or cloud-hosted. */ export function composeServicesNeeded(): Set { const needed = new Set(); + if (isCloud() || isEmbedded()) { + return needed; + } if (getConfigValue(Config.DbProvider) === DbProviderType.Mongo) { needed.add("mongo"); } @@ -88,14 +99,35 @@ export function composeServicesNeeded(): Set { return needed; } -/** True when at least one provider needs a Docker container. */ -export function needsDocker(): boolean { - return composeServicesNeeded().size > 0; +/** Get the currently configured infra mode (or derive from provider settings). */ +export function getInfraMode(): InfraMode { + const stored = getConfigValue(Config.InfraMode); + if (stored === "cloud" || stored === "docker" || stored === "embedded") { + return stored; + } + if ( + getConfigValue(Config.DbProvider) === DbProviderType.Sqlite && + getConfigValue(Config.GraphProvider) === GraphProviderType.Ladybug && + getConfigValue(Config.QueueProvider) === QueueProviderType.Honker + ) { + return "embedded"; + } + return "docker"; +} + +/** True when the active infra mode is cloud-hosted external instances (no Docker). */ +export function isCloud(): boolean { + return getInfraMode() === "cloud"; } /** True when the active provider combo is fully file-based (no Docker). */ export function isEmbedded(): boolean { - return !needsDocker(); + return getInfraMode() === "embedded"; +} + +/** True when at least one provider needs a Docker container. */ +export function needsDocker(): boolean { + return !isCloud() && !isEmbedded() && composeServicesNeeded().size > 0; } /** @@ -109,20 +141,34 @@ const EMBEDDED_PATH_DEFAULTS: ReadonlyArray = [ [Config.QueueDbPath, "queue.db"], ]; -/** Apply one of the two presets to the three provider config keys. */ +const DOCKER_DEFAULTS: ReadonlyArray = [ + [Config.MongoUri, "mongodb://127.0.0.1:27017/bytebell"], + [Config.Neo4jUri, "bolt://127.0.0.1:7687"], + [Config.Neo4jUser, "neo4j"], + [Config.RedisUrl, "redis://127.0.0.1:6379"], +]; + +/** Apply one of the presets to the provider config keys. */ export function applyInfraMode(mode: InfraMode): void { - const providers = mode === "embedded" ? EMBEDDED_PROVIDERS : DOCKER_PROVIDERS; + setConfigValue(Config.InfraMode, mode); + const providers = mode === "embedded" ? EMBEDDED_PROVIDERS : mode === "cloud" ? CLOUD_PROVIDERS : DOCKER_PROVIDERS; setConfigValue(Config.DbProvider, providers.db); setConfigValue(Config.GraphProvider, providers.graph); setConfigValue(Config.QueueProvider, providers.queue); - if (mode !== "embedded") { - return; - } - const home = getBytebellHome(); - for (const [key, filename] of EMBEDDED_PATH_DEFAULTS) { - const current = getConfigValue(key); - if (typeof current === "string" && current.length === 0) { - setConfigValue(key, path.join(home, filename)); + if (mode === "embedded") { + const home = getBytebellHome(); + for (const [key, filename] of EMBEDDED_PATH_DEFAULTS) { + const current = getConfigValue(key); + if (typeof current === "string" && current.length === 0) { + setConfigValue(key, path.join(home, filename)); + } + } + } else if (mode === "docker") { + for (const [key, def] of DOCKER_DEFAULTS) { + const current = getConfigValue(key); + if (typeof current === "string" && current.length === 0) { + setConfigValue(key, def); + } } } } diff --git a/packages/cli/src/keyMap.ts b/packages/cli/src/keyMap.ts index 5481c9f..c69cc51 100644 --- a/packages/cli/src/keyMap.ts +++ b/packages/cli/src/keyMap.ts @@ -1,4 +1,12 @@ -import { LLM_PROVIDERS, LOG_LEVELS, setConfigValue, type LlmProvider, type LogLevel } from "@bb/config"; +import { + LLM_PROVIDERS, + LOG_LEVELS, + INFRA_MODES, + setConfigValue, + type LlmProvider, + type LogLevel, + type InfraMode, +} from "@bb/config"; import { Config, DbProviderType, GraphProviderType, IngestionStrategyType, QueueProviderType } from "@bb/types"; type Setter = (raw: string) => void; @@ -167,6 +175,71 @@ export const KEY_MAP: Record = { redact: false, setter: (s) => setConfigValue(Config.OllamaModel, s), }, + "anthropic-api-key": { + configKey: Config.AnthropicApiKey, + redact: true, + setter: (s) => setConfigValue(Config.AnthropicApiKey, s), + }, + "anthropic-model": { + configKey: Config.AnthropicModel, + redact: false, + setter: (s) => setConfigValue(Config.AnthropicModel, s), + }, + "bedrock-api-key": { + configKey: Config.BedrockApiKey, + redact: true, + setter: (s) => setConfigValue(Config.BedrockApiKey, s), + }, + "bedrock-region": { + configKey: Config.BedrockRegion, + redact: false, + setter: (s) => setConfigValue(Config.BedrockRegion, s), + }, + "bedrock-model": { + configKey: Config.BedrockModel, + redact: false, + setter: (s) => setConfigValue(Config.BedrockModel, s), + }, + "gemini-api-key": { + configKey: Config.GeminiApiKey, + redact: true, + setter: (s) => setConfigValue(Config.GeminiApiKey, s), + }, + "gemini-model": { + configKey: Config.GeminiModel, + redact: false, + setter: (s) => setConfigValue(Config.GeminiModel, s), + }, + "openai-api-key": { + configKey: Config.OpenaiApiKey, + redact: true, + setter: (s) => setConfigValue(Config.OpenaiApiKey, s), + }, + "openai-model": { + configKey: Config.OpenaiModel, + redact: false, + setter: (s) => setConfigValue(Config.OpenaiModel, s), + }, + "openai-base-url": { + configKey: Config.OpenaiBaseUrl, + redact: false, + setter: (s) => setConfigValue(Config.OpenaiBaseUrl, s), + }, + "aws-access-key-id": { + configKey: Config.AwsAccessKeyId, + redact: true, + setter: (s) => setConfigValue(Config.AwsAccessKeyId, s), + }, + "aws-secret-access-key": { + configKey: Config.AwsSecretAccessKey, + redact: true, + setter: (s) => setConfigValue(Config.AwsSecretAccessKey, s), + }, + "aws-session-token": { + configKey: Config.AwsSessionToken, + redact: true, + setter: (s) => setConfigValue(Config.AwsSessionToken, s), + }, "db-provider": { configKey: Config.DbProvider, redact: false, @@ -210,6 +283,12 @@ export const KEY_MAP: Record = { redact: false, setter: (s) => setConfigValue(Config.LadybugPath, s), }, + "infra-mode": { + configKey: Config.InfraMode, + redact: false, + setter: (s) => setConfigValue(Config.InfraMode, parseEnum(s, "infra-mode", INFRA_MODES) as InfraMode), + toggleValues: ["docker", "cloud"], + }, }; export function validKeysList(): string[] { diff --git a/packages/cli/src/llmProviders.ts b/packages/cli/src/llmProviders.ts new file mode 100644 index 0000000..f272dda --- /dev/null +++ b/packages/cli/src/llmProviders.ts @@ -0,0 +1,182 @@ +// SPDX-License-Identifier: AGPL-3.0-only WITH non-commercial-clause +import { Config } from "@bb/types"; +import { getConfigValue } from "@bb/config"; + +// ───────────────────────────────────────────────────────────────────────────── +// Single source of truth for the LLM backends offered at signup. The wizard +// renders from this table and `SetupCommand` writes from it, so adding a +// provider is one entry here — no branching in the Ink components and no +// second list to keep in sync. +// ───────────────────────────────────────────────────────────────────────────── + +export type LlmProviderChoice = "openrouter" | "ollama" | "anthropic" | "bedrock" | "gemini" | "openai"; + +/** + * The string-valued config keys a provider field may bind to. Narrower than + * `Config` so `getConfigValue` returns `string` rather than the full value + * union (which includes numbers and booleans). + */ +export type ProviderConfigKey = + | Config.OpenrouterApiKey + | Config.OpenrouterModel + | Config.AnthropicApiKey + | Config.AnthropicModel + | Config.BedrockApiKey + | Config.BedrockRegion + | Config.BedrockModel + | Config.GeminiApiKey + | Config.GeminiModel + | Config.OpenaiApiKey + | Config.OpenaiModel + | Config.OpenaiBaseUrl + | Config.OllamaUrl + | Config.OllamaModel; + +export interface ProviderField { + /** Field id, also the `KEY_MAP` key used to persist it. */ + cliKey: string; + label: string; + /** Config key the wizard pre-fills from (current value, else schema default). */ + configKey: ProviderConfigKey; + mask?: boolean; + /** Blank is never valid — every listed field is required by its provider. */ + hint: string; +} + +export interface ProviderSpec { + value: LlmProviderChoice; + label: string; + hint: string; + fields: readonly ProviderField[]; + /** Whether this backend can drive `askLLMWithTools` (concept-graph strategy). */ + supportsTools: boolean; +} + +export const LLM_PROVIDER_SPECS: readonly ProviderSpec[] = [ + { + value: "openrouter", + label: "OpenRouter", + hint: "API key required — openrouter.ai/keys · reports real cost · supports all strategies", + supportsTools: true, + fields: [ + { + cliKey: "openrouter-api-key", + label: "API key", + configKey: Config.OpenrouterApiKey, + mask: true, + hint: "sk-or-v1-…", + }, + { + cliKey: "openrouter-model", + label: "Model", + configKey: Config.OpenrouterModel, + hint: "e.g. anthropic/claude-sonnet-5", + }, + ], + }, + { + value: "anthropic", + label: "Anthropic", + hint: "API key required — console.anthropic.com · Claude models direct", + supportsTools: true, + fields: [ + { + cliKey: "anthropic-api-key", + label: "API key", + configKey: Config.AnthropicApiKey, + mask: true, + hint: "sk-ant-…", + }, + { cliKey: "anthropic-model", label: "Model", configKey: Config.AnthropicModel, hint: "e.g. claude-sonnet-5" }, + ], + }, + { + value: "gemini", + label: "Google Gemini", + hint: "API key required — aistudio.google.com/apikey", + supportsTools: true, + fields: [ + { cliKey: "gemini-api-key", label: "API key", configKey: Config.GeminiApiKey, mask: true, hint: "AIza…" }, + { cliKey: "gemini-model", label: "Model", configKey: Config.GeminiModel, hint: "e.g. gemini-2.5-flash" }, + ], + }, + { + value: "openai", + label: "OpenAI / compatible", + hint: "API key required — platform.openai.com · or point base URL at vLLM / LiteLLM / a gateway", + supportsTools: true, + fields: [ + { cliKey: "openai-api-key", label: "API key", configKey: Config.OpenaiApiKey, mask: true, hint: "sk-…" }, + { + cliKey: "openai-model", + label: "Model", + configKey: Config.OpenaiModel, + hint: "exact model id from your provider", + }, + ], + }, + { + value: "bedrock", + label: "AWS Bedrock", + hint: "API key or AWS IAM/instance role · any Bedrock model · billed to your AWS account", + supportsTools: true, + fields: [ + { + cliKey: "bedrock-api-key", + label: "Bedrock API key", + configKey: Config.BedrockApiKey, + mask: true, + hint: "from the Bedrock console", + }, + { cliKey: "bedrock-region", label: "Region", configKey: Config.BedrockRegion, hint: "e.g. us-east-1" }, + { + cliKey: "bedrock-model", + label: "Model id", + configKey: Config.BedrockModel, + // Any Bedrock family works (Converse). Ids are versioned and + // region-dependent, so copy the exact one from the console. + hint: "any family — copy the exact id/ARN from the Bedrock console", + }, + ], + }, + { + value: "ollama", + label: "Ollama", + hint: "local, free, no key — daemon must already be running", + supportsTools: false, + fields: [ + { cliKey: "ollama-url", label: "Ollama URL", configKey: Config.OllamaUrl, hint: "http://localhost:11434" }, + { cliKey: "ollama-model", label: "Model name", configKey: Config.OllamaModel, hint: "e.g. qwen2.5-coder:7b" }, + ], + }, +]; + +export function providerSpec(value: LlmProviderChoice): ProviderSpec { + const found = LLM_PROVIDER_SPECS.find((p) => p.value === value); + if (found === undefined) { + throw new Error(`internal: no provider spec for "${value}"`); + } + return found; +} + +/** Pre-fill every field of every provider from the current config. */ +export function initialProviderValues(): Record { + const out: Record = {}; + for (const spec of LLM_PROVIDER_SPECS) { + for (const field of spec.fields) { + out[field.cliKey] = getConfigValue(field.configKey); + } + } + return out; +} + +export function providerFieldsValid(spec: ProviderSpec, values: Record): boolean { + return spec.fields.every((f) => (values[f.cliKey] ?? "").trim().length > 0); +} + +export function maskSecret(raw: string): string { + if (raw.length === 0) { + return "(none)"; + } + return `${"•".repeat(Math.min(raw.length, 8))}${raw.length > 8 ? "…" : ""}`; +} diff --git a/packages/cli/src/serverSpawn.ts b/packages/cli/src/serverSpawn.ts index 32f1c2a..0e8e145 100644 --- a/packages/cli/src/serverSpawn.ts +++ b/packages/cli/src/serverSpawn.ts @@ -40,7 +40,12 @@ async function tcpReachable(host: string, port: number): Promise { function parseHostPort(uri: string): { host: string; port: number } | null { try { const u = new URL(uri); - const defaultPort = u.protocol === "bolt:" ? 7687 : u.protocol === "redis:" ? 6379 : 27017; + const defaultPort = + u.protocol === "bolt:" || u.protocol === "neo4j:" || u.protocol === "neo4j+s:" + ? 7687 + : u.protocol === "redis:" + ? 6379 + : 27017; const port = u.port !== "" ? Number.parseInt(u.port, 10) : defaultPort; return { host: u.hostname || "127.0.0.1", port }; } catch { diff --git a/packages/config/src/index.ts b/packages/config/src/index.ts index c710ff9..9173292 100644 --- a/packages/config/src/index.ts +++ b/packages/config/src/index.ts @@ -1,5 +1,5 @@ -export { LOG_LEVELS, LLM_PROVIDERS, HINTS, requiredKeysFor } from "./schema.ts"; -export type { BytebellConfig, ConfigValue, ConfigValueMap, LogLevel, LlmProvider } from "./schema.ts"; +export { LOG_LEVELS, LLM_PROVIDERS, INFRA_MODES, HINTS, requiredKeysFor } from "./schema.ts"; +export type { BytebellConfig, ConfigValue, ConfigValueMap, LogLevel, LlmProvider, InfraMode } from "./schema.ts"; export { loadConfig, getConfigValue, isConfigComplete, seedConfig, __isSeeded, __resetSeedForTests } from "./loader.ts"; export type { ConfigCompletenessResult } from "./loader.ts"; diff --git a/packages/config/src/schema-fields.ts b/packages/config/src/schema-fields.ts index ef2b013..0649616 100644 --- a/packages/config/src/schema-fields.ts +++ b/packages/config/src/schema-fields.ts @@ -1,7 +1,7 @@ import type { BytebellConfig } from "./schema.ts"; import { Config } from "@bb/types"; import type { ConfigValue } from "./schema.ts"; -import type { LogLevel, LlmProvider, IngestionStrategy } from "./schema.ts"; +import type { LogLevel, LlmProvider, IngestionStrategy, InfraMode } from "./schema.ts"; export function readField(cfg: BytebellConfig, key: K): ConfigValue { switch (key) { @@ -43,6 +43,32 @@ export function readField(cfg: BytebellConfig, key: K): Config return cfg.ollama_url as ConfigValue; case Config.OllamaModel: return cfg.ollama_model as ConfigValue; + case Config.AnthropicApiKey: + return cfg.anthropic_api_key as ConfigValue; + case Config.AnthropicModel: + return cfg.anthropic_model as ConfigValue; + case Config.BedrockApiKey: + return cfg.bedrock_api_key as ConfigValue; + case Config.BedrockRegion: + return cfg.bedrock_region as ConfigValue; + case Config.BedrockModel: + return cfg.bedrock_model as ConfigValue; + case Config.GeminiApiKey: + return cfg.gemini_api_key as ConfigValue; + case Config.GeminiModel: + return cfg.gemini_model as ConfigValue; + case Config.OpenaiApiKey: + return cfg.openai_api_key as ConfigValue; + case Config.OpenaiModel: + return cfg.openai_model as ConfigValue; + case Config.OpenaiBaseUrl: + return cfg.openai_base_url as ConfigValue; + case Config.AwsAccessKeyId: + return cfg.aws_access_key_id as ConfigValue; + case Config.AwsSecretAccessKey: + return cfg.aws_secret_access_key as ConfigValue; + case Config.AwsSessionToken: + return cfg.aws_session_token as ConfigValue; case Config.ContextWindowLimit: return cfg["context.window.limit"] as ConfigValue; case Config.MaxTokensPerChunk: @@ -83,6 +109,8 @@ export function readField(cfg: BytebellConfig, key: K): Config return cfg.graph_provider as ConfigValue; case Config.QueueProvider: return cfg.queue_provider as ConfigValue; + case Config.InfraMode: + return cfg.infra_mode as ConfigValue; case Config.QueueDbPath: return cfg.queue_db_path as ConfigValue; case Config.SqlitePath: @@ -154,6 +182,32 @@ export function writeField(cfg: BytebellConfig, key: K, value: return { ...cfg, ollama_url: value as string }; case Config.OllamaModel: return { ...cfg, ollama_model: value as string }; + case Config.AnthropicApiKey: + return { ...cfg, anthropic_api_key: value as string }; + case Config.AnthropicModel: + return { ...cfg, anthropic_model: value as string }; + case Config.BedrockApiKey: + return { ...cfg, bedrock_api_key: value as string }; + case Config.BedrockRegion: + return { ...cfg, bedrock_region: value as string }; + case Config.BedrockModel: + return { ...cfg, bedrock_model: value as string }; + case Config.GeminiApiKey: + return { ...cfg, gemini_api_key: value as string }; + case Config.GeminiModel: + return { ...cfg, gemini_model: value as string }; + case Config.OpenaiApiKey: + return { ...cfg, openai_api_key: value as string }; + case Config.OpenaiModel: + return { ...cfg, openai_model: value as string }; + case Config.OpenaiBaseUrl: + return { ...cfg, openai_base_url: value as string }; + case Config.AwsAccessKeyId: + return { ...cfg, aws_access_key_id: value as string }; + case Config.AwsSecretAccessKey: + return { ...cfg, aws_secret_access_key: value as string }; + case Config.AwsSessionToken: + return { ...cfg, aws_session_token: value as string }; case Config.ContextWindowLimit: return { ...cfg, "context.window.limit": value as number }; case Config.MaxTokensPerChunk: @@ -194,6 +248,8 @@ export function writeField(cfg: BytebellConfig, key: K, value: return { ...cfg, graph_provider: value as string }; case Config.QueueProvider: return { ...cfg, queue_provider: value as string }; + case Config.InfraMode: + return { ...cfg, infra_mode: value as InfraMode }; case Config.QueueDbPath: return { ...cfg, queue_db_path: value as string }; case Config.SqlitePath: diff --git a/packages/config/src/schema.ts b/packages/config/src/schema.ts index 19f1c49..1180760 100644 --- a/packages/config/src/schema.ts +++ b/packages/config/src/schema.ts @@ -6,9 +6,12 @@ export { Config }; export const LOG_LEVELS = ["error", "warn", "info", "http", "verbose", "debug", "silly"] as const; export type LogLevel = (typeof LOG_LEVELS)[number]; -export const LLM_PROVIDERS = ["openrouter", "ollama"] as const; +export const LLM_PROVIDERS = ["openrouter", "ollama", "anthropic", "bedrock", "gemini", "openai"] as const; export type LlmProvider = (typeof LLM_PROVIDERS)[number]; +export const INFRA_MODES = ["docker", "cloud", "embedded"] as const; +export type InfraMode = (typeof INFRA_MODES)[number]; + // The PUBLIC strategies the open-source engine ships. A downstream deployment // may set `ingestion.strategy` to a private strategy name this list does not // enumerate, so the stored config value is a free string (validated below). @@ -42,6 +45,22 @@ export const configSchema = z llm_provider: z.enum(LLM_PROVIDERS).default("openrouter"), ollama_url: z.string().default("http://localhost:11434"), ollama_model: z.string().default(""), + anthropic_api_key: z.string().default(""), + anthropic_model: z.string().default("claude-sonnet-5"), + bedrock_api_key: z.string().default(""), + // No default. The region is an endpoint locator, not a credential — a + // wrong one fails as "model not found" rather than "misconfigured", so an + // unset region is caught by the boot gate with a precise hint instead. + bedrock_region: z.string().default(""), + bedrock_model: z.string().default("anthropic.claude-sonnet-5"), + gemini_api_key: z.string().default(""), + gemini_model: z.string().default("gemini-2.5-flash"), + openai_api_key: z.string().default(""), + openai_model: z.string().default(""), + openai_base_url: z.string().default(""), + aws_access_key_id: z.string().default(""), + aws_secret_access_key: z.string().default(""), + aws_session_token: z.string().default(""), "context.window.limit": z.number().int().positive().default(15000), "max.tokens.per.chunk": z.number().int().positive().default(6000), "big.file.concurrency": z.number().int().positive().default(25), @@ -62,6 +81,7 @@ export const configSchema = z db_provider: z.string().default("mongo"), graph_provider: z.string().default("neo4j"), queue_provider: z.string().default("bullmq"), + infra_mode: z.enum(INFRA_MODES).default("docker"), queue_db_path: z.string().default(""), sqlite_path: z.string().default(""), ladybug_path: z.string().default(""), @@ -107,6 +127,19 @@ export type ConfigValueMap = { [Config.LlmProvider]: LlmProvider; [Config.OllamaUrl]: string; [Config.OllamaModel]: string; + [Config.AnthropicApiKey]: string; + [Config.AnthropicModel]: string; + [Config.BedrockApiKey]: string; + [Config.BedrockRegion]: string; + [Config.BedrockModel]: string; + [Config.GeminiApiKey]: string; + [Config.GeminiModel]: string; + [Config.OpenaiApiKey]: string; + [Config.OpenaiModel]: string; + [Config.OpenaiBaseUrl]: string; + [Config.AwsAccessKeyId]: string; + [Config.AwsSecretAccessKey]: string; + [Config.AwsSessionToken]: string; [Config.ContextWindowLimit]: number; [Config.MaxTokensPerChunk]: number; [Config.BigFileConcurrency]: number; @@ -127,6 +160,7 @@ export type ConfigValueMap = { [Config.DbProvider]: string; [Config.GraphProvider]: string; [Config.QueueProvider]: string; + [Config.InfraMode]: InfraMode; [Config.QueueDbPath]: string; [Config.SqlitePath]: string; [Config.LadybugPath]: string; @@ -155,6 +189,17 @@ export const REQUIRED_KEYS: readonly Config[] = [ const PROVIDER_REQUIRED_KEYS: Readonly> = { openrouter: [Config.OpenrouterApiKey], ollama: [Config.OllamaUrl, Config.OllamaModel], + anthropic: [Config.AnthropicApiKey, Config.AnthropicModel], + // Bedrock takes EITHER a bearer API key OR SigV4 credentials OR an ambient + // instance role, so no single credential key is universally required — the + // provider fails with a precise hint when none resolves. Region and model are + // always needed. + bedrock: [Config.BedrockRegion, Config.BedrockModel], + gemini: [Config.GeminiApiKey, Config.GeminiModel], + // Bedrock accepts either a bearer API key or SigV4 credentials, so neither is + // individually required — `resolveBedrockAuth` fails with a precise hint when + // both are absent. Region is always needed. + openai: [Config.OpenaiApiKey, Config.OpenaiModel], }; export function requiredKeysFor(provider: LlmProvider): readonly Config[] { @@ -168,7 +213,7 @@ export const HINTS: Readonly> = { [Config.Neo4jUser]: "bytebell set neo4j-user ", [Config.Neo4jPassword]: "bytebell set neo4j-password ", [Config.RedisUrl]: "bytebell set redis ", - [Config.OpenrouterApiKey]: "bytebell keys set", + [Config.OpenrouterApiKey]: "bytebell set openrouter-api-key ", [Config.OpenrouterModel]: "bytebell models set ", [Config.OpenrouterFallbackModel1]: "bytebell set openrouter-fallback-model-1 ", [Config.OpenrouterFallbackModel2]: "bytebell set openrouter-fallback-model-2 ", @@ -178,9 +223,22 @@ export const HINTS: Readonly> = { [Config.LogLevel]: "bytebell set log-level ", [Config.LogRetentionDays]: "bytebell set log-retention-days ", [Config.LlmCacheEnabled]: "bytebell set llm_cache_enabled ", - [Config.LlmProvider]: "bytebell set llm-provider ", + [Config.LlmProvider]: "bytebell set llm-provider ", [Config.OllamaUrl]: "bytebell set ollama-url ", [Config.OllamaModel]: "bytebell set ollama-model ", + [Config.AnthropicApiKey]: "bytebell set anthropic-api-key ", + [Config.AnthropicModel]: "bytebell set anthropic-model ", + [Config.BedrockApiKey]: "bytebell set bedrock-api-key ", + [Config.BedrockRegion]: "bytebell set bedrock-region ", + [Config.BedrockModel]: "bytebell set bedrock-model ", + [Config.GeminiApiKey]: "bytebell set gemini-api-key ", + [Config.GeminiModel]: "bytebell set gemini-model ", + [Config.OpenaiApiKey]: "bytebell set openai-api-key ", + [Config.OpenaiModel]: "bytebell set openai-model ", + [Config.OpenaiBaseUrl]: "bytebell set openai-base-url ", + [Config.AwsAccessKeyId]: "bytebell set aws-access-key-id ", + [Config.AwsSecretAccessKey]: "bytebell set aws-secret-access-key ", + [Config.AwsSessionToken]: "bytebell set aws-session-token ", [Config.ContextWindowLimit]: "bytebell set context.window.limit ", [Config.MaxTokensPerChunk]: "bytebell set max.tokens.per.chunk ", [Config.BigFileConcurrency]: "bytebell set big.file.concurrency ", @@ -201,6 +259,7 @@ export const HINTS: Readonly> = { [Config.DbProvider]: "bytebell set db-provider ", [Config.GraphProvider]: "bytebell set graph-provider ", [Config.QueueProvider]: "bytebell set queue-provider ", + [Config.InfraMode]: "bytebell set infra-mode ", [Config.QueueDbPath]: "bytebell set queue-db-path ", [Config.SqlitePath]: "bytebell set sqlite-path ", [Config.LadybugPath]: "bytebell set ladybug-path ", diff --git a/packages/errors/src/llm-errors.ts b/packages/errors/src/llm-errors.ts index bc5c00c..ade253e 100644 --- a/packages/errors/src/llm-errors.ts +++ b/packages/errors/src/llm-errors.ts @@ -2,8 +2,15 @@ export class LlmConfigError extends Error { override readonly name = "LlmConfigError"; readonly hint: string; - constructor(hint: string) { - super(`OpenRouter API key is not configured. Run:\n ${hint}`); + /** + * `hint` is the exact `bytebell set …` command that fixes the problem. + * + * The summary is deliberately provider-neutral: this used to hardcode + * "OpenRouter API key is not configured", so a missing Gemini key or Bedrock + * region reported a provider the operator had never selected. + */ + constructor(hint: string, summary = "LLM provider is not fully configured") { + super(`${summary}. Run:\n ${hint}`); this.hint = hint; } } diff --git a/packages/ingest-strategies/src/concept-graph/VERIFY.md b/packages/ingest-strategies/src/concept-graph/VERIFY.md index 9cae218..d9deada 100644 --- a/packages/ingest-strategies/src/concept-graph/VERIFY.md +++ b/packages/ingest-strategies/src/concept-graph/VERIFY.md @@ -12,7 +12,7 @@ default. ## Prerequisites - `bytebell-server` running locally with Mongo, Neo4j, and Redis reachable -- An OpenRouter API key configured (`bytebell keys set`) +- A tool-capable provider configured (e.g. `bytebell set openrouter-api-key `) - A tool-use-capable enrichment model selected (Anthropic Claude Sonnet 4.x / Opus 4.x via OpenRouter — confirmed to support OpenAI-style `tool_calls`) - A small public repo to index (5–50 files is ideal) diff --git a/packages/llm/README.md b/packages/llm/README.md index b1b93ae..7a21d10 100644 --- a/packages/llm/README.md +++ b/packages/llm/README.md @@ -11,13 +11,12 @@ imported by Domain (`@bb/ingest-*`, `@bb/mcp`, future ## Responsibility -Minimal multi-provider LLM call surface for v0. The active backend is -selected by `Config.LlmProvider` (`"openrouter"` default, or -`"ollama"`): +Multi-provider LLM call surface. The active backend is selected by +`Config.LlmProvider` — one of `openrouter` (default), `ollama`, `anthropic`, +`bedrock`, `gemini` — and dispatched through the table in `src/providers.ts`: -- `askLLM(prompt, opts?)` — dispatches to either - `src/openrouter.ts` or `src/ollama.ts` depending on - `Config.LlmProvider`. Returns +- `askLLM(prompt, opts?)` — resolves the provider entry for + `Config.LlmProvider` (or `opts.provider`) and dispatches. Returns `{ content, usage: { model, inputTokens, outputTokens, costUsd } }`. Caller never sees the provider; the result shape is identical across backends. `costUsd` is the provider-reported USD cost for that single @@ -121,13 +120,20 @@ it. The cost ledger described in [docs/arch.md](../../docs/arch.md) is ## Invariants -1. **OpenRouter or local Ollama, nothing else.** No direct - Anthropic / OpenAI / Gemini / Bedrock SDKs. OpenRouter URL is fixed - at `https://openrouter.ai/api/v1/chat/completions`; Ollama URL is - user-configured via `Config.OllamaUrl` (default - `http://localhost:11434`). Provider is selected by - `Config.LlmProvider`, or by `opts.provider` when the caller wants to - override on a per-call basis. +1. **Every backend is an entry in `src/providers.ts`; no vendor SDKs.** + Five backends ship: OpenRouter, Ollama, Anthropic, Bedrock, Gemini. Every + one is plain `fetch` against a documented HTTP endpoint — no + `@anthropic-ai/*`, `@google/*`, or `@aws-sdk/*` dependency, so the Bun + binary stays small and the dependency surface auditable. Provider is + selected by `Config.LlmProvider`, or by `opts.provider` per call. An + unrecognised name throws `LlmConfigError` listing the valid set — it never + silently falls back to another backend. + 1a. **Bedrock auth is a Bedrock API key, not SigV4.** `Authorization: +Bearer ` against + `bedrock-runtime.{region}.amazonaws.com`. Long-term IAM credential signing + is deliberately out of scope: it would pull a signing dependency into the + binary and turn a one-field signup into four. Operators who need SigV4 + should front Bedrock with a gateway or use OpenRouter. 2. **Per-call credential override.** When `opts.apiKey` is set, the OpenRouter call uses it directly and skips `Config.OpenrouterApiKey`. This is the extension point that lets downstream consumers diff --git a/packages/llm/package.json b/packages/llm/package.json index 4a2f8ad..b84654d 100644 --- a/packages/llm/package.json +++ b/packages/llm/package.json @@ -12,11 +12,14 @@ "#src/*": "./src/*" }, "dependencies": { + "@ai-sdk/amazon-bedrock": "^5.0.61", "@bb/config": "workspace:*", + "@bb/db": "workspace:*", "@bb/errors": "workspace:*", "@bb/logger": "workspace:*", - "@bb/db": "workspace:*", "@bb/types": "workspace:*", + "ai": "^7.0.77", + "openai": "^7.5.0", "tiktoken": "^1.0.22" } } diff --git a/packages/llm/src/README.md b/packages/llm/src/README.md index 5cea49a..7da3174 100644 --- a/packages/llm/src/README.md +++ b/packages/llm/src/README.md @@ -7,13 +7,70 @@ package-level contract; this file documents how the source tree is split. - **[index.ts](index.ts)** — public re-exports. The only entry point other packages may import. Exposes `askLLM`, the `AskLlmOptions` type, the - `LlmProviderName` union (`"openrouter" | "ollama"`), plus the JSON + `LlmProviderName` union (six backends), plus the JSON client surface. Anything not re-exported here is internal. - **[client.ts](client.ts)** — the `askLLM` orchestrator. Selects the active provider via `opts.provider ?? getConfigValue(Config.LlmProvider)` - (per-call override beats config), dispatches to `openrouter.ts` or - `ollama.ts`. Consults the filesystem decision cache before issuing a + (per-call override beats config), then dispatches through the + `providers.ts` table. Consults the filesystem decision cache before issuing a request. Throws typed errors via `@bb/errors`. +- **[attempt.ts](attempt.ts)** — per-attempt resilience for the client-side + providers. `retryTransient` retries ONE turn in place on a 429 / 5xx / + timeout (exponential backoff, 3 attempts); `walkChain` walks a model chain, + next model on any failure; `causeMessage` preserves the provider status + + message when wrapping, because the failure classifier reads that string. + OpenRouter needs none of this — it takes a server-side `models: [...]` array + and reroutes internally. Everything else resolves a single-element chain + unless the caller passes `opts.fallbackModels`, so without in-place retry one + 429 is a hard failure for that file, and in a pipeline where any file failure + fails the run that discards an hour of work. BullMQ's `attempts: 3` retries + the whole job and re-bills the files that already succeeded; this does not. +- **[providers.ts](providers.ts)** — `LLM_PROVIDER_ENTRIES`, the + provider dispatch table. One entry per backend + (`resolveChain` / `call` / `reportsCost` / `supportsTools`); + `resolveProviderEntry(name)` throws `LlmConfigError` listing every valid + name rather than silently falling back, so a typo in `llm_provider` fails + loudly instead of billing the wrong account. Adding a backend is one entry + here plus one module — `client.ts` never branches on provider identity. +- **[anthropicMessages.ts](anthropicMessages.ts)** — the Anthropic Messages + wire format, used by the direct Anthropic API. (Bedrock shared this module + until it moved to Converse — that move is what made Bedrock family-agnostic.) + Owns `supportsTemperature()`, which both providers need: current Claude + families reject `temperature` on every platform, and the OpenAI families on + Bedrock reject it while Anthropic / Nova / Llama / Mistral accept it. Without + it the skip-decision gate's `temperature: 0` would hard-fail every scan. Also + refusal detection (`stop_reason: "refusal"` is HTTP 200 with empty content) + and `thinking`-block filtering. +- **[anthropic.ts](anthropic.ts)** — `callAnthropic` / `resolveAnthropicChain`. + `x-api-key` + `anthropic-version: 2023-06-01`, model in the body. +- **[bedrock.ts](bedrock.ts)** — `callBedrock` / `resolveBedrockChain` / + `resolveBedrockAuth`, over Converse via `@ai-sdk/amazon-bedrock`. The one + provider that takes an SDK: Converse has its own request shape and, without a + Bedrock API key, SigV4-signed requests — which must not be hand-rolled. The + SDK also resolves the AWS default credential chain, so an EC2/EKS deployment + authenticates from its instance role. Auth precedence: API key → static SigV4 + credentials → default chain. Covers every Bedrock family plus inference + profiles and ARNs. Clients cached per region + key-prefix, never the secret. + +- **[openaiCompatible.ts](openaiCompatible.ts)** — `openAiCompatibleChat`: one + attempt against any OpenAI-shaped `/chat/completions`. OpenAI, OpenRouter, + Gemini's compatible surface, Bedrock's `/openai/v1` route and every + self-hosted gateway speak this format, so a new provider is a base URL and a + model chain rather than another hand-copied fetch with its own subtly + different error handling. +- **[openai.ts](openai.ts)** — `callOpenAi` / `resolveOpenAiChain` / + `openAiBase`. Direct OpenAI, or any OpenAI-compatible server via + `Config.OpenaiBaseUrl` (vLLM / LiteLLM / an internal gateway). +- **[toolChat.ts](toolChat.ts)** — `toolChat`: one tool-capable turn on any + non-OpenRouter provider, through the `openai` SDK pointed at that provider's + OpenAI-compatible base URL. Tool use used to be OpenRouter-only and threw + everywhere else, which meant `concept-graph` silently vanished on a provider + switch — that is a feature disappearing, not a provider switching. Bedrock's + route here is bearer-authenticated, so tool use needs the API key even when + the main call path uses SigV4. +- \*\*[gemini.ts](gemini.ts) — `callGemini` / `resolveGeminiChain`. + `x-goog-api-key`, `:generateContent`, `systemInstruction` / `contents` + mapping, and `promptFeedback.blockReason` surfaced as a typed error. - **[openrouter.ts](openrouter.ts)** — `callOpenRouter` and `resolveOpenRouterChain`. Resolves the API key (`opts.apiKey ?? getConfigValue(Config.OpenrouterApiKey)`) and the model chain diff --git a/packages/llm/src/anthropic.ts b/packages/llm/src/anthropic.ts new file mode 100644 index 0000000..bf010d1 --- /dev/null +++ b/packages/llm/src/anthropic.ts @@ -0,0 +1,48 @@ +// SPDX-License-Identifier: AGPL-3.0-only WITH non-commercial-clause +import { getConfigValue } from "@bb/config"; +import { Config } from "@bb/types"; +import { LlmConfigError, LlmError } from "@bb/errors"; +import type { AskLlmOptions, AskLlmResult } from "./client.ts"; +import { anthropicMessagesCall, type AnthropicTarget } from "./anthropicMessages.ts"; +import { causeMessage, walkChain } from "./attempt.ts"; + +const ANTHROPIC_URL = "https://api.anthropic.com/v1/messages"; +const ANTHROPIC_VERSION = "2023-06-01"; + +/** + * Primary model plus any caller-supplied fallbacks. The Anthropic API has no + * server-side `models: [...]` fan-out, so the chain is walked client-side. + */ +export function resolveAnthropicChain(opts: AskLlmOptions): string[] { + const apiKey = opts.apiKey ?? getConfigValue(Config.AnthropicApiKey); + if (apiKey.length === 0) { + throw new LlmConfigError("bytebell set anthropic-api-key "); + } + const primary = opts.model ?? getConfigValue(Config.AnthropicModel); + if (primary.length === 0) { + throw new LlmConfigError("bytebell set anthropic-model "); + } + const chain = [primary, ...(opts.fallbackModels ?? [])].map((m) => m.trim()).filter((m) => m.length > 0); + return [...new Set(chain)]; +} + +export async function callAnthropic(prompt: string, opts: AskLlmOptions, timeoutMs: number): Promise { + const chain = resolveAnthropicChain(opts); + const apiKey = opts.apiKey ?? getConfigValue(Config.AnthropicApiKey); + + return walkChain("anthropic", chain, async (model) => { + const target: AnthropicTarget = { + label: "Anthropic", + url: ANTHROPIC_URL, + headers: { "x-api-key": apiKey, "anthropic-version": ANTHROPIC_VERSION }, + model, + }; + try { + return await anthropicMessagesCall(target, prompt, opts, timeoutMs); + } catch (cause: unknown) { + throw cause instanceof LlmError + ? cause + : new LlmError(`anthropic request failed (model=${model}): ${causeMessage(cause)}`, cause); + } + }); +} diff --git a/packages/llm/src/anthropicMessages.ts b/packages/llm/src/anthropicMessages.ts new file mode 100644 index 0000000..50b216b --- /dev/null +++ b/packages/llm/src/anthropicMessages.ts @@ -0,0 +1,177 @@ +// SPDX-License-Identifier: AGPL-3.0-only WITH non-commercial-clause +import { LlmError } from "@bb/errors"; +import { tokenLen } from "./tokenizer.ts"; +import type { AskLlmOptions, AskLlmResult } from "./client.ts"; + +// ───────────────────────────────────────────────────────────────────────────── +// The Anthropic Messages wire format, used by the direct Anthropic API. +// +// Bedrock used to share this module (Anthropic-on-Bedrock accepts the same +// body), but it now speaks Converse instead — that is what makes the Bedrock +// provider family-agnostic. `supportsTemperature` still lives here because both +// providers need it and it is the one rule that spans them. +// ───────────────────────────────────────────────────────────────────────────── + +/** Fallback completion cap. Anthropic requires `max_tokens` — there is no "unset". */ +export const DEFAULT_MAX_COMPLETION_TOKENS = 16384; + +export interface AnthropicTarget { + /** Human label used in error messages. */ + label: string; + url: string; + headers: Record; + model: string; +} + +interface AnthropicRequest { + model: string; + max_tokens: number; + system?: string; + messages: Array<{ role: "user"; content: string }>; + temperature?: number; +} + +interface AnthropicContentBlock { + type: string; + text?: string; +} + +interface AnthropicResponse { + model?: string; + content?: AnthropicContentBlock[]; + stop_reason?: string; + stop_details?: { category?: string; explanation?: string } | null; + usage?: { input_tokens?: number; output_tokens?: number }; +} + +/** + * Claude families that reject `temperature` / `top_p` / `top_k` with a 400 — + * sampling params were removed API-wide on these models. Matters because the + * skip-decision gate calls with `temperature: 0` on every scan, so sending it + * unconditionally would hard-fail every ingest on a current Claude model. + */ +const CLAUDE_REJECTS_SAMPLING = [/opus-5/u, /opus-4-8/u, /opus-4-7/u, /sonnet-5/u, /fable-5/u, /mythos-5/u]; + +/** + * The family segment of a Bedrock model reference. + * + * Handles all three forms operators actually configure: + * `anthropic.claude-…` bare model id + * `us.anthropic.claude-…` cross-region inference profile + * `arn:aws:bedrock:…:inference-profile/us.anthropic.claude-…` profile ARN + * + * ARNs are unwrapped to their trailing resource id first, then the region-group + * prefix is stripped. Missing the ARN case would reject a perfectly valid + * inference profile — the standard way to reach cross-region capacity. + */ +function bedrockFamily(modelId: string): string { + const resource = modelId.startsWith("arn:") ? (modelId.split("/").pop() ?? modelId) : modelId; + return resource.replace(/^(us|eu|apac|us-gov)\./u, ""); +} + +/** + * Whether this model accepts `temperature`. Two independent rules: + * + * - The OpenAI families **on Bedrock** reject it outright ("This model doesn't + * support the temperature field"), while Anthropic, Nova, Llama and Mistral + * take it. Matched on the family segment after any cross-region + * inference-profile prefix, because that prefix is part of the id operators + * configure. + * - Current Claude families reject it on every platform. + */ +export function supportsTemperature(model: string): boolean { + if (bedrockFamily(model).startsWith("openai.")) { + return false; + } + return !CLAUDE_REJECTS_SAMPLING.some((rx) => rx.test(model)); +} + +export function resolveMaxCompletionTokens(opts: AskLlmOptions): number { + const requested = opts.maxCompletionTokens ?? 0; + return requested > 0 ? requested : DEFAULT_MAX_COMPLETION_TOKENS; +} + +export async function anthropicMessagesCall( + target: AnthropicTarget, + prompt: string, + opts: AskLlmOptions, + timeoutMs: number, +): Promise { + const body: AnthropicRequest = { + model: target.model, + max_tokens: resolveMaxCompletionTokens(opts), + messages: [{ role: "user", content: prompt }], + }; + if (opts.systemPrompt !== undefined) { + body.system = opts.systemPrompt; + } + // Silently drop sampling on models that 400 on it rather than failing the run. + if (opts.temperature !== undefined && supportsTemperature(target.model)) { + body.temperature = opts.temperature; + } + + const controller = new AbortController(); + const timer = setTimeout(() => controller.abort(), timeoutMs); + + let response: Response; + try { + response = await fetch(target.url, { + method: "POST", + headers: { "Content-Type": "application/json", ...target.headers }, + body: JSON.stringify(body), + signal: controller.signal, + }); + } catch (cause: unknown) { + if (cause instanceof Error && cause.name === "AbortError") { + throw new LlmError(`${target.label} request timed out after ${timeoutMs}ms`, cause); + } + throw new LlmError(`${target.label} request failed`, cause); + } finally { + clearTimeout(timer); + } + + if (!response.ok) { + const text = await response.text().catch(() => ""); + throw new LlmError(`${target.label} HTTP ${response.status}`, undefined, { + status: response.status, + detail: text.slice(0, 4000), + }); + } + + const json = (await response.json()) as AnthropicResponse; + + // A safety-classifier decline is HTTP 200 with an empty `content` array. + // Surface it as a typed error instead of "empty completion", so the operator + // sees the actual cause. + if (json.stop_reason === "refusal") { + const category = json.stop_details?.category ?? "unspecified"; + throw new LlmError(`${target.label} refused the request (category: ${category})`); + } + + // Concatenate every text block; reasoning models interleave `thinking` + // blocks, which carry no `text` and are skipped. + const content = (json.content ?? []) + .filter((b) => b.type === "text" && typeof b.text === "string") + .map((b) => b.text ?? "") + .join(""); + + if (content.length === 0) { + const reason = json.stop_reason ?? "unknown"; + throw new LlmError(`${target.label} returned empty completion (stop_reason: ${reason})`); + } + + return { + content, + usage: { + model: typeof json.model === "string" && json.model.length > 0 ? json.model : target.model, + inputTokens: + typeof json.usage?.input_tokens === "number" + ? json.usage.input_tokens + : tokenLen((opts.systemPrompt ?? "") + prompt), + outputTokens: typeof json.usage?.output_tokens === "number" ? json.usage.output_tokens : tokenLen(content), + // Anthropic reports no per-call price. `bytebell stats` shows $0 — same + // treatment as Ollama. Never computed client-side. + costUsd: 0, + }, + }; +} diff --git a/packages/llm/src/attempt.ts b/packages/llm/src/attempt.ts new file mode 100644 index 0000000..eddfc3a --- /dev/null +++ b/packages/llm/src/attempt.ts @@ -0,0 +1,130 @@ +// SPDX-License-Identifier: AGPL-3.0-only WITH non-commercial-clause +import { LlmError } from "@bb/errors"; +import { logger } from "@bb/logger"; + +// ───────────────────────────────────────────────────────────────────────────── +// Per-attempt resilience for the client-side providers (Anthropic, Bedrock, +// Gemini, Ollama). OpenRouter does not need any of this — it takes a +// server-side `models: [...]` array and reroutes internally. +// +// Why this exists: the model chain is the only resilience a client-side +// provider has, and it collapses. Every non-OpenRouter backend resolves a +// SINGLE-element chain unless the caller passes `opts.fallbackModels`, so one +// 429 or 5xx is a hard failure for that file — and in a pipeline where any file +// failure fails the run, one blip discards an hour of work. BullMQ's +// `attempts: 3` retries the whole job, re-billing every file that already +// succeeded; this retries just the turn that blipped. +// ───────────────────────────────────────────────────────────────────────────── + +const DEFAULT_ATTEMPTS = 3; +const DEFAULT_BASE_DELAY_MS = 1000; + +export interface RetryTransientOptions { + /** Total attempts including the first. Default 3. */ + attempts?: number; + /** Base backoff in ms; delay = baseDelayMs × 2^(attempt-1). Default 1000. */ + baseDelayMs?: number; + /** Short label for logs (e.g. `"anthropic claude-sonnet-5"`). */ + label?: string; +} + +/** + * The provider's own message, so a wrapped error still says WHY it failed. + * + * Without this an Anthropic 400 (oversized prompt) and an Anthropic 503 reach + * the caller as the identical "request failed". That string is what the + * pipeline persists as its failure detail and what the failure classifier + * reads to decide whether a run is retryable — so losing the cause turns every + * provider error into "unreachable". + */ +export function causeMessage(cause: unknown): string { + if (cause instanceof Error) { + const status = (cause as { status?: unknown }).status; + const prefix = typeof status === "number" ? `HTTP ${status}: ` : ""; + return `${prefix}${cause.message}`; + } + return String(cause); +} + +/** + * True for errors worth retrying: HTTP 429 / 5xx (provider rate-limit or + * transient server error) or an aborted (timed-out) request. Everything else — + * a 400, a refusal, a missing key — is a hard error and must not burn retries. + * Unwraps `cause`, since providers wrap the transport error in an `LlmError`. + */ +function isTransient(err: unknown): boolean { + if (err instanceof Error && err.name === "AbortError") { + return true; + } + // A timeout is wrapped as LlmError with the AbortError as its cause; the + // message is the only thing that survives at the top level. + if (err instanceof LlmError && err.message.includes("timed out")) { + return true; + } + const status = (err as { status?: unknown }).status; + if (typeof status === "number") { + return status === 429 || status >= 500; + } + const cause = (err as { cause?: unknown }).cause; + if (cause !== undefined && cause !== null && cause !== err) { + return isTransient(cause); + } + return false; +} + +function sleep(ms: number): Promise { + return new Promise((resolve) => setTimeout(resolve, ms)); +} + +export async function retryTransient(op: () => Promise, options: RetryTransientOptions = {}): Promise { + const attempts = options.attempts ?? DEFAULT_ATTEMPTS; + const baseDelayMs = options.baseDelayMs ?? DEFAULT_BASE_DELAY_MS; + const label = options.label ?? "llm"; + let lastError: unknown; + for (let attempt = 1; attempt <= attempts; attempt += 1) { + try { + return await op(); + } catch (err: unknown) { + lastError = err; + if (attempt >= attempts || !isTransient(err)) { + throw err; + } + const wait = baseDelayMs * 2 ** (attempt - 1); + logger.warn( + `llm: ${label} transient failure (attempt ${attempt}/${attempts}), retrying in ${wait}ms: ${ + err instanceof Error ? err.message : String(err) + }`, + ); + await sleep(wait); + } + } + throw lastError instanceof Error ? lastError : new Error(String(lastError)); +} + +/** One model attempt, retried in place on a transient failure. */ +export async function attemptWithRetry(label: string, model: string, op: () => Promise): Promise { + return retryTransient(op, { label: `${label} ${model}` }); +} + +/** + * Walk a model chain client-side: one attempt per model (each retried in place + * on transient failures), next model on any failure, last error surfaced when + * the chain is dry. This is what OpenRouter gets server-side from its + * `models: [...]` array; every other backend has to do it here. + */ +export async function walkChain( + label: string, + chain: readonly string[], + attempt: (model: string) => Promise, +): Promise { + let lastError: Error | null = null; + for (const model of chain) { + try { + return await attemptWithRetry(label, model, () => attempt(model)); + } catch (err: unknown) { + lastError = err instanceof Error ? err : new Error(String(err)); + logger.warn(`llm: ${label} attempt failed (model=${model}) — ${lastError.message}; trying next in chain`); + } + } + throw lastError ?? new LlmError(`${label}: model chain exhausted`); +} diff --git a/packages/llm/src/bedrock.ts b/packages/llm/src/bedrock.ts new file mode 100644 index 0000000..63613b2 --- /dev/null +++ b/packages/llm/src/bedrock.ts @@ -0,0 +1,155 @@ +// SPDX-License-Identifier: AGPL-3.0-only WITH non-commercial-clause +import { createAmazonBedrock, type AmazonBedrockProvider } from "@ai-sdk/amazon-bedrock"; +import { generateText } from "ai"; +import { getConfigValue } from "@bb/config"; +import { Config } from "@bb/types"; +import { LlmConfigError, LlmError } from "@bb/errors"; +import { tokenLen } from "./tokenizer.ts"; +import type { AskLlmOptions, AskLlmResult } from "./client.ts"; +import { resolveMaxCompletionTokens, supportsTemperature } from "./anthropicMessages.ts"; +import { causeMessage, walkChain } from "./attempt.ts"; + +// ───────────────────────────────────────────────────────────────────────────── +// Amazon Bedrock via `@ai-sdk/amazon-bedrock`. +// +// This is the one provider that takes an SDK. Bedrock is not an OpenAI-shaped +// `/chat/completions` like the others: Converse has its own request shape and, +// unless a Bedrock API key is configured, SigV4-signed requests. Hand-rolling +// SigV4 is not something to get subtly wrong, and the SDK also resolves the AWS +// default credential chain — which is how a deployment on EC2/EKS authenticates +// from an instance role with no static credentials at all. +// +// Converse covers **every model family on Bedrock** (Anthropic, Nova, Llama, +// Mistral, DeepSeek, the OpenAI models) plus inference profiles and ARNs. What +// it does not normalise is which *parameters* each family accepts — see +// `supportsTemperature`. +// ───────────────────────────────────────────────────────────────────────────── + +export interface BedrockAuth { + region: string; + /** Bearer API key — wins when set. */ + apiKey?: string; + credentials?: { accessKeyId: string; secretAccessKey: string; sessionToken?: string }; +} + +/** + * Auth precedence mirrors the SDK's own: a Bedrock API key wins, else static + * SigV4 credentials when configured, else region alone so the AWS default + * provider chain resolves the task/instance role. Passing `undefined` + * explicitly would override that chain with nothing. + */ +export function resolveBedrockAuth(opts: AskLlmOptions): BedrockAuth { + const region = getConfigValue(Config.BedrockRegion); + if (region.length === 0) { + throw new LlmConfigError("bytebell set bedrock-region "); + } + const apiKey = opts.apiKey ?? getConfigValue(Config.BedrockApiKey); + if (apiKey.length > 0) { + return { region, apiKey }; + } + const accessKeyId = getConfigValue(Config.AwsAccessKeyId); + const secretAccessKey = getConfigValue(Config.AwsSecretAccessKey); + if (accessKeyId.length > 0 && secretAccessKey.length > 0) { + const sessionToken = getConfigValue(Config.AwsSessionToken); + return { + region, + credentials: { accessKeyId, secretAccessKey, ...(sessionToken.length > 0 ? { sessionToken } : {}) }, + }; + } + // No static credential configured — fall through to the AWS default chain + // (instance/task role, shared profile). If nothing resolves there the SDK + // raises its own credential error, which `causeMessage` surfaces intact. + return { region }; +} + +// One cached client per resolved credential set. Keyed by region + a short key +// prefix or the access key id — never the secret. +const clients = new Map(); + +function clientFor(auth: BedrockAuth): AmazonBedrockProvider { + const authKey = + auth.apiKey !== undefined ? `key:${auth.apiKey.slice(0, 8)}` : (auth.credentials?.accessKeyId ?? "default-chain"); + const cacheKey = `${auth.region}|${authKey}`; + const existing = clients.get(cacheKey); + if (existing !== undefined) { + return existing; + } + const client = createAmazonBedrock({ + region: auth.region, + ...(auth.apiKey !== undefined ? { apiKey: auth.apiKey } : {}), + ...(auth.credentials !== undefined + ? { + accessKeyId: auth.credentials.accessKeyId, + secretAccessKey: auth.credentials.secretAccessKey, + ...(auth.credentials.sessionToken !== undefined ? { sessionToken: auth.credentials.sessionToken } : {}), + } + : {}), + }); + clients.set(cacheKey, client); + return client; +} + +export function resolveBedrockChain(opts: AskLlmOptions): string[] { + resolveBedrockAuth(opts); + const primary = opts.model ?? getConfigValue(Config.BedrockModel); + if (primary.length === 0) { + throw new LlmConfigError("bytebell set bedrock-model "); + } + const chain = [primary, ...(opts.fallbackModels ?? [])].map((m) => m.trim()).filter((m) => m.length > 0); + return [...new Set(chain)]; +} + +async function attemptBedrock( + auth: BedrockAuth, + model: string, + prompt: string, + opts: AskLlmOptions, + timeoutMs: number, +): Promise { + const client = clientFor(auth); + const controller = new AbortController(); + const timer = setTimeout(() => controller.abort(), timeoutMs); + + try { + const response = await generateText({ + model: client(model), + ...(opts.systemPrompt !== undefined ? { system: opts.systemPrompt } : {}), + prompt, + // Converse normalises shape, not accepted parameters. + ...(opts.temperature !== undefined && supportsTemperature(model) ? { temperature: opts.temperature } : {}), + maxOutputTokens: resolveMaxCompletionTokens(opts), + abortSignal: controller.signal, + }); + + const content = response.text; + if (content.length === 0) { + throw new LlmError(`Bedrock returned empty completion (model=${model})`); + } + return { + content, + usage: { + model, + inputTokens: response.usage.inputTokens ?? tokenLen(`${opts.systemPrompt ?? ""}${prompt}`), + outputTokens: response.usage.outputTokens ?? tokenLen(content), + // Bedrock reports no per-call price — spend is billed to, and read from, + // the operator's own AWS account. + costUsd: 0, + }, + }; + } catch (cause: unknown) { + if (cause instanceof Error && cause.name === "AbortError") { + throw new LlmError(`Bedrock request timed out after ${timeoutMs}ms (model=${model})`, cause); + } + throw cause instanceof LlmError + ? cause + : new LlmError(`bedrock request failed (model=${model}): ${causeMessage(cause)}`, cause); + } finally { + clearTimeout(timer); + } +} + +export async function callBedrock(prompt: string, opts: AskLlmOptions, timeoutMs: number): Promise { + const auth = resolveBedrockAuth(opts); + const chain = resolveBedrockChain(opts); + return walkChain("bedrock", chain, (model) => attemptBedrock(auth, model, prompt, opts, timeoutMs)); +} diff --git a/packages/llm/src/client.ts b/packages/llm/src/client.ts index be9c645..15623f7 100644 --- a/packages/llm/src/client.ts +++ b/packages/llm/src/client.ts @@ -2,12 +2,11 @@ import { getConfigValue } from "@bb/config"; import { logger } from "@bb/logger"; import { Config } from "@bb/types"; import { computeCacheKey, getCachedDecision, isCacheEnabled, recordDecision, recordHit } from "./cache.ts"; -import { callOllama, resolveOllamaChain } from "./ollama.ts"; -import { callOpenRouter, resolveOpenRouterChain } from "./openrouter.ts"; +import { resolveProviderEntry } from "./providers.ts"; const DEFAULT_TIMEOUT_MS = 360_000; -export type LlmProviderName = "openrouter" | "ollama"; +export type LlmProviderName = "openrouter" | "ollama" | "anthropic" | "bedrock" | "gemini" | "openai"; export interface AskLlmOptions { model?: string; @@ -15,11 +14,12 @@ export interface AskLlmOptions { timeoutMs?: number; systemPrompt?: string; /** - * Per-call override of the OpenRouter API key. When set, takes precedence - * over `Config.OpenrouterApiKey`. Used by downstream consumers (e.g. the - * enterprise wrapper) that resolve per-org credentials at the enqueue - * boundary and pass them through the job payload. Ignored by the Ollama - * provider (which is keyless). + * Per-call override of the active provider's API key. When set, takes + * precedence over that provider's configured key (`Config.OpenrouterApiKey`, + * `Config.AnthropicApiKey`, `Config.BedrockApiKey`, `Config.GeminiApiKey`). + * Used by downstream consumers (e.g. the enterprise wrapper) that resolve + * per-org credentials at the enqueue boundary and pass them through the job + * payload. Ignored by the Ollama provider (which is keyless). */ apiKey?: string; /** @@ -62,8 +62,9 @@ export interface AskLlmUsage { outputTokens: number; /** * Provider-reported cost in USD for this single call. Taken directly from - * the provider's response — `usage.cost` on OpenRouter, `0` for Ollama, - * `0` when the provider omits the field. Never computed client-side. + * the provider's response — `usage.cost` on OpenRouter, `0` on every other + * backend (Ollama is local; Anthropic, Bedrock, and Gemini report tokens but + * not cost). Never computed client-side. See `LlmProviderEntry.reportsCost`. */ costUsd: number; /** @@ -83,7 +84,8 @@ export interface AskLlmResult { export async function askLLM(prompt: string, opts: AskLlmOptions = {}): Promise { const provider: LlmProviderName = opts.provider ?? (getConfigValue(Config.LlmProvider) as LlmProviderName); const timeoutMs = opts.timeoutMs ?? DEFAULT_TIMEOUT_MS; - const chain = provider === "ollama" ? resolveOllamaChain(opts) : resolveOpenRouterChain(opts); + const entry = resolveProviderEntry(provider); + const chain = entry.resolveChain(opts); const cacheOn = isCacheEnabled(); const cacheKey = cacheOn @@ -108,8 +110,7 @@ export async function askLLM(prompt: string, opts: AskLlmOptions = {}): Promise< logger.debug(`llm: cache miss (key=${cacheKey.slice(0, 8)})`); } - const result = - provider === "ollama" ? await callOllama(prompt, opts, timeoutMs) : await callOpenRouter(prompt, opts, timeoutMs); + const result = await entry.call(prompt, opts, timeoutMs); if (cacheOn && cacheKey !== null) { void recordDecision(cacheKey, { diff --git a/packages/llm/src/gemini.ts b/packages/llm/src/gemini.ts new file mode 100644 index 0000000..574462e --- /dev/null +++ b/packages/llm/src/gemini.ts @@ -0,0 +1,147 @@ +// SPDX-License-Identifier: AGPL-3.0-only WITH non-commercial-clause +import { getConfigValue } from "@bb/config"; +import { Config } from "@bb/types"; +import { LlmConfigError, LlmError } from "@bb/errors"; +import { tokenLen } from "./tokenizer.ts"; +import type { AskLlmOptions, AskLlmResult } from "./client.ts"; +import { resolveMaxCompletionTokens } from "./anthropicMessages.ts"; +import { causeMessage, walkChain } from "./attempt.ts"; + +const GEMINI_BASE = "https://generativelanguage.googleapis.com/v1beta/models"; + +interface GeminiPart { + text?: string; +} + +interface GeminiRequest { + systemInstruction?: { parts: GeminiPart[] }; + contents: Array<{ role: "user"; parts: GeminiPart[] }>; + generationConfig: { maxOutputTokens: number; temperature?: number }; +} + +interface GeminiResponse { + modelVersion?: string; + candidates?: Array<{ content?: { parts?: GeminiPart[] }; finishReason?: string }>; + promptFeedback?: { blockReason?: string }; + usageMetadata?: { promptTokenCount?: number; candidatesTokenCount?: number }; +} + +/** + * Primary model plus any caller-supplied fallbacks. Google exposes no + * server-side fallback array, so the chain is walked client-side. + */ +export function resolveGeminiChain(opts: AskLlmOptions): string[] { + const apiKey = opts.apiKey ?? getConfigValue(Config.GeminiApiKey); + if (apiKey.length === 0) { + throw new LlmConfigError("bytebell set gemini-api-key "); + } + const primary = opts.model ?? getConfigValue(Config.GeminiModel); + if (primary.length === 0) { + throw new LlmConfigError("bytebell set gemini-model "); + } + const chain = [primary, ...(opts.fallbackModels ?? [])].map((m) => m.trim()).filter((m) => m.length > 0); + return [...new Set(chain)]; +} + +/** One attempt against one model. */ +async function attemptGemini( + model: string, + prompt: string, + opts: AskLlmOptions, + timeoutMs: number, +): Promise { + const apiKey = opts.apiKey ?? getConfigValue(Config.GeminiApiKey); + + const body: GeminiRequest = { + contents: [{ role: "user", parts: [{ text: prompt }] }], + generationConfig: { maxOutputTokens: resolveMaxCompletionTokens(opts) }, + }; + if (opts.systemPrompt !== undefined) { + body.systemInstruction = { parts: [{ text: opts.systemPrompt }] }; + } + if (opts.temperature !== undefined) { + body.generationConfig.temperature = opts.temperature; + } + + const controller = new AbortController(); + const timer = setTimeout(() => controller.abort(), timeoutMs); + + let response: Response; + try { + response = await fetch(`${GEMINI_BASE}/${encodeURIComponent(model)}:generateContent`, { + method: "POST", + headers: { "Content-Type": "application/json", "x-goog-api-key": apiKey }, + body: JSON.stringify(body), + signal: controller.signal, + }); + } catch (cause: unknown) { + if (cause instanceof Error && cause.name === "AbortError") { + throw new LlmError(`Gemini request timed out after ${timeoutMs}ms`, cause); + } + throw new LlmError("Gemini request failed", cause); + } finally { + clearTimeout(timer); + } + + if (!response.ok) { + const text = await response.text().catch(() => ""); + throw new LlmError(`Gemini HTTP ${response.status}`, undefined, { + status: response.status, + detail: text.slice(0, 4000), + }); + } + + const json = (await response.json()) as GeminiResponse; + + // A safety block is HTTP 200 with no candidates and a `blockReason`. + const blockReason = json.promptFeedback?.blockReason; + if (typeof blockReason === "string" && blockReason.length > 0) { + throw new LlmError(`Gemini blocked the request (reason: ${blockReason})`); + } + + const candidate = json.candidates?.[0]; + const content = (candidate?.content?.parts ?? []) + .map((p) => p.text ?? "") + .filter((t) => t.length > 0) + .join(""); + + if (content.length === 0) { + const reason = candidate?.finishReason ?? "unknown"; + throw new LlmError(`Gemini returned empty completion (finishReason: ${reason})`); + } + + return { + content, + usage: { + model: typeof json.modelVersion === "string" && json.modelVersion.length > 0 ? json.modelVersion : model, + inputTokens: + typeof json.usageMetadata?.promptTokenCount === "number" + ? json.usageMetadata.promptTokenCount + : tokenLen((opts.systemPrompt ?? "") + prompt), + outputTokens: + typeof json.usageMetadata?.candidatesTokenCount === "number" + ? json.usageMetadata.candidatesTokenCount + : tokenLen(content), + // Gemini does not report cost. Same treatment as Ollama / Anthropic. + costUsd: 0, + }, + }; +} + +/** + * Gemini chat completion. One attempt per model (each retried in place on a + * transient 429 / 5xx / timeout), next model on any failure, last error + * surfaced when the chain is dry. + */ +export async function callGemini(prompt: string, opts: AskLlmOptions, timeoutMs: number): Promise { + const chain = resolveGeminiChain(opts); + return walkChain("gemini", chain, async (model) => { + try { + return await attemptGemini(model, prompt, opts, timeoutMs); + } catch (cause: unknown) { + throw cause instanceof LlmError + ? cause + : new LlmError(`gemini request failed (model=${model}): ${causeMessage(cause)}`, cause); + } + }); +} diff --git a/packages/llm/src/openai.ts b/packages/llm/src/openai.ts new file mode 100644 index 0000000..f09bbdd --- /dev/null +++ b/packages/llm/src/openai.ts @@ -0,0 +1,53 @@ +// SPDX-License-Identifier: AGPL-3.0-only WITH non-commercial-clause +import { getConfigValue } from "@bb/config"; +import { Config } from "@bb/types"; +import { LlmConfigError } from "@bb/errors"; +import type { AskLlmOptions, AskLlmResult } from "./client.ts"; +import { supportsTemperature } from "./anthropicMessages.ts"; +import { walkChain } from "./attempt.ts"; +import { openAiCompatibleChat, type OpenAiCompatibleTarget } from "./openaiCompatible.ts"; + +const DEFAULT_OPENAI_BASE = "https://api.openai.com/v1"; + +/** + * Endpoint base. `Config.OpenaiBaseUrl` points this at a self-hosted + * OpenAI-compatible server (vLLM / LiteLLM / an internal gateway), which is the + * cheapest way to reach a model this table does not name. + */ +export function openAiBase(): string { + const configured = getConfigValue(Config.OpenaiBaseUrl); + const base = configured.length > 0 ? configured : DEFAULT_OPENAI_BASE; + return base.endsWith("/") ? base.slice(0, -1) : base; +} + +export function resolveOpenAiChain(opts: AskLlmOptions): string[] { + const apiKey = opts.apiKey ?? getConfigValue(Config.OpenaiApiKey); + if (apiKey.length === 0) { + throw new LlmConfigError("bytebell set openai-api-key "); + } + const primary = opts.model ?? getConfigValue(Config.OpenaiModel); + if (primary.length === 0) { + throw new LlmConfigError("bytebell set openai-model "); + } + const chain = [primary, ...(opts.fallbackModels ?? [])].map((m) => m.trim()).filter((m) => m.length > 0); + return [...new Set(chain)]; +} + +/** + * Direct OpenAI (or any OpenAI-compatible server) chat completion. No + * server-side fallback array, so the chain is walked client-side. + */ +export async function callOpenAi(prompt: string, opts: AskLlmOptions, timeoutMs: number): Promise { + const chain = resolveOpenAiChain(opts); + const apiKey = opts.apiKey ?? getConfigValue(Config.OpenaiApiKey); + + return walkChain("openai", chain, async (model) => { + const target: OpenAiCompatibleTarget = { + label: "OpenAI", + url: `${openAiBase()}/chat/completions`, + apiKey, + allowTemperature: supportsTemperature(model), + }; + return openAiCompatibleChat(target, model, opts, prompt, timeoutMs); + }); +} diff --git a/packages/llm/src/openaiCompatible.ts b/packages/llm/src/openaiCompatible.ts new file mode 100644 index 0000000..1c716d2 --- /dev/null +++ b/packages/llm/src/openaiCompatible.ts @@ -0,0 +1,117 @@ +// SPDX-License-Identifier: AGPL-3.0-only WITH non-commercial-clause +import { LlmError } from "@bb/errors"; +import { causeMessage } from "./attempt.ts"; +import { tokenLen } from "./tokenizer.ts"; +import type { AskLlmOptions, AskLlmResult } from "./client.ts"; + +// ───────────────────────────────────────────────────────────────────────────── +// One attempt against any OpenAI-shaped `/chat/completions` endpoint. +// +// OpenAI, OpenRouter, Gemini's OpenAI-compatible surface, Bedrock's +// `/openai/v1` route, and every self-hosted gateway (vLLM / LiteLLM) speak this +// wire format. The differences between them are a base URL, a bearer token, and +// which optional parameters a given model tolerates — not a request shape. +// Keeping the transport in one place is what lets a new provider be a base URL +// and a model chain rather than another hand-copied fetch with its own subtly +// different error handling. +// ───────────────────────────────────────────────────────────────────────────── + +export interface OpenAiCompatibleTarget { + /** Provider name, used in error messages so a failure names its origin. */ + label: string; + /** Full endpoint URL, including `/chat/completions`. */ + url: string; + apiKey: string; + /** Extra request headers. */ + headers?: Record; + /** Some models reject `temperature` outright — see `supportsTemperature`. */ + allowTemperature?: boolean; + /** `max_completion_tokens` on newer surfaces, `max_tokens` on OpenRouter. */ + tokenCapField?: "max_tokens" | "max_completion_tokens"; +} + +interface ApiResponse { + choices?: { message?: { content?: string | null } }[]; + usage?: { prompt_tokens?: number; completion_tokens?: number; cost?: number }; +} + +export async function openAiCompatibleChat( + target: OpenAiCompatibleTarget, + model: string, + opts: AskLlmOptions, + prompt: string, + timeoutMs: number, +): Promise { + const messages: { role: "system" | "user"; content: string }[] = []; + if (opts.systemPrompt !== undefined) { + messages.push({ role: "system", content: opts.systemPrompt }); + } + messages.push({ role: "user", content: prompt }); + + const capField = target.tokenCapField ?? "max_completion_tokens"; + const controller = new AbortController(); + const timer = setTimeout(() => controller.abort(), timeoutMs); + + let response: Response; + try { + response = await fetch(target.url, { + method: "POST", + headers: { + Authorization: `Bearer ${target.apiKey}`, + "Content-Type": "application/json", + ...(target.headers ?? {}), + }, + body: JSON.stringify({ + model, + messages, + ...(opts.temperature !== undefined && target.allowTemperature !== false + ? { temperature: opts.temperature } + : {}), + ...(opts.maxCompletionTokens !== undefined && opts.maxCompletionTokens > 0 + ? { [capField]: opts.maxCompletionTokens } + : {}), + }), + signal: controller.signal, + }); + } catch (cause: unknown) { + if (cause instanceof Error && cause.name === "AbortError") { + throw new LlmError(`${target.label} request timed out after ${timeoutMs}ms (model=${model})`, cause); + } + throw new LlmError(`${target.label} request failed (model=${model}): ${causeMessage(cause)}`, cause); + } finally { + clearTimeout(timer); + } + + if (!response.ok) { + const text = await response.text().catch(() => ""); + throw new LlmError(`${target.label} HTTP ${response.status} (model=${model})`, undefined, { + status: response.status, + detail: text.slice(0, 4000), + }); + } + + let body: ApiResponse; + try { + body = (await response.json()) as ApiResponse; + } catch (cause: unknown) { + throw new LlmError(`${target.label} returned a non-JSON body (model=${model})`, cause); + } + + const content = body.choices?.[0]?.message?.content; + if (typeof content !== "string" || content.length === 0) { + throw new LlmError(`${target.label} returned empty completion (model=${model})`); + } + const promptText = messages.map((m) => m.content).join("\n"); + return { + content, + usage: { + model, + inputTokens: typeof body.usage?.prompt_tokens === "number" ? body.usage.prompt_tokens : tokenLen(promptText), + outputTokens: + typeof body.usage?.completion_tokens === "number" ? body.usage.completion_tokens : tokenLen(content), + // Only OpenRouter reports a per-call price; everywhere else spend lives in + // the operator's own provider account and is read from there. + costUsd: typeof body.usage?.cost === "number" ? body.usage.cost : 0, + }, + }; +} diff --git a/packages/llm/src/openrouter.ts b/packages/llm/src/openrouter.ts index c1c54c9..63f3250 100644 --- a/packages/llm/src/openrouter.ts +++ b/packages/llm/src/openrouter.ts @@ -7,7 +7,7 @@ import { openRouterRawChat, type OpenRouterMessageInput } from "./openrouterChat export function resolveOpenRouterChain(opts: AskLlmOptions): string[] { const apiKey = opts.apiKey ?? getConfigValue(Config.OpenrouterApiKey); if (apiKey.length === 0) { - throw new LlmConfigError("bytebell keys set"); + throw new LlmConfigError("bytebell set openrouter-api-key "); } const model = opts.model ?? getConfigValue(Config.OpenrouterModel); const fallbackSlots = opts.fallbackModels ?? [ diff --git a/packages/llm/src/providers.ts b/packages/llm/src/providers.ts new file mode 100644 index 0000000..af8d2f4 --- /dev/null +++ b/packages/llm/src/providers.ts @@ -0,0 +1,93 @@ +// SPDX-License-Identifier: AGPL-3.0-only WITH non-commercial-clause +import { LlmConfigError } from "@bb/errors"; +import type { AskLlmOptions, AskLlmResult, LlmProviderName } from "./client.ts"; +import { callOllama, resolveOllamaChain } from "./ollama.ts"; +import { callOpenRouter, resolveOpenRouterChain } from "./openrouter.ts"; +import { callAnthropic, resolveAnthropicChain } from "./anthropic.ts"; +import { callBedrock, resolveBedrockChain } from "./bedrock.ts"; +import { callGemini, resolveGeminiChain } from "./gemini.ts"; +import { callOpenAi, resolveOpenAiChain } from "./openai.ts"; + +// ───────────────────────────────────────────────────────────────────────────── +// Provider dispatch table. Adding a backend is one entry here plus one module — +// `askLLM` never branches on provider identity. Mirrors the registry shape used +// by @bb/db, @bb/graph-db, and @bb/queue, minus the runtime `register` call: +// every LLM backend is in-tree and keyless to construct, so a compile-time map +// is enough and keeps the cross-cutting tier free of a connect() lifecycle. +// ───────────────────────────────────────────────────────────────────────────── + +export interface LlmProviderEntry { + /** + * Models this call will try, in order. Single-element for every backend + * except OpenRouter, which has a native `models: [...]` fan-out. Also the + * validation seam — each implementation throws `LlmConfigError` with the + * exact `bytebell set …` hint when its credentials are missing. + */ + resolveChain: (opts: AskLlmOptions) => string[]; + call: (prompt: string, opts: AskLlmOptions, timeoutMs: number) => Promise; + /** True when the backend reports real spend. Drives `bytebell stats` honesty. */ + reportsCost: boolean; + /** + * True when the backend can drive `askLLMWithTools` (and therefore the + * `concept-graph` strategy). Everything except Ollama exposes an OpenAI-shaped + * `tools` / `tool_calls` surface; Ollama stays out because tool-format support + * varies per locally-pulled model and we cannot check it. + */ + supportsTools: boolean; +} + +export const LLM_PROVIDER_ENTRIES: Readonly> = { + openrouter: { + resolveChain: resolveOpenRouterChain, + call: callOpenRouter, + reportsCost: true, + supportsTools: true, + }, + ollama: { + resolveChain: resolveOllamaChain, + call: callOllama, + reportsCost: false, + supportsTools: false, + }, + anthropic: { + resolveChain: resolveAnthropicChain, + call: callAnthropic, + reportsCost: false, + supportsTools: true, + }, + bedrock: { + resolveChain: resolveBedrockChain, + call: callBedrock, + reportsCost: false, + // Tool use routes through Bedrock's OpenAI-compatible `/openai/v1` surface, + // which is bearer-authenticated — so it needs `bedrock_api_key`, not SigV4. + supportsTools: true, + }, + gemini: { + resolveChain: resolveGeminiChain, + call: callGemini, + reportsCost: false, + supportsTools: true, + }, + openai: { + resolveChain: resolveOpenAiChain, + call: callOpenAi, + reportsCost: false, + supportsTools: true, + }, +}; + +export const LLM_PROVIDER_NAMES: readonly LlmProviderName[] = Object.keys(LLM_PROVIDER_ENTRIES) as LlmProviderName[]; + +/** + * Resolve a provider by name. Throws rather than silently falling back — a + * typo in `llm_provider` that quietly routed every call to a different backend + * (and a different bill) is worse than a boot failure with a precise hint. + */ +export function resolveProviderEntry(name: string): LlmProviderEntry { + const entry = LLM_PROVIDER_ENTRIES[name as LlmProviderName]; + if (entry === undefined) { + throw new LlmConfigError(`bytebell set llm-provider <${LLM_PROVIDER_NAMES.join("|")}>`); + } + return entry; +} diff --git a/packages/llm/src/toolChat.ts b/packages/llm/src/toolChat.ts new file mode 100644 index 0000000..e688efc --- /dev/null +++ b/packages/llm/src/toolChat.ts @@ -0,0 +1,169 @@ +// SPDX-License-Identifier: AGPL-3.0-only WITH non-commercial-clause +import OpenAI from "openai"; +import { getConfigValue } from "@bb/config"; +import { Config } from "@bb/types"; +import { LlmConfigError, LlmError } from "@bb/errors"; +import { tokenLen } from "./tokenizer.ts"; +import type { AskLlmOptions, AskLlmUsage, LlmProviderName } from "./client.ts"; +import { supportsTemperature } from "./anthropicMessages.ts"; +import { causeMessage } from "./attempt.ts"; +import { openAiBase } from "./openai.ts"; +import type { OpenRouterMessageInput, OpenRouterToolCall, OpenRouterToolDef } from "./openrouterChat.ts"; + +// ───────────────────────────────────────────────────────────────────────────── +// One tool-capable chat turn, on whichever provider the deployment runs. +// +// Tool use used to be OpenRouter-only, and off OpenRouter the loop simply threw +// — which meant the `concept-graph` strategy silently became unavailable the +// moment an operator switched backend. That is not a provider switch; it is a +// feature disappearing. So this dispatches instead. +// +// Every provider below exposes an **OpenAI-shaped `tools` / `tool_calls` +// surface**, which is why the non-OpenRouter branch is one shared client call +// rather than four more request builders. OpenRouter keeps its own path +// (`openRouterRawChat`) because it alone carries the server-side `models[]` +// fallback chain, the `provider` routing rules, and a reported `usage.cost`. +// ───────────────────────────────────────────────────────────────────────────── + +/** OpenAI-compatible base URL per provider. */ +function baseUrlFor(provider: LlmProviderName): string { + switch (provider) { + case "gemini": + return "https://generativelanguage.googleapis.com/v1beta/openai/"; + case "anthropic": + // Anthropic publishes an OpenAI-compatible layer on the same host. + return "https://api.anthropic.com/v1/"; + case "bedrock": { + const region = getConfigValue(Config.BedrockRegion); + if (region.length === 0) { + throw new LlmConfigError("bytebell set bedrock-region "); + } + // `bedrock.ts` calls Converse through the AI SDK, which covers every model + // family but exposes no OpenAI-shaped `tool_calls` block. AWS publishes + // `/openai/v1` on the same host, which does — and takes the Bedrock API + // key as a plain bearer token. + return `https://bedrock-runtime.${region}.amazonaws.com/openai/v1`; + } + case "openai": + return openAiBase(); + default: + throw new LlmError(`toolChat: provider "${provider}" has no OpenAI-compatible surface`); + } +} + +/** The bearer credential for a provider's OpenAI-compatible surface. */ +function apiKeyFor(provider: LlmProviderName, opts: AskLlmOptions): string { + if (opts.apiKey !== undefined && opts.apiKey.length > 0) { + return opts.apiKey; + } + switch (provider) { + case "gemini": + return requireKey(getConfigValue(Config.GeminiApiKey), "bytebell set gemini-api-key "); + case "anthropic": + return requireKey(getConfigValue(Config.AnthropicApiKey), "bytebell set anthropic-api-key "); + case "bedrock": + // The OpenAI-compatible route is bearer-authenticated — SigV4 does not + // apply here, so tool use on Bedrock needs the API key specifically. + return requireKey( + getConfigValue(Config.BedrockApiKey), + "bytebell set bedrock-api-key — Bedrock tool use requires the bearer API key (SigV4 is not supported on the OpenAI-compatible route)", + ); + case "openai": + return requireKey(getConfigValue(Config.OpenaiApiKey), "bytebell set openai-api-key "); + default: + throw new LlmError(`toolChat: provider "${provider}" has no credential mapping`); + } +} + +function requireKey(value: string, hint: string): string { + if (value.length === 0) { + throw new LlmConfigError(hint); + } + return value; +} + +// One cached client per (provider, key-prefix, base) — never keyed on the secret. +const clients = new Map(); + +function clientFor(provider: LlmProviderName, opts: AskLlmOptions): OpenAI { + const apiKey = apiKeyFor(provider, opts); + const baseURL = baseUrlFor(provider); + const cacheKey = `${provider}|${baseURL}|${apiKey.slice(0, 8)}`; + const existing = clients.get(cacheKey); + if (existing !== undefined) { + return existing; + } + const client = new OpenAI({ apiKey, baseURL }); + clients.set(cacheKey, client); + return client; +} + +export interface ToolChatResult { + message: OpenRouterMessageInput & { tool_calls?: OpenRouterToolCall[] }; + usage: AskLlmUsage; + finishReason: string | null; +} + +/** + * One tool turn against a non-OpenRouter provider. + * + * `OpenRouterMessageInput` and `OpenRouterToolDef` are structurally the OpenAI + * shapes — same roles, same `tool_calls` / `tool_call_id` fields — so the + * conversion is a cast at the boundary rather than a rewrite. + */ +export async function toolChat( + provider: LlmProviderName, + model: string, + messages: OpenRouterMessageInput[], + opts: AskLlmOptions, + timeoutMs: number, + tools?: OpenRouterToolDef[], + toolChoice?: "auto" | "required", +): Promise { + const client = clientFor(provider, opts); + + try { + const completion = await client.chat.completions.create( + { + model, + messages: messages as unknown as OpenAI.Chat.Completions.ChatCompletionMessageParam[], + ...(tools !== undefined && tools.length > 0 + ? { + tools: tools as unknown as OpenAI.Chat.Completions.ChatCompletionTool[], + tool_choice: toolChoice ?? "auto", + } + : {}), + ...(opts.temperature !== undefined && supportsTemperature(model) ? { temperature: opts.temperature } : {}), + ...(opts.maxCompletionTokens !== undefined && opts.maxCompletionTokens > 0 + ? { max_completion_tokens: opts.maxCompletionTokens } + : {}), + }, + { timeout: timeoutMs }, + ); + + const choice = completion.choices[0]; + if (choice === undefined) { + throw new LlmError(`${provider} returned no choices (model=${model})`); + } + const promptText = messages + .map((m) => (typeof m.content === "string" ? m.content : "")) + .filter((t) => t.length > 0) + .join("\n"); + + return { + message: choice.message as unknown as OpenRouterMessageInput & { tool_calls?: OpenRouterToolCall[] }, + finishReason: choice.finish_reason ?? null, + usage: { + model: completion.model.length > 0 ? completion.model : model, + inputTokens: completion.usage?.prompt_tokens ?? tokenLen(promptText), + outputTokens: completion.usage?.completion_tokens ?? tokenLen(choice.message.content ?? ""), + // Only OpenRouter reports a per-call price. + costUsd: 0, + }, + }; + } catch (cause: unknown) { + throw cause instanceof LlmError || cause instanceof LlmConfigError + ? cause + : new LlmError(`${provider} tool turn failed (model=${model}): ${causeMessage(cause)}`, cause); + } +} diff --git a/packages/llm/src/toolLoop.ts b/packages/llm/src/toolLoop.ts index 29f81d3..85e4ddc 100644 --- a/packages/llm/src/toolLoop.ts +++ b/packages/llm/src/toolLoop.ts @@ -1,6 +1,13 @@ import { LlmError } from "@bb/errors"; import { logger } from "@bb/logger"; +import { getConfigValue } from "@bb/config"; +import { Config } from "@bb/types"; import { resolveOpenRouterChain } from "./openrouter.ts"; +import { resolveAnthropicChain } from "./anthropic.ts"; +import { resolveBedrockChain } from "./bedrock.ts"; +import { resolveGeminiChain } from "./gemini.ts"; +import { resolveOpenAiChain } from "./openai.ts"; +import { toolChat, type ToolChatResult } from "./toolChat.ts"; import { openRouterRawChat, type OpenRouterMessageInput, type OpenRouterToolDef } from "./openrouterChat.ts"; import type { AskLLMWithToolsOptions, @@ -9,7 +16,7 @@ import type { ToolDefinition, ToolInvocation, } from "./toolTypes.ts"; -import type { AskLlmOptions, AskLlmUsage } from "./client.ts"; +import type { AskLlmOptions, AskLlmUsage, LlmProviderName } from "./client.ts"; const DEFAULT_PER_REQUEST_TIMEOUT_MS = 120_000; const DEFAULT_MAX_TOOL_RESULT_CHARS = 20_000; @@ -37,12 +44,24 @@ const TRUNCATED_MARKER = "\n…[truncated]"; // ───────────────────────────────────────────────────────────────────────────── export async function askLLMWithTools(opts: AskLLMWithToolsOptions): Promise { - const provider = opts.provider ?? "openrouter"; - if (provider !== "openrouter") { - throw new LlmError(`askLLMWithTools: provider "${provider}" does not support tool use`); + // Default to the *configured* provider, not to OpenRouter. Defaulting to + // OpenRouter made a non-OpenRouter deployment fail inside + // `resolveOpenRouterChain` with "bytebell keys set" — an error about a key + // the operator deliberately never set. Now the capability guard below fires + // first and names the real problem. + const provider: LlmProviderName = opts.provider ?? (getConfigValue(Config.LlmProvider) as LlmProviderName); + if (!providerSupportsTools(provider)) { + throw new LlmError( + `askLLMWithTools: provider "${provider}" does not support tool use. ` + + `The concept-graph strategy requires it — either run ` + + `\`bytebell set ingestion.strategy flat-folder\`, or switch to a ` + + `tool-capable provider.`, + ); } const subOpts = buildSubOpts(opts); - const chain = resolveOpenRouterChain(subOpts); + // OpenRouter keeps its own chain resolver: it carries the key check, the + // capped 3-model server-side chain, and its typed LlmConfigError. + const chain = provider === "openrouter" ? resolveOpenRouterChain(subOpts) : resolveToolChain(provider, subOpts); const perRequestTimeoutMs = opts.perRequestTimeoutMs ?? DEFAULT_PER_REQUEST_TIMEOUT_MS; const maxToolResultChars = opts.maxToolResultChars ?? DEFAULT_MAX_TOOL_RESULT_CHARS; const toolDefs = toOpenRouterTools(opts.tools); @@ -77,7 +96,7 @@ export async function askLLMWithTools(opts: AskLLMWithToolsOptions): Promise { + if (provider === "openrouter") { + return openRouterRawChat(messages, chain, opts, timeoutMs, toolDefs, toolChoice); + } + const model = chain[0] ?? ""; + return toolChat(provider, model, messages, opts, timeoutMs, toolDefs, toolChoice); +} diff --git a/packages/server/src/index.ts b/packages/server/src/index.ts index 9675380..555e068 100755 --- a/packages/server/src/index.ts +++ b/packages/server/src/index.ts @@ -3,7 +3,7 @@ import { writeFile } from "node:fs/promises"; import path from "node:path"; import express from "express"; import { Config, DbProviderType, GraphProviderType, QueueProviderType, type Config as ConfigEnum } from "@bb/types"; -import { getBytebellHome, getConfigValue, HINTS } from "@bb/config"; +import { getBytebellHome, getConfigValue, HINTS, requiredKeysFor, type LlmProvider } from "@bb/config"; import { connectDb } from "@bb/db"; import { connectGraph, indexesGraph } from "@bb/graph-db"; import { connectQueue, resumeOrphans } from "@bb/queue"; @@ -32,13 +32,15 @@ import { registerRoutes } from "./routes.ts"; import { installShutdownHandlers } from "./shutdown.ts"; import { reconcileLegacyLayout } from "./legacyLayout.ts"; +// Infra keys only. The LLM credentials are provider-dependent and come from +// `requiredKeysFor(llm_provider)` — hardcoding the OpenRouter key here blocked +// boot for every deployment that deliberately chose a different backend. const REQUIRED: ConfigEnum[] = [ Config.MongoUri, Config.RedisUrl, Config.Neo4jUri, Config.Neo4jUser, Config.Neo4jPassword, - Config.OpenrouterApiKey, ]; function checkRequiredConfig(): void { @@ -48,7 +50,11 @@ function checkRequiredConfig(): void { const graphProvider = getConfigValue(Config.GraphProvider); const queueProvider = getConfigValue(Config.QueueProvider); - const required = [...REQUIRED]; + const llmProvider = getConfigValue(Config.LlmProvider); + // requiredKeysFor() returns the shared infra keys plus the active provider's + // credentials; REQUIRED already covers the infra half, so take the delta. + const llmKeys = requiredKeysFor(llmProvider as LlmProvider).filter((k) => !REQUIRED.includes(k)); + const required = [...REQUIRED, ...llmKeys]; const remove = (key: ConfigEnum): void => { const idx = required.indexOf(key); if (idx !== -1) { diff --git a/packages/types/src/config.ts b/packages/types/src/config.ts index 78d569b..fa16bea 100644 --- a/packages/types/src/config.ts +++ b/packages/types/src/config.ts @@ -18,6 +18,21 @@ export enum Config { LlmProvider = "llm_provider", OllamaUrl = "ollama_url", OllamaModel = "ollama_model", + AnthropicApiKey = "anthropic_api_key", + AnthropicModel = "anthropic_model", + BedrockApiKey = "bedrock_api_key", + BedrockRegion = "bedrock_region", + BedrockModel = "bedrock_model", + GeminiApiKey = "gemini_api_key", + GeminiModel = "gemini_model", + OpenaiApiKey = "openai_api_key", + OpenaiModel = "openai_model", + /** Override for self-hosted OpenAI-compatible servers (vLLM / LiteLLM / gateway). */ + OpenaiBaseUrl = "openai_base_url", + /** Bedrock SigV4 auth — used when `bedrock_api_key` is unset. */ + AwsAccessKeyId = "aws_access_key_id", + AwsSecretAccessKey = "aws_secret_access_key", + AwsSessionToken = "aws_session_token", ContextWindowLimit = "context.window.limit", MaxTokensPerChunk = "max.tokens.per.chunk", BigFileConcurrency = "big.file.concurrency", @@ -38,6 +53,7 @@ export enum Config { DbProvider = "db_provider", GraphProvider = "graph_provider", QueueProvider = "queue_provider", + InfraMode = "infra_mode", QueueDbPath = "queue_db_path", SqlitePath = "sqlite_path", LadybugPath = "ladybug_path", @@ -80,6 +96,20 @@ export enum QueueProviderType { Bullmq = "bullmq", Honker = "honker", } + +/** + * The PUBLIC LLM backends the open-source engine ships. `llm_provider` is a + * free string in the config schema, so a downstream deployment may select a + * backend this enum does not enumerate. + */ +export enum LlmProviderType { + OpenRouter = "openrouter", + Ollama = "ollama", + Anthropic = "anthropic", + Bedrock = "bedrock", + Gemini = "gemini", + OpenAi = "openai", +} /** * The PUBLIC ingestion strategies. `flat-folder` is the historic default that * produces `:Repo` + `:Folder` summaries via per-folder LLM passes.