diff --git a/README.md b/README.md index 4875c62..cb244f7 100644 --- a/README.md +++ b/README.md @@ -57,28 +57,58 @@ Supply credentials through a secured environment or protected files, not secret- | Default environment variable | Purpose | | --- | --- | | `JEV_API_KEY` | Upstream Jev API access for decisions and the MCP server's backend | +| `GOOGLE_API_KEY`, then `GEMINI_API_KEY` | Google Gemini Decisions API access when the `gemini` provider is selected | | `JEVWISE_MCP_TOKEN` | Optional override for local MCP authentication; otherwise generated and privately stored | +### Providers + +| Provider | Service | Key | +| --- | --- | --- | +| `jev` (default) | TypeSafe Jev API | `JEV_API_KEY` | +| `gemini` | Google Gemini Decisions API (`gemini-decision-1`) | `GOOGLE_API_KEY`, then `GEMINI_API_KEY` | +| `codex`, `claude` | Reserved, not implemented | | + +Select one per call with `--provider gemini`, or for every call with `provider = "gemini"` under `[general]` in your configuration. Each provider reads only its own keys, so a Jev key never reaches Google and the reverse. See [configuration](docs/configuration.md#gemini-decisions-provider). + ### Upstream API key Set `JEV_API_KEY` in your secured environment, or reference **any exported variable** in your selected TOML configuration: ```toml +[provider.jev] api_key = "$ABCD_TOKEN" # Equivalent: api_key = "${ABCD_TOKEN}" ``` - `ABCD_TOKEN` is only an example; custom names such as `JEV_MCP_API_KEY` also work. Export the referenced variable before starting Jevwise. -- Only TOML `api_key` expands environment references. Unset variables expand to empty; Jev service initialization requires a nonblank key. +- Only the TOML `api_key` expands environment references. Unset variables expand to empty; Jev service initialization requires a nonblank key. - Changed `--api-key` flags override nonblank API environment values, which override TOML. Direct environment and flag values stay literal. - API fallback names, in order: `JEVWISE_API_KEY`, `TS_JEV_API_KEY`, `TYPESAFE_API_KEY`. The standalone Go SDK retains `TYPESAFE_API_KEY` as its environment fallback. See [configuration](docs/configuration.md). +### Google API key + +Jevwise sends the key in the `x-goog-api-key` header; it does not use OAuth or Application Default Credentials (ADC) at runtime. Get a key in one of two ways: + +| Way | Best for | How | +| --- | --- | --- | +| 1. Personal (AI Studio) | Trying it out, personal use | Sign in at [aistudio.google.com/api-keys](https://aistudio.google.com/api-keys) and create a key. AI Studio sets up a Google Cloud project if you have none. | +| 2. Organization project (Terraform) | Teams, billing and key restrictions under an organization | Use ADC once to let Terraform enable the Gemini API, link billing and create a key restricted to `generativelanguage.googleapis.com`. Follow [Google API key via Terraform](docs/gcp_adc.md). | + +```shell +export GEMINI_API_KEY='...' # or GOOGLE_API_KEY, which wins if both are set +jevwise decide --provider gemini --prompt "What is 2 + 2?" --option 4 --option 5 +``` + +- Restrict the key to the Generative Language API, and keep it out of command arguments and stored files, like the Jev key. +- A paid project needs a positive Gemini prepaid balance in AI Studio; at $0 its keys stop working. See the [Terraform guide](docs/gcp_adc.md#7-buy-gemini-prepaid-credits). +- Work accounts may block key creation by organization policy; the error names the constraint. + ### Local MCP authentication - Use a separate bearer token: **never reuse the upstream API key or give it to the agent**. - With no token override, `jevwise mcp` generates a random token in `/agent.token` and reuses it across restarts. Only your upstream API key needs manual setup. - `jevwise mcp token-path` prints the automatic file path, never the secret; see [agent onboarding](docs/mcp-agents.md) to load it into the client environment. -- Register the installed client with `jevwise mcp install codex` or `jevwise mcp install claude`. Both default to user-wide settings; custom ports use `--listen IP:PORT` or `mcp_port` in [configuration](docs/configuration.md). Registration stores only an environment reference, not the token. See [agent onboarding](docs/mcp-agents.md#3-choose-a-client). +- Register the installed client with `jevwise mcp install codex` or `jevwise mcp install claude`. Both default to user-wide settings; custom ports use `--listen IP:PORT` or `port` under `[mcp]` in [configuration](docs/configuration.md). Registration stores only an environment reference, not the token. See [agent onboarding](docs/mcp-agents.md#3-choose-a-client). - Overrides take precedence: `--token-file /path/to/protected/agent-token` > `JEVWISE_MCP_TOKEN` (including aliases) > automatic token. Explicit values are never copied into the automatic file. - MCP fallback names, in order: `JEVWISE_MCP_AUTH_TOKEN`, `JEV_MCP_TOKEN`. Invalid nonempty tokens fail without fallback; tokens must be printable ASCII without whitespace. - Keep secrets out of shell history, logs and stored agent configuration. Follow [local MCP setup](docs/mcp.md) and [agent onboarding](docs/mcp-agents.md) for protected files and client environment references. @@ -88,7 +118,8 @@ api_key = "$ABCD_TOKEN" | Guide | Contents | | --- | --- | | [CLI usage](docs/usage.md) | Decisions and skill setup | -| [Configuration](docs/configuration.md) | Precedence, API keys, local config and editor | +| [Configuration](docs/configuration.md) | Precedence, API keys, providers, local config and editor | +| [Google API key via Terraform](docs/gcp_adc.md) | Organization project key for the Gemini provider | | [Go SDK](docs/sdk.md) | Quickstart, request options and retries | | [Development](docs/development.md) | Source builds, platform builds and tests | | [Debugging](docs/debug.md) | Debug logging and safe diagnostics | diff --git a/config.example.toml b/config.example.toml index 86b5efb..1445120 100644 --- a/config.example.toml +++ b/config.example.toml @@ -1,13 +1,23 @@ # Optional CLI configuration. Prefer an environment variable for credentials. -# Copy to ~/.config/ts-jev/config.toml, or select with --config. -api_key = "" -base_url = "https://api.typesafe.ai" -model = "jev-latest" +# Copy to ~/.config/jevwise/jevwise.toml, or select with --config. +[general] provider = "jev" -timeout = "10s" -editor = "vim" -# Local MCP server: literal loopback IP (no scheme) and port. -mcp_base_url = "127.0.0.1" -mcp_port = 8080 # Structured debug logs on stderr; --debug or --verbose overrides. debug = false +editor = "vim" + +# Local MCP server: literal loopback IP (no scheme) and port. +[mcp] +host = "127.0.0.1" +port = 8080 + +[provider.jev] +api_key = "$JEV_API_KEY" +base_url = "https://api.typesafe.ai" +model = "jev-latest" +timeout = "10s" + +# Google Gemini Decisions: select with provider = "gemini" or --provider gemini. +[provider.gemini] +api_key = "$GEMINI_API_KEY" +model = "gemini-decision-1" diff --git a/docs/configuration.md b/docs/configuration.md index 97ef5d4..748f5aa 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -6,27 +6,87 @@ Precedence: **changed flags > nonblank environment > TOML > defaults**. These names apply to the CLI; the standalone Go SDK retains its `TYPESAFE_API_KEY` fallback. -| Setting | Flag | Environment | Default | +Settings are grouped into tables. `[general]` selects the provider, which reads its settings from `[provider.]` (`jev`, `gemini`, `codex` or `claude`); other provider tables are ignored. + +```toml +[general] +provider = "jev" +debug = false +editor = "vim" + +[mcp] +host = "127.0.0.1" +port = 8080 + +[provider.jev] +api_key = "$JEV_API_KEY" +base_url = "https://api.typesafe.ai" +model = "jev-latest" +timeout = "10s" +``` + +| TOML key | Flag | Environment | Default | | --- | --- | --- | --- | -| API key | `--api-key` | `JEV_API_KEY`; aliases: `JEVWISE_API_KEY`, `TS_JEV_API_KEY`, `TYPESAFE_API_KEY` | Required for Jev | -| Base URL | `--base-url` | `TS_JEV_BASE_URL`, then `TYPESAFE_BASE_URL` | `https://api.typesafe.ai` | -| Model | `--model` | `TS_JEV_MODEL`, then `TYPESAFE_DEFAULT_MODEL` | `jev-latest` | -| Provider | `--provider` | `TS_JEV_PROVIDER` | `jev` | -| Operation timeout | `--timeout` | `TS_JEV_TIMEOUT` | `10s` | -| MCP host | `--listen` (with port) | `TS_JEV_MCP_BASE_URL` | `127.0.0.1` | -| MCP port | `--listen` (with host) | `TS_JEV_MCP_PORT` | `8080` | -| Debug logging | `--debug`, `--verbose` | `TS_JEV_DEBUG` | `false` | - -- `mcp_base_url` is a literal loopback IP without scheme or port (`127.0.0.1`, `::1`); `http://`, `localhost` and non-loopback hosts fail. `mcp_port` is an integer from 1 to 65535 (`mcp_port = 8080` or `"8080"`). `jevwise mcp` and `jevwise mcp install` read only these two keys for the address; a changed `--listen` overrides both, so `jevwise mcp --listen 127.0.0.1:9090` equals `jevwise mcp` with `mcp_port = 9090`. +| `general.provider` | `--provider` | `TS_JEV_PROVIDER` | `jev` | +| `general.debug` | `--debug`, `--verbose` | `TS_JEV_DEBUG` | `false` | +| `general.editor` | `config edit --editor` | Not read | `vim` | +| `mcp.host` | `--listen` (with port) | `TS_JEV_MCP_BASE_URL` | `127.0.0.1` | +| `mcp.port` | `--listen` (with host) | `TS_JEV_MCP_PORT` | `8080` | +| `provider..api_key` | `--api-key` | See provider table below | Required for Jev and Gemini | +| `provider..base_url` | `--base-url` | See provider table below | See provider table below | +| `provider..model` | `--model` | See provider table below | See provider table below | +| `provider..timeout` | `--timeout` | `TS_JEV_TIMEOUT` | `10s` | + +Provider environment names and defaults. Each provider reads only its own names, so a Jev key or URL never reaches Gemini, and the reverse: + +| Provider | API key environment | Base URL environment, default | Model environment, default | +| --- | --- | --- | --- | +| `jev` (also `codex`, `claude`) | `JEV_API_KEY`; aliases: `JEVWISE_API_KEY`, `TS_JEV_API_KEY`, `TYPESAFE_API_KEY` | `TS_JEV_BASE_URL`, then `TYPESAFE_BASE_URL`; `https://api.typesafe.ai` | `TS_JEV_MODEL`, then `TYPESAFE_DEFAULT_MODEL`; `jev-latest` | +| `gemini` | `GOOGLE_API_KEY`, then `GEMINI_API_KEY` (as in google-genai) | `GOOGLE_GEMINI_BASE_URL`; `https://generativelanguage.googleapis.com` | None; `gemini-decision-1` | + +- Changed flags apply to the selected provider. Other environment names are unchanged from the flat layout, so `TS_JEV_MCP_BASE_URL` still sets `mcp.host`. + +### Gemini Decisions provider + +Select Google's [Gemini Decisions API](https://blog.google/innovation-and-ai/technology/developers-tools/gemini-decisions-api/) (`gemini-decision-1`) for `decide` and MCP: + +```toml +[general] +provider = "gemini" + +[provider.gemini] +api_key = "$GEMINI_API_KEY" +``` + +- Or select it per call with `--provider gemini`, keeping the Jev settings in place. The key comes from `GOOGLE_API_KEY`, then `GEMINI_API_KEY`, then TOML; never from Jev names. +- Each decision sends `POST /v1beta/decisions` with one `choice` question: the prompt is the shared `input`, every option is a criterion with an empty description, and a fixed instruction asks for the best-fitting option. At most 255 options. +- The response carries no model name, so output reports the configured model. Probabilities and confidence are returned as received, never normalized. +- Inputs blocked by Google's content filters fail the decision. Upstream error messages are not shown or logged, as they may echo input. +- The endpoint is `v1beta`: Google may change it. Ordinary decisions bill input tokens to your Google account. +- `mcp.host` is a literal loopback IP without scheme or port (`127.0.0.1`, `::1`); `http://`, `localhost` and non-loopback hosts fail. `mcp.port` is an integer from 1 to 65535 (`port = 8080` or `"8080"`). `jevwise mcp` and `jevwise mcp install` read only these two keys for the address; a changed `--listen` overrides both, so `jevwise mcp --listen 127.0.0.1:9090` equals `jevwise mcp` with `port = 9090` under `[mcp]`. + +### Upgrading a flat configuration + +Files from earlier releases put every key at the top level (`api_key = ...`, `mcp_port = ...`). They are no longer read: commands that load settings fail and ask you to migrate. Convert the selected file once: + +```sh +jevwise config migrate # selected local or global file +jevwise config migrate --config PATH # an explicit file +``` + +- Keys move to their tables unchanged: `$NAME` references stay literal and values are not validated, so you can fix them afterwards with `jevwise config edit`. `mcp_base_url` becomes `mcp.host` and `mcp_port` becomes `mcp.port`. +- The original is kept beside the file as `.flat.bak` (owner-only). It may contain your API key: delete it once the migrated file is correct. Migration refuses to run if that backup exists, and refuses links, unknown keys, files mixing both layouts, and values other than strings, integers or booleans; errors name keys, never values. +- Comments are not carried over. A file already using tables is left unchanged. +- `config edit`, `version` and `mcp status`/`stop` still run on a flat file, so it can be repaired or migrated. - Explicit file: `--config /path/to/config.toml`. Missing or invalid explicit files fail. - Otherwise: current-directory `.jevwise.toml`, then `$XDG_CONFIG_HOME/jevwise/jevwise.toml` or `$HOME/.config/jevwise/jevwise.toml`. Only a missing global file is allowed. The selected local file replaces rather than merges with global settings; no ancestor-directory search or visible-name fallback. - On Windows, the default global path is `%USERPROFILE%\.config\jevwise\jevwise.toml` when `XDG_CONFIG_HOME` is unset. - Quoted `--config "~/.config/jevwise/jevwise.toml"` paths expand the current user's home directory for init, view, edit and runtime loading. `~otheruser` and shell/environment evaluation of paths are not supported. An XDG path beginning with `~/` is supported too. - Trust the current directory before running decision/MCP commands: local settings can change the upstream destination. Environment/changed flags still win. Invalid, oversized (over 1 MiB), linked or nonregular local files fail without global fallback; help/version do not load them, except that `version` and other commands read only the `debug` key and ignore an invalid or unreadable file. - Start with [config.example.toml](../config.example.toml). Prefer environment credentials rather than command-line secrets. -- TOML `api_key` supports `$NAME`/`${NAME}`, for example `api_key = "$JEV_API_KEY"` or `api_key = "${JEV_MCP_API_KEY}"`. Runtime loading expands references and trims the result; undefined variables become empty. +- TOML `provider..api_key` supports `$NAME`/`${NAME}`, for example `api_key = "$JEV_API_KEY"` or `api_key = "${JEV_MCP_API_KEY}"`. Runtime loading expands references and trims the result; undefined variables become empty. - API environment names above are checked left to right, skipping blank values. `JEV_API_KEY` is for the upstream API; `JEVWISE_MCP_TOKEN` is separate local MCP authentication. Never reuse the same credential. -- TOML `api_key` supports `$NAME`/`${NAME}`, for example `api_key = "${MY_JEV_KEY}"`. Runtime loading expands references and trims the result; undefined variables become empty. +- TOML `provider..api_key` supports `$NAME`/`${NAME}`, for example `api_key = "${MY_JEV_KEY}"`. Runtime loading expands references and trims the result; undefined variables become empty. - Custom environment names work through TOML references, for example `api_key = "${JEV_MCP_API_KEY}"`. Export the variable in the process environment before starting Jevwise. Exporting `JEV_MCP_API_KEY` alone is not an automatic API-key override; nonblank `TS_JEV_API_KEY`/`TYPESAFE_API_KEY` still take precedence over the file. - Local discovery never loads `config.toml`, `configs.toml`, YAML files or ancestor configs. Only `.jevwise.toml` is a local candidate; visible `jevwise.toml` is not auto-loaded. Legacy `ts-jev/config.toml` is no longer discovered: migrate manually or select it with `--config`. An explicit `--config PATH` can still select another trusted filename. - `JEV_MCP_API_KEY` is your chosen upstream-key variable when referenced by `api_key`; it is not the separate MCP agent bearer token (`JEV_MCP_TOKEN` or `--token-file`). @@ -41,11 +101,12 @@ jevwise config view # Selected file/defaults, API key redacted jevwise config view --config /path/to/config.toml jevwise config edit # Selected local/global file, vim by default jevwise config edit --editor nvim # One-shot override, also for repairs +jevwise config migrate # Convert a flat file to grouped tables ``` - Init never overwrites existing files, directories or links. It writes a template, not your environment credentials; Unix file permissions are requested as `0600`, new parent directories as `0700`. Use a trusted parent with safe ownership/ACLs: mode bits alone do not guarantee privacy, including inherited Unix ACLs. Windows users must manage ACLs separately. -- `--config PATH` also selects an init destination; do not combine it with `--local` or pass a blank path. View ignores environment/decision flag overrides, unknown keys and comments; invalid files fail safely. It is not a dump of effective runtime settings. Only the `api_key` field is automatically redacted; do not store credentials in other displayed settings. -- Set `editor = "nvim"` (or another executable name/path) in TOML to change the editor. Selection is `config edit --editor` > selected file's `editor` > `vim` on PATH; `$EDITOR`/`$VISUAL` are not read. No editor is bundled and no shell is used. Arguments are not parsed: use a trusted wrapper executable if options such as `--wait` are needed. +- `--config PATH` also selects an init destination; do not combine it with `--local` or pass a blank path. View prints the `[general]`, `[mcp]` and selected `[provider.]` tables. It ignores environment/decision flag overrides, unknown keys, other provider tables and comments; invalid files fail safely. It is not a dump of effective runtime settings. Only the `api_key` field is automatically redacted; do not store credentials in other displayed settings. +- Set `editor = "nvim"` (or another executable name/path) under `[general]` to change the editor. Selection is `config edit --editor` > selected file's `general.editor` (or a flat file's `editor`) > `vim` on PATH; `$EDITOR`/`$VISUAL` are not read. No editor is bundled and no shell is used. Arguments are not parsed: use a trusted wrapper executable if options such as `--wait` are needed. - Edit forwards terminal input/output and waits for the direct editor process. It uses the command context, not the decision timeout; missing editors/nonzero exits fail safely. Invalid TOML falls back to Vim for repair; wrong-typed editor settings or oversized files can be repaired using `--editor`. It refuses missing files, directories and final symlinks. - Go-copy pipe waits are bounded after process exit/cancellation. Only the direct editor process is managed, not wrapper descendants; arbitrary blocking custom readers/writers cannot be forcibly interrupted. - Trust local editor settings and wrappers before editing: they select a program to execute. Editor terminal output intentionally may show the file you are editing. macOS/Linux support assumes an installed editor; no Windows editor availability/behavior is promised. diff --git a/docs/debug.md b/docs/debug.md index df90089..56ef0a0 100644 --- a/docs/debug.md +++ b/docs/debug.md @@ -10,7 +10,7 @@ jevwise --debug mcp jevwise mcp --verbose # Same logging as --debug ``` -- Debug logging is opt-in via `--debug`, its `--verbose` alias, `TS_JEV_DEBUG=true` or TOML `debug = true`, uses `github.com/phuslu/log`, and writes JSON to stderr. Normal stdout stays unchanged; logs can be redirected with `2>debug.log`. +- Debug logging is opt-in via `--debug`, its `--verbose` alias, `TS_JEV_DEBUG=true` or `debug = true` under `[general]`, uses `github.com/phuslu/log`, and writes JSON to stderr. Normal stdout stays unchanged; logs can be redirected with `2>debug.log`. - Covers command/stage outcomes and timing, config loading, API attempts/retries/status codes, release verification/replacement, skills, MCP requests/lifecycle and authenticated management. - Never logs credentials, prompts/options/results, user-supplied URLs/paths, raw headers/bodies, or raw error causes. Release checks may log their fixed public GitHub endpoint and strictly numeric rate-limit metadata. Treat timing/status diagnostics as operationally sensitive when sharing logs. - Detached background mode traces the launcher only; child output remains isolated. There is no persistent debug log file. Use foreground MCP for request diagnostics, then restart without `--debug` when finished. @@ -18,7 +18,7 @@ jevwise mcp --verbose # Same logging as --debug ## MCP startup failures -- Either flag enables diagnostics; `--debug=false --verbose` still logs. A changed flag overrides `TS_JEV_DEBUG` and TOML `debug`, so `--debug=false` disables a `debug = true` setting. Without a flag, an invalid or unreadable setting means no debug trace; the detached background child ignores the setting. +- Either flag enables diagnostics; `--debug=false --verbose` still logs. A changed flag overrides `TS_JEV_DEBUG` and `general.debug`, so `--debug=false` disables a `debug = true` setting. Without a flag, an invalid or unreadable setting means no debug trace; the detached background child ignores the setting. - Foreground startup logs classify missing/invalid upstream API keys, unsupported providers, client configuration errors and failed/empty factories, without showing values or raw causes. Known credential failures also print actionable errors without enabling debugging. -- Set a Jev API key through `TS_JEV_API_KEY`, `TYPESAFE_API_KEY` or TOML `api_key`. The separate MCP agent bearer token is not the upstream key; see [configuration](configuration.md) and [MCP credentials](mcp.md#credentials-state-and-safety). +- Set a Jev API key through `TS_JEV_API_KEY`, `TYPESAFE_API_KEY` or `api_key` under `[provider.jev]`. The separate MCP agent bearer token is not the upstream key; see [configuration](configuration.md) and [MCP credentials](mcp.md#credentials-state-and-safety). - If neither flag is listed by `jevwise mcp --help`, check `jevwise version` and which binary is on PATH. Older published binaries may predate logging even when the current source supports it. For Homebrew-managed installs, use `brew upgrade jevwise` once a release containing these changes is available, not the standalone updater. diff --git a/docs/gcp_adc.md b/docs/gcp_adc.md new file mode 100644 index 0000000..b2a99bf --- /dev/null +++ b/docs/gcp_adc.md @@ -0,0 +1,172 @@ +# Google API + +Gemini API key via Terraform +Terraform that links billing and creates a restricted Gemini API key in the existing project `YOUR_PROJECT_ID`. + +The Terraform files live in your own infrastructure repository; this page is the runbook. Replace `YOUR_PROJECT_ID`, `YOUR_ACCOUNT_EMAIL` and `YOUR_BILLING_ACCOUNT_NAME` with your values. + +## What Terraform manages + +| File | Resources | +|---|---| +| `main.tf` | `locals` (including `project_id`) | +| `policy.tf` | Enables `apikeys` and `generativelanguage`, creates the API key | +| `billing.tf` | Looks up the billing account, links it to the project | +| `outputs.tf` | `api_key_id`, `api_key` (sensitive), `billing_account` | + +The key is restricted to `generativelanguage.googleapis.com`. + +## What stays manual + +| Item | Why | +|---|---| +| Quota project on ADC | Terraform cannot set your local credentials | +| Enable `serviceusage` and `cloudbilling` APIs | Data sources read at plan time, so these must exist before the first apply | +| IAM roles for your user | Needed before Terraform can run | +| Gemini API prepaid credits | Bought in AI Studio, not available in Terraform | + +## Prerequisites + +- `gcloud` CLI and `terraform` (1.5 or newer if you use import blocks) +- Account: `YOUR_ACCOUNT_EMAIL` +- An existing project: `YOUR_PROJECT_ID` +- An existing billing account named `YOUR_BILLING_ACCOUNT_NAME` + +## 1. Authenticate + +```shell +gcloud auth application-default login + +# [optional] impersonate a service account instead +gcloud auth application-default login \ + --impersonate-service-account=SA_EMAIL +``` + +Check which account ADC uses (the `email` field): + +```shell +TOKEN=$(gcloud auth application-default print-access-token) +curl -s "https://oauth2.googleapis.com/tokeninfo?access_token=$TOKEN" +``` + +## 2. Grant roles + +| Role | Where | Why | +|---|---|---| +| `roles/serviceusage.serviceUsageConsumer` | Project | Use the project as quota project | +| `roles/billing.viewer` | Billing account | Look up the account by name | +| `roles/billing.user` | Billing account | Link the project to it | + +Your user also needs permission to manage services and API keys on the project (for example Owner or Editor). + +```shell +gcloud projects add-iam-policy-binding YOUR_PROJECT_ID \ + --member="user:YOUR_ACCOUNT_EMAIL" \ + --role="roles/serviceusage.serviceUsageConsumer" +``` + +The billing roles are granted on the billing account in the console: Billing > Account management > Add principal. + +## 3. Set the quota project + +ADC has no quota project by default. Without one, Google uses a shared project where APIs are disabled (`SERVICE_DISABLED`). + +```shell +gcloud auth application-default set-quota-project YOUR_PROJECT_ID + +export USER_PROJECT_OVERRIDE=true +export GOOGLE_BILLING_PROJECT=YOUR_PROJECT_ID +``` + +## 4. Enable the bootstrap APIs + +Terraform needs these before the first apply. Terraform enables `apikeys` and `generativelanguage` itself. + +```shell +gcloud services enable \ + serviceusage.googleapis.com \ + cloudbilling.googleapis.com \ + --project YOUR_PROJECT_ID +``` + +If `serviceusage` is not enabled yet, enable it in the console first: +https://console.developers.google.com/apis/api/serviceusage.googleapis.com/overview?project=YOUR_PROJECT_ID + +## 5. Import the existing billing link (optional) + +If billing was already linked by hand, import it once so Terraform knows about it: + +```shell +terraform init +terraform import google_billing_project_info.this projects/YOUR_PROJECT_ID +``` + +## 6. Apply + +```shell +terraform plan -out=apikey.tfplan +terraform apply "apikey.tfplan" +``` + +Notes: + +- The billing link has `prevent_destroy = true`. Destroying it could unlink billing and break the key. +- The billing account lookup needs an exact, unique `display_name`. + +## 7. Buy Gemini prepaid credits + +Linking billing is **not enough**. The Gemini API only serves requests with a positive prepaid balance. + +1. Open [AI Studio](https://aistudio.google.com) > Billing. +2. Select the project `YOUR_PROJECT_ID`. +3. Buy prepaid credits. +4. Turn on auto-reload. + +Notes: + +- Google Cloud credits do not apply until you have a positive prepaid balance. +- At $0, all API keys on that billing account stop working at once. + +## 8. Get and test the key + +```shell +# print the key (sensitive output) +terraform output -raw api_key + +# list models with the key +KEY=$(terraform output -raw api_key) +curl -s "https://generativelanguage.googleapis.com/v1beta/models?key=$KEY" +``` + +A JSON list of models means the key works. + +Security notes: + +- The key is stored in plain text in the Terraform state. Use a remote backend with restricted access. +- Never commit state files or the key. + +## Outputs + +| Output | Sensitive | Description | +|---|---|---| +| `api_key_id` | No | Resource ID of the API key | +| `api_key` | Yes | The key string | +| `billing_account` | No | Billing account ID linked to the project | + +## Troubleshooting + +| Error | Cause | Fix | +|---|---|---| +| 403 `SERVICE_DISABLED` on a Google API | No quota project on ADC | Step 3 | +| 403 `IAM_PERMISSION_DENIED` on billing | Missing billing role | Step 2 | +| Permission error listing projects in the billing lookup | No permission to list projects | Set `lookup_projects = false` in `billing.tf` (already set) | +| `Duplicate data configuration` | Same data source defined twice | Keep it in `billing.tf` only | +| 403 `SERVICE_DISABLED` on the key call | Gemini API not enabled | Re-run apply, check `policy.tf` | +| 403 `API_KEY_SERVICE_BLOCKED` | Key restriction does not match the API called | Add the API to `api_targets` | +| 403 right after key creation | Propagation delay | Wait 2 to 5 minutes | +| 402 Payment Required | No Gemini prepaid balance | Step 7 | +| `xargs: command line cannot be assembled, too long` | Token too long for `xargs` | Use the `TOKEN=$(...)` form in step 1 | + +### If key creation is blocked by an org policy + +The error will name the constraint. The old override resource was removed because it was not needed. If you ever need it, apply the override with `roles/orgpolicy.policyAdmin` and enable `orgpolicy.googleapis.com`. diff --git a/docs/mcp-agents.md b/docs/mcp-agents.md index 0b674ee..700f4e8 100644 --- a/docs/mcp-agents.md +++ b/docs/mcp-agents.md @@ -62,7 +62,7 @@ jevwise mcp install claude --scope project # Shared .mcp.json, requires client a - Both default to user-wide settings. Requires the selected `codex` or `claude` executable on PATH; Claude Code must support `--bare`. Standard client configuration overrides (`CODEX_HOME`, `CLAUDE_CONFIG_DIR`) are honored. - Uses native registration commands, preserving unrelated settings. An existing `jevwise` entry is refused; inspect it before intentionally removing/updating it with the client's own commands. Configuration assumes trusted clients and cooperative writers; do not run competing installers/settings edits concurrently. -- Stores `JEVWISE_MCP_TOKEN` as an environment reference, never its value. The installer does not export the token into your shell, read Jev configuration other than `mcp_base_url` and `mcp_port`, start MCP, verify connectivity or make a paid decision. Use section 2 to load the token before launching your client. The endpoint comes from `--listen`, else `mcp_base_url` and `mcp_port`; it must match the running server and does not change the server port. +- Stores `JEVWISE_MCP_TOKEN` as an environment reference, never its value. The installer does not export the token into your shell, read Jev configuration other than `[mcp]` `host` and `port`, start MCP, verify connectivity or make a paid decision. Use section 2 to load the token before launching your client. The endpoint comes from `--listen`, else `[mcp]` `host` and `port`; it must match the running server and does not change the server port. - Child processes do not inherit API keys or MCP tokens. Client output is bounded and withheld, including diagnostics; failures may have partially registered an entry, so inspect settings before retrying. `--timeout` bounds registration (default `10s`), not a server/model request. Only the direct process is managed, not wrapper descendants. - Manual setup remains available below. Installed registration is not proof of a live authenticated connection; verify via `/mcp` without calling `decide`. diff --git a/docs/mcp.md b/docs/mcp.md index 56f92d0..9b414b6 100644 --- a/docs/mcp.md +++ b/docs/mcp.md @@ -33,7 +33,7 @@ jevwise mcp stop | `jevwise mcp install codex` / `claude` | Register the HTTP endpoint using an installed client CLI; no server/model startup or token export. | | `jevwise mcp token-path` | Prints only the automatic token path; does not read credentials, create files or report explicit overrides. | | `jevwise mcp stop` | Authenticates the exact instance and requests shutdown; an acknowledgement is not a promise that cleanup has already finished. | -| `--listen` | Literal loopback IP and nonzero port; overrides `mcp_base_url` and `mcp_port` from [configuration](configuration.md); default `127.0.0.1:8080`. | +| `--listen` | Literal loopback IP and nonzero port; overrides `[mcp]` `host` and `port` from [configuration](configuration.md); default `127.0.0.1:8080`. | | `--runtime-dir` | Private background state directory; default `os.UserCacheDir()/jevwise-mcp`. Use the same override for start, status and stop. | | `--token-file` | Explicit protected agent-token file, overriding `JEVWISE_MCP_TOKEN`; unreadable or unsafe files fail without fallback. | @@ -52,7 +52,7 @@ jevwise mcp stop For token-safe Codex and Claude Code setup, discovery checks and cleanup, follow the [local agent onboarding guide](mcp-agents.md). -Connect an HTTP-capable local agent to `http://127.0.0.1:8080/mcp`, supplying `Authorization: Bearer `. The only tool is `decide`: send `prompt` and at least two distinct `options`. Results contain Jev probabilities and optional usage; ordinary decisions can incur upstream API charges. +Connect an HTTP-capable local agent to `http://127.0.0.1:8080/mcp`, supplying `Authorization: Bearer `. The only tool is `decide`: send `prompt` and at least two distinct `options`. Results contain the configured provider's probabilities (Jev or Gemini Decisions) and optional usage; ordinary decisions can incur upstream API charges. - Stateless Streamable HTTP uses POST with SSE responses, not legacy `/sse` or standalone GET streams. GET/DELETE do not create sessions. Supported revisions: 2026-07-28, 2025-11-25, 2025-06-18, 2025-03-26 and 2024-11-05; JSON-RPC batches are rejected. - Only literal loopback addresses are supported. Requests need the exact configured Host and, when supplied, the exact `http://IP:port` Origin. Proxy headers do not establish trust; this is not a public-network deployment mode. @@ -65,7 +65,7 @@ Connect an HTTP-capable local agent to `http://127.0.0.1:8080/mcp`, supplying `A | Credential | Purpose | Where it belongs | | --- | --- | --- | -| Upstream API key | Server-to-Jev decisions | Existing decision configuration; never give it to the agent. | +| Upstream API key | Server-to-provider decisions (Jev or Gemini) | Existing decision configuration; never give it to the agent. | | Agent bearer token | Agent-to-local `/mcp` access | Automatic private `agent.token`, `JEVWISE_MCP_TOKEN`, or an explicit protected token file. | | Management key | Status/stop and instance verification | Separate private runtime file; never give it to the agent or transmit it as a bearer token. | diff --git a/internal/cli/config.go b/internal/cli/config.go index 98c6026..2eefd18 100644 --- a/internal/cli/config.go +++ b/internal/cli/config.go @@ -93,6 +93,33 @@ func newConfig() *cobra.Command { } return nil }} - cmd.AddCommand(init, view, newConfigEdit(runConfigEditor)) + migrate := &cobra.Command{Use: "migrate", Short: "Convert a flat configuration file to grouped tables, keeping a backup", Args: configNoArgs, + RunE: func(cmd *cobra.Command, _ []string) error { + if err := cmd.Context().Err(); err != nil { + return err + } + path, err := configPathFlag(cmd) + if err != nil { + return err + } + path, _, err = config.ResolvePath(path) + if err != nil { + return err + } + debuglog.Event(cmd.Context(), "config.migrate.rewrite") + backup, err := config.Migrate(cmd.Context(), path) + message := fmt.Sprintf("Migrated configuration: %q\nOriginal kept at: %q\nComments were not carried over; check the result with jevwise config view.\nThe backup may contain your API key: delete it once the result is correct.\n", path, backup) + if errors.Is(err, config.ErrNotFlat) { + message, err = fmt.Sprintf("Configuration already uses grouped tables; nothing changed: %q\n", path), nil + } + if err != nil { + return err + } + if n, err := fmt.Fprint(cmd.OutOrStdout(), message); err != nil || n != len(message) { + return errors.New("configuration migrated but output failed") + } + return nil + }} + cmd.AddCommand(init, view, migrate, newConfigEdit(runConfigEditor)) return cmd } diff --git a/internal/cli/config_edit_test.go b/internal/cli/config_edit_test.go index 61e345f..08ac725 100644 --- a/internal/cli/config_edit_test.go +++ b/internal/cli/config_edit_test.go @@ -39,16 +39,16 @@ func TestConfigEditSelectionStreamsAndRepair(t *testing.T) { if mode == "explicit" { path = filepath.Join(t.TempDir(), "selected file.toml") args = append(args, "--config", path) - _ = os.WriteFile(config.LocalName, []byte("editor='wrong-local'"), 0600) + _ = os.WriteFile(config.LocalName, []byte("[general]\neditor='wrong-local'"), 0600) } if mode == "local" || mode == "global" || mode == "explicit" { - content, want = "editor='configured editor'", "configured editor" + content, want = "[general]\neditor='configured editor'", "configured editor" } if mode == "malformed" { - content = "api_key='private-secret'\n[" + content = "[provider.jev]\napi_key='private-secret'\n[" } if mode == "override" { - content, want = "editor=42", "override-editor" + content, want = "[general]\neditor=42", "override-editor" args = append(args, "--editor", want) } _ = os.MkdirAll(filepath.Dir(path), 0700) @@ -88,7 +88,7 @@ func TestConfigEditInvalidRequestsHelpAndSafeFailure(t *testing.T) { t.Fatal("invalid edit accepted or leaked", err) } } - _ = os.WriteFile(config.LocalName, []byte("editor='vim'"), 0600) + _ = os.WriteFile(config.LocalName, []byte("[general]\neditor='vim'"), 0600) if _, _, err := execute(editorRoot(t, func(context.Context, string, string, io.Reader, io.Writer, io.Writer) error { return errors.New("private-secret") }), "config", "edit"); err == nil || strings.Contains(err.Error(), "private-secret") { @@ -139,7 +139,7 @@ func TestConfigEditorProcessNoShellAndCancellation(t *testing.T) { func TestConfigEditRejectsBadEditorAndCancelledRequests(t *testing.T) { configEnvironment(t) - _ = os.WriteFile(config.LocalName, []byte("editor=42"), 0600) + _ = os.WriteFile(config.LocalName, []byte("[general]\neditor=42"), 0600) for _, args := range [][]string{{"config", "edit"}, {"config", "edit", "--editor="}, {"config", "edit", "--config", "."}} { if _, _, err := execute(editorRoot(t, func(context.Context, string, string, io.Reader, io.Writer, io.Writer) error { t.Fatal("invalid editor request launched") diff --git a/internal/cli/config_expansion_test.go b/internal/cli/config_expansion_test.go index b34b1c0..2d2cbd0 100644 --- a/internal/cli/config_expansion_test.go +++ b/internal/cli/config_expansion_test.go @@ -17,7 +17,7 @@ func TestDecideRejectsMissingOrBlankExpandedKey(t *testing.T) { configEnvironment(t) const name = "JEVWISE_TEST_DECISION_KEY" path := filepath.Join(t.TempDir(), "config.toml") - if err := os.WriteFile(path, []byte("api_key = '${"+name+"}'"), 0600); err != nil { + if err := os.WriteFile(path, []byte("[provider.jev]\napi_key = '${"+name+"}'"), 0600); err != nil { t.Fatal(err) } for _, value := range []string{"", " \t\n "} { @@ -32,7 +32,7 @@ func TestDecideRejectsMissingOrBlankExpandedKey(t *testing.T) { func TestHiddenConfigCustomKeyAndView(t *testing.T) { configEnvironment(t) t.Setenv("JEV_MCP_API_KEY", "synthetic-custom-key") - if err := os.WriteFile(config.LocalName, []byte("api_key='${JEV_MCP_API_KEY}'\nmodel='hidden-model'"), 0600); err != nil { + if err := os.WriteFile(config.LocalName, []byte("[provider.jev]\napi_key='${JEV_MCP_API_KEY}'\nmodel='hidden-model'"), 0600); err != nil { t.Fatal(err) } if err := os.WriteFile("configs.toml", []byte("[invalid TOML"), 0600); err != nil { @@ -73,7 +73,7 @@ func TestConfigGlobalHomeAndTildeInitViewEdit(t *testing.T) { if _, err := os.Stat(path); err != nil { t.Fatal("init created literal tilde directory instead of global file") } - if err := os.WriteFile(path, []byte("model='home-global'\napi_key='$JEV_MCP_API_KEY'"), 0600); err != nil { + if err := os.WriteFile(path, []byte("[provider.jev]\nmodel='home-global'\napi_key='$JEV_MCP_API_KEY'"), 0600); err != nil { t.Fatal(err) } for _, args := range [][]string{{"config", "view"}, {"config", "view", "--config", literal}} { diff --git a/internal/cli/config_migrate_test.go b/internal/cli/config_migrate_test.go new file mode 100644 index 0000000..3525837 --- /dev/null +++ b/internal/cli/config_migrate_test.go @@ -0,0 +1,55 @@ +package cli + +import ( + "context" + "io" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/bitbrew-dev/jevwise/internal/config" +) + +func TestConfigMigrateCommandAndFlatHints(t *testing.T) { + configEnvironment(t) + path := filepath.Join(t.TempDir(), "config.toml") + if err := os.WriteFile(path, []byte("api_key = 'private-key'\nmcp_port = 9090\n"), 0600); err != nil { + t.Fatal(err) + } + if _, _, err := execute(configRoot(t), "mcp", "--config", path); err == nil || !strings.Contains(err.Error(), "jevwise config migrate") || strings.Contains(err.Error(), "private") { + t.Fatal("flat file not rejected with migrate hint", err) + } + if out, _, err := execute(NewRoot(), "version", "--config", path); err != nil || !strings.Contains(out, "commit:") { + t.Fatal("version failed on a flat file", err) + } + if _, _, err := execute(NewRoot(), "mcp", "status", "--runtime-dir", t.TempDir(), "--config", path); err != nil && strings.Contains(err.Error(), "migrate") { + t.Fatal("mcp status loaded the flat file", err) + } + edited := "" + edit := newConfigEdit(func(_ context.Context, editor, target string, _ io.Reader, _, _ io.Writer) error { + edited = editor + " " + target + return nil + }) + root := configRoot(t) + configCmd, _, _ := root.Find([]string{"config"}) + for _, child := range configCmd.Commands() { + if child.Name() == "edit" { + configCmd.RemoveCommand(child) + } + } + configCmd.AddCommand(edit) + if _, _, err := execute(root, "config", "edit", "--config", path); err != nil || edited != "vim "+path { + t.Fatal("config edit cannot open a flat file", edited, err) + } + out, _, err := execute(configRoot(t), "config", "migrate", "--config", path) + if err != nil || !strings.Contains(out, "Migrated configuration") || !strings.Contains(out, config.BackupSuffix) || strings.Contains(out, "private") { + t.Fatal("migrate output wrong or leaked", out, err) + } + if view, _, err := execute(configRoot(t), "config", "view", "--config", path); err != nil || !strings.Contains(view, "[mcp]\nhost = \"127.0.0.1\"\nport = 9090\n") || strings.Contains(view, "private") { + t.Fatal("migrated view wrong or leaked", view, err) + } + if out, _, err := execute(configRoot(t), "config", "migrate", "--config", path); err != nil || !strings.Contains(out, "nothing changed") { + t.Fatal("second migration not a no-op", out, err) + } +} diff --git a/internal/cli/config_test.go b/internal/cli/config_test.go index d02d74b..74e94bd 100644 --- a/internal/cli/config_test.go +++ b/internal/cli/config_test.go @@ -41,7 +41,7 @@ func TestConfigInitTargetsNoOverwriteAndAutoLoad(t *testing.T) { path = filepath.Join(t.TempDir(), "selected.toml") args = append(args, "--config", path) } else { - _ = os.WriteFile(config.LocalName, []byte("model='local'"), 0600) + _ = os.WriteFile(config.LocalName, []byte("[provider.jev]\nmodel='local'"), 0600) } out, stderr, err := execute(configRoot(t), args...) data, readErr := os.ReadFile(path) @@ -82,13 +82,13 @@ func TestConfigHelpValidationAndSafeView(t *testing.T) { t.Fatal("invalid config request accepted or leaked", err) } } - _ = os.WriteFile(config.LocalName, []byte("api_key='private-secret'\nmodel='file-model'"), 0600) + _ = os.WriteFile(config.LocalName, []byte("[provider.jev]\napi_key='private-secret'\nmodel='file-model'"), 0600) t.Setenv("TS_JEV_MODEL", "environment-model") out, _, err := execute(configRoot(t), "config", "view", "--api-key=flag-secret") if err != nil || strings.Contains(out, "secret") || strings.Contains(out, "environment-model") || !strings.Contains(out, "file-model") { t.Fatal("view exposed credentials or effective overrides", err) } - _ = os.WriteFile(config.LocalName, []byte("api_key='private-secret'\n["), 0600) + _ = os.WriteFile(config.LocalName, []byte("[provider.jev]\napi_key='private-secret'\n["), 0600) if out, _, err := execute(configRoot(t), "config", "view"); err == nil || out != "" || strings.Contains(err.Error(), "private-secret") { t.Fatal("parser diagnostic leaked", err) } diff --git a/internal/cli/debug_config_test.go b/internal/cli/debug_config_test.go index 06b6470..e213a56 100644 --- a/internal/cli/debug_config_test.go +++ b/internal/cli/debug_config_test.go @@ -15,7 +15,7 @@ func TestDebugFromConfigAndFlagOverride(t *testing.T) { t.Fatal(err) } } - write("debug = true\n") + write("[general]\ndebug = true\n") for _, tt := range []struct { args []string enabled bool @@ -30,7 +30,7 @@ func TestDebugFromConfigAndFlagOverride(t *testing.T) { t.Fatal("debug setting precedence changed", tt.args, err) } } - write("debug = false\n") + write("[general]\ndebug = false\n") t.Setenv("TS_JEV_DEBUG", "true") if _, stderr, err := execute(NewRoot(), "version", "--config", path); err != nil || stderr == "" { t.Fatal("environment did not enable debug", err) @@ -41,7 +41,7 @@ func TestDebugFromConfigAndFlagOverride(t *testing.T) { t.Fatal("background child enabled debug from environment") } t.Setenv("TS_JEV_DEBUG", "") - write("debug = 'maybe'\n") + write("[general]\ndebug = 'maybe'\n") if out, stderr, err := execute(NewRoot(), "version", "--config", path); err != nil || stderr != "" || !strings.Contains(out, "commit:") { t.Fatal("invalid debug setting broke or logged version", err) } diff --git a/internal/cli/decide.go b/internal/cli/decide.go index 3e4eea0..f2224c7 100644 --- a/internal/cli/decide.go +++ b/internal/cli/decide.go @@ -9,6 +9,7 @@ import ( "github.com/bitbrew-dev/jevwise/internal/config" "github.com/bitbrew-dev/jevwise/internal/debuglog" + "github.com/bitbrew-dev/jevwise/internal/gemini" "github.com/bitbrew-dev/jevwise/internal/service" "github.com/bitbrew-dev/jevwise/pkg/typesafe" "github.com/spf13/cobra" @@ -29,7 +30,7 @@ func (e *decisionError) Error() string { return e.message } func (e *decisionError) Unwrap() error { return e.cause } func defaultServiceFactory(cfg config.Config) (service.DecisionService, func(), error) { - if cfg.Provider != "jev" { + if cfg.Provider != "jev" && cfg.Provider != "gemini" { svc, err := service.New(cfg.Provider, nil) return svc, nil, err } @@ -42,6 +43,14 @@ func defaultServiceFactory(cfg config.Config) (service.DecisionService, func(), return nil, nil, &initializationError{kind: "api_key_invalid"} } } + if cfg.Provider == "gemini" { + client, err := gemini.New(gemini.Options{APIKey: cfg.APIKey, BaseURL: cfg.BaseURL, Model: cfg.Model, Timeout: cfg.Timeout}) + if err != nil { + return nil, nil, &initializationError{kind: "client_configuration", cause: err} + } + svc, err := service.NewGemini(client.Choose, cfg.Model) + return svc, nil, err + } client, err := typesafe.NewClient(typesafe.ClientOptions{APIKey: cfg.APIKey, BaseURL: cfg.BaseURL, DefaultModel: cfg.Model, Timeout: cfg.Timeout}) if err != nil { @@ -59,7 +68,7 @@ func newDecide(factory ServiceFactory) *cobra.Command { var prompt string var options []string var stdin bool - cmd := &cobra.Command{Use: "decide", Short: "Return Jev's probabilities for repeated options", + cmd := &cobra.Command{Use: "decide", Short: "Return the provider's probabilities for repeated options", Args: func(_ *cobra.Command, args []string) error { if len(args) != 0 { return errors.New("decide does not accept positional arguments") diff --git a/internal/cli/decide_gemini_test.go b/internal/cli/decide_gemini_test.go new file mode 100644 index 0000000..ef56c55 --- /dev/null +++ b/internal/cli/decide_gemini_test.go @@ -0,0 +1,37 @@ +package cli + +import ( + "encoding/json" + "io" + "net/http" + "net/http/httptest" + "strings" + "testing" +) + +func TestDecideWithGeminiUsesOnlyGoogleCredentials(t *testing.T) { + decisionEnvironment(t) + t.Setenv("JEV_API_KEY", "private-jev-key") + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + body, _ := io.ReadAll(r.Body) + if r.URL.Path != "/v1beta/decisions" || r.Header.Get("x-goog-api-key") != "fixture-google-key" || r.Header.Get("Authorization") != "" || strings.Contains(string(body), "private-jev-key") || !strings.Contains(string(body), `"model":"gemini-decision-1"`) { + t.Error("unexpected Gemini request") + } + _, _ = io.WriteString(w, `{"answers":{"decision":{"type":"choice","choice":"Go","confidence":0.8,"probabilities":{"Go":0.8,"Rust":0.2}}},"usage":{"total_input_tokens":7,"total_output_tokens":0}}`) + })) + defer server.Close() + t.Setenv("GOOGLE_API_KEY", "fixture-google-key") + t.Setenv("GOOGLE_GEMINI_BASE_URL", server.URL) + out, stderr, err := execute(NewRoot(), "decide", "--provider", "gemini", "--prompt", "Pick a language", "--option", "Go", "--option", "Rust") + var response struct { + Model, Choice string + Probabilities map[string]float64 + } + if err != nil || stderr != "" || json.Unmarshal([]byte(out), &response) != nil || response.Model != "gemini-decision-1" || response.Choice != "Go" || response.Probabilities["Rust"] != 0.2 { + t.Fatal("Gemini decision failed", out, err) + } + t.Setenv("GOOGLE_API_KEY", "") + if _, _, err := execute(NewRoot(), "decide", "--provider", "gemini", "--prompt", "p", "--option", "A", "--option", "B"); err == nil || !strings.Contains(err.Error(), "API key is required") { + t.Fatal("Gemini used a non-Google key", err) + } +} diff --git a/internal/cli/decide_test.go b/internal/cli/decide_test.go index c080419..b059516 100644 --- a/internal/cli/decide_test.go +++ b/internal/cli/decide_test.go @@ -26,7 +26,7 @@ func (f decideFunc) Decide(ctx context.Context, req service.Request) (service.Re func decisionEnvironment(t *testing.T) { t.Helper() t.Setenv("XDG_CONFIG_HOME", t.TempDir()) - for _, key := range []string{"JEV_API_KEY", "JEVWISE_API_KEY", "JEVWISE_MCP_TOKEN", "JEVWISE_MCP_AUTH_TOKEN", "JEV_MCP_TOKEN", "TS_JEV_API_KEY", "TS_JEV_MODEL", "TS_JEV_PROVIDER", "TS_JEV_BASE_URL", "TS_JEV_TIMEOUT", "TS_JEV_MCP_BASE_URL", "TS_JEV_MCP_PORT", "TS_JEV_DEBUG", "TYPESAFE_API_KEY", "TYPESAFE_DEFAULT_MODEL", "TYPESAFE_BASE_URL"} { + for _, key := range []string{"JEV_API_KEY", "JEVWISE_API_KEY", "JEVWISE_MCP_TOKEN", "JEVWISE_MCP_AUTH_TOKEN", "JEV_MCP_TOKEN", "TS_JEV_API_KEY", "TS_JEV_MODEL", "TS_JEV_PROVIDER", "TS_JEV_BASE_URL", "TS_JEV_TIMEOUT", "TS_JEV_MCP_BASE_URL", "TS_JEV_MCP_PORT", "TS_JEV_DEBUG", "GOOGLE_API_KEY", "GEMINI_API_KEY", "GOOGLE_GEMINI_BASE_URL", "TYPESAFE_API_KEY", "TYPESAFE_DEFAULT_MODEL", "TYPESAFE_BASE_URL"} { t.Setenv(key, "") } } diff --git a/internal/cli/initialization.go b/internal/cli/initialization.go index 8a8dc87..2d90b2c 100644 --- a/internal/cli/initialization.go +++ b/internal/cli/initialization.go @@ -21,7 +21,7 @@ func (e *initializationError) Error() string { case "api_key_invalid": return "API key must contain printable ASCII without whitespace; check the selected credential" default: - return "cannot initialize Jev client: check base URL, model and timeout" + return "cannot initialize decision client: check base URL, model and timeout" } } func (e *initializationError) Unwrap() error { return e.cause } diff --git a/internal/cli/mcp.go b/internal/cli/mcp.go index 45b055b..d22e6b6 100644 --- a/internal/cli/mcp.go +++ b/internal/cli/mcp.go @@ -63,7 +63,7 @@ func newMCPWithBackground(factory ServiceFactory, run mcpRunner, start mcpStarte return err } if token == "" { - if cfg.Provider != "jev" { + if !mcpProvider(cfg.Provider) { return initializationFailure(ctx, "provider is not implemented", service.ErrNotImplemented) } if cfg.APIKey == "" { @@ -127,7 +127,7 @@ func newMCPWithBackground(factory ServiceFactory, run mcpRunner, start mcpStarte } return nil }} - cmd.Flags().StringVar(&address, "listen", "127.0.0.1:8080", "Literal loopback IP:port for /mcp; overrides mcp_base_url and mcp_port") + cmd.Flags().StringVar(&address, "listen", "127.0.0.1:8080", "Literal loopback IP:port for /mcp; overrides [mcp] host and port") cmd.Flags().BoolVar(&background, "background", false, "Start an independent MCP background process") cmd.PersistentFlags().StringVar(&tokenFile, "token-file", "", "Read an owner-private bearer token file instead of JEVWISE_MCP_TOKEN") cache, _ := os.UserCacheDir() @@ -150,8 +150,8 @@ func mcpEnvironmentToken() string { return "" } -// mcpListenAddress keeps a changed --listen ahead of the mcp_base_url and -// mcp_port settings, which are resolved without loading any other setting. +// mcpListenAddress keeps a changed --listen ahead of the [mcp] host and +// port settings, which are resolved without loading any other setting. func mcpListenAddress(cmd *cobra.Command, listen string) (string, error) { if !cmd.Flags().Changed("listen") { path, err := cmd.Flags().GetString("config") @@ -164,3 +164,6 @@ func mcpListenAddress(cmd *cobra.Command, listen string) (string, error) { } return mcpserver.ValidateAddress(listen) } + +// mcpProvider reports whether MCP can serve decisions for the provider. +func mcpProvider(provider string) bool { return provider == "jev" || provider == "gemini" } diff --git a/internal/cli/mcp_background.go b/internal/cli/mcp_background.go index c70d955..08ae26a 100644 --- a/internal/cli/mcp_background.go +++ b/internal/cli/mcp_background.go @@ -20,7 +20,7 @@ import ( type mcpStarter func(context.Context, string, daemon.Bootstrap, func(daemon.Bootstrap) error, func(context.Context, bool) error) (daemon.StartResult, error) func runMCPBackground(cmd *cobra.Command, cfg config.Config, address, agentToken, runtimeDir string, start mcpStarter) error { - if cfg.Provider != "jev" { + if !mcpProvider(cfg.Provider) { return &decisionError{"provider is not implemented", service.ErrNotImplemented} } if cfg.Validate() != nil || cfg.Timeout > time.Duration(1<<63-1)-30*time.Second || mcpserver.ValidateToken(agentToken) != nil { diff --git a/internal/cli/mcp_install.go b/internal/cli/mcp_install.go index 8a583e8..6f3c86c 100644 --- a/internal/cli/mcp_install.go +++ b/internal/cli/mcp_install.go @@ -22,7 +22,7 @@ func newMCPInstall(run agentCommand) *cobra.Command { } cmd := &cobra.Command{Use: "install", Short: "Register Jevwise with an installed MCP client", Args: noArgs, RunE: func(cmd *cobra.Command, _ []string) error { return cmd.Help() }} - cmd.PersistentFlags().StringVar(&address, "listen", "127.0.0.1:8080", "Server's literal loopback IP:port (same as mcp --listen); overrides mcp_base_url and mcp_port") + cmd.PersistentFlags().StringVar(&address, "listen", "127.0.0.1:8080", "Server's literal loopback IP:port (same as mcp --listen); overrides [mcp] host and port") cmd.PersistentFlags().StringVar(&scope, "scope", "user", "Client settings scope: user; Claude also supports local or project") _ = cmd.RegisterFlagCompletionFunc("scope", func(cmd *cobra.Command, _ []string, prefix string) ([]string, cobra.ShellCompDirective) { choices := []string{"user"} diff --git a/internal/cli/mcp_listen_test.go b/internal/cli/mcp_listen_test.go index 6e40546..07cf85f 100644 --- a/internal/cli/mcp_listen_test.go +++ b/internal/cli/mcp_listen_test.go @@ -26,11 +26,11 @@ func TestMCPListenFromConfigMatchesFlag(t *testing.T) { decisionEnvironment(t) t.Setenv("JEV_MCP_TOKEN", "fixture-agent-token") factory := func(config.Config) (service.DecisionService, func(), error) { return mcpFixtureService(t), nil, nil } - path := mcpListenConfig(t, "api_key = 'fixture-upstream-key'\nmcp_base_url = '127.0.0.1'\nmcp_port = 9090\n") + path := mcpListenConfig(t, "[mcp]\nhost = '127.0.0.1'\nport = 9090\n[provider.jev]\napi_key = 'fixture-upstream-key'\n") for _, args := range [][]string{ {"mcp", "--config", path}, {"mcp", "--config", path, "--listen", "127.0.0.1:9090"}, - {"mcp", "--config", mcpListenConfig(t, "api_key = 'fixture-upstream-key'\nmcp_port = 7070\n"), "--listen", "127.0.0.1:9090"}, + {"mcp", "--config", mcpListenConfig(t, "[mcp]\nport = 7070\n[provider.jev]\napi_key = 'fixture-upstream-key'\n"), "--listen", "127.0.0.1:9090"}, } { address := "" run := func(ctx context.Context, _ service.DecisionService, _ time.Duration, options mcpserver.RunOptions) error { @@ -45,8 +45,8 @@ func TestMCPListenFromConfigMatchesFlag(t *testing.T) { t.Fatal("invalid MCP setting started runner") return nil } - for _, content := range []string{"mcp_base_url = 'http://127.0.0.1'\n", "mcp_port = 0\n"} { - if _, _, err := execute(mcpRoot(factory, run), "mcp", "--config", mcpListenConfig(t, "api_key = 'k'\n"+content)); err == nil { + for _, content := range []string{"[mcp]\nhost = 'http://127.0.0.1'\n", "[mcp]\nport = 0\n"} { + if _, _, err := execute(mcpRoot(factory, run), "mcp", "--config", mcpListenConfig(t, "[provider.jev]\napi_key = 'k'\n"+content)); err == nil { t.Fatal("invalid MCP setting accepted", content) } } @@ -58,8 +58,8 @@ func TestMCPInstallListenFromConfig(t *testing.T) { args []string want string }{ - {"mcp_port = 9090\n", nil, "http://127.0.0.1:9090/mcp"}, - {"mcp_port = 9090\ntimeout = 'invalid'\n", []string{"--listen", "127.0.0.1:7070"}, "http://127.0.0.1:7070/mcp"}, + {"[mcp]\nport = 9090\n", nil, "http://127.0.0.1:9090/mcp"}, + {"[mcp]\nport = 9090\n[provider.jev]\ntimeout = 'invalid'\n", []string{"--listen", "127.0.0.1:7070"}, "http://127.0.0.1:7070/mcp"}, } { url := "" run := func(_ context.Context, _ string, args ...string) ([]byte, error) { @@ -72,7 +72,7 @@ func TestMCPInstallListenFromConfig(t *testing.T) { t.Fatal("install endpoint not resolved from config or flag", url, err) } } - if _, _, err := execute(installerRoot(t, nil), "mcp", "install", "claude", "--config", mcpListenConfig(t, "mcp_base_url = 'localhost'\n")); err == nil { + if _, _, err := execute(installerRoot(t, nil), "mcp", "install", "claude", "--config", mcpListenConfig(t, "[mcp]\nhost = 'localhost'\n")); err == nil { t.Fatal("install accepted non-literal MCP host") } } diff --git a/internal/cli/mcp_process_test.go b/internal/cli/mcp_process_test.go index ba32d55..e4828d9 100644 --- a/internal/cli/mcp_process_test.go +++ b/internal/cli/mcp_process_test.go @@ -29,7 +29,9 @@ func TestMain(m *testing.M) { if len(os.Args) == 2 && os.Args[1] == daemon.ChildArgument && os.Getenv("JEVWISE_PROCESS_FIXTURE") == "1" { os.Exit(runProcessFixture()) } - _ = os.Unsetenv("TS_JEV_DEBUG") + for _, name := range []string{"TS_JEV_DEBUG", "GOOGLE_API_KEY", "GEMINI_API_KEY", "GOOGLE_GEMINI_BASE_URL"} { + _ = os.Unsetenv(name) + } isolated, err := os.MkdirTemp("", "jevwise-cli-config-") if err != nil { os.Exit(1) diff --git a/internal/cli/root.go b/internal/cli/root.go index 79cf476..9ffa00e 100644 --- a/internal/cli/root.go +++ b/internal/cli/root.go @@ -52,7 +52,7 @@ func NewRootWithFactory(factory ServiceFactory) *cobra.Command { flags.Bool("debug", false, "Write safe structured debug logs to stderr") flags.Bool("verbose", false, "Alias for --debug: write safe structured debug logs to stderr") flags.String("config", "", "Path to TOML configuration") - flags.String("provider", "jev", "Decision provider: jev, codex or claude") + flags.String("provider", "jev", "Decision provider: jev (TypeSafe Jev API), gemini (Google Gemini Decisions API), codex or claude (reserved, not implemented)") flags.String("api-key", "", "Jev API key (prefer JEV_API_KEY or config)") flags.String("base-url", "https://api.typesafe.ai", "Jev API base URL") flags.String("model", "jev-latest", "Jev model") diff --git a/internal/cli/root_test.go b/internal/cli/root_test.go index 38755cb..196ed1c 100644 --- a/internal/cli/root_test.go +++ b/internal/cli/root_test.go @@ -93,7 +93,7 @@ func TestConfigHelperAndIsolatedRoots(t *testing.T) { func TestConfigErrorsAreDeferredAndSafe(t *testing.T) { path := filepath.Join(t.TempDir(), "broken.toml") - if err := os.WriteFile(path, []byte("api_key = private-secret\n"), 0600); err != nil { + if err := os.WriteFile(path, []byte("[provider.jev]\napi_key = private-secret\n"), 0600); err != nil { t.Fatal(err) } for _, args := range [][]string{{"check", "--config", path}, {"check", "--timeout", "invalid"}, {"check", "--provider", "unknown"}} { diff --git a/internal/config/config.go b/internal/config/config.go index 5edc053..923073a 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -2,14 +2,15 @@ package config import ( - "bytes" "errors" "fmt" "net/url" "os" + "slices" "strings" "time" + "github.com/bitbrew-dev/jevwise/internal/gemini" "github.com/spf13/pflag" "github.com/spf13/viper" ) @@ -20,17 +21,18 @@ type Config struct { Timeout time.Duration } +// Providers lists the accepted provider names; each has a [provider.] table. +var Providers = []string{"jev", "gemini", "codex", "claude"} + +var errProvider = errors.New("provider must be jev, gemini, codex or claude") + // Load applies changed flags > nonblank environment > TOML > defaults. // An empty path selects ./.jevwise.toml, then the Jevwise global file. // Only a missing default global file is optional; selected files never merge. +// Provider settings come from [provider.]; flat files fail. // TOML API keys expand $NAME/${NAME}; environment and flag keys stay literal. // Neither global Viper state nor the supplied flags are mutated. func Load(path string, flags *pflag.FlagSet) (Config, error) { - v := viper.New() - defaults := defaultValues() - for key, value := range defaults { - v.SetDefault(key, value) - } path, required, err := ResolvePath(path) if err != nil { return Config{}, err @@ -39,58 +41,96 @@ func Load(path string, flags *pflag.FlagSet) (Config, error) { if err != nil && (required || !errors.Is(err, os.ErrNotExist)) { return Config{}, errors.New("cannot read configuration: file missing, unreadable or unsafe") } - // Parse only these selected bytes. Never use Viper search paths or ReadInConfig. - v.SetConfigType("toml") - if err == nil && v.ReadConfig(bytes.NewReader(data)) != nil { - return Config{}, errors.New("cannot read configuration: invalid TOML") - } - upstream := map[string]string{"api_key": "TYPESAFE_API_KEY", "base_url": "TYPESAFE_BASE_URL", "model": "TYPESAFE_DEFAULT_MODEL"} - env := make(map[string]any) - for key := range defaults { - names := []string{"TS_JEV_" + strings.ToUpper(key), upstream[key]} - if key == "api_key" { - names = []string{"JEV_API_KEY", "JEVWISE_API_KEY", "TS_JEV_API_KEY", "TYPESAFE_API_KEY"} + v := viper.New() + if err == nil { + if v, err = readSettings(data); err != nil { + return Config{}, err } - for _, name := range names { - if value := strings.TrimSpace(os.Getenv(name)); value != "" { - env[key] = value - break - } + } + values := make(map[string]string) + provider, _, err := resolveSetting(v, flags, "", "provider", "general.provider") + if err != nil { + return Config{}, err + } + if !slices.Contains(Providers, provider) { + return Config{}, errProvider + } + values["provider"] = provider + for _, key := range []string{"api_key", "base_url", "model", "timeout"} { + value, fromFile, err := resolveSetting(v, flags, provider, key, "provider."+provider+"."+key) + if err != nil { + return Config{}, err } - if flags != nil { - if flag := flags.Lookup(strings.ReplaceAll(key, "_", "-")); flag != nil { - if err := v.BindPFlag(key, flag); err != nil { - return Config{}, fmt.Errorf("bind %s flag: %w", key, err) - } - } + if key == "api_key" && fromFile { + value = strings.TrimSpace(os.ExpandEnv(value)) } + values[key] = value } - // Merge instead of Set so changed bound flags retain their higher priority. - if err := v.MergeConfigMap(env); err != nil { - return Config{}, errors.New("cannot apply environment configuration") - } - apiKeyFromFile := env["api_key"] == nil + return fromValues(values) +} + +// resolveSetting applies changed flag > nonblank environment > TOML path > default +// for one setting of the given provider and reports whether it came from the file. +func resolveSetting(v *viper.Viper, flags *pflag.FlagSet, provider, key, path string) (string, bool, error) { if flags != nil { - if flag := flags.Lookup("api-key"); flag != nil && flag.Changed { - apiKeyFromFile = false + if flag := flags.Lookup(strings.ReplaceAll(key, "_", "-")); flag != nil && flag.Changed { + return strings.TrimSpace(flag.Value.String()), false, nil } } - values := make(map[string]string) - for key := range defaults { - value, ok := v.Get(key).(string) - if !ok { - return Config{}, fmt.Errorf("%s must be a string", key) + for _, name := range envNames(provider, key) { + if value := strings.TrimSpace(os.Getenv(name)); value != "" { + return value, false, nil } - if key == "api_key" && apiKeyFromFile { - value = os.ExpandEnv(value) + } + if value, ok, err := fileString(v, path); ok || err != nil { + return value, ok, err + } + return defaultValues(provider)[key], false, nil +} + +// fileString reads one string setting from the parsed file only. +func fileString(v *viper.Viper, path string) (string, bool, error) { + raw := v.Get(path) + if raw == nil { + return "", false, nil + } + value, ok := raw.(string) + if !ok { + return "", false, fmt.Errorf("%s must be a string", path) + } + return strings.TrimSpace(value), true, nil +} + +// envNames keeps credentials and destinations per provider: Jev names never +// configure Gemini, and Gemini names follow google-genai (GOOGLE_API_KEY wins). +func envNames(provider, key string) []string { + if provider == "gemini" { + switch key { + case "api_key": + return []string{"GOOGLE_API_KEY", "GEMINI_API_KEY"} + case "base_url": + return []string{"GOOGLE_GEMINI_BASE_URL"} + case "model": + return nil } - values[key] = strings.TrimSpace(value) } - return fromValues(values) + switch key { + case "api_key": + return []string{"JEV_API_KEY", "JEVWISE_API_KEY", "TS_JEV_API_KEY", "TYPESAFE_API_KEY"} + case "base_url": + return []string{"TS_JEV_BASE_URL", "TYPESAFE_BASE_URL"} + case "model": + return []string{"TS_JEV_MODEL", "TYPESAFE_DEFAULT_MODEL"} + } + return []string{"TS_JEV_" + strings.ToUpper(key)} } -func defaultValues() map[string]string { - return map[string]string{"api_key": "", "base_url": "https://api.typesafe.ai", "model": "jev-latest", "provider": "jev", "timeout": "10s"} +func defaultValues(provider string) map[string]string { + values := map[string]string{"api_key": "", "base_url": "https://api.typesafe.ai", "model": "jev-latest", "provider": "jev", "timeout": "10s"} + if provider == "gemini" { + values["base_url"], values["model"] = gemini.DefaultBaseURL, gemini.DefaultModel + } + return values } func fromValues(values map[string]string) (Config, error) { @@ -120,10 +160,8 @@ func (c Config) Validate() error { if strings.TrimSpace(c.Model) == "" { return errors.New("model must not be blank") } - switch c.Provider { - case "jev", "codex", "claude": - default: - return errors.New("provider must be jev, codex or claude") + if !slices.Contains(Providers, c.Provider) { + return errProvider } return nil } diff --git a/internal/config/config_test.go b/internal/config/config_test.go index 4d026df..1601a0d 100644 --- a/internal/config/config_test.go +++ b/internal/config/config_test.go @@ -15,7 +15,7 @@ import ( func cleanEnv(t *testing.T) { t.Helper() t.Chdir(t.TempDir()) - for _, name := range []string{"JEV_API_KEY", "JEVWISE_API_KEY", "TS_JEV_API_KEY", "TS_JEV_BASE_URL", "TS_JEV_MODEL", "TS_JEV_PROVIDER", "TS_JEV_TIMEOUT", "TYPESAFE_API_KEY", "TYPESAFE_BASE_URL", "TYPESAFE_DEFAULT_MODEL"} { + for _, name := range []string{"JEV_API_KEY", "JEVWISE_API_KEY", "TS_JEV_API_KEY", "TS_JEV_BASE_URL", "TS_JEV_MODEL", "TS_JEV_PROVIDER", "TS_JEV_TIMEOUT", "TYPESAFE_API_KEY", "TYPESAFE_BASE_URL", "TYPESAFE_DEFAULT_MODEL", "GOOGLE_API_KEY", "GEMINI_API_KEY", "GOOGLE_GEMINI_BASE_URL"} { t.Setenv(name, "") } t.Setenv("XDG_CONFIG_HOME", t.TempDir()) @@ -45,8 +45,8 @@ func TestDefaultsAndDiscovery(t *testing.T) { } xdg := filepath.Join(os.Getenv("XDG_CONFIG_HOME"), "jevwise", GlobalName) home := filepath.Join(os.Getenv("HOME"), ".config", "jevwise", GlobalName) - writeConfig(t, xdg, `model = "xdg"`) - writeConfig(t, home, `model = "home"`) + writeConfig(t, xdg, `[provider.jev]`+"\n"+`model = "xdg"`) + writeConfig(t, home, `[provider.jev]`+"\n"+`model = "home"`) got, err = Load("", nil) if err != nil || got.Model != "xdg" { t.Fatalf("XDG: %+v %v", got, err) @@ -67,7 +67,7 @@ func TestDefaultsAndDiscovery(t *testing.T) { func TestPrecedenceAndIsolation(t *testing.T) { cleanEnv(t) - path := writeConfig(t, filepath.Join(t.TempDir(), "config.toml"), "api_key='file'\nbase_url='https://file.test'\nmodel='file'\nprovider='codex'\ntimeout='1s'") + path := writeConfig(t, filepath.Join(t.TempDir(), "config.toml"), "[general]\nprovider='codex'\n[provider.codex]\napi_key='file'\nbase_url='https://file.test'\nmodel='file'\ntimeout='1s'") file, err := Load(path, nil) if err != nil || file.APIKey != "file" || file.Provider != "codex" || file.Timeout != time.Second { t.Fatalf("file: %+v %v", file, err) @@ -116,7 +116,7 @@ func TestInvalidFilesAndValues(t *testing.T) { if _, err := Load(filepath.Join(t.TempDir(), "missing.toml"), nil); err == nil { t.Fatal("explicit missing file accepted") } - for _, content := range []string{"api_key = 'secret'\n[", "timeout='garbage'", "timeout='0s'", "timeout='-1s'", "timeout=10", "base_url='/relative'", "base_url='ftp://example.com'", "base_url='https://user:secret@example.com'", "base_url='https://example.com?q=secret'", "provider='other'", "model=' '", "api_key=42"} { + for _, content := range []string{"[provider.jev]\napi_key = 'secret'\n[", "[provider.jev]\ntimeout='garbage'", "[provider.jev]\ntimeout='0s'", "[provider.jev]\ntimeout='-1s'", "[provider.jev]\ntimeout=10", "[provider.jev]\nbase_url='/relative'", "[provider.jev]\nbase_url='ftp://example.com'", "[provider.jev]\nbase_url='https://user:secret@example.com'", "[provider.jev]\nbase_url='https://example.com?q=secret'", "[general]\nprovider='other'", "[provider.jev]\nmodel=' '", "[provider.jev]\napi_key=42"} { t.Run(content, func(t *testing.T) { path := writeConfig(t, filepath.Join(t.TempDir(), "config.toml"), content) if _, err := Load(path, nil); err == nil || strings.Contains(err.Error(), "secret") { @@ -138,3 +138,59 @@ func TestInvalidFilesAndValues(t *testing.T) { t.Fatal("invalid discovered file accepted") } } + +func TestProviderTablesAreIsolated(t *testing.T) { + cleanEnv(t) + path := writeConfig(t, filepath.Join(t.TempDir(), "config.toml"), "[provider.jev]\napi_key='jev-only-key'\nmodel='jev-model'\n[provider.codex]\nmodel='codex-model'\n[provider.gemini]\napi_key='gemini-only-key'\nmodel='gemini-decision-1'\n") + flags := pflag.NewFlagSet("test", pflag.ContinueOnError) + flags.String("provider", "jev", "") + if err := flags.Parse([]string{"--provider=codex"}); err != nil { + t.Fatal(err) + } + codex, err := Load(path, flags) + if err != nil || codex.Provider != "codex" || codex.Model != "codex-model" || codex.APIKey != "" { + t.Fatal("selected provider read another provider table", err) + } + jev, err := Load(path, nil) + if err != nil || jev.Model != "jev-model" || jev.APIKey != "jev-only-key" { + t.Fatal("default provider table not used or unknown provider table rejected", err) + } + text, err := View(path) + if err != nil || strings.Contains(text, "gemini") || strings.Contains(text, "codex-model") { + t.Fatal("view printed an unselected provider table", err) + } +} + +func TestGeminiSettingsNeverUseJevCredentialsOrDestinations(t *testing.T) { + cleanEnv(t) + for name, value := range map[string]string{"JEV_API_KEY": "jev-key", "TYPESAFE_API_KEY": "typesafe-key", "TS_JEV_BASE_URL": "https://jev.test", "TYPESAFE_DEFAULT_MODEL": "jev-model"} { + t.Setenv(name, value) + } + dir := t.TempDir() + path := writeConfig(t, filepath.Join(dir, "config.toml"), "[general]\nprovider='gemini'\n") + cfg, err := Load(path, nil) + if err != nil || cfg.APIKey != "" || cfg.BaseURL != "https://generativelanguage.googleapis.com" || cfg.Model != "gemini-decision-1" || cfg.Timeout != 10*time.Second { + t.Fatal("gemini picked up Jev settings or lost its defaults", err) + } + t.Setenv("GEMINI_API_KEY", "gemini-key") + if cfg, err := Load(path, nil); err != nil || cfg.APIKey != "gemini-key" { + t.Fatal("GEMINI_API_KEY ignored", err) + } + t.Setenv("GOOGLE_API_KEY", "google-key") + t.Setenv("GOOGLE_GEMINI_BASE_URL", "https://gemini.test") + if cfg, err := Load(path, nil); err != nil || cfg.APIKey != "google-key" || cfg.BaseURL != "https://gemini.test" { + t.Fatal("GOOGLE_API_KEY must win and GOOGLE_GEMINI_BASE_URL must apply", err) + } + jevPath := writeConfig(t, filepath.Join(dir, "jev.toml"), "[provider.gemini]\napi_key='file-gemini-key'\n") + if cfg, err := Load(jevPath, nil); err != nil || cfg.APIKey != "jev-key" || cfg.BaseURL != "https://jev.test" || cfg.Model != "jev-model" { + t.Fatal("Jev picked up Gemini settings", err) + } + t.Setenv("GOOGLE_API_KEY", "") + t.Setenv("GEMINI_API_KEY", "") + writeConfig(t, path, "[general]\nprovider='gemini'\n[provider.gemini]\napi_key='$GEMINI_FILE_KEY'\nmodel='gemini-decision-2'\n") + t.Setenv("GEMINI_FILE_KEY", "file-gemini-key") + text, viewErr := View(path) + if cfg, err := Load(path, nil); err != nil || viewErr != nil || cfg.APIKey != "file-gemini-key" || cfg.Model != "gemini-decision-2" || !strings.Contains(text, "[provider.gemini]\napi_key = \"[redacted]\"\nbase_url = \"https://generativelanguage.googleapis.com\"") { + t.Fatal("gemini table not used by load or view", err, viewErr, text) + } +} diff --git a/internal/config/debug.go b/internal/config/debug.go index 53a2499..8bc1136 100644 --- a/internal/config/debug.go +++ b/internal/config/debug.go @@ -1,7 +1,6 @@ package config import ( - "bytes" "errors" "os" "strconv" @@ -10,9 +9,9 @@ import ( "github.com/spf13/viper" ) -var errDebug = errors.New("debug must be true or false") +var errDebug = errors.New("general.debug must be true or false") -// Debug resolves the debug setting (nonblank TS_JEV_DEBUG > TOML > false). It reads +// Debug resolves general.debug (nonblank TS_JEV_DEBUG > TOML > false). It reads // no other settings; only a missing file falls back to false. func Debug(path string) (bool, error) { resolved, _, err := ResolvePath(path) @@ -24,9 +23,10 @@ func Debug(path string) (bool, error) { return false, errors.New("cannot read configuration: file unreadable or unsafe") } v := viper.New() - v.SetConfigType("toml") - if err == nil && v.ReadConfig(bytes.NewReader(data)) != nil { - return false, errors.New("cannot read configuration: invalid TOML") + if err == nil { + if v, err = readSettings(data); err != nil { + return false, err + } } enabled, err := debugValue(v) if err != nil { @@ -41,7 +41,7 @@ func Debug(path string) (bool, error) { } func debugValue(v *viper.Viper) (bool, error) { - switch raw := v.Get("debug").(type) { + switch raw := v.Get("general.debug").(type) { case nil: return false, nil case bool: diff --git a/internal/config/debug_test.go b/internal/config/debug_test.go index 38668bb..814a4dd 100644 --- a/internal/config/debug_test.go +++ b/internal/config/debug_test.go @@ -15,18 +15,18 @@ func TestDebugPrecedenceAndValidation(t *testing.T) { } path := filepath.Join(dir, "config.toml") for content, want := range map[string]bool{ - "debug = true\n": true, - "debug = false\n": false, - "debug = 'true'\n": true, - "timeout = 'invalid'\ndebug = true\n": true, - "model = 'unrelated'\n": false, + "[general]\ndebug = true\n": true, + "[general]\ndebug = false\n": false, + "[general]\ndebug = 'true'\n": true, + "[general]\ndebug = true\n[provider.jev]\ntimeout = 'invalid'\n": true, + "[provider.jev]\nmodel = 'unrelated'\n": false, } { if enabled, err := Debug(writeConfig(t, path, content)); err != nil || enabled != want { t.Fatal("wrong debug value", content, err) } } t.Setenv("TS_JEV_DEBUG", "false") - if enabled, err := Debug(writeConfig(t, path, "debug = true\n")); err != nil || enabled { + if enabled, err := Debug(writeConfig(t, path, "[general]\ndebug = true\n")); err != nil || enabled { t.Fatal("environment did not override TOML", err) } t.Setenv("TS_JEV_DEBUG", "maybe") @@ -34,7 +34,7 @@ func TestDebugPrecedenceAndValidation(t *testing.T) { t.Fatal("invalid environment debug accepted") } t.Setenv("TS_JEV_DEBUG", "") - for _, content := range []string{"debug = 'maybe'\n", "debug = 1\n", "debug = [\n"} { + for _, content := range []string{"[general]\ndebug = 'maybe'\n", "[general]\ndebug = 1\n", "[general]\ndebug = [\n"} { if _, err := Debug(writeConfig(t, path, content)); err == nil { t.Fatal("invalid debug accepted", content) } diff --git a/internal/config/discovery_test.go b/internal/config/discovery_test.go index 1018b34..ebc6380 100644 --- a/internal/config/discovery_test.go +++ b/internal/config/discovery_test.go @@ -20,18 +20,18 @@ func TestDiscoveryAllowlistsHiddenLocalAndNamedGlobal(t *testing.T) { t.Fatal("unrelated file or custom environment override loaded", err) } global, _ := GlobalPath() - writeConfig(t, global, "model='global'\napi_key='global-only-key'") + writeConfig(t, global, "[provider.jev]\nmodel='global'\napi_key='global-only-key'") cfg, err = Load("", nil) if err != nil || cfg.Model != "global" { t.Fatal("named Jevwise global file not selected", err) } - writeConfig(t, LocalName, "model='local'\napi_key='$JEV_MCP_API_KEY'") + writeConfig(t, LocalName, "[provider.jev]\nmodel='local'\napi_key='$JEV_MCP_API_KEY'") cfg, err = Load("", nil) path, required, pathErr := ResolvePath("") if err != nil || pathErr != nil || !required || filepath.Base(path) != LocalName || cfg.Model != "local" || cfg.APIKey != "custom-variable-is-not-an-override" { t.Fatal("local file or custom reference not selected", err) } - writeConfig(t, LocalName, "model='local-only'") + writeConfig(t, LocalName, "[provider.jev]\nmodel='local-only'") cfg, err = Load("", nil) if err != nil || cfg.APIKey != "" { t.Fatal("local and global files merged", err) @@ -47,7 +47,7 @@ func TestGlobalHomeAndLiteralTildePaths(t *testing.T) { t.Setenv("XDG_CONFIG_HOME", "") home := os.Getenv("HOME") expected := filepath.Join(home, ".config", "jevwise", GlobalName) - writeConfig(t, expected, "model='home-global'\napi_key='${JEV_MCP_API_KEY}'") + writeConfig(t, expected, "[provider.jev]\nmodel='home-global'\napi_key='${JEV_MCP_API_KEY}'") t.Setenv("JEV_MCP_API_KEY", "synthetic-custom-key") for _, path := range []string{"", "~/.config/jevwise/jevwise.toml", expected} { cfg, err := Load(path, nil) @@ -84,7 +84,7 @@ func TestGlobalDirectorySymlinkSupportsDotfiles(t *testing.T) { if err := os.Symlink(target, filepath.Dir(global)); err != nil { t.Fatal(err) } - writeConfig(t, filepath.Join(target, GlobalName), "model='dotfiles'") + writeConfig(t, filepath.Join(target, GlobalName), "[provider.jev]\nmodel='dotfiles'") cfg, err := Load("", nil) if err != nil || cfg.Model != "dotfiles" { t.Fatal("trusted dotfiles directory symlink rejected", err) diff --git a/internal/config/editor.go b/internal/config/editor.go index 3a54960..b0b47a6 100644 --- a/internal/config/editor.go +++ b/internal/config/editor.go @@ -1,26 +1,26 @@ package config import ( - "bytes" "errors" "strings" - - "github.com/spf13/viper" ) -// Editor reads only the selected file's editor setting, not environment/SDK -// configuration. Invalid TOML falls back to vim so the file can be repaired. +// Editor reads only the selected file's general.editor setting, not environment/SDK +// configuration. Invalid TOML falls back to vim and a flat file uses its editor key, +// so either can still be opened for repair or migration. func Editor(path string) (string, error) { data, err := ReadFile(path) if err != nil { return "", errors.New("cannot read configuration to choose editor; use --editor to repair") } - v := viper.New() - v.SetConfigType("toml") - if err := v.ReadConfig(bytes.NewReader(data)); err != nil { + v, err := parseTOML(data) + if err != nil { return "vim", nil } - value := v.Get("editor") + value := v.Get("general.editor") + if isFlat(v) { + value = v.Get("editor") + } if value == nil { return "vim", nil } diff --git a/internal/config/editor_test.go b/internal/config/editor_test.go index cdfdcda..2c0d153 100644 --- a/internal/config/editor_test.go +++ b/internal/config/editor_test.go @@ -15,10 +15,10 @@ func TestEditorUsesOnlyFileAndCanRepairInvalidTOML(t *testing.T) { content, want string invalid bool }{ - {"", "vim", false}, {"editor='nvim'", "nvim", false}, {"editor=' '", "vim", false}, - {"editor='/some/editor with spaces'", "/some/editor with spaces", false}, - {"model=42\neditor='nano'", "nano", false}, {"api_key='private-secret'\n[", "vim", false}, - {"editor=42", "", true}, + {"", "vim", false}, {"[general]\neditor='nvim'", "nvim", false}, {"[general]\neditor=' '", "vim", false}, + {"[general]\neditor='/some/editor with spaces'", "/some/editor with spaces", false}, + {"[general]\neditor='nano'\n[provider.jev]\nmodel=42", "nano", false}, {"[provider.jev]\napi_key='private-secret'\n[", "vim", false}, + {"[general]\neditor=42", "", true}, } { path := writeConfig(t, filepath.Join(t.TempDir(), "config.toml"), tc.content) got, err := Editor(path) diff --git a/internal/config/env_names_test.go b/internal/config/env_names_test.go index 4d210dd..1912efc 100644 --- a/internal/config/env_names_test.go +++ b/internal/config/env_names_test.go @@ -9,7 +9,7 @@ import ( func TestAPIEnvironmentNamesPrecedence(t *testing.T) { cleanEnv(t) - path := writeConfig(t, filepath.Join(t.TempDir(), "config.toml"), `api_key = "file-key"`) + path := writeConfig(t, filepath.Join(t.TempDir(), "config.toml"), `[provider.jev]`+"\n"+`api_key = "file-key"`) names := []string{"JEV_API_KEY", "JEVWISE_API_KEY", "TS_JEV_API_KEY", "TYPESAFE_API_KEY"} for i, name := range names { t.Setenv(name, name+"-fixture") @@ -48,7 +48,7 @@ func TestCustomTOMLAPIReferenceNotMCPAuthentication(t *testing.T) { t.Setenv("JEV_MCP_API_KEY", "upstream-fixture") t.Setenv("JEVWISE_MCP_TOKEN", "agent-fixture") for _, reference := range []string{"$JEV_MCP_API_KEY", "${JEV_MCP_API_KEY}"} { - path := writeConfig(t, filepath.Join(t.TempDir(), "config.toml"), "api_key = '"+reference+"'") + path := writeConfig(t, filepath.Join(t.TempDir(), "config.toml"), "[provider.jev]\napi_key = '"+reference+"'") cfg, err := Load(path, nil) if err != nil || cfg.APIKey != "upstream-fixture" { t.Fatal("custom TOML API key did not expand", err) diff --git a/internal/config/expansion_test.go b/internal/config/expansion_test.go index 13bef18..defeb5f 100644 --- a/internal/config/expansion_test.go +++ b/internal/config/expansion_test.go @@ -31,14 +31,14 @@ func TestLoadExpandsOnlyFileAPIKeyAndTrimsResult(t *testing.T) { t.Fatal(err) } } - writeConfig(t, path, "api_key = '"+test.file+"'") + writeConfig(t, path, "[provider.jev]\napi_key = '"+test.file+"'") cfg, err := Load(path, nil) if err != nil || cfg.APIKey != test.want { t.Fatal("wrong file key resolution", err) } }) } - writeConfig(t, path, "api_key = '$"+name+"'") + writeConfig(t, path, "[provider.jev]\napi_key = '$"+name+"'") t.Setenv(name, "file-expanded-key") t.Setenv("TS_JEV_API_KEY", " $"+name+" ") flags := pflag.NewFlagSet("test", pflag.ContinueOnError) @@ -79,7 +79,7 @@ func TestViewTemplateAndReferenceIndependentOfEnvironment(t *testing.T) { t.Fatal("fresh template cannot be viewed", err) } const name = "JEVWISE_TEST_VIEW_KEY" - writeConfig(t, path, "api_key = '${"+name+"}'") + writeConfig(t, path, "[provider.jev]\napi_key = '${"+name+"}'") t.Setenv(name, "") before, err := View(path) if err != nil { diff --git a/internal/config/files.go b/internal/config/files.go index 5fd7888..06e0c37 100644 --- a/internal/config/files.go +++ b/internal/config/files.go @@ -1,7 +1,7 @@ package config import ( - "bytes" + "cmp" "context" "crypto/rand" _ "embed" @@ -10,10 +10,9 @@ import ( "io" "os" "path/filepath" + "slices" "strconv" "strings" - - "github.com/spf13/viper" ) //go:embed template.toml @@ -79,29 +78,42 @@ func fileValues(path string) (map[string]string, error) { if err != nil { return nil, errors.New("cannot read configuration file; initialize it first") } - v := viper.New() - v.SetConfigType("toml") - if err := v.ReadConfig(bytes.NewReader(data)); err != nil { - return nil, errors.New("cannot read configuration: invalid TOML") - } - defaults := defaultValues() - defaults["editor"] = "vim" - values := make(map[string]string) - for key, value := range defaults { - v.SetDefault(key, value) - value, ok := v.Get(key).(string) - if !ok { - return nil, errors.New("configuration settings must be strings") + v, err := readSettings(data) + if err != nil { + return nil, err + } + provider, _, err := fileString(v, "general.provider") + if err != nil { + return nil, err + } + values := defaultValues(cmp.Or(provider, "jev")) + values["editor"] = "vim" + set := func(key, path string) error { + value, ok, err := fileString(v, path) + if ok { + values[key] = value + } + return err + } + if err := set("provider", "general.provider"); err != nil { + return nil, err + } + if !slices.Contains(Providers, values["provider"]) { + return nil, errProvider + } + for _, key := range []string{"api_key", "base_url", "model", "timeout"} { + if err := set(key, "provider."+values["provider"]+"."+key); err != nil { + return nil, err } - values[key] = strings.TrimSpace(value) + } + if err := set("editor", "general.editor"); err != nil { + return nil, err } mcp, err := mcpValues(v, false) if err != nil { return nil, err } - for key, value := range mcp { - values[key] = value - } + values["host"], values["port"] = mcp["host"], mcp["port"] debug, err := debugValue(v) if err != nil { return nil, err @@ -110,9 +122,10 @@ func fileValues(path string) (map[string]string, error) { return values, nil } -// View shows known file values/defaults, never environment overrides, arbitrary -// extra keys or comments. Only api_key is redacted; URL userinfo/query/fragment -// are refused before display. Other settings must not contain credentials. +// View shows known file values/defaults as grouped tables, never environment +// overrides, arbitrary extra keys, other provider tables or comments. Only api_key +// is redacted; URL userinfo/query/fragment are refused before display. Other +// settings must not contain credentials. func View(path string) (string, error) { values, err := fileValues(path) if err != nil { @@ -123,12 +136,25 @@ func View(path string) (string, error) { } values["api_key"] = "[redacted]" var out strings.Builder - for _, key := range []string{"api_key", "base_url", "model", "provider", "timeout", "editor", "mcp_base_url", "mcp_port", "debug"} { - value, _ := json.Marshal(values[key]) - if key == "mcp_port" || key == "debug" { - value = []byte(values[key]) + for i, section := range []struct { + name string + keys []string + }{ + {"general", []string{"provider", "debug", "editor"}}, + {"mcp", []string{"host", "port"}}, + {"provider." + values["provider"], []string{"api_key", "base_url", "model", "timeout"}}, + } { + if i > 0 { + out.WriteString("\n") + } + out.WriteString("[" + section.name + "]\n") + for _, key := range section.keys { + value, _ := json.Marshal(values[key]) + if key == "port" || key == "debug" { + value = []byte(values[key]) + } + out.WriteString(key + " = " + string(value) + "\n") } - out.WriteString(key + " = " + string(value) + "\n") } return out.String(), nil } diff --git a/internal/config/files_test.go b/internal/config/files_test.go index 76f2924..869f8a0 100644 --- a/internal/config/files_test.go +++ b/internal/config/files_test.go @@ -65,7 +65,7 @@ func TestInitPrivateTemplateNoOverwriteOrCredentialCopy(t *testing.T) { func TestViewOnlyKnownFileValuesAndRedactsSecrets(t *testing.T) { cleanEnv(t) t.Setenv("TS_JEV_MODEL", "environment-model") - path := writeConfig(t, filepath.Join(t.TempDir(), "config.toml"), "API_KEY='file-secret'\nmodel='file-model'\nextra='extra-secret'\n# comment-secret\n") + path := writeConfig(t, filepath.Join(t.TempDir(), "config.toml"), "extra='extra-secret'\n# comment-secret\n[provider.jev]\nAPI_KEY='file-secret'\nmodel='file-model'\n") text, err := View(path) if err != nil || !strings.Contains(text, `model = "file-model"`) || !strings.Contains(text, `editor = "vim"`) || !strings.Contains(text, "[redacted]") { t.Fatal("wrong file view", err) @@ -75,7 +75,7 @@ func TestViewOnlyKnownFileValuesAndRedactsSecrets(t *testing.T) { t.Fatal("view leaked non-file or secret data") } } - for _, content := range []string{"api_key='parse-secret'\n[", "base_url='https://user:url-secret@example.com'", "editor=42"} { + for _, content := range []string{"[provider.jev]\napi_key='parse-secret'\n[", "[provider.jev]\nbase_url='https://user:url-secret@example.com'", "[general]\neditor=42"} { writeConfig(t, path, content) if out, err := View(path); out != "" || err == nil || strings.Contains(err.Error(), "secret") { t.Fatal("invalid file accepted or leaked", err) diff --git a/internal/config/mcp.go b/internal/config/mcp.go index 6bfe60f..0d4b190 100644 --- a/internal/config/mcp.go +++ b/internal/config/mcp.go @@ -1,7 +1,6 @@ package config import ( - "bytes" "errors" "net/netip" "os" @@ -13,18 +12,17 @@ import ( // Default MCP listen settings. The server binds only literal loopback IPs. const ( - DefaultMCPBaseURL = "127.0.0.1" - DefaultMCPPort = 8080 + DefaultMCPHost = "127.0.0.1" + DefaultMCPPort = 8080 ) -var errMCPPort = errors.New("mcp_port must be an integer from 1 to 65535") +var errHost = errors.New("mcp.host must be a literal loopback IP without scheme or port, such as 127.0.0.1") +var errMCPPort = errors.New("mcp.port must be an integer from 1 to 65535") -// MCPAddress resolves mcp_base_url and mcp_port (nonblank environment > TOML > -// defaults) into a canonical loopback IP:port. It reads no other settings, so only a +// MCPAddress resolves [mcp] host and port (nonblank TS_JEV_MCP_BASE_URL and +// TS_JEV_MCP_PORT > TOML > defaults) into a canonical loopback IP:port. It reads no other settings, so only a // missing file falls back to defaults; unsafe files, invalid TOML or values fail. func MCPAddress(path string) (string, error) { - v := viper.New() - v.SetConfigType("toml") resolved, _, err := ResolvePath(path) if err != nil { return "", err @@ -33,48 +31,54 @@ func MCPAddress(path string) (string, error) { if err != nil && !errors.Is(err, os.ErrNotExist) { return "", errors.New("cannot read configuration: file unreadable or unsafe") } - if err == nil && v.ReadConfig(bytes.NewReader(data)) != nil { - return "", errors.New("cannot read configuration: invalid TOML") + v := viper.New() + if err == nil { + if v, err = readSettings(data); err != nil { + return "", err + } } values, err := mcpValues(v, true) if err != nil { return "", err } - return mcpAddress(values["mcp_base_url"], values["mcp_port"]) + return mcpAddress(values["host"], values["port"]) } +// mcpEnvNames maps [mcp] keys to their environment overrides, unchanged since the flat layout. +var mcpEnvNames = map[string]string{"host": "TS_JEV_MCP_BASE_URL", "port": "TS_JEV_MCP_PORT"} + // mcpValues reads both keys, accepting a TOML integer or string port, and // validates the result. Environment overrides apply only when env is true. func mcpValues(v *viper.Viper, env bool) (map[string]string, error) { - values := map[string]string{"mcp_base_url": DefaultMCPBaseURL, "mcp_port": strconv.Itoa(DefaultMCPPort)} + values := map[string]string{"host": DefaultMCPHost, "port": strconv.Itoa(DefaultMCPPort)} for key := range values { - switch raw := v.Get(key).(type) { + switch raw := v.Get("mcp." + key).(type) { case nil: case string: values[key] = strings.TrimSpace(raw) case int64: - if key != "mcp_port" { - return nil, errors.New("mcp_base_url must be a string") + if key != "port" { + return nil, errHost } values[key] = strconv.FormatInt(raw, 10) case int: - if key != "mcp_port" { - return nil, errors.New("mcp_base_url must be a string") + if key != "port" { + return nil, errHost } values[key] = strconv.Itoa(raw) default: - if key == "mcp_port" { + if key == "port" { return nil, errMCPPort } - return nil, errors.New("mcp_base_url must be a string") + return nil, errHost } if env { - if value := strings.TrimSpace(os.Getenv("TS_JEV_" + strings.ToUpper(key))); value != "" { + if value := strings.TrimSpace(os.Getenv(mcpEnvNames[key])); value != "" { values[key] = value } } } - if _, err := mcpAddress(values["mcp_base_url"], values["mcp_port"]); err != nil { + if _, err := mcpAddress(values["host"], values["port"]); err != nil { return nil, err } return values, nil @@ -83,7 +87,7 @@ func mcpValues(v *viper.Viper, env bool) (map[string]string, error) { func mcpAddress(host, port string) (string, error) { addr, err := netip.ParseAddr(host) if err != nil || !addr.IsLoopback() || addr.Zone() != "" { - return "", errors.New("mcp_base_url must be a literal loopback IP without scheme or port, such as 127.0.0.1") + return "", errHost } n, err := strconv.ParseUint(port, 10, 16) if err != nil || n == 0 { diff --git a/internal/config/mcp_test.go b/internal/config/mcp_test.go index 1b79ab9..86e5a06 100644 --- a/internal/config/mcp_test.go +++ b/internal/config/mcp_test.go @@ -22,23 +22,23 @@ func TestMCPAddressPrecedence(t *testing.T) { t.Fatal("missing file did not fall back to defaults", got, err) } for content, want := range map[string]string{ - "mcp_port = 9090\n": "127.0.0.1:9090", - "mcp_port = \"9091\"\n": "127.0.0.1:9091", - "mcp_base_url = \"::1\"\nmcp_port = 9092\n": "[::1]:9092", - "mcp_base_url = \" 127.0.0.2 \"\n": "127.0.0.2:8080", - "model = 'unrelated'\ntimeout = 'invalid'\n": "127.0.0.1:8080", + "[mcp]\nport = 9090\n": "127.0.0.1:9090", + "[mcp]\nport = \"9091\"\n": "127.0.0.1:9091", + "[mcp]\nhost = \"::1\"\nport = 9092\n": "[::1]:9092", + "[mcp]\nhost = \" 127.0.0.2 \"\n": "127.0.0.2:8080", + "[provider.jev]\nmodel = 'unrelated'\ntimeout = 'invalid'\n": "127.0.0.1:8080", } { path := writeConfig(t, filepath.Join(dir, "config.toml"), content) if got, err := MCPAddress(path); err != nil || got != want { t.Fatal("wrong address", content, got, err) } } - if err := os.Symlink(writeConfig(t, filepath.Join(dir, "target.toml"), "mcp_port = 9090\n"), LocalName); err == nil { + if err := os.Symlink(writeConfig(t, filepath.Join(dir, "target.toml"), "[mcp]\nport = 9090\n"), LocalName); err == nil { if _, err := MCPAddress(""); err == nil { t.Fatal("linked local configuration accepted") } } - path := writeConfig(t, filepath.Join(dir, "config.toml"), "mcp_port = 9090\n") + path := writeConfig(t, filepath.Join(dir, "config.toml"), "[mcp]\nport = 9090\n") t.Setenv("TS_JEV_MCP_PORT", "9093") if got, err := MCPAddress(path); err != nil || got != "127.0.0.1:9093" { t.Fatal("environment did not override TOML", got, err) @@ -49,9 +49,9 @@ func TestMCPAddressRejectsInvalidValues(t *testing.T) { mcpEnv(t) dir := t.TempDir() for _, content := range []string{ - "mcp_base_url = 'http://127.0.0.1'\n", "mcp_base_url = 'localhost'\n", "mcp_base_url = '0.0.0.0'\n", - "mcp_base_url = '127.0.0.1:9090'\n", "mcp_base_url = ''\n", "mcp_base_url = 1\n", - "mcp_port = 0\n", "mcp_port = 65536\n", "mcp_port = -1\n", "mcp_port = 'http'\n", "mcp_port = true\n", "mcp_port = [", + "[mcp]\nhost = 'http://127.0.0.1'\n", "[mcp]\nhost = 'localhost'\n", "[mcp]\nhost = '0.0.0.0'\n", + "[mcp]\nhost = '127.0.0.1:9090'\n", "[mcp]\nhost = ''\n", "[mcp]\nhost = 1\n", + "[mcp]\nport = 0\n", "[mcp]\nport = 65536\n", "[mcp]\nport = -1\n", "[mcp]\nport = 'http'\n", "[mcp]\nport = true\n", "[mcp]\nport = [", } { path := writeConfig(t, filepath.Join(dir, "config.toml"), content) if got, err := MCPAddress(path); err == nil || got != "" { @@ -64,13 +64,13 @@ func TestViewAndTemplateIncludeMCPSettings(t *testing.T) { mcpEnv(t) path := writeConfig(t, filepath.Join(t.TempDir(), "config.toml"), string(template)) text, err := View(path) - if err != nil || !strings.Contains(text, "mcp_base_url = \"127.0.0.1\"\n") || !strings.Contains(text, "mcp_port = 8080\n") { + if err != nil || !strings.Contains(text, "[mcp]\nhost = \"127.0.0.1\"\nport = 8080\n") { t.Fatal("view or template lacks MCP settings", err, text) } if got, err := MCPAddress(path); err != nil || got != "127.0.0.1:8080" { t.Fatal("template MCP settings invalid", got, err) } - writeConfig(t, path, "mcp_port = 0\n") + writeConfig(t, path, "[mcp]\nport = 0\n") if _, err := View(path); err == nil { t.Fatal("view accepted invalid MCP port") } diff --git a/internal/config/migrate.go b/internal/config/migrate.go new file mode 100644 index 0000000..5ac13c1 --- /dev/null +++ b/internal/config/migrate.go @@ -0,0 +1,175 @@ +package config + +import ( + "bytes" + "context" + "crypto/rand" + "encoding/json" + "errors" + "fmt" + "os" + "path/filepath" + "slices" + "strconv" + "strings" + + "github.com/spf13/viper" +) + +// ErrNotFlat reports a file that already uses grouped tables; nothing is changed. +var ErrNotFlat = errors.New("configuration already uses grouped tables; nothing to migrate") + +// BackupSuffix names the private copy Migrate keeps beside the original file. +const BackupSuffix = ".flat.bak" + +type target struct{ section, key string } + +var migration = map[string]target{ + "provider": {"general", "provider"}, "debug": {"general", "debug"}, "editor": {"general", "editor"}, + "mcp_base_url": {"mcp", "host"}, "mcp_port": {"mcp", "port"}, + "api_key": {"provider", "api_key"}, "base_url": {"provider", "base_url"}, "model": {"provider", "model"}, "timeout": {"provider", "timeout"}, +} + +// Migrate rewrites a flat configuration file as grouped tables and returns the +// backup path. It moves keys without validating or expanding values, so $NAME +// references stay literal and invalid values can be fixed afterwards. Comments are +// dropped. Links, unknown or mixed keys, existing backups and values other than +// strings, integers or booleans are refused. Errors name keys, never values. +func Migrate(ctx context.Context, path string) (string, error) { + if ctx == nil { + return "", errors.New("configuration context is required") + } + if err := ctx.Err(); err != nil { + return "", err + } + if info, err := os.Lstat(path); err != nil { + return "", errors.New("cannot read configuration file; nothing to migrate") + } else if !info.Mode().IsRegular() { + return "", errors.New("configuration must be a regular file, not a link; migrate the link target instead") + } + data, err := ReadFile(path) + if err != nil { + return "", errors.New("cannot read configuration file; nothing to migrate") + } + v, err := parseTOML(data) + if err != nil { + return "", err + } + if !isFlat(v) { + return "", ErrNotFlat + } + text, err := groupedTOML(v) + if err != nil { + return "", err + } + if err := ctx.Err(); err != nil { + return "", err + } + return path + BackupSuffix, replaceWithBackup(path, data, text) +} + +func groupedTOML(v *viper.Viper) ([]byte, error) { + provider := "jev" + if name, ok := v.Get("provider").(string); ok && slices.Contains(Providers, strings.TrimSpace(name)) { + provider = strings.TrimSpace(name) + } + lines := map[string][]string{} + for _, key := range v.AllKeys() { + to, ok := migration[key] + if !ok { + return nil, fmt.Errorf("cannot migrate %q: unknown or mixed setting; move it by hand", key) + } + value, err := tomlValue(v.Get(key)) + if err != nil { + return nil, fmt.Errorf("cannot migrate %q: value must be a string, integer or boolean", key) + } + lines[to.section] = append(lines[to.section], to.key+" = "+value) + } + var out bytes.Buffer + out.WriteString("# Migrated from the flat layout; comments were not carried over.\n") + for _, section := range []string{"general", "mcp", "provider"} { + entries := lines[section] + if len(entries) == 0 { + continue + } + slices.SortFunc(entries, func(a, b string) int { return order(a) - order(b) }) + name := section + if section == "provider" { + name = "provider." + provider + } + out.WriteString("\n[" + name + "]\n" + strings.Join(entries, "\n") + "\n") + } + return out.Bytes(), nil +} + +func order(line string) int { + key, _, _ := strings.Cut(line, " ") + return slices.Index([]string{"provider", "debug", "editor", "host", "port", "api_key", "base_url", "model", "timeout"}, key) +} + +func tomlValue(raw any) (string, error) { + switch value := raw.(type) { + case string: + var buf bytes.Buffer + encoder := json.NewEncoder(&buf) + encoder.SetEscapeHTML(false) + if err := encoder.Encode(value); err != nil { + return "", err + } + return strings.TrimSuffix(buf.String(), "\n"), nil + case bool: + return strconv.FormatBool(value), nil + case int64: + return strconv.FormatInt(value, 10), nil + case int: + return strconv.Itoa(value), nil + } + return "", errors.New("unsupported value") +} + +// replaceWithBackup keeps the original bytes in a new private backup, then +// atomically renames a staged private file over the configuration. A failed +// rewrite removes its backup so the unchanged original can be migrated again. +func replaceWithBackup(path string, original, grouped []byte) (err error) { + root, err := os.OpenRoot(filepath.Dir(path)) + if err != nil { + return err + } + defer root.Close() + name := filepath.Base(path) + if err := writePrivate(root, name+BackupSuffix, original); err != nil { + if errors.Is(err, os.ErrExist) { + return errors.New("a migration backup already exists; move it away before migrating again") + } + return err + } + stage := ".config-" + rand.Text() + ".tmp" + defer func() { + if cleanup := root.Remove(stage); cleanup != nil && !errors.Is(cleanup, os.ErrNotExist) { + err = errors.Join(err, errors.New("cannot clean configuration staging file")) + } + if err != nil { + _ = root.Remove(name + BackupSuffix) + } + }() + if err := writePrivate(root, stage, grouped); err != nil { + return err + } + return root.Rename(stage, name) +} + +func writePrivate(root *os.Root, name string, data []byte) error { + file, err := root.OpenFile(name, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0600) + if err != nil { + return err + } + if _, err := file.Write(data); err != nil { + _ = file.Close() + return err + } + if err := file.Sync(); err != nil { + _ = file.Close() + return err + } + return file.Close() +} diff --git a/internal/config/migrate_test.go b/internal/config/migrate_test.go new file mode 100644 index 0000000..a6883da --- /dev/null +++ b/internal/config/migrate_test.go @@ -0,0 +1,114 @@ +package config + +import ( + "context" + "errors" + "os" + "path/filepath" + "runtime" + "strings" + "testing" +) + +const flatFixture = "# comment\nAPI_KEY = \"$JEV_TOKEN\"\nbase_url = \"https://flat.test\"\nmodel = \"flat-model\"\nprovider = \"codex\"\ntimeout = \"garbage\"\neditor = \"nano\"\nmcp_base_url = \"127.0.0.2\"\nmcp_port = \"9090\"\ndebug = true\n" + +func TestFlatLayoutFailsWithMigrateHint(t *testing.T) { + mcpEnv(t) + t.Setenv("TS_JEV_DEBUG", "") + path := writeConfig(t, filepath.Join(t.TempDir(), "config.toml"), "model = 'private-flat'\n") + _, loadErr := Load(path, nil) + _, mcpErr := MCPAddress(path) + _, debugErr := Debug(path) + _, viewErr := View(path) + for _, err := range []error{loadErr, mcpErr, debugErr, viewErr} { + if !errors.Is(err, ErrFlatLayout) || !strings.Contains(err.Error(), "jevwise config migrate") || strings.Contains(err.Error(), "private") { + t.Fatal("flat layout not rejected with hint", err) + } + } + writeConfig(t, path, "[provider]\njev = { model = 'grouped' }\n") + if cfg, err := Load(path, nil); err != nil || cfg.Model != "grouped" { + t.Fatal("provider table treated as flat", err) + } + if editor, err := Editor(writeConfig(t, path, "editor = 'nano'\n")); err != nil || editor != "nano" { + t.Fatal("flat file cannot be opened for repair", editor, err) + } +} + +func TestMigrateRoundTripKeepsBackupAndLiterals(t *testing.T) { + mcpEnv(t) + t.Setenv("TS_JEV_DEBUG", "") + t.Setenv("JEV_TOKEN", "expanded-token") + path := writeConfig(t, filepath.Join(t.TempDir(), "config.toml"), flatFixture) + backup, err := Migrate(context.Background(), path) + if err != nil || backup != path+BackupSuffix { + t.Fatal("migration failed", err) + } + data, _ := os.ReadFile(path) + saved, _ := os.ReadFile(backup) + text := string(data) + for _, want := range []string{"[general]\nprovider = \"codex\"\ndebug = true\neditor = \"nano\"\n", "[mcp]\nhost = \"127.0.0.2\"\nport = \"9090\"\n", "[provider.codex]\napi_key = \"$JEV_TOKEN\"\nbase_url = \"https://flat.test\"\nmodel = \"flat-model\"\ntimeout = \"garbage\"\n"} { + if !strings.Contains(text, want) { + t.Fatal("unexpected migrated layout", text) + } + } + if string(saved) != flatFixture || strings.Contains(text, "# comment") { + t.Fatal("backup changed or comment carried over") + } + for _, name := range []string{path, backup} { + if info, _ := os.Stat(name); runtime.GOOS != "windows" && info.Mode().Perm() != 0600 { + t.Fatal("migrated files are not private", name) + } + } + writeConfig(t, path, strings.Replace(text, "garbage", "1s", 1)) + cfg, err := Load(path, nil) + if err != nil || cfg.Provider != "codex" || cfg.APIKey != "expanded-token" || cfg.Model != "flat-model" || cfg.BaseURL != "https://flat.test" { + t.Fatal("migrated file loads different values", err) + } + if address, err := MCPAddress(path); err != nil || address != "127.0.0.2:9090" { + t.Fatal("migrated MCP address changed", address, err) + } + if enabled, err := Debug(path); err != nil || !enabled { + t.Fatal("migrated debug changed", err) + } + if _, err := Migrate(context.Background(), path); !errors.Is(err, ErrNotFlat) { + t.Fatal("grouped file migrated again", err) + } +} + +func TestMigrateRefusesWithoutChanging(t *testing.T) { + dir := t.TempDir() + path := filepath.Join(dir, "config.toml") + for _, content := range []string{ + "model = 'm'\nunknown = 'private-value'\n", "model = 'm'\n[mcp]\nport = 1\n", + "model = ['private-value']\n", "model = 'm'\n[", + } { + writeConfig(t, path, content) + _, err := Migrate(context.Background(), path) + after, _ := os.ReadFile(path) + if err == nil || string(after) != content || strings.Contains(err.Error(), "private") { + t.Fatal("invalid migration changed file or leaked", content, err) + } + if _, statErr := os.Stat(path + BackupSuffix); !errors.Is(statErr, os.ErrNotExist) { + t.Fatal("refused migration left a backup") + } + } + writeConfig(t, path, "model = 'm'\n") + writeConfig(t, path+BackupSuffix, "earlier") + if _, err := Migrate(context.Background(), path); err == nil || !strings.Contains(err.Error(), "backup already exists") { + t.Fatal("existing backup replaced", err) + } + if saved, _ := os.ReadFile(path + BackupSuffix); string(saved) != "earlier" { + t.Fatal("existing backup changed") + } + link := filepath.Join(dir, "link.toml") + if os.Symlink(writeConfig(t, filepath.Join(dir, "target.toml"), "model = 'm'\n"), link) == nil { + if _, err := Migrate(context.Background(), link); err == nil { + t.Fatal("link migrated") + } + } + ctx, cancel := context.WithCancel(context.Background()) + cancel() + if _, err := Migrate(ctx, path); !errors.Is(err, context.Canceled) { + t.Fatal("canceled migration ran", err) + } +} diff --git a/internal/config/paths_test.go b/internal/config/paths_test.go index bf02348..52e1d54 100644 --- a/internal/config/paths_test.go +++ b/internal/config/paths_test.go @@ -13,8 +13,8 @@ import ( func TestLocalDiscoveryPrecedenceAndNoMerge(t *testing.T) { cleanEnv(t) global, _ := GlobalPath() - writeConfig(t, global, "model='global'\napi_key='global-secret'") - writeConfig(t, LocalName, "model='local'") + writeConfig(t, global, "[provider.jev]\nmodel='global'\napi_key='global-secret'") + writeConfig(t, LocalName, "[provider.jev]\nmodel='local'") cfg, err := Load("", nil) if err != nil || cfg.Model != "local" || cfg.APIKey != "" { t.Fatal("local file did not replace global", err) @@ -49,10 +49,10 @@ func TestLocalDiscoveryFailsClosedAndDoesNotSearchAncestors(t *testing.T) { t.Run(mode, func(t *testing.T) { cleanEnv(t) global, _ := GlobalPath() - writeConfig(t, global, "model='global'") + writeConfig(t, global, "[provider.jev]\nmodel='global'") switch mode { case "invalid": - writeConfig(t, LocalName, "api_key='private-secret'\n[") + writeConfig(t, LocalName, "[provider.jev]\napi_key='private-secret'\n[") case "oversize": writeConfig(t, LocalName, strings.Repeat("x", MaxFileBytes+1)) case "directory": @@ -71,7 +71,7 @@ func TestLocalDiscoveryFailsClosedAndDoesNotSearchAncestors(t *testing.T) { }) } cleanEnv(t) - writeConfig(t, LocalName, "model='ancestor'") + writeConfig(t, LocalName, "[provider.jev]\nmodel='ancestor'") // A nested directory of the current project must not inherit its file. cwd, _ := os.Getwd() child := filepath.Join(cwd, "child") diff --git a/internal/config/settings.go b/internal/config/settings.go new file mode 100644 index 0000000..ca89cfb --- /dev/null +++ b/internal/config/settings.go @@ -0,0 +1,54 @@ +package config + +import ( + "bytes" + "errors" + + "github.com/spf13/viper" +) + +var errInvalidTOML = errors.New("cannot read configuration: invalid TOML") + +// parseTOML parses only the selected bytes into a fresh Viper instance. It never +// uses Viper search paths, ReadInConfig, environment binding or defaults. +func parseTOML(data []byte) (*viper.Viper, error) { + v := viper.New() + v.SetConfigType("toml") + if err := v.ReadConfig(bytes.NewReader(data)); err != nil { + return nil, errInvalidTOML + } + return v, nil +} + +// ErrFlatLayout reports a pre-grouped configuration file. It never includes values. +var ErrFlatLayout = errors.New("configuration uses the old flat layout; run `jevwise config migrate` to convert it to [general], [mcp] and [provider.jev] tables") + +// flatKeys are the top-level settings of the flat layout used before grouped tables. +var flatKeys = []string{"api_key", "base_url", "model", "provider", "timeout", "editor", "mcp_base_url", "mcp_port", "debug"} + +// isFlat reports whether any flat-layout key appears at the top level. A +// top-level provider table belongs to the grouped layout; a provider value does not. +func isFlat(v *viper.Viper) bool { + for _, key := range flatKeys { + if !v.InConfig(key) { + continue + } + if _, table := v.Get(key).(map[string]any); key == "provider" && table { + continue + } + return true + } + return false +} + +// readSettings parses grouped configuration and rejects the flat layout. +func readSettings(data []byte) (*viper.Viper, error) { + v, err := parseTOML(data) + if err != nil { + return nil, err + } + if isFlat(v) { + return nil, ErrFlatLayout + } + return v, nil +} diff --git a/internal/config/template.toml b/internal/config/template.toml index 2327386..766bdbc 100644 --- a/internal/config/template.toml +++ b/internal/config/template.toml @@ -1,12 +1,21 @@ # Optional CLI configuration. Prefer environment credentials. +[general] +provider = "jev" +debug = false +editor = "vim" + +# Local MCP server: literal loopback IP (no scheme) and port. +[mcp] +host = "127.0.0.1" +port = 8080 + +[provider.jev] api_key = "" base_url = "https://api.typesafe.ai" model = "jev-latest" -provider = "jev" timeout = "10s" -editor = "vim" -# Local MCP server: literal loopback IP (no scheme) and port. -mcp_base_url = "127.0.0.1" -mcp_port = 8080 -# Structured debug logs on stderr; --debug or --verbose overrides. -debug = false + +# Google Gemini Decisions: select with provider = "gemini" or --provider gemini. +[provider.gemini] +api_key = "" +model = "gemini-decision-1" diff --git a/internal/daemon/validation.go b/internal/daemon/validation.go index 667241b..3428a9e 100644 --- a/internal/daemon/validation.go +++ b/internal/daemon/validation.go @@ -18,7 +18,7 @@ import ( // Runtime-directory privacy and exclusive ownership are checked during startup. func ValidateBootstrap(boot Bootstrap) error { invalid := errors.New("invalid background bootstrap configuration") - if boot.Config.Validate() != nil || boot.Config.Provider != "jev" || boot.Config.APIKey == "" || boot.Config.Timeout > time.Duration(1<<63-1)-30*time.Second { + if boot.Config.Validate() != nil || boot.Config.Provider != "jev" && boot.Config.Provider != "gemini" || boot.Config.APIKey == "" || boot.Config.Timeout > time.Duration(1<<63-1)-30*time.Second { return invalid } for _, value := range []string{boot.Config.APIKey, boot.Config.BaseURL, boot.Config.Model, boot.Config.Provider} { diff --git a/internal/daemon/validation_test.go b/internal/daemon/validation_test.go index 9b063f0..4cf88a7 100644 --- a/internal/daemon/validation_test.go +++ b/internal/daemon/validation_test.go @@ -26,6 +26,11 @@ func TestBootstrapValidationIsPure(t *testing.T) { if err := ValidateBootstrap(boot); err != nil || boot != before { t.Fatal("valid bootstrap changed or failed", err) } + boot.Config.Provider = "gemini" + if err := ValidateBootstrap(boot); err != nil { + t.Fatal("gemini bootstrap rejected", err) + } + boot = before if _, err := os.Lstat(boot.RuntimeDir); !errors.Is(err, os.ErrNotExist) { t.Fatal("validation created runtime storage") } diff --git a/internal/gemini/client.go b/internal/gemini/client.go new file mode 100644 index 0000000..1e733e1 --- /dev/null +++ b/internal/gemini/client.go @@ -0,0 +1,286 @@ +// Package gemini is a minimal client for one choice question on the Gemini +// Decisions API (POST /v1beta/decisions). It follows pkg/typesafe safety rules: +// no redirects, bounded responses, redacted errors and deadline-bound retries. +package gemini + +import ( + "bytes" + "cmp" + "context" + "encoding/json" + "errors" + "fmt" + "io" + "math" + "math/rand/v2" + "net/http" + "net/url" + "regexp" + "slices" + "strings" + "time" + + "github.com/bitbrew-dev/jevwise/internal/debuglog" + typesafe "github.com/bitbrew-dev/jevwise/pkg/typesafe" +) + +const ( + // DefaultBaseURL is the Gemini Developer API host. + DefaultBaseURL = "https://generativelanguage.googleapis.com" + // DefaultModel is the Gemini Decisions model. + DefaultModel = "gemini-decision-1" + // MaxOptions is the upstream limit for one choice question. + MaxOptions = 255 + + endpointPath = "/v1beta/decisions" + questionName = "decision" + maxResponseBytes = 8 << 20 + maxRetries = 2 + backoffInitial = 500 * time.Millisecond + backoffMax = 5 * time.Second + maxRetryAfter = 30 * time.Second +) + +// Options configures a Client. Redirects are always refused. +type Options struct { + APIKey, BaseURL, Model string + Timeout time.Duration +} + +// Client sends choice questions with one API key, host, model and per-attempt timeout. +type Client struct { + apiKey, endpoint, model string + timeout time.Duration + http http.Client +} + +// ChoiceRequest is one shared input with distinct option labels. +type ChoiceRequest struct { + Input, Instructions string + Options []string +} + +// Result holds the service distribution, never normalized locally, and the +// reported token counts when present. +type Result struct { + Choice string + Confidence float64 + Probabilities map[string]float64 + InputTokens, OutputTokens *int +} + +// APIError is a non-2xx response. Upstream messages are dropped: they may echo input. +type APIError struct { + StatusCode int + Code string +} + +func (e *APIError) Error() string { + return strings.TrimSpace(fmt.Sprintf("gemini: API error %d %s", e.StatusCode, e.Code)) +} + +// BlockedError reports input rejected by content filters, by status or prompt_feedback. +type BlockedError struct{ Reason string } + +func (e *BlockedError) Error() string { + return "gemini: input blocked by content filters (" + e.Reason + ")" +} + +// ErrInvalidResponse reports a response that does not match the request. +var ErrInvalidResponse = errors.New("gemini: invalid decision response") + +// Transport failures hide URLs and causes, which may carry request details. +var ( + errTimeout = errors.New("gemini: request timed out") + errConnection = errors.New("gemini: connection failed") +) + +// New validates options. The key must be printable ASCII; the base URL must be an +// absolute HTTP(S) URL without credentials, query or fragment. +func New(options Options) (*Client, error) { + key := strings.TrimSpace(options.APIKey) + if key == "" || strings.IndexFunc(key, func(r rune) bool { return r < 0x21 || r > 0x7e }) >= 0 { + return nil, errors.New("gemini: API key must be nonblank printable ASCII") + } + base := strings.TrimRight(cmp.Or(options.BaseURL, DefaultBaseURL), "/") + u, err := url.Parse(base) + if err != nil || u.Hostname() == "" || (u.Scheme != "http" && u.Scheme != "https") || u.User != nil || u.RawQuery != "" || u.Fragment != "" { + return nil, errors.New("gemini: base URL must be an absolute HTTP(S) URL without credentials, query or fragment") + } + if options.Timeout <= 0 { + return nil, errors.New("gemini: timeout must be positive") + } + c := &Client{apiKey: key, endpoint: base + endpointPath, model: cmp.Or(strings.TrimSpace(options.Model), DefaultModel), timeout: options.Timeout} + c.http.CheckRedirect = func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse } + return c, nil +} + +type question struct { + Type string `json:"type"` + Instructions string `json:"instructions"` + Criteria map[string]string `json:"criteria"` +} + +type response struct { + Answers map[string]struct { + Type string `json:"type"` + Choice string `json:"choice"` + Confidence *float64 `json:"confidence"` + Probabilities map[string]float64 `json:"probabilities"` + } `json:"answers"` + Usage struct { + Input *int `json:"total_input_tokens"` + Output *int `json:"total_output_tokens"` + } `json:"usage"` + Feedback struct { + BlockReason string `json:"block_reason"` + } `json:"prompt_feedback"` +} + +// Choose asks one choice question with an empty description for every label. +func (c *Client) Choose(ctx context.Context, r ChoiceRequest) (Result, error) { + if len(r.Options) < 2 || len(r.Options) > MaxOptions || strings.TrimSpace(r.Instructions) == "" { + return Result{}, fmt.Errorf("gemini: a choice needs instructions and 2 to %d options", MaxOptions) + } + criteria := make(map[string]string, len(r.Options)) + for _, option := range r.Options { + criteria[option] = "" + } + if len(criteria) != len(r.Options) { + return Result{}, errors.New("gemini: options must be distinct") + } + body, err := json.Marshal(map[string]any{"model": c.model, "input": r.Input, + "questions": map[string]question{questionName: {"choice", r.Instructions, criteria}}}) + if err != nil { + return Result{}, err + } + payload, err := c.send(ctx, body) + if err != nil { + return Result{}, err + } + var decoded response + if json.Unmarshal(payload, &decoded) != nil { + return Result{}, ErrInvalidResponse + } + if reason := decoded.Feedback.BlockReason; reason != "" { + return Result{}, &BlockedError{Reason: safeCode(reason)} + } + answer, ok := decoded.Answers[questionName] + if !ok || answer.Type != "choice" || answer.Confidence == nil || !validDistribution(answer.Choice, answer.Probabilities, r.Options) { + return Result{}, ErrInvalidResponse + } + return Result{Choice: answer.Choice, Confidence: *answer.Confidence, Probabilities: answer.Probabilities, + InputTokens: decoded.Usage.Input, OutputTokens: decoded.Usage.Output}, nil +} + +func validDistribution(choice string, probabilities map[string]float64, options []string) bool { + if len(probabilities) != len(options) || !slices.Contains(options, choice) { + return false + } + for _, option := range options { + p, ok := probabilities[option] + if !ok || math.IsNaN(p) || p < 0 || p > 1 { + return false + } + } + return true +} + +// send retries 408, 409, 429, 5xx, connection and timeout failures at most twice, +// never past the caller's deadline. Other statuses, including 400, fail at once. +func (c *Client) send(ctx context.Context, body []byte) ([]byte, error) { + for attempt := 0; ; attempt++ { + payload, headers, err := c.once(ctx, body) + if err == nil || attempt == maxRetries || !retryable(err) { + return payload, err + } + delay := min(backoffInitial<= 500 + } + return errors.Is(err, errTimeout) || errors.Is(err, errConnection) +} + +func (c *Client) once(ctx context.Context, body []byte) (_ []byte, _ http.Header, resultErr error) { + finish := debuglog.Trace(ctx, "gemini.request") + defer func() { finish(resultErr) }() + if err := ctx.Err(); err != nil { + return nil, nil, err + } + attemptCtx, cancel := context.WithTimeout(ctx, c.timeout) + defer cancel() + req, err := http.NewRequestWithContext(attemptCtx, http.MethodPost, c.endpoint, bytes.NewReader(body)) + if err != nil { + return nil, nil, errors.New("gemini: cannot build request") + } + req.Header.Set("x-goog-api-key", c.apiKey) + req.Header.Set("Content-Type", "application/json") + req.Header.Set("Accept", "application/json") + res, err := c.http.Do(req) + if err != nil { + if ctxErr := ctx.Err(); ctxErr != nil { + return nil, nil, ctxErr + } + if attemptCtx.Err() != nil { + return nil, nil, errTimeout + } + return nil, nil, errConnection + } + defer res.Body.Close() + debuglog.Count(ctx, "gemini.response.status", res.StatusCode) + payload, err := io.ReadAll(io.LimitReader(res.Body, maxResponseBytes+1)) + if err != nil || len(payload) > maxResponseBytes { + return nil, res.Header, ErrInvalidResponse + } + if res.StatusCode < 200 || res.StatusCode > 299 { + return nil, res.Header, apiError(res.StatusCode, payload) + } + return payload, res.Header, nil +} + +func apiError(status int, payload []byte) error { + var body struct { + Error struct { + Code json.RawMessage `json:"code"` + Status string `json:"status"` + } `json:"error"` + } + _ = json.Unmarshal(payload, &body) + code := body.Error.Status + var text string + if json.Unmarshal(body.Error.Code, &text) == nil && text != "" { + code = text + } + if status == http.StatusBadRequest && strings.EqualFold(code, "safety") { + return &BlockedError{Reason: "safety"} + } + return &APIError{StatusCode: status, Code: safeCode(code)} +} + +// safeCode keeps short identifier-like codes and replaces anything else. +func safeCode(code string) string { + if !codePattern.MatchString(code) { + return "other" + } + return code +} + +var codePattern = regexp.MustCompile(`^[A-Za-z0-9_-]{1,64}$`) diff --git a/internal/gemini/client_test.go b/internal/gemini/client_test.go new file mode 100644 index 0000000..224eaa2 --- /dev/null +++ b/internal/gemini/client_test.go @@ -0,0 +1,163 @@ +package gemini + +import ( + "context" + "encoding/json" + "errors" + "io" + "net/http" + "net/http/httptest" + "strings" + "sync/atomic" + "testing" + "time" +) + +const fixtureKey = "fixture-gemini-key" + +func fixture(t *testing.T, handler http.HandlerFunc) (*Client, *atomic.Int32) { + t.Helper() + calls := &atomic.Int32{} + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + calls.Add(1) + handler(w, r) + })) + t.Cleanup(server.Close) + client, err := New(Options{APIKey: fixtureKey, BaseURL: server.URL + "/", Model: "fixture-model", Timeout: time.Second}) + if err != nil { + t.Fatal(err) + } + return client, calls +} + +func choiceBody(choice string, probabilities string) string { + return `{"answers":{"decision":{"type":"choice","choice":"` + choice + `","confidence":0.7,"probabilities":` + probabilities + `}},"usage":{"total_input_tokens":12,"total_output_tokens":0}}` +} + +var request = ChoiceRequest{Input: "fixture input", Instructions: "Pick one.", Options: []string{"A", "B"}} + +func TestChooseWireShapeAndAnswer(t *testing.T) { + client, calls := fixture(t, func(w http.ResponseWriter, r *http.Request) { + var body struct { + Model, Input string + Questions map[string]question + } + data, _ := io.ReadAll(r.Body) + if r.Method != http.MethodPost || r.URL.Path != "/v1beta/decisions" || r.Header.Get("x-goog-api-key") != fixtureKey || r.Header.Get("Authorization") != "" || json.Unmarshal(data, &body) != nil { + t.Error("unexpected request line or headers") + } + q := body.Questions["decision"] + if body.Model != "fixture-model" || body.Input != "fixture input" || q.Type != "choice" || q.Instructions != "Pick one." || len(q.Criteria) != 2 || q.Criteria["A"] != "" || !strings.Contains(string(data), `"criteria":{"A":"","B":""}`) { + t.Error("unexpected request body", string(data)) + } + _, _ = io.WriteString(w, choiceBody("B", `{"A":0.3,"B":0.7}`)) + }) + answer, err := client.Choose(context.Background(), request) + if err != nil || answer.Choice != "B" || answer.Confidence != 0.7 || answer.Probabilities["A"] != 0.3 || *answer.InputTokens != 12 || *answer.OutputTokens != 0 || calls.Load() != 1 { + t.Fatal("unexpected answer", answer, err) + } +} + +func TestChooseRejectsInvalidResponses(t *testing.T) { + for _, body := range []string{ + `not json`, `{}`, `{"answers":{"decision":{"type":"noul","noul":0.5,"confidence":0.5}}}`, + choiceBody("C", `{"A":0.3,"B":0.7}`), choiceBody("A", `{"A":0.3}`), choiceBody("A", `{"A":0.3,"B":0.7,"C":0}`), + choiceBody("A", `{"A":1.3,"B":0.7}`), `{"answers":{"decision":{"type":"choice","choice":"A","probabilities":{"A":0.5,"B":0.5}}}}`, + } { + client, _ := fixture(t, func(w http.ResponseWriter, _ *http.Request) { _, _ = io.WriteString(w, body) }) + if _, err := client.Choose(context.Background(), request); !errors.Is(err, ErrInvalidResponse) { + t.Fatal("invalid response accepted", body, err) + } + } +} + +func TestChooseBlockedAndAPIErrorsAreSafeAndNotRetried(t *testing.T) { + for _, tt := range []struct { + status int + body string + blocked bool + }{ + {200, `{"prompt_feedback":{"block_reason":"prohibited_content"}}`, true}, + {400, `{"error":{"code":"safety","message":"Request blocked"}}`, true}, + {400, `{"error":{"code":400,"status":"INVALID_ARGUMENT","message":"bad key ` + fixtureKey + ` and input"}}`, false}, + {403, `not json`, false}, + } { + client, calls := fixture(t, func(w http.ResponseWriter, _ *http.Request) { + w.WriteHeader(tt.status) + _, _ = io.WriteString(w, tt.body) + }) + _, err := client.Choose(context.Background(), request) + var blocked *BlockedError + var api *APIError + if errors.As(err, &blocked) != tt.blocked || !tt.blocked && (!errors.As(err, &api) || api.StatusCode != tt.status) || calls.Load() != 1 { + t.Fatal("wrong error or retried", tt.status, err) + } + if api != nil && (strings.Contains(err.Error(), fixtureKey) || strings.Contains(err.Error(), "input") || tt.status == 400 && api.Code != "INVALID_ARGUMENT") { + t.Fatal("API error leaked key or message", err) + } + } +} + +func TestChooseRetriesTransientFailures(t *testing.T) { + attempts := 0 + client, calls := fixture(t, func(w http.ResponseWriter, _ *http.Request) { + if attempts++; attempts < 3 { + w.Header().Set("Retry-After", "0") + w.WriteHeader(http.StatusServiceUnavailable) + return + } + _, _ = io.WriteString(w, choiceBody("A", `{"A":0.6,"B":0.4}`)) + }) + if answer, err := client.Choose(context.Background(), request); err != nil || answer.Choice != "A" || calls.Load() != 3 { + t.Fatal("transient failures not retried", err, calls.Load()) + } + client, calls = fixture(t, func(w http.ResponseWriter, _ *http.Request) { + w.Header().Set("Retry-After", "0") + w.WriteHeader(http.StatusInternalServerError) + }) + var api *APIError + if _, err := client.Choose(context.Background(), request); !errors.As(err, &api) || calls.Load() != 3 { + t.Fatal("retry count not bounded", err, calls.Load()) + } + client, calls = fixture(t, func(w http.ResponseWriter, _ *http.Request) { + w.Header().Set("Retry-After", "10") + w.WriteHeader(http.StatusTooManyRequests) + }) + ctx, cancel := context.WithTimeout(context.Background(), 200*time.Millisecond) + defer cancel() + start := time.Now() + if _, err := client.Choose(ctx, request); err == nil || calls.Load() != 1 || time.Since(start) > time.Second { + t.Fatal("retry waited past the caller deadline", err) + } +} + +func TestRedirectsAreRefusedWithoutForwardingKey(t *testing.T) { + forwarded := false + target := httptest.NewServer(http.HandlerFunc(func(_ http.ResponseWriter, r *http.Request) { forwarded = r.Header.Get("x-goog-api-key") != "" })) + defer target.Close() + client, _ := fixture(t, func(w http.ResponseWriter, r *http.Request) { + http.Redirect(w, r, target.URL, http.StatusTemporaryRedirect) + }) + var api *APIError + if _, err := client.Choose(context.Background(), request); !errors.As(err, &api) || api.StatusCode != http.StatusTemporaryRedirect || forwarded { + t.Fatal("redirect followed or key forwarded", err) + } +} + +func TestValidation(t *testing.T) { + for _, options := range []Options{{Timeout: 1}, {APIKey: "a key", Timeout: 1}, {APIKey: "k", BaseURL: "ftp://x.test", Timeout: 1}, {APIKey: "k", BaseURL: "https://u:p@x.test", Timeout: 1}, {APIKey: "k"}} { + if _, err := New(options); err == nil { + t.Fatal("invalid options accepted", options.BaseURL) + } + } + client, err := New(Options{APIKey: "k", Timeout: time.Second}) + if err != nil || client.endpoint != DefaultBaseURL+"/v1beta/decisions" || client.model != DefaultModel { + t.Fatal("defaults not applied", err) + } + many := strings.Split(strings.Repeat("x,", MaxOptions)+"y", ",") + for _, r := range []ChoiceRequest{{Instructions: "i", Options: []string{"A"}}, {Instructions: "i", Options: []string{"A", "A"}}, {Instructions: " ", Options: []string{"A", "B"}}, {Instructions: "i", Options: many}} { + if _, err := client.Choose(context.Background(), r); err == nil { + t.Fatal("invalid request accepted", len(r.Options)) + } + } +} diff --git a/internal/mcpserver/tool.go b/internal/mcpserver/tool.go index c158ac2..42e7538 100644 --- a/internal/mcpserver/tool.go +++ b/internal/mcpserver/tool.go @@ -67,7 +67,7 @@ func New(lifetime context.Context, svc service.DecisionService, timeout time.Dur return nil, &toolError{"decision server lifetime has ended", err} } server := mcp.NewServer(&mcp.Implementation{Name: "jevwise", Version: buildinfo.Version}, &mcp.ServerOptions{Logger: slog.New(slog.NewTextHandler(io.Discard, nil))}) - server.AddTool(&mcp.Tool{Name: "decide", Description: "Choose among distinct options using Jev probabilities.", + server.AddTool(&mcp.Tool{Name: "decide", Description: "Choose among distinct options using the configured decision provider's probabilities (Jev or Gemini Decisions).", InputSchema: inputSchema, OutputSchema: outputSchema}, decisionHandler(lifetime, svc, timeout)) return server, nil } @@ -141,15 +141,15 @@ func decisionHandler(lifetime context.Context, svc service.DecisionService, time return toolFailure("decision canceled or timed out", ctxErr) } if err != nil { - return toolFailure("cannot obtain Jev decision", err) + return toolFailure("cannot obtain decision", err) } // The concrete Response fields match outputSchema; Marshal rejects non-JSON numbers. output, err := json.Marshal(response) if err != nil { - return toolFailure("cannot encode Jev decision", err) + return toolFailure("cannot encode decision", err) } if len(output) > outputLimit { - return toolFailure("cannot encode Jev decision", errors.New("decision output is too large")) + return toolFailure("cannot encode decision", errors.New("decision output is too large")) } if err := callError(); err != nil { return toolFailure("decision canceled or timed out", err) diff --git a/internal/service/gemini.go b/internal/service/gemini.go new file mode 100644 index 0000000..5e90a38 --- /dev/null +++ b/internal/service/gemini.go @@ -0,0 +1,65 @@ +package service + +import ( + "context" + "errors" + "strings" + + "github.com/bitbrew-dev/jevwise/internal/gemini" + typesafe "github.com/bitbrew-dev/jevwise/pkg/typesafe" +) + +// GeminiInstructions is the single instruction sent with every Gemini choice; the +// prompt is the shared input, so it is billed once. +const GeminiInstructions = "Choose the option that best answers or fits the input." + +// ChooseFunc permits an injected Gemini client. +type ChooseFunc func(context.Context, gemini.ChoiceRequest) (gemini.Result, error) + +type geminiService struct { + choose ChooseFunc + model string +} + +// NewGemini adapts Gemini Decisions. Responses carry no model, so the configured +// model is reported instead. +func NewGemini(choose ChooseFunc, model string) (DecisionService, error) { + if choose == nil { + return nil, errors.New("gemini requires a Gemini Decisions client") + } + return geminiService{choose, model}, nil +} + +func (s geminiService) Decide(ctx context.Context, r Request) (Response, error) { + if err := r.Validate(); err != nil { + return Response{}, err + } + if len(r.Options) > gemini.MaxOptions { + return Response{}, errors.New("gemini accepts at most 255 options") + } + labels := make([]string, len(r.Options)) + for i, option := range r.Options { + labels[i] = strings.TrimSpace(option) + } + result, err := s.choose(ctx, gemini.ChoiceRequest{Input: r.Prompt, Instructions: GeminiInstructions, Options: labels}) + if err != nil { + return Response{}, err + } + out := Response{Model: s.model, Choice: result.Choice, Confidence: result.Confidence, + Probabilities: make(map[string]float64, len(result.Probabilities))} + for label, probability := range result.Probabilities { + out.Probabilities[label] = probability + } + if result.InputTokens != nil || result.OutputTokens != nil { + out.Usage = &typesafe.Usage{InputTokens: copyInt(result.InputTokens), OutputTokens: copyInt(result.OutputTokens)} + } + return out, nil +} + +func copyInt(n *int) *int { + if n == nil { + return nil + } + value := *n + return &value +} diff --git a/internal/service/gemini_test.go b/internal/service/gemini_test.go new file mode 100644 index 0000000..28c0ae4 --- /dev/null +++ b/internal/service/gemini_test.go @@ -0,0 +1,59 @@ +package service + +import ( + "context" + "errors" + "strings" + "testing" + + "github.com/bitbrew-dev/jevwise/internal/gemini" +) + +func TestGeminiAdapterSendsPromptAsInputAndReportsConfiguredModel(t *testing.T) { + tokens := 9 + result := gemini.Result{Choice: "B", Confidence: 0.6, Probabilities: map[string]float64{"A": 0.4, "B": 0.6}, InputTokens: &tokens} + svc, err := NewGemini(func(_ context.Context, r gemini.ChoiceRequest) (gemini.Result, error) { + if r.Input != " exact prompt\n" || r.Instructions != GeminiInstructions || strings.Join(r.Options, "|") != "A|B" { + t.Error("unexpected Gemini request") + } + return result, nil + }, "fixture-model") + if err != nil { + t.Fatal(err) + } + out, err := svc.Decide(context.Background(), Request{Prompt: " exact prompt\n", Options: []string{" A ", "B"}}) + if err != nil || out.Model != "fixture-model" || out.Choice != "B" || out.Confidence != 0.6 || out.Probabilities["A"] != 0.4 || *out.Usage.InputTokens != 9 || out.Usage.OutputTokens != nil { + t.Fatal("unexpected decision", out, err) + } + result.Probabilities["A"], tokens = 0, 1 + if out.Probabilities["A"] != 0.4 || *out.Usage.InputTokens != 9 { + t.Fatal("response shares client data") + } +} + +func TestGeminiAdapterValidationAndErrors(t *testing.T) { + if _, err := NewGemini(nil, "m"); err == nil { + t.Fatal("nil client accepted") + } + cause := errors.New("fixture failure") + calls := 0 + svc, _ := NewGemini(func(context.Context, gemini.ChoiceRequest) (gemini.Result, error) { + calls++ + return gemini.Result{}, cause + }, "m") + many := make([]string, gemini.MaxOptions+1) + for i := range many { + many[i] = strings.Repeat("x", i+1) + } + for _, r := range []Request{{Prompt: " ", Options: []string{"A", "B"}}, {Prompt: "p", Options: []string{"A", "A"}}, {Prompt: "p", Options: many}} { + if _, err := svc.Decide(context.Background(), r); err == nil || calls != 0 { + t.Fatal("invalid request reached client") + } + } + if _, err := svc.Decide(context.Background(), Request{Prompt: "p", Options: []string{"A", "B"}}); !errors.Is(err, cause) { + t.Fatal("client error lost", err) + } + if _, err := New("gemini", nil); err == nil { + t.Fatal("gemini selected without its client") + } +} diff --git a/internal/service/service.go b/internal/service/service.go index a02edff..f6cc0f2 100644 --- a/internal/service/service.go +++ b/internal/service/service.go @@ -70,6 +70,8 @@ func New(provider string, send SystemOneFunc) (DecisionService, error) { return nil, errors.New("jev requires a System One client") } return jev{send}, nil + case "gemini": + return nil, errors.New("gemini requires NewGemini") case "codex", "claude": return nil, fmt.Errorf("%s: %w", provider, ErrNotImplemented) default: diff --git a/internal/service/service_test.go b/internal/service/service_test.go index 1595731..0b5c0fd 100644 --- a/internal/service/service_test.go +++ b/internal/service/service_test.go @@ -51,13 +51,13 @@ func TestProviderSelectionNeverInvokesDeferredProviders(t *testing.T) { t.Fatal("placeholder or unknown provider invoked client") return nil, nil } - for _, provider := range []string{"codex", "claude", "unknown-secret"} { + for _, provider := range []string{"gemini", "codex", "claude", "unknown-secret"} { for _, callback := range []SystemOneFunc{nil, send} { svc, err := New(provider, callback) if svc != nil || err == nil || strings.Contains(err.Error(), "unknown-secret") { t.Fatalf("provider failure = %v, %v", svc, err) } - if provider != "unknown-secret" && !errors.Is(err, ErrNotImplemented) { + if provider != "unknown-secret" && provider != "gemini" && !errors.Is(err, ErrNotImplemented) { t.Fatalf("missing placeholder sentinel: %v", err) } } diff --git a/jevwise.toml b/jevwise.toml index ed2d791..10f7702 100644 --- a/jevwise.toml +++ b/jevwise.toml @@ -1,12 +1,16 @@ # Optional CLI configuration. Prefer environment credentials. +[general] +provider = "jev" +debug = false +editor = "vim" + +# Local MCP server: literal loopback IP (no scheme) and port. +[mcp] +host = "127.0.0.1" +port = 8080 + +[provider.jev] api_key = "$JEV_TOKEN" base_url = "https://api.typesafe.ai" model = "jev-latest" -provider = "jev" timeout = "10s" -editor = "vim" -# Local MCP server: literal loopback IP (no scheme) and port. -mcp_base_url = "127.0.0.1" -mcp_port = 8080 -# Structured debug logs on stderr; --debug or --verbose overrides. -debug = false