diff --git a/CHANGELOG.md b/CHANGELOG.md index 0a285371..b326a1d9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,5 @@ --- -latest_version: 3.4.11 +latest_version: 3.4.12 released: 2026-07-12 --- @@ -10,6 +10,16 @@ Format follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). > **Versioning:** CLI version is tracked in workspace `Cargo.toml`. v3.x is the Rust port of [v2.x (TypeScript/Bun)](https://github.com/onebrain-ai/onebrain). `v3.0.0-alpha.1` is the first user-facing alpha (binary artifacts published to GitHub Releases for 7 platforms). +## [3.4.12] — 2026-07-12 — Self-healing: run smooth under a running daemon + +Theme: no command, `serve`, or MCP call should error or fail to run because a daemon is (or isn't) holding the single-process redb lock. + +- `token gain` works under a running daemon: default/`--by`/`--history`/`--reset` read the lock-free JSONL raw log; `--all-time`/`--since` route through the daemon's `/api/token/gain` — even across a **version skew**, so it works right after an upgrade without restarting the daemon ([#258](https://github.com/onebrain-ai/onebrain-cli/issues/258)) +- A genuinely contended rollup open (or a daemon too old to serve the route) now reports the shared `E_ENGINE_BUSY` (exit 77) with an actionable hint, instead of a raw redb `Database already open` error at exit 1 ([#258](https://github.com/onebrain-ai/onebrain-cli/issues/258)) +- `serve` now **reuses or starts** a daemon (restarting a stale/version-mismatched one) instead of an engine-less foreground standalone — so the Token-Gain dashboard is populated rather than dark; the explicit `--port`/`--dir` standalone escape hatch now also opens its token cache ([#257](https://github.com/onebrain-ai/onebrain-cli/issues/257), [#258](https://github.com/onebrain-ai/onebrain-cli/issues/258)) +- `search vsearch` is daemon-routable: vector-only search routes through the daemon's new `/api/vault/search?mode=vec` instead of failing `E_ENGINE_BUSY` while an `onebrain mcp` session holds the engine ([#258](https://github.com/onebrain-ai/onebrain-cli/issues/258)) +- Fixes: `doctor`'s scoped-key lookup no longer panics on empty segments; remove dead test code; migrate a `chrono` `DateTime::from_timestamp` deprecation (Copilot autoreview) + ## [3.4.11] — 2026-07-12 — Token-opt seam fixes Closes the three real seams that post-ship verification of v3.4.10 surfaced — none was caught by the epic's gates because no test exercised those exact cross-component paths — plus polish. diff --git a/Cargo.lock b/Cargo.lock index 5cfdb7a4..bfb440ed 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -2065,7 +2065,7 @@ checksum = "384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe" [[package]] name = "onebrain-cache" -version = "3.4.11" +version = "3.4.12" dependencies = [ "chrono", "onebrain-core", @@ -2080,7 +2080,7 @@ dependencies = [ [[package]] name = "onebrain-cli" -version = "3.4.11" +version = "3.4.12" dependencies = [ "anyhow", "assert_cmd", @@ -2133,7 +2133,7 @@ dependencies = [ [[package]] name = "onebrain-core" -version = "3.4.11" +version = "3.4.12" dependencies = [ "chrono", "indexmap", @@ -2150,7 +2150,7 @@ dependencies = [ [[package]] name = "onebrain-fs" -version = "3.4.11" +version = "3.4.12" dependencies = [ "anyhow", "chrono", @@ -2182,7 +2182,7 @@ dependencies = [ [[package]] name = "onebrain-search" -version = "3.4.11" +version = "3.4.12" dependencies = [ "anyhow", "fastembed", @@ -2199,7 +2199,7 @@ dependencies = [ [[package]] name = "onebrain-token" -version = "3.4.11" +version = "3.4.12" dependencies = [ "chrono", "pretty_assertions", diff --git a/Cargo.toml b/Cargo.toml index f27a61b2..27692041 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -4,7 +4,7 @@ default-members = ["crates/*"] resolver = "2" [workspace.package] -version = "3.4.11" +version = "3.4.12" edition = "2021" license = "MIT OR Apache-2.0" authors = ["OneBrain Contributors"] diff --git a/crates/onebrain-cli/src/commands/daemon_client.rs b/crates/onebrain-cli/src/commands/daemon_client.rs index 4e8e4d78..ebeb6e11 100644 --- a/crates/onebrain-cli/src/commands/daemon_client.rs +++ b/crates/onebrain-cli/src/commands/daemon_client.rs @@ -1001,6 +1001,42 @@ pub fn discover_matching(expected_vault: Option<&Path>) -> Result, +) -> Result> { + let path = discovery_path()?; + let info = match DaemonInfo::read(&path) { + Ok(Some(info)) => info, + Ok(None) => return Ok(None), + Err(_) => { + let _ = DaemonInfo::remove(&path); + return Ok(None); + } + }; + // Vault match still gates adoption (never touch another vault's daemon), but + // version is deliberately IGNORED here. + let expected = vault_expectation(expected_vault); + if vault_decision(info.vault.as_deref(), &expected) == VaultDecision::Restart { + return Ok(None); + } + let handle = DaemonHandle::new(info); + Ok(handle.is_live().then_some(handle)) +} + /// Ensure a daemon is running at our version and return a connected handle. /// /// Fast path: [`discover`] returns an existing live daemon. Otherwise spawn @@ -1684,6 +1720,58 @@ mod tests { assert_eq!(still.vault, srv.info.vault, "record must be left intact"); } + /// #258 Gap 4: `discover_same_vault_any_version` adopts a same-vault LIVE + /// daemon even when its version differs from ours (the upgrade-skew case), + /// while `discover_matching` version-rejects the very same record. A + /// different vault is still declined (never disturb another vault's daemon). + #[cfg(unix)] + #[test] + fn discover_same_vault_any_version_adopts_across_version_skew() { + let vault = tempdir().unwrap(); + let cache = tempdir().unwrap(); + let home = tempdir().unwrap(); + let _env = crate::test_env::set_vars(&[ + ("ONEBRAIN_CACHE_DIR", cache.path().as_os_str()), + ("HOME", home.path().as_os_str()), + ]); + let mut srv = start_live_server(vault.path(), "dc-anyver-token-1234567890"); + // Pin a version DIFFERENT from ours (the post-upgrade skew). + srv.info.version = "0.0.1-old".to_string(); + let path = discovery_path().unwrap(); + srv.info.write(&path).unwrap(); + + // Version-gated discovery rejects the skewed daemon … + assert!( + discover_matching(Some(vault.path())).unwrap().is_none(), + "discover_matching must version-reject the skewed daemon" + ); + // … but the any-version variant adopts it (same vault, live). + assert!( + discover_same_vault_any_version(Some(vault.path())) + .unwrap() + .is_some(), + "any-version discovery must adopt a same-vault skewed daemon (Gap 4)" + ); + // A DIFFERENT vault is still declined. + let other = tempdir().unwrap(); + assert!( + discover_same_vault_any_version(Some(other.path())) + .unwrap() + .is_none(), + "must not adopt a different-vault daemon" + ); + } + + #[cfg(unix)] + #[test] + fn discover_same_vault_any_version_none_without_a_record() { + let home = tempdir().unwrap(); + let _env = crate::test_env::set_var("HOME", home.path()); + assert!(discover_same_vault_any_version(Some(home.path())) + .unwrap() + .is_none()); + } + /// HEADLINE acceptance (Track 2c): with a daemon holding the engine, a /// client can write a note, reindex it THROUGH the daemon, and then `get` / /// `status` see it — i.e. in-session indexing works and CLI-during-session diff --git a/crates/onebrain-cli/src/commands/mcp.rs b/crates/onebrain-cli/src/commands/mcp.rs index bb8b31d4..4b72371d 100644 --- a/crates/onebrain-cli/src/commands/mcp.rs +++ b/crates/onebrain-cli/src/commands/mcp.rs @@ -570,9 +570,11 @@ fn degrade_vec_error(has_lex: bool, result: anyhow::Result>) -> anyhow: } /// The daemon `mode` string for a typed sub-query. The daemon's -/// `GET /api/vault/search` speaks only `lex` (BM25) or `hybrid` (lex + vector), -/// so a `vec`/`hyde` sub-query maps to `hybrid` (which embeds the query and -/// fuses with lex inside the daemon), and `lex` maps to `lex`. +/// `GET /api/vault/search` speaks `lex` (BM25), `hybrid` (lex + vector), and — +/// since v3.4.12 — `vec` (vector-only). The MCP `query` tool deliberately still +/// maps a `vec`/`hyde` sub-query to `hybrid` (which embeds the query and fuses +/// with lex inside the daemon for RRF), and `lex` to `lex`; the standalone +/// `vec` mode is what the CLI `search vsearch` verb routes through, not this. fn daemon_mode_for(t: &SubQueryType) -> &'static str { match t { SubQueryType::Lex => "lex", diff --git a/crates/onebrain-cli/src/commands/search_query.rs b/crates/onebrain-cli/src/commands/search_query.rs index b072d0db..bc33994e 100644 --- a/crates/onebrain-cli/src/commands/search_query.rs +++ b/crates/onebrain-cli/src/commands/search_query.rs @@ -254,21 +254,52 @@ fn rerank_unreranked_hint( } } +/// Route `vsearch` through a warm daemon that holds this vault's engine, if any +/// (#258 Gap 3). Returns `Some(result)` when it routed, `None` to fall through +/// to the direct engine open. Semantic build only — daemon search routing +/// (`route_to_daemon`/`run_query_via_daemon`) is compiled there; the lex-only +/// build has no vec route, so this always returns `None`. +#[cfg(feature = "semantic")] +fn try_vsearch_via_daemon( + resolved: &onebrain_core::ResolvedVault, + args: &SearchQueryArgs, + mode: &OutputMode, +) -> Option> { + route_to_daemon(resolved) + .map(|handle| run_query_via_daemon(&handle, resolved, "vec", args, mode)) +} + +#[cfg(not(feature = "semantic"))] +fn try_vsearch_via_daemon( + _resolved: &onebrain_core::ResolvedVault, + _args: &SearchQueryArgs, + _mode: &OutputMode, +) -> Option> { + None +} + /// `onebrain search vsearch` — vector-only semantic search. Also embeds the /// query text (model-download point on first use). /// -/// NOT daemon-routable: the warm daemon's `/api/vault/search` exposes only -/// `lex` + `hybrid` (no vector-only mode), so vsearch always opens the engine -/// directly. When a daemon is holding the engine (an active `onebrain mcp` -/// session), this therefore degrades to T1's honest `E_ENGINE_BUSY` rather than -/// silently returning hybrid results under a different ranking — use `query` -/// (hybrid, daemon-routable) if you need results while a session is live. A -/// dedicated vec-only daemon endpoint is a follow-up (out of scope for 2c). +/// Daemon-routable since v3.4.12 (#258 Gap 3): when a warm daemon holds the +/// engine (an active `onebrain mcp` session), vsearch routes through its +/// `/api/vault/search?mode=vec` endpoint instead of degrading to +/// `E_ENGINE_BUSY`. Only when the daemon serves this exact vault; otherwise it +/// opens the engine directly as before (`$ONEBRAIN_NO_DAEMON` forces direct). pub fn run_vsearch( vault_flag: Option, mode: &OutputMode, args: &SearchQueryArgs, ) -> Result<()> { + // Warm-daemon path: route vector-only search through a daemon holding this + // vault's engine, so it works WITHOUT hitting redb's single-writer lock. + // Semantic-build only (daemon search routing is compiled there); the lex-only + // build has no vec route and falls straight to the direct open below. + let resolved = crate::vault_ctx::require(vault_flag.clone())?; + if let Some(result) = try_vsearch_via_daemon(&resolved, args, mode) { + return result; + } + let (mut engine, resolved) = open_engine(vault_flag)?; let raw_min = apply_rerank_overrides(&mut engine, args); let vault_info = crate::vault_ctx::info_from(&resolved); @@ -439,10 +470,10 @@ fn run_query_via_daemon( // interpretation either. rerank_unreranked_hint_data(&hits, rerank_enabled) }; - let command = if daemon_mode == "hybrid" { - "search.query" - } else { - "search.lex" + let command = match daemon_mode { + "hybrid" => "search.query", + "vec" => "search.vec", + _ => "search.lex", }; emit_hits_data( command, diff --git a/crates/onebrain-cli/src/commands/serve.rs b/crates/onebrain-cli/src/commands/serve.rs index b42533f5..8830967c 100644 --- a/crates/onebrain-cli/src/commands/serve.rs +++ b/crates/onebrain-cli/src/commands/serve.rs @@ -8,12 +8,16 @@ //! only difference is the shutdown trigger (Ctrl-C here, SIGTERM there). See //! the build-level design `2026-06-04-daemon-serve-design.md` §2–4. //! -//! **Daemon-aware since v3.4.8 (#197, design §2's "step 2b"):** when a daemon -//! is already serving THIS vault, `serve` does not bind a second listener (the -//! two share port 6789 by design, and both would want the engine) — it prints -//! the daemon's webui URL, honours `--open`, and exits. Explicit `--port` / -//! `--dir` flags (or a set `$ONEBRAIN_BIND`) always mean a standalone server -//! (see [`plan_serve`]). +//! **Daemon-aware since v3.4.8 (#197); reuse-or-start since v3.4.12 (#258):** +//! `serve` reuses a daemon already serving THIS vault, and — when none is +//! running — STARTS one (restarting a stale/version-mismatched one) via +//! [`daemon_client::ensure_running`], then prints that daemon's webui URL, +//! honours `--open`, and exits. It never binds a second listener next to a +//! daemon (the two share port 6789 by design, and both would want the engine), +//! and the started daemon holds the engine + token cache so the dashboard is +//! populated. Explicit `--port` / `--dir` flags (or a set `$ONEBRAIN_BIND`) — +//! and `$ONEBRAIN_NO_DAEMON` — always mean a foreground standalone server (see +//! [`plan_serve`]). //! //! **Localhost-only since v3.4.8 (#205):** the `--host` flag was REMOVED — //! every listener binds `127.0.0.1`, same as the daemon. Remote access goes @@ -83,16 +87,18 @@ enum ServePlan { /// A live daemon already serves this vault — print its webui URL (and /// `--open` it); do NOT bind a second listener or open a second engine. OpenDaemon { url: String }, - /// No matching daemon (or the user asked for a specific listener) — start - /// the foreground server exactly as before. + /// The user asked for a specific listener (`--port`/`--dir`/`$ONEBRAIN_BIND`), + /// the daemon kill switch is set, or starting a daemon failed — run the + /// foreground server directly. Standalone, } -/// Decide between routing to an existing daemon and standalone serving. +/// Decide between routing to a daemon and standalone serving. /// -/// `daemon` is the discovery record of a live, version- AND vault-matched -/// daemon (`daemon_client::discover_matching` already applied every guard — -/// a mismatch arrives here as `None`). `explicit_listener` is `true` when the +/// `daemon` is the record of the daemon `serve` reuses or just started for this +/// vault (`daemon_client::ensure_running` — `None` only when routing is disabled, +/// the user forced a listener, or the start failed). `explicit_listener` is +/// `true` when the /// user passed `--port` or `--dir`, or set `$ONEBRAIN_BIND`: they asked for a /// SPECIFIC standalone listener, so a daemon never hijacks that (the /// standalone bind will fail loudly on a port conflict rather than silently @@ -165,7 +171,10 @@ fn non_loopback_warning(host: &IpAddr) -> String { /// dashboard and the standalone escape hatch. fn build_daemon_banner(url: &str) -> String { let mut out = String::new(); - out.push_str(§ion("🌐", "Daemon already serving this vault")); + // "serving" (not "already serving") — `serve` may have just STARTED this + // daemon via ensure_running, so the banner must read correctly on both the + // reuse and the just-started paths. + out.push_str(§ion("🌐", "Daemon serving this vault")); out.push('\n'); out.push_str(&item("URL", url)); out.push('\n'); @@ -187,19 +196,33 @@ pub fn run(args: &ServeArgs, _mode: &OutputMode) -> Result<()> { let resolved = crate::vault_ctx::require(args.vault_dir.clone())?; let vault_root = resolved.root.as_path().to_path_buf(); - // Daemon-aware routing (#197): if a live daemon already serves THIS vault, - // don't bind a second listener (shared port, and two engine owners would - // collide on the redb lock) — hand the user the daemon's webui URL instead. - // PASSIVE discovery only (`discover_matching`): a version/vault mismatch or - // a dead record yields `None` and we serve standalone — `serve` never - // starts, stops, or restarts a daemon. `$ONEBRAIN_NO_DAEMON` (the CLI-wide - // routing kill switch) skips discovery entirely. Explicit listener flags — - // and a set `$ONEBRAIN_BIND` (a container operator NEEDS a reachable - // listener, not a redirect to a loopback-bound daemon) — skip it too. + // Daemon-aware routing (#197, self-healing since v3.4.12): `serve` reuses a + // running daemon for THIS vault, and — when none is running — STARTS one + // (or restarts a stale/version-mismatched one) via `ensure_running`, then + // hands the user that daemon's webui URL. This makes `serve` "always run: + // reuse-or-start", and the started daemon holds the engine + token cache, so + // the Token-Gain dashboard is populated (fixes #257 for the default path) + // instead of the old engine-less foreground standalone. + // + // Two engine owners would collide on redb's single-writer lock, so we never + // bind a second listener next to a daemon. If starting a daemon genuinely + // fails (e.g. the engine is still locked by an exiting process), we fall + // back to a standalone foreground server rather than erroring. + // + // `$ONEBRAIN_NO_DAEMON` (the CLI-wide routing kill switch) skips this + // entirely → standalone. Explicit listener flags — and a set + // `$ONEBRAIN_BIND` (a container operator NEEDS a reachable listener, not a + // redirect to a loopback-bound daemon) — skip it too. let bind_override = bind_env(); let explicit_listener = args.port.is_some() || args.dir.is_some() || bind_override.is_some(); let daemon_info = if wants_daemon_routing(explicit_listener) { - daemon_client::discover_matching(Some(&vault_root))?.map(|handle| handle.info().clone()) + match daemon_client::ensure_running(Some(&vault_root)) { + Ok(handle) => Some(handle.info().clone()), + Err(e) => { + tracing::warn!(error = %e, "could not reuse or start a daemon; serving standalone"); + None + } + } } else { None }; @@ -237,6 +260,12 @@ pub fn run(args: &ServeArgs, _mode: &OutputMode) -> Result<()> { // so it opens the search engine per-request as before rather than // holding it. Only the daemon (`daemon __run`) sets `hold_engine`. hold_engine: false, + // …but DO open the token cache so this standalone escape-hatch server + // (reached only via --port/--dir/$ONEBRAIN_BIND now that the default + // path starts a daemon) still populates the Token-Gain dashboard (#257) + // rather than 503-ing. Best-effort: a daemon-held token.redb → the + // routes degrade to "unavailable". + open_token_cache: true, }; // Jupyter-style URL — the token rides in the query string so a copy-paste @@ -639,7 +668,7 @@ mod tests { #[test] fn daemon_banner_carries_url_and_hint() { let b = build_daemon_banner("http://127.0.0.1:6789/?token=abc"); - assert!(b.contains("🌐 Daemon already serving this vault"), "{b:?}"); + assert!(b.contains("🌐 Daemon serving this vault"), "{b:?}"); assert!( b.contains(" URL http://127.0.0.1:6789/?token=abc"), "{b:?}" diff --git a/crates/onebrain-cli/src/commands/token_check.rs b/crates/onebrain-cli/src/commands/token_check.rs index af41a069..e6e058ef 100644 --- a/crates/onebrain-cli/src/commands/token_check.rs +++ b/crates/onebrain-cli/src/commands/token_check.rs @@ -611,9 +611,22 @@ mod tests { for stream in listener.incoming() { let Ok(mut stream) = stream else { continue }; std::thread::spawn(move || { - let mut buf = [0u8; 8192]; - let n = stream.read(&mut buf).unwrap_or(0); - let req = String::from_utf8_lossy(&buf[..n]); + // Accumulate until the end of the HTTP headers (`\r\n\r\n`) so + // a request split across TCP segments isn't misread from a + // single partial `read` (Copilot). + let mut req_bytes = Vec::with_capacity(8192); + let mut chunk = [0u8; 1024]; + while req_bytes.len() < 8192 { + let n = stream.read(&mut chunk).unwrap_or(0); + if n == 0 { + break; + } + req_bytes.extend_from_slice(&chunk[..n]); + if req_bytes.windows(4).any(|w| w == b"\r\n\r\n") { + break; + } + } + let req = String::from_utf8_lossy(&req_bytes); if req.starts_with("GET /api/health") { let _ = stream.write_all( b"HTTP/1.1 200 OK\r\ncontent-length: 0\r\nconnection: close\r\n\r\n", diff --git a/crates/onebrain-cli/src/commands/token_gain.rs b/crates/onebrain-cli/src/commands/token_gain.rs index d402f87b..209cc5cb 100644 --- a/crates/onebrain-cli/src/commands/token_gain.rs +++ b/crates/onebrain-cli/src/commands/token_gain.rs @@ -4,11 +4,11 @@ //! `--history` (raw JSONL tail), `--reset --label` epoch archiving, and //! `--rebuild` rollup recovery. //! -//! No live traffic writes `GainEvent`s yet in this PR — Track 4 wires the -//! MCP/CLI/daemon surfaces through `onebrain_token::run_funnel`. This -//! command is the reporting/administration surface over whatever -//! `token.redb` + the raw JSONL log already contain; against a fresh vault -//! every mode below reports zeroes, which is the correct, honest answer. +//! Live surfaces (`search get`, the MCP `get`/`query`, the read-hook) write +//! `GainEvent`s through `onebrain_token::run_funnel`; this command is the +//! reporting/administration surface over whatever `token.redb` + the raw JSONL +//! log already contain. Against a fresh vault every mode below reports zeroes, +//! which is the correct, honest answer. //! //! Renders exclusively through [`emit`] / [`Envelope`] — no hand-rolled //! printer (design §5d: every new command goes through the canonical @@ -21,8 +21,13 @@ use anyhow::{Context, Result}; use chrono::Datelike; use serde::{Deserialize, Serialize}; +use onebrain_core::path::ResolvedVault; + use crate::cli::TokenGainArgs; -use crate::commands::search_common::{collection_cache_dir, resolve_collection}; +use crate::commands::daemon_client::{self, DaemonHandle}; +use crate::commands::search_common::{ + collection_cache_dir, daemon_routing_disabled, resolve_collection, +}; use crate::output::{emit, Envelope, OutputMode}; use onebrain_token::gain::{pivot, rollup}; use onebrain_token::{ @@ -213,6 +218,109 @@ pub(crate) struct TokenGainData { archived_to: Option, } +/// The actionable, exit-77 error for a genuinely contended `token.redb`. Typed +/// as [`onebrain_core::CoreError::EngineBusy`] so it maps to `E_ENGINE_BUSY` / +/// exit 77 — the SAME transient-lock code `search query`/`vsearch`/`get` use +/// (`search_common::map_engine_open_error`), instead of a plain exit-1 generic +/// error that scripts can't tell apart from a real failure. +fn rollup_busy_error(detail: &str) -> anyhow::Error { + anyhow::Error::new(onebrain_core::CoreError::EngineBusy(detail.to_string())) +} + +const ROLLUP_LOCK_HINT: &str = + "token.redb is held by another onebrain process (a running daemon owns the rollup DB). \ + The default summary, --by, --history and --reset read the raw log directly and work \ + regardless; only --all-time, --since and --rebuild need the rollup DB — retry once that \ + process exits (e.g. `onebrain daemon stop`)."; + +/// Open the Tier-2 rollup DB (`token.redb`) directly, for the modes that +/// genuinely need it (`--rebuild`, and the Direct leg of `--all-time`/`--since` +/// when no daemon holds the lock). redb is single-process, so a running +/// `onebrain daemon` — the sole `token.redb` owner — makes this open fail with +/// `DatabaseAlreadyOpen`. When that's the cause we surface the actionable, +/// exit-77 [`rollup_busy_error`] (issue #258); any other open failure passes +/// through unchanged. +fn open_rollup_db_direct(tok_dir: &Path) -> Result { + std::fs::create_dir_all(tok_dir).context("creating token cache dir")?; + Database::create(redb_path(tok_dir)) + .context("opening token.redb") + .map_err(|e| { + if onebrain_search::error::is_redb_lock_error(&e) { + rollup_busy_error(ROLLUP_LOCK_HINT) + } else { + e + } + }) +} + +/// Resolve a rollup-backed pivot (`--all-time` / `--since`). The daemon is the +/// single owner of `token.redb`, so when a same-vault daemon holds it we route +/// the pivot through its `GET /api/token/gain` route rather than open the redb +/// ourselves — a Direct open would only hit the exclusive lock (issue #258). +/// +/// Routing adopts the daemon REGARDLESS of version +/// ([`daemon_client::discover_same_vault_any_version`]), because the gain route +/// returns the version-stable [`PivotResult`]: this closes the +/// upgrade-without-restart gap (#258 Gap 4) where a still-running old daemon +/// holds the lock. With no same-vault daemon there's no contention, so we open +/// Direct. `ONEBRAIN_NO_DAEMON` forces the Direct leg (operator kill-switch + +/// deterministic tests), matching every other search surface. +fn resolve_rollup_pivot( + resolved: &ResolvedVault, + tok_dir: &Path, + by: Option<&str>, + query: &PivotQuery, +) -> Result { + if !daemon_routing_disabled() { + if let Ok(Some(handle)) = + daemon_client::discover_same_vault_any_version(Some(resolved.root.as_path())) + { + // `None` = the daemon vanished mid-call (transport error) → it no + // longer holds the lock, so fall THROUGH to a now-safe Direct open + // (self-healing, #258 Gap 8). `Some` = a real pivot; `Err` = a + // too-old daemon that still holds the lock (actionable) or a decode + // failure. + if let Some(pivot) = daemon_rollup_pivot(&handle, by, query.since.as_deref())? { + return Ok(pivot); + } + } + } + let db = open_rollup_db_direct(tok_dir)?; + rollup::ensure_tables(&db).context("ensuring rollup tables")?; + pivot::query(&db, query).context("querying gain rollups") +} + +/// Fetch the rollup pivot from a same-vault daemon's `GET /api/token/gain`. +/// +/// - `Ok(Some(pivot))` — the route answered; decode the version-stable +/// [`PivotResult`] (a decode failure is a hard `Err`, never a silent Direct +/// fallback that could mask a real wire break). +/// - `Ok(None)` — a TRANSPORT error: the daemon died between discovery and this +/// call, so it no longer holds the lock; the caller falls through to a Direct +/// open (self-healing, #258 Gap 8). +/// - `Err(EngineBusy)` — a 404: the daemon is too old to serve the route but +/// still holds the lock, so Direct would fail too; surface the actionable +/// exit-77 error rather than the raw redb error. +fn daemon_rollup_pivot( + handle: &DaemonHandle, + by: Option<&str>, + since: Option<&str>, +) -> Result> { + match handle.token_gain(by, since, false) { + Ok(Some(json)) => serde_json::from_value::(json) + .map(Some) + .context("decoding the daemon's /api/token/gain pivot response"), + Ok(None) => Err(rollup_busy_error( + "the running onebrain daemon is too old to serve `token gain` (no \ + /api/token/gain route) and holds token.redb — restart it \ + (`onebrain daemon stop`) to pick up this version, then retry.", + )), + // Transport failure: the daemon is gone, so it released the lock. Signal + // a fall-through to Direct rather than erroring. + Err(_) => Ok(None), + } +} + pub fn run(vault_flag: Option, mode: &OutputMode, args: &TokenGainArgs) -> Result<()> { let (resolved, collection) = resolve_collection(vault_flag)?; let vault_info = crate::vault_ctx::info_from(&resolved); @@ -221,9 +329,11 @@ pub fn run(vault_flag: Option, mode: &OutputMode, args: &TokenGainArgs) let cache_dir = collection_cache_dir(&collection); let tok_dir = token_dir(&cache_dir); let gdir = gain_dir(&tok_dir); - std::fs::create_dir_all(&tok_dir).context("creating token cache dir")?; - let db = Database::create(redb_path(&tok_dir)).context("opening token.redb")?; - rollup::ensure_tables(&db).context("ensuring rollup tables")?; + // `token.redb` (the Tier-2 rollup) is opened LAZILY, only in the branches + // that need it (`--rebuild`, and the Direct leg of `--all-time`/`--since`). + // The default summary, `--by`, `--history` and `--reset` read the lock-free + // JSONL raw log (the source of truth), so they work even while a daemon holds + // the redb lock — the #258 fix. // `--json` is a local shorthand (mirrors `doctor --json` / `update // --json`) — it still renders through the SAME `emit`/`Envelope` @@ -235,6 +345,12 @@ pub fn run(vault_flag: Option, mode: &OutputMode, args: &TokenGainArgs) }; if args.rebuild { + // A rebuild WRITES the rollup tables, so it can't route to the daemon's + // read-only gain route — it needs exclusive redb access. Under a running + // daemon this surfaces the actionable "held by another process" message + // rather than the raw redb lock error. + let db = open_rollup_db_direct(&tok_dir)?; + rollup::ensure_tables(&db).context("ensuring rollup tables")?; let stats = rollup::rebuild(&gdir, &db).context("rebuilding rollups from the raw gain log")?; // The post-rebuild summary is the all-time cumulative rollup (every @@ -348,7 +464,9 @@ pub fn run(vault_flag: Option, mode: &OutputMode, args: &TokenGainArgs) .context("reading current-epoch gain log")?; pivot::query_events(&events, &query) } else { - pivot::query(&db, &query).context("querying gain rollups")? + // Rollup-backed (`--all-time` / `--since`): route to the daemon when it + // holds token.redb, else open Direct (the #258 fix). + resolve_rollup_pivot(&resolved, &tok_dir, args.by.as_deref(), &query)? }; // month/year buckets on a current-epoch report can't span the archived // epoch — flag it so the report never implies a cross-epoch bucket it @@ -846,4 +964,483 @@ mod tests { let dir = tempfile::tempdir().unwrap(); assert!(read_reset_marker(dir.path()).is_none()); } + + // ── #258: token gain under a daemon that holds token.redb ────────────── + // The daemon is the single owner of `token.redb` (redb is single-process). + // Before the fix, `token gain` opened the redb eagerly at the top of `run`, + // so EVERY mode hard-errored under a warm daemon. These prove: (a) the + // lock-free modes work while the lock is held, (b) the rollup modes route + // to the daemon, and (c) a genuine Direct lock reports an actionable message + // instead of the raw redb error. + + use onebrain_token::PivotRow; + #[cfg(unix)] + use std::io::{Read, Write}; + #[cfg(unix)] + use std::net::TcpListener; + + fn default_args() -> TokenGainArgs { + TokenGainArgs { + by: None, + all_time: false, + since: None, + history: false, + json: true, + reset: false, + label: None, + rebuild: false, + } + } + + /// A vault with a search collection + isolated cache dir — enough for + /// `resolve_collection` / `collection_cache_dir` to resolve `token.redb`. + fn gain_vault() -> (tempfile::TempDir, tempfile::TempDir) { + let vault = tempfile::tempdir().unwrap(); + std::fs::write( + vault.path().join("onebrain.yml"), + "search:\n collection: token-gain-seam\n", + ) + .unwrap(); + let cache = tempfile::tempdir().unwrap(); + (vault, cache) + } + + /// The token dir + `token.redb` path for the seam collection under the + /// currently-set `ONEBRAIN_CACHE_DIR` (call after the env is set). + fn seam_tok_dir() -> PathBuf { + token_dir(&collection_cache_dir("token-gain-seam")) + } + + /// Open + hold `token.redb`, standing in for the running daemon that owns + /// it. The returned handle keeps the exclusive lock until dropped. + fn hold_redb_lock(tok_dir: &Path) -> Database { + std::fs::create_dir_all(tok_dir).unwrap(); + Database::create(redb_path(tok_dir)).unwrap() + } + + /// Seed rollup rows into `token.redb`, then DROP the handle so its lock is + /// released before the code under test opens its own. + fn seed_rollup(tok_dir: &Path, evs: &[(i64, Surface, u64, u64)]) { + std::fs::create_dir_all(tok_dir).unwrap(); + let db = Database::create(redb_path(tok_dir)).unwrap(); + for (ts, surface, before, after) in evs { + rollup::update( + &db, + &GainEvent { + ts: *ts, + surface: *surface, + transform: "whitespace".to_string(), + level: OptLevel::Conservative, + bytes_before: *before, + bytes_after: *after, + cache: CacheKind::None, + session_token: None, + }, + ) + .unwrap(); + } + } + + #[test] + fn default_summary_succeeds_while_a_holder_locks_redb() { + let (vault, cache) = gain_vault(); + let _env = crate::test_env::set_vars(&[ + ("HOME", vault.path().as_os_str()), // no daemon.json here + ("ONEBRAIN_CACHE_DIR", cache.path().as_os_str()), + ("ONEBRAIN_NO_DAEMON", std::ffi::OsStr::new("1")), + ]); + let _held = hold_redb_lock(&seam_tok_dir()); + + // Pre-#258 this hard-errored ("Database already open. Cannot acquire + // lock."). Now the default summary reads the lock-free JSONL → Ok. + let out = run( + Some(vault.path().to_path_buf()), + &OutputMode::Json { pretty: false }, + &default_args(), + ); + assert!( + out.is_ok(), + "default `token gain` must work under a held redb lock: {out:?}" + ); + } + + #[test] + fn history_succeeds_while_a_holder_locks_redb() { + let (vault, cache) = gain_vault(); + let _env = crate::test_env::set_vars(&[ + ("HOME", vault.path().as_os_str()), + ("ONEBRAIN_CACHE_DIR", cache.path().as_os_str()), + ("ONEBRAIN_NO_DAEMON", std::ffi::OsStr::new("1")), + ]); + let _held = hold_redb_lock(&seam_tok_dir()); + + let args = TokenGainArgs { + history: true, + ..default_args() + }; + let out = run( + Some(vault.path().to_path_buf()), + &OutputMode::Json { pretty: false }, + &args, + ); + assert!( + out.is_ok(), + "`token gain --history` reads the raw log and must work under a lock: {out:?}" + ); + } + + #[test] + fn all_time_direct_under_a_lock_reports_actionable_message_not_raw_redb() { + let (vault, cache) = gain_vault(); + let _env = crate::test_env::set_vars(&[ + ("HOME", vault.path().as_os_str()), + ("ONEBRAIN_CACHE_DIR", cache.path().as_os_str()), + ("ONEBRAIN_NO_DAEMON", std::ffi::OsStr::new("1")), // force the Direct leg + ]); + let tok_dir = seam_tok_dir(); + let _held = hold_redb_lock(&tok_dir); + + let resolved = crate::vault_ctx::require(Some(vault.path().to_path_buf())).unwrap(); + let err = resolve_rollup_pivot(&resolved, &tok_dir, None, &PivotQuery::default()) + .expect_err("a Direct rollup open against a held lock must error"); + // Typed as CoreError::EngineBusy → E_ENGINE_BUSY / exit 77, the SAME + // transient-lock code every sibling search surface uses (not a raw redb + // error at generic exit 1). + let core = err + .downcast_ref::() + .expect("the lock error must be a typed CoreError, not a plain anyhow error"); + assert_eq!(core.error_code(), "E_ENGINE_BUSY", "must map to exit 77"); + let msg = format!("{err:#}"); + assert!( + msg.contains("token.redb is held by another onebrain process"), + "actionable hint present: {msg}" + ); + assert!(msg.contains("onebrain daemon stop"), "{msg}"); + } + + #[test] + fn all_time_direct_returns_the_seeded_rollup_when_no_daemon() { + let (vault, cache) = gain_vault(); + let _env = crate::test_env::set_vars(&[ + ("HOME", vault.path().as_os_str()), // no daemon.json → Direct + ("ONEBRAIN_CACHE_DIR", cache.path().as_os_str()), + ("ONEBRAIN_NO_DAEMON", std::ffi::OsStr::new("1")), + ]); + let tok_dir = seam_tok_dir(); + seed_rollup( + &tok_dir, + &[ + (1_783_728_000, Surface::CliSearch, 1000, 400), + (1_783_900_800, Surface::McpQuery, 500, 100), + ], + ); + + let resolved = crate::vault_ctx::require(Some(vault.path().to_path_buf())).unwrap(); + let pivot = + resolve_rollup_pivot(&resolved, &tok_dir, None, &PivotQuery::default()).unwrap(); + assert_eq!(pivot.totals.count, 2); + assert_eq!(pivot.totals.bytes_before, 1500); + assert_eq!(pivot.totals.bytes_after, 500); + } + + // ── daemon-routing (Complete scope): --all-time/--since via the route ── + + #[cfg(unix)] + fn write_daemon_json(home: &Path, vault: &Path, port: u16, token: &str) { + write_daemon_json_versioned(home, vault, port, token, env!("CARGO_PKG_VERSION")); + } + + /// Like [`write_daemon_json`] but pins the recorded daemon `version` — the + /// Gap 4 test needs a version DIFFERENT from ours to prove the routed read + /// adopts a version-skewed same-vault daemon. + #[cfg(unix)] + fn write_daemon_json_versioned( + home: &Path, + vault: &Path, + port: u16, + token: &str, + version: &str, + ) { + let run_dir = home.join(".onebrain").join("run"); + std::fs::create_dir_all(&run_dir).unwrap(); + let info = daemon_client::DaemonInfo { + port, + token: token.to_string(), + pid: std::process::id(), + version: version.to_string(), + vault: daemon_client::canonical_vault_id(vault), + }; + std::fs::write( + run_dir.join("daemon.json"), + serde_json::to_vec_pretty(&info).unwrap(), + ) + .unwrap(); + } + + type CapturedReqs = std::sync::Arc>>; + + /// A minimal HTTP/1.1 responder for the two routes `resolve_rollup_pivot` + /// exercises: `GET /api/health` (so daemon discovery adopts it) and `GET + /// /api/token/gain`. It records each non-health request's first line (so a + /// test can assert `by=`/`since=` forwarding) and serves `gain_body` with + /// `gain_status` — EXCEPT `gain_status == 0`, which drops the connection + /// with no response, simulating a daemon that died mid-call (a transport + /// error → Gap 8 Direct-fallback). + #[cfg(unix)] + fn start_fake_gain_daemon(gain_status: u16, gain_body: String) -> (u16, CapturedReqs) { + let listener = TcpListener::bind("127.0.0.1:0").unwrap(); + let port = listener.local_addr().unwrap().port(); + let reqs: CapturedReqs = std::sync::Arc::new(std::sync::Mutex::new(Vec::new())); + let reqs_bg = reqs.clone(); + std::thread::spawn(move || { + for stream in listener.incoming() { + let Ok(mut stream) = stream else { continue }; + let body = gain_body.clone(); + let reqs = reqs_bg.clone(); + std::thread::spawn(move || { + // Accumulate until the end of the HTTP headers (`\r\n\r\n`) so + // a request split across TCP segments isn't misread from a + // single partial `read` (Copilot). + let mut req_bytes = Vec::with_capacity(8192); + let mut chunk = [0u8; 1024]; + while req_bytes.len() < 8192 { + let n = stream.read(&mut chunk).unwrap_or(0); + if n == 0 { + break; + } + req_bytes.extend_from_slice(&chunk[..n]); + if req_bytes.windows(4).any(|w| w == b"\r\n\r\n") { + break; + } + } + let req = String::from_utf8_lossy(&req_bytes).to_string(); + if req.starts_with("GET /api/health") { + let _ = stream.write_all( + b"HTTP/1.1 200 OK\r\ncontent-length: 0\r\nconnection: close\r\n\r\n", + ); + return; + } + reqs.lock() + .unwrap() + .push(req.lines().next().unwrap_or("").to_string()); + if gain_status == 0 { + // Drop the connection with no response → transport error. + return; + } + let status_line = match gain_status { + 200 => "HTTP/1.1 200 OK", + 404 => "HTTP/1.1 404 Not Found", + _ => "HTTP/1.1 500 Internal Server Error", + }; + let resp = format!( + "{status_line}\r\ncontent-type: application/json\r\ncontent-length: {}\r\nconnection: close\r\n\r\n{}", + body.len(), + body + ); + let _ = stream.write_all(resp.as_bytes()); + }); + } + }); + (port, reqs) + } + + /// A single-surface pivot body for the fake daemon (`count`/bytes on one + /// `cli_search` row). + #[cfg(unix)] + fn pivot_body(count: u64) -> String { + serde_json::to_string(&PivotResult { + rows: vec![PivotRow { + time: None, + dim: Some("cli_search".to_string()), + totals: PivotTotals { + bytes_before: 2000, + bytes_after: 500, + count, + }, + }], + totals: PivotTotals { + bytes_before: 2000, + bytes_after: 500, + count, + }, + }) + .unwrap() + } + + #[cfg(unix)] + #[test] + fn all_time_routes_through_the_daemon_that_holds_the_lock() { + let (vault, cache) = gain_vault(); + let (port, _reqs) = start_fake_gain_daemon(200, pivot_body(3)); + let _env = crate::test_env::set_vars(&[ + ("HOME", vault.path().as_os_str()), + ("ONEBRAIN_CACHE_DIR", cache.path().as_os_str()), + ]); + write_daemon_json(vault.path(), vault.path(), port, "gain-daemon-token-123"); + + // Hold the LOCAL redb locked: a Direct open would return the wrong data + // (empty) or error — so this test can pass ONLY by routing to the daemon. + let tok_dir = seam_tok_dir(); + let _held = hold_redb_lock(&tok_dir); + + let resolved = crate::vault_ctx::require(Some(vault.path().to_path_buf())).unwrap(); + let pivot = + resolve_rollup_pivot(&resolved, &tok_dir, None, &PivotQuery::default()).unwrap(); + assert_eq!( + pivot.totals.count, 3, + "must reflect the daemon's rollup, not the locked local db" + ); + assert_eq!(pivot.rows.len(), 1); + assert_eq!(pivot.rows[0].dim.as_deref(), Some("cli_search")); + } + + /// #258 Gap 4: after a CLI upgrade the still-running OLD daemon holds + /// token.redb. `discover_matching` would version-reject it → Direct-open → + /// lock error. Routing must adopt the version-skewed same-vault daemon + /// anyway (its gain route + PivotResult are version-stable). The local redb + /// is held locked so the ONLY way to succeed is routing. + #[cfg(unix)] + #[test] + fn all_time_routes_to_a_version_mismatched_same_vault_daemon() { + let (vault, cache) = gain_vault(); + let (port, _reqs) = start_fake_gain_daemon(200, pivot_body(7)); + let _env = crate::test_env::set_vars(&[ + ("HOME", vault.path().as_os_str()), + ("ONEBRAIN_CACHE_DIR", cache.path().as_os_str()), + ]); + // A daemon at a DIFFERENT version than ours (the upgrade-skew case). + write_daemon_json_versioned(vault.path(), vault.path(), port, "tok", "0.0.1-old"); + let tok_dir = seam_tok_dir(); + let _held = hold_redb_lock(&tok_dir); + + let resolved = crate::vault_ctx::require(Some(vault.path().to_path_buf())).unwrap(); + let pivot = + resolve_rollup_pivot(&resolved, &tok_dir, None, &PivotQuery::default()).unwrap(); + assert_eq!( + pivot.totals.count, 7, + "must route to the version-skewed same-vault daemon (Gap 4), not Direct-open the locked db" + ); + } + + /// #258 Gap 8: the daemon dies between discovery and the gain call → a + /// transport error. It no longer holds the lock, so the read must fall + /// through to a now-safe Direct open (self-healing), NOT surface an error. + #[cfg(unix)] + #[test] + fn daemon_gone_mid_call_falls_back_to_direct() { + let (vault, cache) = gain_vault(); + let (port, _reqs) = start_fake_gain_daemon(0, String::new()); // drops the gain call + let _env = crate::test_env::set_vars(&[ + ("HOME", vault.path().as_os_str()), + ("ONEBRAIN_CACHE_DIR", cache.path().as_os_str()), + ]); + write_daemon_json(vault.path(), vault.path(), port, "tok"); + let tok_dir = seam_tok_dir(); + // No lock held (the daemon "died"); seed the local rollup Direct will read. + seed_rollup(&tok_dir, &[(1_783_728_000, Surface::CliSearch, 1000, 400)]); + + let resolved = crate::vault_ctx::require(Some(vault.path().to_path_buf())).unwrap(); + let pivot = + resolve_rollup_pivot(&resolved, &tok_dir, None, &PivotQuery::default()).unwrap(); + assert_eq!( + pivot.totals.count, 1, + "a transport error must fall back to a now-unlocked Direct open (Gap 8)" + ); + } + + /// R3: prove `by`/`since` are actually forwarded onto the daemon route URL + /// (the fake daemon records the request line). + #[cfg(unix)] + #[test] + fn by_and_since_are_forwarded_to_the_daemon_route() { + let (vault, cache) = gain_vault(); + let empty = serde_json::to_string(&PivotResult { + rows: vec![], + totals: PivotTotals::default(), + }) + .unwrap(); + let (port, reqs) = start_fake_gain_daemon(200, empty); + let _env = crate::test_env::set_vars(&[ + ("HOME", vault.path().as_os_str()), + ("ONEBRAIN_CACHE_DIR", cache.path().as_os_str()), + ]); + write_daemon_json(vault.path(), vault.path(), port, "tok"); + let tok_dir = seam_tok_dir(); + let _held = hold_redb_lock(&tok_dir); + + let resolved = crate::vault_ctx::require(Some(vault.path().to_path_buf())).unwrap(); + let query = PivotQuery { + time: None, + dim: None, + since: Some("2026-01-01".to_string()), + }; + let _ = resolve_rollup_pivot(&resolved, &tok_dir, Some("surface"), &query).unwrap(); + + let captured = reqs.lock().unwrap().clone(); + assert!( + captured.iter().any(|r| r.contains("since=2026-01-01")), + "since must be forwarded: {captured:?}" + ); + assert!( + captured.iter().any(|r| r.contains("by=surface")), + "by must be forwarded: {captured:?}" + ); + } + + #[cfg(unix)] + #[test] + fn daemon_too_old_for_the_gain_route_is_actionable() { + let (vault, cache) = gain_vault(); + let (port, _reqs) = start_fake_gain_daemon(404, "not found".to_string()); + let _env = crate::test_env::set_vars(&[ + ("HOME", vault.path().as_os_str()), + ("ONEBRAIN_CACHE_DIR", cache.path().as_os_str()), + ]); + write_daemon_json(vault.path(), vault.path(), port, "gain-daemon-token-123"); + let tok_dir = seam_tok_dir(); + let _held = hold_redb_lock(&tok_dir); + + let resolved = crate::vault_ctx::require(Some(vault.path().to_path_buf())).unwrap(); + let err = resolve_rollup_pivot(&resolved, &tok_dir, None, &PivotQuery::default()) + .expect_err("a 404 gain route while the daemon holds the lock must error"); + // Typed EngineBusy (exit 77), actionable, and points at the fix. + assert_eq!( + err.downcast_ref::() + .expect("typed CoreError") + .error_code(), + "E_ENGINE_BUSY" + ); + let msg = format!("{err:#}"); + assert!(msg.contains("too old to serve"), "{msg}"); + assert!(msg.contains("onebrain daemon stop"), "{msg}"); + } + + /// R3: `--rebuild` is a redb WRITE that can't route to the read-only daemon + /// route — under a held lock it must surface the actionable exit-77 error. + #[test] + fn rebuild_under_a_lock_reports_engine_busy() { + let (vault, cache) = gain_vault(); + let _env = crate::test_env::set_vars(&[ + ("HOME", vault.path().as_os_str()), + ("ONEBRAIN_CACHE_DIR", cache.path().as_os_str()), + ("ONEBRAIN_NO_DAEMON", std::ffi::OsStr::new("1")), + ]); + let _held = hold_redb_lock(&seam_tok_dir()); + let args = TokenGainArgs { + rebuild: true, + ..default_args() + }; + let err = run( + Some(vault.path().to_path_buf()), + &OutputMode::Json { pretty: false }, + &args, + ) + .expect_err("rebuild needs exclusive redb; under a lock it must error"); + assert_eq!( + err.downcast_ref::() + .expect("typed CoreError") + .error_code(), + "E_ENGINE_BUSY" + ); + } } diff --git a/crates/onebrain-cli/src/server/mod.rs b/crates/onebrain-cli/src/server/mod.rs index 8db345a6..bc12e621 100644 --- a/crates/onebrain-cli/src/server/mod.rs +++ b/crates/onebrain-cli/src/server/mod.rs @@ -99,6 +99,15 @@ pub struct ServeConfig { /// before), since a foreground `serve` is short-lived and not the canonical /// engine owner. pub hold_engine: bool, + /// When `true`, open the `token.redb` cache at router-build time even + /// without holding the engine — so a standalone `serve` (the explicit + /// `--port`/`--dir` escape hatch) can serve `/api/token/*` and light up the + /// Token-Gain dashboard (#257) instead of 503-ing. `hold_engine` already + /// implies opening the token cache (the daemon), so this only matters for + /// the engine-less standalone path. Best-effort: if `token.redb` is held by + /// another process the open fails and the routes degrade to "unavailable", + /// never a second racing opener. The unit-test router leaves it `false`. + pub open_token_cache: bool, } impl ServeConfig { @@ -118,6 +127,7 @@ impl ServeConfig { port, token, hold_engine: false, + open_token_cache: false, } } @@ -260,7 +270,7 @@ pub fn build_router_with_state(cfg: ServeConfig) -> (Router, Arc) { // guard so only the warm daemon owns `token.redb`. A failure to open // (never-indexed vault, no collection) leaves it `None` and the token // routes degrade gracefully — they never fall back to a per-request open. - let token_cache = if cfg.hold_engine { + let token_cache = if cfg.hold_engine || cfg.open_token_cache { cfg.vault_root .as_deref() .and_then(token_api::open_held_token_cache) diff --git a/crates/onebrain-cli/src/server/search.rs b/crates/onebrain-cli/src/server/search.rs index 5370b8e4..788415d3 100644 --- a/crates/onebrain-cli/src/server/search.rs +++ b/crates/onebrain-cli/src/server/search.rs @@ -2,14 +2,16 @@ //! `onebrain-search` engine (the SAME index the CLI `onebrain search …` //! verbs and the native MCP server use). //! -//! Two modes: +//! Three modes: //! //! ```text //! mode=lex → LexIndex (BM25 keyword, no model, fast as-you-type) //! mode=hybrid → Engine::query (lex + vector, one query embedding) -//! (lex-only / --no-default-features builds have no -//! embedder, so hybrid degrades to the same LexIndex path -//! as mode=lex — see `run_hybrid` below) +//! mode=vec → Engine::vector_search (vector-only, one query embedding) — +//! routes `onebrain search vsearch` through a warm daemon (#258) +//! (lex-only / --no-default-features builds have no embedder, so +//! hybrid degrades to the same LexIndex path as mode=lex — see +//! `run_hybrid` below — and vec errors, having no lex analogue) //! ``` //! //! Read-only translator: it returns vault-relative paths the existing @@ -45,7 +47,8 @@ use onebrain_search::lex::LexIndex; pub(crate) struct SearchQuery { /// The user's search text. q: String, - /// `lex` (BM25 keyword, the default) or `hybrid` (keyword + semantic). + /// `lex` (BM25 keyword, the default), `hybrid` (keyword + semantic), or + /// `vec` (vector-only — the `search vsearch` daemon route, #258 Gap 3). #[serde(default)] mode: Option, /// Max hits to return. When absent, falls back to the vault's @@ -122,6 +125,7 @@ pub(crate) async fn get_vault_search( let query = q.q.trim().to_string(); let mode: &'static str = match q.mode.as_deref() { Some("hybrid") => "hybrid", + Some("vec") => "vec", _ => "lex", }; @@ -212,10 +216,13 @@ fn run_search( } // lex never opens redb (tantivy only), so it stays on the standalone index - // even with a held engine — no lock, no contention. Only hybrid reuses the - // held (sole redb-owning) engine. + // even with a held engine — no lock, no contention. hybrid AND vec reuse the + // held (sole redb-owning) engine so the CLI/mcp can search vector-only + // through the daemon instead of hitting `E_ENGINE_BUSY` (#258 Gap 3). if mode == "hybrid" { run_hybrid_held(engine, &cache_dir, root, query, top_k, min_candidates) + } else if mode == "vec" { + run_vec_held(engine, &cache_dir, root, query, top_k, min_candidates) } else { run_lex(&cache_dir, query, top_k) } @@ -278,7 +285,111 @@ fn run_hybrid_held( run_lex(cache_dir, query, top_k) } -/// Synchronous native search. `mode` is `"hybrid"` or `"lex"`. +/// Vector-only search against the daemon's held engine (#258 Gap 3). Mirrors +/// [`run_hybrid_held`] but ranks purely by embedding cosine (+ optional rerank) +/// via `Engine::vector_search`, so `onebrain search vsearch` can route through a +/// warm daemon instead of failing with `E_ENGINE_BUSY` while an mcp session +/// holds the engine. +#[cfg(feature = "semantic")] +fn run_vec_held( + engine: &SharedEngine, + _cache_dir: &Path, + root: &Path, + query: &str, + top_k: usize, + min_candidates: Option, +) -> anyhow::Result> { + let mut engine = engine.lock().unwrap_or_else(|p| p.into_inner()); + if let Some(min_candidates) = min_candidates { + engine.set_rerank_min_candidates(min_candidates); + } + if engine.status(root)?.doc_count == 0 { + return Ok(vec![]); + } + Ok(engine + .vector_search(query, top_k)? + .into_iter() + .filter(|h| !h.doc_path.is_empty()) + .map(|h| SearchHit { + title: if h.heading_path.is_empty() { + title_from_path(&h.doc_path) + } else { + h.heading_path.clone() + }, + path: h.doc_path, + score: h.score, + snippet: h.snippet, + rerank_score: h.rerank_score, + }) + .collect()) +} + +/// Lex-only build: no embedder, so a vector-only search can't run and (unlike +/// hybrid) has no lex analogue to degrade to — surface a clear error rather than +/// silently returning keyword results under a `vec` label. +#[cfg(not(feature = "semantic"))] +fn run_vec_held( + _engine: &SharedEngine, + _cache_dir: &Path, + _root: &Path, + _query: &str, + _top_k: usize, + _min_candidates: Option, +) -> anyhow::Result> { + anyhow::bail!("vector-only search is unavailable in this lex-only build") +} + +/// Vector-only per-request search (standalone `serve`, no held engine) — the +/// #258 Gap 3 analogue of [`run_hybrid`]. Same empty-index guard + engine setup +/// (exclude + rerank), ranking via `Engine::vector_search`. +#[cfg(feature = "semantic")] +fn run_vec( + cache_dir: &Path, + root: &Path, + query: &str, + top_k: usize, + min_candidates: Option, +) -> anyhow::Result> { + let config = onebrain_core::load_vault_config_at(root)?; + let mut engine = Engine::open(cache_dir, &config.search.embed_model)?; + engine.set_exclude_patterns(config.search.exclude.clone()); + engine.set_rerank_settings(rerank_settings_from_config(&config.search.reranker)); + if let Some(min_candidates) = min_candidates { + engine.set_rerank_min_candidates(min_candidates); + } + if engine.status(root)?.doc_count == 0 { + return Ok(vec![]); + } + Ok(engine + .vector_search(query, top_k)? + .into_iter() + .filter(|h| !h.doc_path.is_empty()) + .map(|h| SearchHit { + title: if h.heading_path.is_empty() { + title_from_path(&h.doc_path) + } else { + h.heading_path.clone() + }, + path: h.doc_path, + score: h.score, + snippet: h.snippet, + rerank_score: h.rerank_score, + }) + .collect()) +} + +#[cfg(not(feature = "semantic"))] +fn run_vec( + _cache_dir: &Path, + _root: &Path, + _query: &str, + _top_k: usize, + _min_candidates: Option, +) -> anyhow::Result> { + anyhow::bail!("vector-only search is unavailable in this lex-only build") +} + +/// Synchronous native search. `mode` is `"hybrid"`, `"vec"`, or `"lex"`. fn run_native( root: &Path, query: &str, @@ -298,6 +409,8 @@ fn run_native( if mode == "hybrid" { run_hybrid(&cache_dir, root, query, top_k, min_candidates) + } else if mode == "vec" { + run_vec(&cache_dir, root, query, top_k, min_candidates) } else { run_lex(&cache_dir, query, top_k) } @@ -441,8 +554,37 @@ mod tests { let _env = crate::test_env::set_var("ONEBRAIN_CACHE_DIR", cache.path()); let result_lex = run_native(dir.path(), "anything", "lex", TOP_K, None); let result_hybrid = run_native(dir.path(), "anything", "hybrid", TOP_K, None); + let result_vec = run_native(dir.path(), "anything", "vec", TOP_K, None); assert!(result_lex.unwrap().is_empty()); assert!(result_hybrid.unwrap().is_empty()); + assert!( + result_vec.unwrap().is_empty(), + "vec on a no-index vault → empty" + ); + } + + /// #258 Gap 3: `run_native` (per-request — the standalone `serve` path) + /// dispatches `mode=vec` to `run_vec` and short-circuits on an EXISTING but + /// empty index, download-free. Covers the per-request vec body (the no-index + /// case above returns at the guard before reaching `run_vec`). + #[cfg(feature = "semantic")] + #[test] + fn run_native_vec_empty_index_is_download_free_empty() { + let vault = tempfile::tempdir().unwrap(); + std::fs::write( + vault.path().join("onebrain.yml"), + "search:\n collection: rn-vec-empty\n", + ) + .unwrap(); + let cache = tempfile::tempdir().unwrap(); + let _env = crate::test_env::set_var("ONEBRAIN_CACHE_DIR", cache.path()); + let cache_dir = collection_cache_dir(&collection_name_readonly(vault.path()).unwrap()); + { + let mut lex = LexIndex::open(&index_artifact_path(&cache_dir, "tantivy")).unwrap(); + lex.commit().unwrap(); + } + let hits = run_native(vault.path(), "anything", "vec", TOP_K, None).unwrap(); + assert!(hits.is_empty(), "empty index → no vec hits, no download"); } #[test] @@ -853,6 +995,33 @@ mod tests { ); } + /// #258 Gap 3: `mode="vec"` reaches `run_vec_held` (not the lex fallback) and + /// short-circuits on an empty index — download-free, so the daemon can serve + /// vector-only search for a CLI `vsearch` during an mcp session. + #[cfg(feature = "semantic")] + #[test] + fn run_search_vec_held_empty_index_is_download_free_empty() { + let vault = tempfile::tempdir().unwrap(); + std::fs::write( + vault.path().join("onebrain.yml"), + "search:\n collection: rs-vec-empty\n", + ) + .unwrap(); + let cache = tempfile::tempdir().unwrap(); + let _env = crate::test_env::set_var("ONEBRAIN_CACHE_DIR", cache.path()); + let cache_dir = collection_cache_dir(&collection_name_readonly(vault.path()).unwrap()); + { + let mut lex = LexIndex::open(&index_artifact_path(&cache_dir, "tantivy")).unwrap(); + lex.commit().unwrap(); + } + let engine = crate::server::internal::open_held_engine(vault.path()).unwrap(); + let hits = run_search(Some(&engine), vault.path(), "anything", "vec", TOP_K, None).unwrap(); + assert!( + hits.is_empty(), + "empty index must yield no vec hits, no download" + ); + } + #[test] fn map_search_failure_engine_busy_is_503() { // A per-request `Engine::open` that hits the redb lock surfaces as the diff --git a/crates/onebrain-cli/src/server/token_api.rs b/crates/onebrain-cli/src/server/token_api.rs index 32e2086d..30950141 100644 --- a/crates/onebrain-cli/src/server/token_api.rs +++ b/crates/onebrain-cli/src/server/token_api.rs @@ -612,6 +612,33 @@ mod tests { assert_eq!(resp.status(), StatusCode::SERVICE_UNAVAILABLE); } + /// #257: a standalone `serve` (`open_token_cache = true`, no held engine) + /// opens the token cache, so the Gain route answers 200 with an honest empty + /// pivot instead of 503 — the dashboard lights up on the `--port` escape + /// hatch, not just under the daemon. + #[tokio::test] + async fn gain_route_ok_with_open_token_cache_and_no_held_engine() { + let (vault, cache) = vault_and_cache(); + let _env = crate::test_env::set_var("ONEBRAIN_CACHE_DIR", cache.path()); + let mut cfg = + ServeConfig::localhost(Some(vault.path().to_path_buf()), 0, TOKEN.to_string(), None); + cfg.open_token_cache = true; // standalone serve, engine NOT held + let router = build_router(cfg); + let resp = router + .oneshot( + Request::get("/api/token/gain?by=surface") + .header("x-onebrain-token", TOKEN) + .body(Body::empty()) + .unwrap(), + ) + .await + .unwrap(); + assert_eq!(resp.status(), StatusCode::OK); + let bytes = resp.into_body().collect().await.unwrap().to_bytes(); + let v: serde_json::Value = serde_json::from_slice(&bytes).unwrap(); + assert_eq!(v["rows"], serde_json::json!([]), "no data → empty pivot"); + } + #[tokio::test] async fn ledger_check_empty_path_400() { let (vault, cache) = vault_and_cache(); diff --git a/crates/onebrain-cli/tests/doctor_integration.rs b/crates/onebrain-cli/tests/doctor_integration.rs index 8ff56279..5bc2176e 100644 --- a/crates/onebrain-cli/tests/doctor_integration.rs +++ b/crates/onebrain-cli/tests/doctor_integration.rs @@ -1975,8 +1975,8 @@ fn doctor_fix_text_mode_partial_outcome_renders_distinct_glyph() { /// sharing a leaf name across different blocks (e.g. `search.reranker.model` /// / `token_optimization.model`). fn find_scoped_key_line(lines: &[&str], segments: &[&str]) -> Option { - let top = segments[0]; - let key = segments.last().unwrap(); + let top = *segments.first()?; + let key = segments.last()?; let key_prefix = format!("{key}:"); let mut current_top: Option<&str> = None; for (i, l) in lines.iter().enumerate() { diff --git a/crates/onebrain-fs/src/init/mod.rs b/crates/onebrain-fs/src/init/mod.rs index 8892ec87..dfa70d5b 100644 --- a/crates/onebrain-fs/src/init/mod.rs +++ b/crates/onebrain-fs/src/init/mod.rs @@ -510,8 +510,6 @@ pub use wizard::{ #[cfg(test)] mod tests { use super::*; - use std::cell::RefCell; - use std::rc::Rc; use std::sync::{Arc, Mutex}; use tempfile::tempdir; @@ -710,8 +708,6 @@ mod tests { fn confirm_fn_directory_no_aborts_before_creating_anything() { let d = tempdir().unwrap(); let (mut opts, _stdout_buf) = test_opts(d.path()); - // Counter to track which question we're on - let count = Rc::new(RefCell::new(0_u32)); // FnMut closures with shared state — can't use Rc/RefCell with Send, // so use a plain Mutex + Box::new + move. let counter = Arc::new(Mutex::new(0_u32)); @@ -721,7 +717,6 @@ mod tests { *c += 1; false // always decline })); - let _ = count; // silence unused let r = run_init(opts).unwrap(); assert!(r.ok); diff --git a/crates/onebrain-token/src/gain/rollup.rs b/crates/onebrain-token/src/gain/rollup.rs index e4d42434..a3e11c5c 100644 --- a/crates/onebrain-token/src/gain/rollup.rs +++ b/crates/onebrain-token/src/gain/rollup.rs @@ -72,17 +72,17 @@ impl RollupValue { /// segment. `pub(crate)` so [`super::pivot::query_events`] buckets raw /// events on the exact same day boundary the rollup does. pub(crate) fn day_key(ts: i64) -> String { - let dt = DateTime::::from_timestamp(ts, 0).unwrap_or_else(Utc::now); + let dt = DateTime::from_timestamp(ts, 0).unwrap_or_else(Utc::now); format!("{:04}-{:02}-{:02}", dt.year(), dt.month(), dt.day()) } fn month_key(ts: i64) -> String { - let dt = DateTime::::from_timestamp(ts, 0).unwrap_or_else(Utc::now); + let dt = DateTime::from_timestamp(ts, 0).unwrap_or_else(Utc::now); format!("{:04}-{:02}", dt.year(), dt.month()) } fn year_key(ts: i64) -> String { - let dt = DateTime::::from_timestamp(ts, 0).unwrap_or_else(Utc::now); + let dt = DateTime::from_timestamp(ts, 0).unwrap_or_else(Utc::now); format!("{:04}", dt.year()) } diff --git a/docs/daemon.md b/docs/daemon.md index 189ff75f..70f8c226 100644 --- a/docs/daemon.md +++ b/docs/daemon.md @@ -44,7 +44,7 @@ The daemon runs the **same static surface as [`serve`](serve.md)**: with `$ONEBR Two ways to open it without ever reading `daemon.json` by hand: - **`onebrain daemon status`** prints the clickable `http://127.0.0.1:PORT/?token=TOKEN` as part of its dashboard (below). -- **`onebrain serve --open`** detects a daemon already serving the current vault and opens the browser at the daemon's URL instead of binding a second listener — see [serve.md](serve.md). +- **`onebrain serve --open`** reuses a daemon already serving the current vault — or starts one when none is running (v3.4.12) — and opens the browser at that daemon's URL instead of binding a second listener — see [serve.md](serve.md). ## `daemon status` — a real dashboard @@ -79,7 +79,7 @@ When a daemon is **already running and serves the same vault**, the CLI search v Notes: - **Passive: route-only, never start or restart.** The CLI uses `daemon_client::discover_matching`, NOT the MCP path's active `discover()`/`ensure_running()`. A plain `onebrain search …` never spawns a daemon and never stops/restarts one — it routes only to a daemon that is ALREADY up (the contention an MCP session creates). This is the load-bearing difference from the MCP path, which owns the daemon's lifecycle and restarts it on a version/vault mismatch. The CLI must never disrupt a daemon serving another vault. - **Vault-match guard.** The machine runs one daemon on a fixed port, so a daemon started for vault A must never answer a `--vault B` request. `discover_matching` reads the daemon's canonical bound vault from `daemon.json` (`DaemonInfo.vault`, written at bind) and routes only when `vault_decision` says it matches the caller's vault AND `version_decision` matches AND the daemon passes the engine-independent `/api/health` liveness probe. On ANY mismatch it declines (direct open) and **leaves the daemon.json record untouched** — no stop, no remove. -- **`search search` (lex)** and **`search vsearch` (vector-only)** are NOT daemon-routed: lex uses the standalone tantivy index (no redb, no contention), and the daemon exposes no vector-only search mode, so vsearch opens directly (honest `E_ENGINE_BUSY` while a session holds the engine — use `query` for results mid-session). +- **`search search` (lex)** is NOT daemon-routed: it uses the standalone tantivy index (no redb, no contention). **`search vsearch` (vector-only)** IS daemon-routed since v3.4.12 (#258 Gap 3): it routes through the daemon's `GET /api/vault/search?mode=vec` — passively, via `route_to_daemon` like the other search verbs — so it returns results while a session holds the engine instead of an `E_ENGINE_BUSY`. - **A routed request stays honest when the daemon holds no engine.** During the upgrade-transition window a *pre-3.4.6* `onebrain mcp` can still own the redb lock while the new warm daemon runs; the daemon is then engine-less and its `/api/internal/*` + `/api/vault/search` routes return `503`. The CLI classifies that 503 as `E_ENGINE_BUSY` (exit 77) for `query`/`get`/`reindex` and `busy: true` for `status` — identical to the direct-open fallback — so a routed verb never leaks an internal error mid-transition. (This is unambiguous because the CLI only routes to a *vault-matched* daemon; a vault-less daemon's "no vault bound" 503 is never reached — see [ADR 0023](decisions/0023-warm-daemon-mcp-search.md) Track 3.) - **Kill switch.** Set `ONEBRAIN_NO_DAEMON=1` to disable all CLI daemon routing (every verb opens the engine directly, the pre-daemon behaviour). @@ -142,7 +142,7 @@ Three recommended paths, in increasing order of setup: Notes: - Set `ONEBRAIN_TOKEN` (**≥ 32 chars**, e.g. `openssl rand -hex 16`) in the daemon's environment for a **stable, bookmarkable URL** across daemon restarts — otherwise every restart mints a fresh token and saved URLs go stale. The value must match the charset `[A-Za-z0-9_-]`; a too-short or invalid value is a hard error that prevents the daemon from starting. Unset the variable to get a generated token instead. -- Do **not** run a standalone `serve` alongside a running daemon on the default port — they share port 6789 and the engine lock (a plain `serve` detects the daemon and routes instead of binding; forcing a second listener needs an explicit `--port`). The `serve --host` flag no longer exists (#205): the only non-loopback bind is the container-scoped `ONEBRAIN_BIND` env var, which warns loudly that it speaks plaintext HTTP (see [serve.md](serve.md#containers--self-host--onebrain_bind)). +- Do **not** force a standalone `serve` alongside a running daemon on the default port — they share port 6789 and the engine lock (a plain `serve` reuses or starts the daemon and routes to it instead of binding; forcing a second listener needs an explicit `--port`). The `serve --host` flag no longer exists (#205): the only non-loopback bind is the container-scoped `ONEBRAIN_BIND` env var, which warns loudly that it speaks plaintext HTTP (see [serve.md](serve.md#containers--self-host--onebrain_bind)). ## Lifecycle diff --git a/docs/decisions/0032-self-healing-daemon-fallback.md b/docs/decisions/0032-self-healing-daemon-fallback.md new file mode 100644 index 00000000..26f8a545 --- /dev/null +++ b/docs/decisions/0032-self-healing-daemon-fallback.md @@ -0,0 +1,29 @@ +# 0032 — Self-healing daemon fallback: token gain, serve, and vsearch under redb contention + +- **Status:** accepted +- **Date:** 2026-07-12 +- **Supersedes (in part):** [0023](0023-warm-daemon-mcp-search.md) (the "`vsearch` is not daemon-routed" mapping) and the `serve` "never starts/stops/restarts a daemon" stance. + +## Context + +redb is single-process: exactly one process may open `engine.redb` / `token.redb` at a time. A warm `onebrain daemon` (or `onebrain mcp`) holds those locks for its whole lifetime. Post-ship testing of the token-optimization epic (v3.4.10/11) surfaced that several surfaces still hard-errored, failed to run, or served degraded results whenever a daemon was (or wasn't) holding a lock — the opposite of "runs smooth" (#258, #257): + +- `onebrain token gain` opened `token.redb` eagerly, so **every** mode errored `Database already open. Cannot acquire lock.` under a running daemon. +- `onebrain serve` with no daemon ran an engine-less foreground standalone — no token cache, so the Token-Gain dashboard was dark (#257); against a different-version daemon it hit a port/lock conflict. +- `onebrain search vsearch` had no daemon route, so it returned `E_ENGINE_BUSY` while an mcp session held the engine. + +Most surfaces already self-healed (search `query`/`get`/`reindex` Direct-fall-back + `with_retry` respawn; `mcp` auto-starts via `ensure_running`; `token check` fails open). These three were the gaps. + +## Decision + +- **Two-tier reads bypass the lock.** `token gain`'s default summary / `--by` / `--history` / `--reset` read the lock-free JSONL raw log (Tier-1 source of truth, ADR 0030); only `--all-time` / `--since` / `--rebuild` touch the redb rollup. The rollup DB is opened **lazily**, only in the branches that need it. +- **Read verbs route across version skew, never restart.** `token gain --all-time/--since` and `search vsearch` route to a **same-vault** daemon regardless of its version (`discover_same_vault_any_version` / passive `route_to_daemon`), because the routes they use return version-stable shapes (`PivotResult`, `SearchHit`). This closes the upgrade-without-restart gap: right after a CLI upgrade the still-running old daemon holds the lock, and the read still works without the user restarting it. A CLI **read** verb must never stop/restart another session's daemon — that stays `ensure_running`/`mcp`/`serve`'s job. +- **`serve` is an active lifecycle owner: reuse-or-start.** `serve` reuses a matching daemon, else **starts** one (restarting a stale/version-mismatched one) via `ensure_running`, then hands over that daemon's URL. The started daemon holds the engine + token cache, so the dashboard is populated. The explicit `--port`/`--dir`/`$ONEBRAIN_BIND` escape hatch still runs a foreground standalone — which now also opens its own token cache (`ServeConfig.open_token_cache`) so its dashboard isn't dark either. +- **New `vec` daemon mode.** `GET /api/vault/search?mode=vec` runs `Engine::vector_search` against the held (or per-request) engine, mirroring the `hybrid` path; `search vsearch` routes through it. +- **One transient-lock signal.** A genuinely contended open (no daemon to route to, or a too-old daemon) maps to `CoreError::EngineBusy` → `E_ENGINE_BUSY` / exit 77 — the same code `query`/`vsearch`/`get` already use — with an actionable "retry after `onebrain daemon stop`" hint, never a raw redb error at exit 1. + +## Consequences + +- `serve` can now restart the machine's single daemon (e.g. to replace a version-mismatched one), which briefly disrupts a cross-vault mcp session bound to that daemon — the session self-heals via its own `with_retry` reconnect. This is the accepted trade for "serve always runs and its dashboard is populated"; true multi-vault daemons are the #230 follow-up. +- `vsearch`'s daemon path uses the generic unreranked hint rather than the direct path's cosine-confidence fold-in (advisory text only; hit data + ordering are identical). A per-query `min_score` on the daemon path can only tighten past the daemon's config floor — identical to the already-shipped `query` daemon path. +- The hit-mapping closure is now duplicated four ways (`run_hybrid`/`run_hybrid_held`/`run_vec`/`run_vec_held`); extraction is a low-priority follow-up. diff --git a/docs/decisions/README.md b/docs/decisions/README.md index 249214de..fe8ff4b3 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -59,5 +59,6 @@ ADRs are immutable once accepted — if a decision changes, write a new ADR that | [0029](0029-token-cache-redb.md) | Token cache on redb: memoization + already-sent ledger + generation counter | accepted | | [0030](0030-gain-telemetry-raw-plus-rollups.md) | Gain telemetry: raw JSONL keep-everything + precomputed rollups + epoch reset | accepted | | [0031](0031-vault-read-ledger-gate-hook.md) | Vault-read ledger gate hook: deny only repeat-unchanged, fail-open, default off | accepted | +| [0032](0032-self-healing-daemon-fallback.md) | Self-healing daemon fallback: token gain / serve / vsearch under redb contention | accepted | > These ADRs distill the public-facing rationale; the full design notes live in the project tracker. Numbers are stable IDs assigned at authoring time — see each ADR's **Date** for chronology. diff --git a/docs/serve.md b/docs/serve.md index 9455277f..83a8aa01 100644 --- a/docs/serve.md +++ b/docs/serve.md @@ -15,7 +15,7 @@ The web UI is **embedded in the binary** — a release `onebrain` ships the late - **Loopback only** (`127.0.0.1:6789`). There is no bind-address flag; for remote access put an encrypted tunnel in front — see [Remote access in daemon.md](daemon.md#remote-access). - **Hardened surface** — confined to the vault (tooling dirs like `.git`/`.claude` are refused), script-carrying files forced to download, a strict CSP, and the agent subprocess never inherits the daemon token. See [Security & trust model](install.md#security--trust-model). -**Daemon-aware (v3.4.8):** `serve` and the [daemon](daemon.md) share port 6789 by design (one surface, one port), so if a daemon is already serving the current vault, `serve` doesn't bind a second listener — it prints the daemon's token-bearing URL, honours `--open`, and exits. That makes `onebrain serve --open` the one command that always lands you in the web UI, whether or not a daemon is up, with no token knowledge required. Passing `--port` or `--dir` (or setting `ONEBRAIN_BIND`) always means a standalone server (you asked for a specific listener), and `ONEBRAIN_NO_DAEMON=1` disables the daemon detection. `serve` never starts, stops, or restarts a daemon. +**Daemon-aware (v3.4.8; reuse-or-start since v3.4.12):** `serve` and the [daemon](daemon.md) share port 6789 by design (one surface, one port). `serve` **reuses** a daemon already serving the current vault — and, when none is running, **starts** one (restarting a stale or version-mismatched daemon) — then prints that daemon's token-bearing URL, honours `--open`, and exits. Because the started daemon holds the engine + token cache, the Token-Gain dashboard is populated. That makes `onebrain serve --open` the one command that always lands you in the web UI, whether or not a daemon is up, with no token knowledge required. Passing `--port` or `--dir` (or setting `ONEBRAIN_BIND`) always means a standalone foreground server (you asked for a specific listener), and `ONEBRAIN_NO_DAEMON=1` disables daemon reuse/start. Like `onebrain mcp`, `serve` is an active daemon-lifecycle owner (it may start/restart the machine's single daemon); the passive CLI search verbs never do. ## Containers / self-host — `ONEBRAIN_BIND`