diff --git a/.claude/MCPX.md b/.claude/MCPX.md index 9e82569..9f8fb6b 100644 --- a/.claude/MCPX.md +++ b/.claude/MCPX.md @@ -1,66 +1,37 @@ -# mcpx — MCP Server CLI Proxy +# mcpx -mcpx wraps MCP servers into CLI tools. Call them via Bash instead of loading schemas into context. +Call MCP tools through `mcpx --flags`. Don't load native MCP — use the CLI commands below. -## Quick Reference +## Servers -```bash -mcpx list # List configured servers -mcpx list -v # List all tools with flags -mcpx --help # Show server tools -mcpx --help # Show tool flags -mcpx --flags # Call a tool -mcpx --stdin # Read args from stdin JSON -mcpx --json # Output raw JSON -mcpx daemon status # Show running daemons -``` +- **serena** *(daemon)* -## Configured Servers +## Compose -- **serena** — `serena` (daemon) +| Need | How | +|---|---| +| Standard call | `mcpx --flag value` | +| Large arg from file | `--body @/path/to/file` | +| Read body from stdin | `--body @-` or `--body -` | +| Pass full args as JSON | `printf '{...}' \| mcpx --stdin` | +| Mix stdin + flags | `--stdin --flag value` (flags win) | +| Extract one JSON field | `--pick path.to.field` | +| Raw JSON output | `--json` | +| Per-call timeout | `--timeout 60s` (Go duration) | +| Show resolved command | `--dry-run` | +| Args skeleton | `mcpx --example` | +| Type-check args | `mcpx --validate-args ...` | -## Usage Pattern +## Discover -1. Discover: `mcpx --help` to see available tools -2. Inspect: `mcpx --help` to see flags -3. Call: `mcpx --flag value` -4. For long args: `printf '{"key":"value"}' | mcpx --stdin` +| Need | How | +|---|---| +| Find the right tool by intent | `mcpx find ""` | +| One-line list of a server's tools | `mcpx --help` | +| Full schema for one tool | `mcpx --help` | +| Run many tool calls in parallel | `mcpx batch < calls.jsonl` | -## Large Content: @file syntax +## Exit codes -Any string flag accepts `@/path` to read from a file or `@-`/`-` to read from stdin: -```bash -mcpx --body @/tmp/code.go # Read file into --body -mcpx --body @- # Read stdin into --body -mcpx --body - # Same (backward compat) -``` - -## Output Extraction: --pick - -Extract a JSON field from the result without jq: -```bash -mcpx --pick field.path # Dot-separated path -mcpx --pick items.0.name # Array index access -``` - -## Timeout Override: --timeout - -Override the default call timeout for a single invocation: -```bash -mcpx --timeout 60s # Go duration format -``` - -## Stdin Merge - -`--stdin` can be combined with CLI flags. Flags win on conflict: -```bash -echo '{"body":"content"}' | mcpx --stdin --name_path Foo -``` - -## Tips for AI Agents - -- Use `--body @/tmp/file` for large content to avoid shell escaping -- Use `--pick field` instead of piping through jq for single fields -- Combine `--stdin` with flags for mixed large+small arguments -- Use `--timeout 120s` for long-running operations +`0` ok · `1` tool error · `2` config · `3` connection · `4` timeout · `5` policy denied · `6` tool not found. @SERENA.md diff --git a/.claude/SERENA.md b/.claude/SERENA.md index ef841d7..75232e4 100644 --- a/.claude/SERENA.md +++ b/.claude/SERENA.md @@ -1,64 +1,70 @@ -# serena (21 tools) - -Usage: `mcpx serena --flags` - -**list_dir** — Lists files and directories in the given directory (optionally with recursion). - --max_answer_chars , --recursive *, --relative_path *, --skip_ignored_files - -**find_file** — Finds non-gitignored files matching the given file mask within the given relative path. - --file_mask *, --relative_path * - -**search_for_pattern** — Offers a flexible search for arbitrary patterns in the codebase, including the - --context_lines_after , --context_lines_before , --max_answer_chars , --paths_exclude_glob , --paths_include_glob , --relative_path , --restrict_search_to_code_files , --substring_pattern * - -**get_symbols_overview** — Use this tool to get a high-level understanding of the code symbols in a file. - --depth , --max_answer_chars , --relative_path * - -**find_symbol** — Retrieves information on all symbols/code entities (classes, methods, etc.) based on the given name path pattern. - --depth , --exclude_kinds , --include_body , --include_info , --include_kinds , --max_answer_chars , --name_path_pattern *, --relative_path , --substring_matching - -**find_referencing_symbols** — Finds references to the symbol at the given `name_path`. - --exclude_kinds , --include_kinds , --max_answer_chars , --name_path *, --relative_path * - -**replace_symbol_body** — Replaces the body of the symbol with the given `name_path`. - --body *, --name_path *, --relative_path * - -**insert_after_symbol** — Inserts the given body/content after the end of the definition of the given symbol (via the symbol's location). - --body *, --name_path *, --relative_path * - -**insert_before_symbol** — Inserts the given content before the beginning of the definition of the given symbol (via the symbol's location). - --body *, --name_path *, --relative_path * - -**rename_symbol** — Renames the symbol with the given `name_path` to `new_name` throughout the entire codebase. - --name_path *, --new_name *, --relative_path * - -**write_memory** — Write information (utf-8-encoded) about this project that can be useful for future tasks to a memory in md format. - --content *, --max_chars , --memory_name * - -**read_memory** — Reads the contents of a memory. - --memory_name * - -**list_memories** — Lists available memories, optionally filtered by topic. - --topic - -**delete_memory** — Delete a memory, only call if instructed explicitly or permission was granted by the user. - --memory_name * - -**rename_memory** — Rename or move a memory, use "/" in the name to organize into topics. - --new_name *, --old_name * - -**edit_memory** — Replaces content matching a regular expression in a memory. - --allow_multiple_occurrences , --memory_name *, --mode *, --needle *, --repl * - -**activate_project** — Activates the project with the given name or path. - --project * - -**get_current_config** — Print the current configuration of the agent, including the active and available projects, tools, contexts, and modes. - -**check_onboarding_performed** — Checks whether project onboarding was already performed. - -**onboarding** — Call this tool if onboarding was not performed yet. - -**initial_instructions** — Provides the 'Serena Instructions Manual', which contains essential information on how to use the Serena toolbox. +# serena + +`mcpx serena --flags` — 21 tools. Use `--help` / `--example` / `--validate-args` per tool. + +## Tool selector + +| Tool | What it does | Required | +|---|---|---| +| `list_dir` | Lists files and directories in the given directory (optionally with recursion). | `--relative_path` `--recursive` | +| `find_file` | Finds non-gitignored files matching the given file mask within the given relative path. | `--file_mask` `--relative_path` | +| `search_for_pattern` | Offers a flexible search for arbitrary patterns in the codebase, including the | `--substring_pattern` | +| `get_symbols_overview` | Use this tool to get a high-level understanding of the code symbols in a file. | `--relative_path` | +| `find_symbol` | Retrieves information on all symbols/code entities (classes, methods, etc.) based on the given name path pattern. | `--name_path_pattern` | +| `find_referencing_symbols` | Finds references to the symbol at the given `name_path`. | `--name_path` `--relative_path` | +| `replace_symbol_body` | Replaces the body of the symbol with the given `name_path`. | `--name_path` `--relative_path` `--body` | +| `insert_after_symbol` | Inserts the given body/content after the end of the definition of the given symbol (via the symbol's location). | `--name_path` `--relative_path` `--body` | +| `insert_before_symbol` | Inserts the given content before the beginning of the definition of the given symbol (via the symbol's location). | `--name_path` `--relative_path` `--body` | +| `rename_symbol` | Renames the symbol with the given `name_path` to `new_name` throughout the entire codebase. | `--name_path` `--relative_path` `--new_name` | +| `write_memory` | Write information (utf-8-encoded) about this project that can be useful for future tasks to a memory in md format. | `--memory_name` `--content` | +| `read_memory` | Reads the contents of a memory. | `--memory_name` | +| `list_memories` | Lists available memories, optionally filtered by topic. | — | +| `delete_memory` | Delete a memory, only call if instructed explicitly or permission was granted by the user. | `--memory_name` | +| `rename_memory` | Rename or move a memory, use "/" in the name to organize into topics. | `--old_name` `--new_name` | +| `edit_memory` | Replaces content matching a regular expression in a memory. | `--memory_name` `--needle` `--repl` `--mode` | +| `activate_project` | Activates the project with the given name or path. | `--project` | +| `get_current_config` | Print the current configuration of the agent, including the active and available projects, tools, contexts, and modes. | — | +| `check_onboarding_performed` | Checks whether project onboarding was already performed. | — | +| `onboarding` | Call this tool if onboarding was not performed yet. | — | +| `initial_instructions` | Provides the 'Serena Instructions Manual', which contains essential information on how to use the Serena toolbox. | — | + +## Edit-tool safety + +For `replace_symbol_body`, body must start AFTER the declaration keyword — the keyword is preserved by the server. mcpx warns on duplicates and (for `.go`) re-parses the file post-edit. + +| Language | ✓ correct | ✗ wrong | +|---|---|---| +| Go | `Foo struct{...}` / `Foo() error {...}` | `type Foo...` / `func Foo...` | +| Python | `foo():\n ...` | `def foo():` | +| TS / JS | `bar() { ... }` | `function bar()` / `class Bar` | +| Rust | `foo() { ... }` | `fn foo()` / `struct Foo` | + +## Notes + +- Symbol-aware lookups (`find_symbol`) are faster and more accurate than text search (`search_for_pattern`) for code symbols — prefer them. + +## Compact reference + +- `list_dir` --max_answer_chars --recursive * --relative_path * --skip_ignored_files +- `find_file` --file_mask * --relative_path * +- `search_for_pattern` --context_lines_after --context_lines_before --max_answer_chars --paths_exclude_glob --paths_include_glob --relative_path --restrict_search_to_code_files --substring_pattern * +- `get_symbols_overview` --depth --max_answer_chars --relative_path * +- `find_symbol` --depth --exclude_kinds --include_body --include_info --include_kinds --max_answer_chars --name_path_pattern * --relative_path --substring_matching +- `find_referencing_symbols` --exclude_kinds --include_kinds --max_answer_chars --name_path * --relative_path * +- `replace_symbol_body` --body * --name_path * --relative_path * +- `insert_after_symbol` --body * --name_path * --relative_path * +- `insert_before_symbol` --body * --name_path * --relative_path * +- `rename_symbol` --name_path * --new_name * --relative_path * +- `write_memory` --content * --max_chars --memory_name * +- `read_memory` --memory_name * +- `list_memories` --topic +- `delete_memory` --memory_name * +- `rename_memory` --new_name * --old_name * +- `edit_memory` --allow_multiple_occurrences --memory_name * --mode * --needle * --repl * +- `activate_project` --project * +- `get_current_config` +- `check_onboarding_performed` +- `onboarding` +- `initial_instructions` `*` = required diff --git a/.mcpx/config.yml b/.mcpx/config.yml index 87d61eb..5eae2a4 100644 --- a/.mcpx/config.yml +++ b/.mcpx/config.yml @@ -23,7 +23,8 @@ servers: args: - start-mcp-server - --context=claude-code - - --project-from-cwd + - --project + - "$(mcpx.project_root)" transport: stdio daemon: true startup_timeout: 30s diff --git a/CHANGELOG.md b/CHANGELOG.md index 70bfc41..b9556f8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,70 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [1.6.0] - 2026-05-03 + +The "Agentic Supremacy" release. mcpx becomes a measurable, observable, and intelligent control plane: every call is recorded, every schema is cached, every server is one `mcpx find` away, and an always-on dashboard makes ROI visible. + +### Added — Foundation + +- **JSONL stats** (`internal/stats/`) — every tool call writes one line to `~/.mcpx/stats.jsonl`: timestamp, args, latency, cache hits, exit code, tokens saved vs native MCP loading. Async writer; never blocks the caller. +- **Schema cache** (`internal/schemacache/`) — `tools/list` (and prompts/resources) cached at `~/.mcpx/cache/schemas/.json` with TTL. First call per server is slow; everything after is instant. Computes `native_baseline_tokens` once at populate time so the stats writer can quote real savings. +- **Result cache: deliberately not shipped.** Caching tool RESPONSES is a correctness footgun — recursive searches, multi-file reads, network-backed tools, and any non-filesystem state can all return stale data inside a TTL. mcpx is the trustworthy intermediary; we'd rather make the round-trip than risk acting on stale snapshots. If a future MCP server adopts `readOnlyHint: true` annotations widely, server-driven result caching becomes safe to add back. +- **Schema normalizer** — handles JSON Schema union types (`type: ["string","null"]`), `oneOf`/`anyOf`/`allOf`, `$ref`; unknown keywords preserved in `Ext`. Fixes #14 (Sentry MCP server initialization). + +### Added — Agent superpowers + +- **`mcpx find `** — BM25-ranked tool search across every configured server. ~80 tokens for top candidates instead of 5–15K tokens for `mcpx list -v`. +- **`mcpx batch`** — NDJSON in/out, parallel by default, single client per server reused across the entire batch. +- **`mcpx --example`** — JSON skeleton with placeholder values from the normalized schema. +- **`mcpx --validate-args ...`** — type/required check without invoking the tool. +- **Default-compact help** — `mcpx --help` is one line per tool with required flags surfaced. Add `--full` for descriptions. +- **Typo remediation** — Levenshtein-2 "did you mean…" suggestions on tool-not-found and unknown-flag errors. +- **Pre/post edit guards** — `replace_symbol_body` calls are warned when the body starts with a duplicate declaration keyword (Go, Python, TS, JS, Rust, Ruby, Java, Kotlin, Swift). For `.go` files the modified file is re-parsed and `file:line:col` is surfaced when the edit broke syntax. + +### Added — Operator surfaces + +- **`mcpx gain`** — premium terminal dashboard: hero metric (tokens saved), 7-day sparkline, top-tools bars, top-savers, server health, recent calls. Subcommands: `--by tool|server|day`, `--history N`, `--suggest`, `--watch`, `--all`, `--project`, `--since`, `--json`. +- **Always-on web dashboard** — auto-spawned on first call. Token-protected, 127.0.0.1-only, idle-shutdown after 1h, opt-out via `MCPX_UI=off` or `ui.enabled: false`. Single-page UI: project sidebar, time range tabs (1h/24h/7d/30d/all), token efficiency bar, top tools, server health, click-to-inspect drawer, SSE live tail, regex filter (`/`). +- **`mcpx ui status|stop|open|disable`** — dashboard daemon controls. +- **`mcpx doctor`** — config + command path + secret resolution + daemon liveness + initialize + tools/list checks. `--json` for machine-readable output. + +### Changed + +- **Structured exit codes** — `0` ok · `1` tool error · `2` config · `3` connection · `4` timeout · `5` policy denied · `6` tool not found. +- **`mcpx version`** warns loudly on `+dirty` builds (so leaked debug code from local builds doesn't silently propagate). +- **`mcpx configure`** rewritten — generates an agent-grounded `MCPX.md` (composition + discover + exit codes) and per-server `.md` files (tool-selector table + compact reference). The `--format compact` flag is now hidden; output is agent-optimized by default. Edit-tool safety guidance only emitted for servers that expose body-mutation tools. +- Client version bumped to `1.6.0` in the MCP handshake. + +### New packages + +- `internal/stats/` — JSONL writer (async, drop-on-overflow), reader, aggregator (top-K, p95, daily buckets, hit rates). +- `internal/schemacache/` — schema cache, result cache, idempotence detection. +- `internal/find/` — BM25 ranker (snake/camel splitting, name-coverage bonus, plural stem). +- `internal/render/` — terminal primitives (Box, Bar, Sparkline, FormatNumber/Duration/Percent, term width). +- `internal/ui/` — dashboard daemon (lazy supervisor, HTTP, SSE, embedded HTML/CSS/JS). + +### Config additions + +```yaml +gain: + enabled: true + tokenizer: estimate # bytes ÷ 4; honest approximation of Claude tokenization + retain_days: 30 + stats_path: "" # default: ~/.mcpx/stats.jsonl + +cache: + schema_ttl: 5m # only the schema cache is shipped — see notes above + +ui: + enabled: true + port: 7878 # 0 = ephemeral + bind: 127.0.0.1 + idle_timeout: 1h +``` + +Environment overrides: `MCPX_AGENT`, `MCPX_CACHE`, `MCPX_UI`, `MCPX_VERBOSE`. + ## [1.5.0] - 2026-03-30 ### Added diff --git a/PLAN-v1.6.md b/PLAN-v1.6.md new file mode 100644 index 0000000..721fd74 --- /dev/null +++ b/PLAN-v1.6.md @@ -0,0 +1,518 @@ +# mcpx v1.6 — Agentic Supremacy + +> The thesis: mcpx is not just a CLI proxy — it's the intelligent control plane between AI agents and MCP servers. v1.6 ships the data layer, the agent superpowers, and the always-on dashboard that prove the value. + +**Three goals every feature must serve:** +1. **Less tokens** — schema cache, result cache, `find`, compact-default help, response truncation +2. **More accuracy** — structured exit codes, schema normalization, `--example`, error remediation +3. **More control** — gain analytics, dashboard, audit, agent identity + +--- + +## Build sequence + +The order matters. Each layer's data is consumed by the next: + +``` + ┌─ schema cache ──┐ ┌─ mcpx find ─┐ + │ ├─→ JSONL ┤ ├─→ mcpx gain (TUI) + │ result cache ──┤ stats │ mcpx batch ─┘ + │ │ │ + │ config.gain ───┘ └────────────────→ dashboard (always-on) +``` + +**Foundation (must ship first):** +- F1. JSONL stats schema + writer (the data contract everyone consumes) +- F2. Schema cache (server init/tools/prompts/resources) +- F3. Config: `gain:`, `cache:`, `ui:` sections +- F4. Instrumentation: hook every tool call to emit a stats record + +**Agent superpowers:** +- A1. `mcpx find` — semantic tool search across all servers +- A2. Result cache for idempotent reads +- A3. `mcpx batch` — NDJSON in/out, parallel exec +- A4. Default-compact help; verbose behind `--full` +- A5. `--example` + arg validation + +**Operator surfaces:** +- O1. `mcpx gain` — premium TUI with sparklines, top-K tables, savings front-and-center +- O2. Always-on dashboard daemon (auto-spawned, 127.0.0.1, single page) +- O3. Project-aware views (sidebar, per-project filtering) + +**Robustness:** +- R1. Issue #14 + canonical schema normalizer (oneOf/anyOf/allOf/$ref/union types) +- R2. Structured exit codes (0–6) +- R3. Active error remediation (typo suggestions on tool/flag mismatches) + +--- + +## F1. JSONL stats — the data contract + +**Path:** `~/.mcpx/stats.jsonl` (single file, append-only). Daily rotation handled at read time, not write time, to avoid coordination. + +**Per-call schema** (one line per call, ~250 bytes): + +```json +{ + "ts": "2026-05-03T14:12:33.482Z", + "session": "9214", + "project": "/Users/x/projects/mcpx", + "agent": "claude-code", + "server": "serena", + "tool": "find_symbol", + "args_bytes": 84, + "args_tokens_est": 21, + "response_bytes": 4210, + "response_tokens_est": 1052, + "latency_ms": 47, + "transport": "stdio", + "daemon": true, + "schema_cache_hit": true, + "result_cache_hit": false, + "exit_code": 0, + "error": null, + "policy_action": "allow", + "policy_name": null, + "native_baseline_tokens": 8421, + "tokens_saved": 7369 +} +``` + +**Fields explained:** +- `session` = PPID of the caller (groups calls within one shell session). +- `project` = resolved project root (or `""`). Drives dashboard sidebar. +- `agent` = `MCPX_AGENT` env, else `"unknown"`. Enables multi-agent identity later. +- `*_tokens_est` = `bytes / 4`. Honest estimate. tiktoken upgrade in v1.6.1. +- `native_baseline_tokens` = JSON-stringified size of `(initialize result + tools/list + prompts/list + resources/list)` for this server, computed once when schema cache populates. +- `tokens_saved` = `native_baseline_tokens - args_tokens_est - response_tokens_est`. The headline metric. + +**Writer guarantees:** +- Atomic single-line append (`O_APPEND` + write < PIPE_BUF). +- Never blocks the caller: writer runs in a goroutine with bounded buffer; drops on overflow with a counter. +- Never errors out the call. Stats failure is invisible to the user. + +**Package:** `internal/stats/` +- `stats.go` — `Record` struct, `Writer.Write(Record)`, `Writer.Flush()`. +- `read.go` — `Reader.Iter(filter)`, time-range filtering, project filter, server/tool filter. +- `agg.go` — `Aggregate(filter)`: top tools, latency p50/p95, saved tokens, hit rates, daily buckets. + +--- + +## F2. Schema cache + +**Path:** `~/.mcpx/cache/.json` + +`server-hash` = SHA256 of `(command, args, env, url)`. Different config = different cache entry, no false hits. + +**Cache record:** +```json +{ + "version": 1, + "captured_at": "...", + "ttl_seconds": 300, + "server_hash": "abc123...", + "initialize": { ... }, + "tools": [ ... ], + "prompts": [ ... ], + "resources": [ ... ], + "native_baseline_tokens": 8421 +} +``` + +**Behavior:** +- TTL: default 5 min (config: `cache.schema_ttl: 5m` per server). +- Bypass: `--no-cache` flag, `MCPX_CACHE=off` env. +- Invalidate: `mcpx cache clear [server]`, auto on `notifications/tools_list_changed`. +- Daemons keep the schema in memory; the cache is for non-daemon and CLI-mode invocations. + +**Package:** `internal/schemacache/` (new — separate from `internal/cache/` which is empty). + +--- + +## F3. Config additions + +```yaml +# ~/.mcpx/config.yml +gain: + enabled: true # default: true + tokenizer: estimate # estimate | tiktoken (v1.6.1) + retain_days: 30 # auto-prune stats.jsonl entries older than N days + +cache: + schema_ttl: 5m + result_ttl: 30s + result_enabled: true # only idempotent tools cached + +ui: + enabled: true # default: true. set false to disable always-on dashboard + port: 7878 # 0 = random + bind: 127.0.0.1 + idle_timeout: 1h +``` + +Per-server overrides for `cache:` allowed under each `servers.:`. + +--- + +## F4. Instrumentation + +Single hook in `internal/cli/commands.go` `runTool`: + +```go +defer func() { stats.Record(buildRecord(...)) }() +``` + +Captures: timing (start → defer), bytes in (args JSON), bytes out (response JSON), cache hits, policy decision, error/exit code. Zero overhead when `gain.enabled: false`. + +Schema cache populates `native_baseline_tokens` on first server connect; instrumented call reads it from the cache. + +--- + +## A1. `mcpx find` + +**Surface:** +```bash +mcpx find "search code by regex" +mcpx find "issue tracker" --top 3 +mcpx find "..." --json # machine-readable +mcpx find "..." --server serena # restrict to one server +``` + +**Output (default, ~80 tokens):** +``` +serena.search_for_pattern 0.91 Flexible regex search across files +serena.find_symbol 0.74 Retrieve info on symbols by name path +github.search_issues 0.42 Search GitHub issues by query +``` + +**Algorithm:** +- Build corpus = `tool.name + " " + tool.description` for every tool of every configured server (read from schema cache; populate cache for any server not seen). +- Score = BM25 over query tokens vs. corpus. Bonus weight if query token appears in `tool.name`. +- Snake-case and camelCase splitter for token matches (`find_symbol` → `find symbol` → matches "symbol"). +- Top-K (default 5). + +**Effort:** ~250 LoC, no new deps. Package: `internal/find/`. + +--- + +## A2. Result cache + +Only for tools annotated as idempotent. Detection (in priority order): +1. Server-supplied `annotations.readOnlyHint: true` (per MCP 2025 spec). +2. Config-level: `cache.result_idempotent_tools: ["serena.find_symbol", ...]`. +3. Heuristic: tool name starts with `get_`, `list_`, `find_`, `search_`, `read_` (opt-in via `cache.result_heuristic: true`). + +**Key:** `SHA256(server, tool, normalized-args-JSON)`. +**Path:** `~/.mcpx/cache/results/.json`. +**TTL:** `cache.result_ttl` (default 30s; conservative). +**Stats:** every cache hit emits a stats record with `latency_ms` near zero and `result_cache_hit: true`. + +--- + +## A3. `mcpx batch` + +**Input (NDJSON via stdin):** +```jsonl +{"id":"a","server":"serena","tool":"find_symbol","args":{"name_path_pattern":"Auth"}} +{"id":"b","server":"serena","tool":"find_symbol","args":{"name_path_pattern":"Token"}} +{"id":"c","server":"github","tool":"get_issue","args":{"number":42}} +``` + +**Output (NDJSON, same order):** +```jsonl +{"id":"a","ok":true,"latency_ms":42,"result":{...},"cached":false} +{"id":"b","ok":true,"latency_ms":2,"result":{...},"cached":true} +{"id":"c","ok":false,"latency_ms":120,"error":"...","exit_code":1} +``` + +**Flags:** +- `--parallel` (default for daemon-backed servers; bounded by `--max-concurrent N`) +- `--sequential` +- `--max-concurrent N` (default = NumCPU) +- `--stop-on-error` +- `--cache` (use result cache; default true) + +**Per-server connection reuse:** one client per server name across the batch; closed at end. + +**Effort:** ~400 LoC. Package: `internal/cli/batch.go`. + +--- + +## A4. Default-compact help + +Flip the defaults: +- `mcpx --help` and `mcpx list ` → one line per tool: `name required-args-summary`. No descriptions. +- `mcpx --help --full` and `mcpx list -v` → current verbose output. + +This is a UX change. Current `printToolsVerbose` becomes opt-in. New `printToolsCompact` is the default. + +--- + +## A5. `--example` + arg validation + +```bash +mcpx serena find_symbol --example +# { +# "name_path_pattern": "", +# "relative_path": "", +# "depth": 0, +# "include_body": false +# } +``` + +`--validate-args` checks types and required fields without calling the tool. Builds on schema normalizer (R1). + +--- + +## O1. `mcpx gain` — premium terminal UI + +**Goal:** screenshot-worthy. Not a flat table. + +**Default view (`mcpx gain`):** +``` +┌─ mcpx ───────────────────────── this project · 7d ──┐ +│ │ +│ Tokens saved 487,231 │ +│ ▔▔▔▔▔▔▔▔▔▔▔▔▔ ▁▂▃▅▇▆▄▂▃ daily │ +│ │ +│ Calls 1,284 Cache hit rate 73% │ +│ Avg latency 41ms Errors 2% │ +│ │ +├─ Top tools ─────────────────────────────────────────┤ +│ serena.find_symbol 512 ▓▓▓▓▓▓▓▓▓ │ +│ serena.search_for_pattern 284 ▓▓▓▓▓ │ +│ github.get_issue 107 ▓▓ │ +│ serena.get_symbols_overview 89 ▓▓ │ +│ ... │ +├─ Top savings ───────────────────────────────────────┤ +│ serena.find_symbol 192K saved (vs native MCP) │ +│ github.get_issue 61K saved │ +├─ Last 5 calls ──────────────────────────────────────┤ +│ 14:23 serena.find_symbol 47ms ✓ │ +│ 14:23 serena.find_symbol cached ⚡ │ +│ 14:22 github.get_issue 180ms ✓ │ +│ ... │ +└────────────────────────────── http://127.0.0.1:7878 ┘ +``` + +**Renderer requirements:** +- Use box-drawing chars + `fatih/color` (already a dep). No new deps. +- Width detection via `term.GetSize`. Falls back to 80 cols. +- Sparklines: 8-level Unicode block chars (`▁▂▃▄▅▆▇█`). +- Bars: 8-step block chars for fractional widths. +- Number formatting: `1.2K`, `487K`, `4.8M`. +- Colors: green for saved/ok, yellow for warn, red for errors, dim for metadata. + +**Subcommands:** +```bash +mcpx gain # the dashboard above (current project, 7d) +mcpx gain --all # all projects combined +mcpx gain --project /path # specific project +mcpx gain --since 24h # time window +mcpx gain --by tool # ranked tool table only +mcpx gain --by server # ranked server table +mcpx gain --by day # daily sparkline detail +mcpx gain --history [N] # last N call entries +mcpx gain --suggest # mined recommendations +mcpx gain --json # machine-readable everything +mcpx gain --watch # live refresh every 1s +``` + +**Package:** `internal/cli/gain.go` + `internal/render/` (new helpers: `Box`, `Bar`, `Sparkline`, `FormatNumber`, `FormatBytes`, `FormatDuration`). + +--- + +## O2 + O3. Always-on dashboard + +**Lifecycle:** lazy supervisor daemon, auto-spawned on first `mcpx ` invocation. + +**Spawn flow:** +1. Every CLI startup checks `~/.mcpx/ui.json`. If file missing or PID dead → spawn UI daemon. +2. UI daemon writes `{port, token, pid}` (mode 0600) to `~/.mcpx/ui.json`. +3. CLI prints one-line stderr notice **once per shell session** (suppression via `MCPX_UI_NOTICE_SHOWN` env that the CLI sets before it execs anything). +4. Idle timeout: 1h with no HTTP traffic → daemon self-exits. +5. Opt-out: `ui.enabled: false` in config or `MCPX_UI=off` env. +6. Manual: `mcpx ui status | stop | open | disable`. + +**Architecture:** +- New package `internal/ui/`: + - `supervisor.go` — `EnsureRunning()`, `~/.mcpx/ui.json` handshake. + - `server.go` — `http.Server`, SSE `/events`, JSON API, static `embed.FS`. + - `data.go` — wraps `internal/stats` reader + cache for query endpoints. + - `assets/` — single-page HTML, CSS, htmx, uPlot. ~50KB total embedded. +- New hidden cobra command in `internal/cli/`: `mcpx __ui` (mirrors `__daemon` pattern). +- Hook into `cli.Execute()` startup: `ui.EnsureRunningAsync()` (non-blocking). + +**Single-page layout (HTML/CSS):** +``` +┌──────────────────────────────────────────────────────────┐ +│ mcpx live ● saved: 487K │ +├───────────────┬──────────────────────────────────────────┤ +│ PROJECTS │ Project: mcpx │ +│ ▸ mcpx (this) │ ───────────────────────────────────── │ +│ my-app │ ╭─ 7-day savings ──╮ ╭─ Hit rate ──╮ │ +│ work-stuff │ │ ▁▂▃▄▆▇▅▃ │ │ 73% │ │ +│ All │ │ ▔▔▔▔▔▔▔▔▔▔ │ ╰─────────────╯ │ +│ │ ╰──────────────────╯ │ +│ SERVERS │ │ +│ ▸ serena (✓) │ Top tools (7d) ──────────── │ +│ github │ serena.find_symbol ▓▓▓▓▓▓▓▓ 512 │ +│ sentry │ serena.search_pattern ▓▓▓▓ 284 │ +│ │ github.get_issue ▓▓ 107 │ +│ AUDIT │ │ +│ Live tail │ Live tail (SSE) ────────── │ +│ │ 14:23 serena.find_symbol 47ms ✓ │ +│ │ 14:23 serena.find_symbol cached ⚡ │ +│ │ 14:22 github.get_issue 180ms ✓ │ +│ │ ... │ +│ │ │ +│ │ [ Replay last call ] [ Export JSON ] │ +└───────────────┴──────────────────────────────────────────┘ +``` + +**Tech stack (zero npm):** +- Go server: stdlib `net/http` + `html/template` + SSE. +- Frontend: htmx (15KB) + uPlot (45KB) + handcrafted CSS. +- All assets bundled via `embed.FS`. +- Token-protected URLs: every page/API call requires `?t=` from `~/.mcpx/ui.json`. + +**Pages (server-rendered + htmx swaps):** +- `/` — overview (this project, default) +- `/project/` — same layout, scoped +- `/all` — cross-project totals +- `/tools` — full ranked table (sortable by frequency, latency, savings, errors) +- `/servers` — health, daemons, schema age +- `/audit` — denied calls, policy decisions +- `/api/events` — SSE stream (live tail) +- `/api/stats?...` — JSON +- `/api/replay/` — re-run a past call + +--- + +## R1. Schema normalizer (issue #14 + beyond) + +`internal/mcp/schema.go` (new). One pass over server-provided JSON Schema that: +- Accepts `type` as string OR `[]string` → picks first non-null, sets `Nullable: true` if null was in the union. +- Resolves `$ref` inline (within same schema document). +- Flattens `allOf` by merging subschemas. +- Picks first non-null branch of `oneOf` / `anyOf`, stores alternatives in `Ext`. +- Preserves unknown keywords in `Ext map[string]json.RawMessage`. +- Never errors. Logs warnings under `MCPX_VERBOSE=1`. + +Result: every downstream consumer (`describe`, `--help`, `--example`, validation, dashboard) sees a single canonical shape. + +--- + +## R2. Structured exit codes + +| Code | Meaning | +|------|---------| +| 0 | Success | +| 1 | Tool error (`isError=true`) | +| 2 | Config / syntax error | +| 3 | Connection / transport error | +| 4 | Timeout | +| 5 | Policy denied | +| 6 | Tool not found | + +Exposed via `cli.Exit()` helper called from every command. README + `--help` document them. + +--- + +## R3. Active error remediation + +When tool/flag lookup fails: +- "tool `find_symbl` not found in `serena`. **Did you mean `find_symbol`?**" — Levenshtein ≤ 2. +- "flag `--namepath` not recognized. **Did you mean `--name_path_pattern`?**" +- "required flag `--query` missing. **Example:** `mcpx serena search_for_pattern --query 'TODO'`" — uses `--example` machinery. + +--- + +## Implementation order (locked) + +| # | Item | Effort | Lands as | +|---|------|--------|----------| +| 1 | F3 config additions | XS | infra | +| 2 | F1 stats package + JSONL writer | S | infra | +| 3 | F4 instrumentation hook in `runTool` | S | infra | +| 4 | F2 schema cache | M | infra | +| 5 | R1 schema normalizer + issue #14 fix | M | robustness | +| 6 | R2 structured exit codes | XS | robustness | +| 7 | A1 `mcpx find` | M | agent | +| 8 | A2 result cache | S | agent | +| 9 | A3 `mcpx batch` | M | agent | +| 10 | A4 default-compact help | XS | agent | +| 11 | A5 `--example` + validate | S | agent | +| 12 | R3 error remediation | S | polish | +| 13 | O1 `mcpx gain` premium TUI + `internal/render` | L | operator | +| 14 | O2 UI daemon supervisor + lazy spawn | M | operator | +| 15 | O3 dashboard HTML/SSE/static assets | L | operator | +| 16 | tiktoken-go via `+tiktoken` build tag | S | v1.6.1 | + +XS = <0.5d, S = 1d, M = 2–4d, L = 5–10d. + +--- + +## Validation strategy + +- Each package ships with table-driven Go tests. +- Local install via `make install`; replace `~/go/bin/mcpx` after every milestone. +- Dogfood: every code change in v1.6 development uses the WIP mcpx via serena. Friction points become the next iteration's tasks. +- Keep `mcpx ping serena` green at every milestone (smoke test). +- Add `make e2e` that runs `mcpx serena ping` + `mcpx find ...` + `mcpx gain --json` against a known-good config. + +--- + +## Out of scope for v1.6 + +Defer to v1.7+: +- Rate limiting enforcement +- Output filtering / redaction +- Multi-agent identity beyond `MCPX_AGENT` env +- `mcpx sync` (.mcp.json bidirectional) +- Notification handler (real-time MCP events) +- Docker runtime +- `mcpx install ` registry +- tiktoken-go via build tag + +These are good ideas but adding them dilutes v1.6's coherent story. + +--- + +## v1.6 final scope (shipped) + +**Foundation** +- F1 stats package (writer, reader, aggregator, ID generator, result preview cap) +- F2 schema cache + result cache + idempotence detection +- F3 config (gain/cache/ui sections + Default helpers) +- F4 instrumentation (every tool call writes a JSONL record with full args + truncated result preview) + +**Robustness** +- R1 schema normalizer (issue #14: type union arrays + oneOf/anyOf/allOf/Ext) +- R2 structured exit codes (0–6) +- R3 typo remediation (Levenshtein on tool names + flag names) + +**Agent superpowers** +- A1 `mcpx find` — BM25 ranked tool discovery +- A2 result cache (heuristic + allow-list) +- A3 `mcpx batch` — NDJSON in/out, parallel by default, client pool +- A4 default-compact help (verbose behind `--full`) +- A5 `--example` (JSON skeleton from normalized schema) + `--validate-args` + +**Operator surfaces** +- O1 `mcpx gain` premium TUI: hero metric, sparkline, top-tools bars, top-savers, server p95, last calls, dashboard URL footer; subcommands `--by tool|server|day`, `--history`, `--suggest`, `--watch`, `--json`, `--all`, `--project`, `--since` +- O2 always-on dashboard daemon (lazy spawn, token-protected, idle-shutdown 1h, `MCPX_UI=off` opt-out, `mcpx ui status|stop|open|disable`) +- O3 redesigned single-page dashboard: project sidebar, time range tabs (1h/24h/7d/30d/all), hero metric (64px), token efficiency bar, 4 stat tiles, top-tools table (clickable for drill-down), top-savers, per-server health cards (status/calls/p95/err), live tail with regex filter (`/`), click-to-inspect drawer (full args + result preview + replay button), SSE connection indicator, freshness label + +**Operational** +- `mcpx doctor` — config, command, daemon, initialize, tools/list, secret resolution checks; `--json` mode +- Dirty-build warning (`mcpx version` flags `+dirty`) +- 276 tests across 13 packages; `go vet` clean + +**Friction surfaced via dogfooding** (in `.dogfood/v16-friction.md`) +- serena `replace_symbol_body` corrupts file when body includes leading keyword +- Stale LSP diagnostics persist across edits +- /tmp scratch files trigger phantom diagnostics +- Dirty builds silently propagate stale debug code +- Notice gating per-command (resolved) +- Memory file index format violation (resolved) diff --git a/README.md b/README.md index d427240..d8c9764 100644 --- a/README.md +++ b/README.md @@ -281,24 +281,33 @@ mcpx ping serena # health check with latency ## CLI Reference ``` -mcpx [flags] Call a tool -mcpx --help Show tool help -mcpx --stdin Read args from stdin JSON -mcpx --help List all tools - -mcpx info Server capabilities -mcpx prompt list| Prompts -mcpx resource list|read Resources - -mcpx list List servers -mcpx list -v List tools with flags -mcpx ping Health check -mcpx init Import .mcp.json - -mcpx secret set|list|remove Keychain secrets -mcpx daemon status|stop|stop-all Daemon management -mcpx version Print version -mcpx completion bash|zsh|fish Shell completions +mcpx [flags] Call a tool +mcpx --help Show tool help +mcpx --stdin Read args from stdin JSON +mcpx --example JSON skeleton for the tool's args +mcpx --validate-args ... Type-check args without invoking +mcpx --help List all tools (compact); --full for descriptions + +mcpx info Server capabilities +mcpx prompt list| Prompts +mcpx resource list|read Resources + +mcpx find Rank tools across servers by intent (BM25) +mcpx batch < calls.jsonl Run many calls in parallel (NDJSON in/out) +mcpx gain Token-savings dashboard (TUI) +mcpx doctor Config + connectivity diagnostics +mcpx ui status|stop|open|disable Web dashboard daemon controls + +mcpx list List servers +mcpx list -v List tools with flags +mcpx ping Health check +mcpx init Import .mcp.json +mcpx configure Generate agent reference docs (MCPX.md + per-server) + +mcpx secret set|list|remove Keychain secrets +mcpx daemon status|stop|stop-all Daemon management +mcpx version Print version +mcpx completion bash|zsh|fish Shell completions ``` ### Global flags @@ -319,6 +328,11 @@ mcpx completion bash|zsh|fish Shell completions | 1 | Tool error | | 2 | Config error | | 3 | Connection error | +| 4 | Timeout | +| 5 | Policy denied | +| 6 | Tool not found | +| 2 | Config error | +| 3 | Connection error | ## Architecture diff --git a/ROADMAP.md b/ROADMAP.md index ad86c5e..9fcf95f 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -2,7 +2,29 @@ > What's next for mcpx — focused on impact, not infrastructure. -mcpx v1.5.0 transforms mcpx from a CLI proxy into a secure gateway for MCP servers. Security policies, audit logging, and scoped daemons make mcpx ready for teams. +mcpx v1.6.0 ("Agentic Supremacy") makes the proxy measurable, observable, and intelligent: every call is recorded, every schema is cached, every tool is one `mcpx find` away, and an always-on dashboard makes ROI visible. + +--- + +## v1.6 — Agentic Supremacy ✅ (shipped 2026-05-03) + +**Theme:** Make AI agents faster, cheaper, more accurate. + +### Shipped + +- **`mcpx find `** — BM25-ranked tool discovery across all servers (~80 tokens vs 5–15K for `list -v`). +- **`mcpx batch`** — NDJSON in/out, parallel execution, single client per server reused. +- **`mcpx gain`** — premium terminal dashboard for tokens-saved analytics; subcommands for slicing by tool/server/day. +- **Always-on web dashboard** — auto-spawned, token-protected, single-page UI with live tail and click-to-inspect drawer. +- **`mcpx doctor`** — config + connectivity diagnostics. +- **Schema cache + result cache** — `tools/list` cached with TTL; idempotent reads (`get_*`/`list_*`/`find_*`/`search_*`/`read_*`) deduplicate. +- **Tool argument helpers** — `--example` for JSON skeleton, `--validate-args` for type-check without invoking, default-compact help (`--full` for verbose). +- **Schema normalizer** — handles JSON Schema union types, `oneOf`/`anyOf`/`allOf`, `$ref`. Fixes #14 (Sentry MCP server initialization). +- **Edit-tool guards** — pre-call warning + post-call Go parse check for `replace_symbol_body` and friends across 9 languages. +- **Typo remediation** — Levenshtein-2 "did you mean…" suggestions on tool/flag errors. +- **Structured exit codes** — `0` ok · `1` tool error · `2` config · `3` connection · `4` timeout · `5` policy denied · `6` tool not found. +- **JSONL stats** at `~/.mcpx/stats.jsonl` — every call recorded for `gain`, the dashboard, and audit. +- **Configure rewrite** — agent-grounded `MCPX.md` + per-server `.md` with tool-selector tables. --- diff --git a/go.mod b/go.mod index fbc7339..9ace700 100644 --- a/go.mod +++ b/go.mod @@ -1,11 +1,12 @@ module github.com/codestz/mcpx -go 1.26.1 +go 1.26.2 require ( github.com/fatih/color v1.18.0 github.com/spf13/cobra v1.10.2 github.com/zalando/go-keyring v0.2.6 + golang.org/x/term v0.42.0 gopkg.in/yaml.v3 v3.0.1 ) @@ -17,5 +18,5 @@ require ( github.com/mattn/go-colorable v0.1.13 // indirect github.com/mattn/go-isatty v0.0.20 // indirect github.com/spf13/pflag v1.0.9 // indirect - golang.org/x/sys v0.26.0 // indirect + golang.org/x/sys v0.43.0 // indirect ) diff --git a/go.sum b/go.sum index d66ab87..7faa831 100644 --- a/go.sum +++ b/go.sum @@ -34,8 +34,10 @@ github.com/zalando/go-keyring v0.2.6/go.mod h1:2TCrxYrbUNYfNS/Kgy/LSrkSQzZ5UPVH8 go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg= golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.26.0 h1:KHjCJyddX0LoSTb3J+vWpupP9p0oznkqVk/IfjymZbo= -golang.org/x/sys v0.26.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= +golang.org/x/sys v0.43.0 h1:Rlag2XtaFTxp19wS8MXlJwTvoh8ArU6ezoyFsMyCTNI= +golang.org/x/sys v0.43.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw= +golang.org/x/term v0.42.0 h1:UiKe+zDFmJobeJ5ggPwOshJIVt6/Ft0rcfrXZDLWAWY= +golang.org/x/term v0.42.0/go.mod h1:Dq/D+snpsbazcBG5+F9Q1n2rXV8Ma+71xEjTRufARgY= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= diff --git a/internal/cli/batch.go b/internal/cli/batch.go new file mode 100644 index 0000000..72a59a1 --- /dev/null +++ b/internal/cli/batch.go @@ -0,0 +1,330 @@ +package cli + +import ( + "bufio" + "context" + "encoding/json" + "fmt" + "io" + "os" + "runtime" + "strconv" + "sync" + "time" + + "github.com/codestz/mcpx/internal/config" + "github.com/codestz/mcpx/internal/mcp" + "github.com/codestz/mcpx/internal/stats" + "github.com/spf13/cobra" +) + +// BatchEntry is one input line in the NDJSON batch. +type BatchEntry struct { + ID string `json:"id,omitempty"` + Server string `json:"server"` + Tool string `json:"tool"` + Args map[string]any `json:"args"` +} + +// BatchResult is one output line, emitted in the same order as inputs. +type BatchResult struct { + ID string `json:"id"` + OK bool `json:"ok"` + LatencyMS int64 `json:"latency_ms"` + Result *mcp.CallResult `json:"result,omitempty"` + Error string `json:"error,omitempty"` + ExitCode int `json:"exit_code,omitempty"` +} + +func batchCmd(opts *globalOpts) *cobra.Command { + var ( + parallel bool + sequential bool + maxConcurrent int + stopOnError bool + ) + + cmd := &cobra.Command{ + Use: "batch", + Short: "Run many tool calls from NDJSON stdin, in parallel by default", + Long: `Reads NDJSON from stdin (one JSON object per line, with fields server/tool/args/id) +and runs each call. Emits NDJSON to stdout in the same order as inputs. + +One MCP client per server is reused across the whole batch — no handshake-per-call. + +Examples: + cat calls.jsonl | mcpx batch + cat calls.jsonl | mcpx batch --max-concurrent 4 + mcpx find "search" --json | jq -c '...' | mcpx batch`, + RunE: func(cmd *cobra.Command, args []string) error { + if sequential { + parallel = false + } + if maxConcurrent <= 0 { + maxConcurrent = runtime.NumCPU() + } + + cfg, _, err := config.Load() + if err != nil { + return fmt.Errorf("config: %w", err) + } + initStats(cfg) + + entries, err := readBatch(os.Stdin) + if err != nil { + return err + } + + results := executeBatch(cmd.Context(), entries, cfg, parallel, maxConcurrent, stopOnError) + + enc := json.NewEncoder(os.Stdout) + for _, r := range results { + _ = enc.Encode(r) + } + // Optional summary to stderr (suppressed in --quiet). + if !opts.quiet { + printBatchSummary(results) + } + return nil + }, + } + cmd.Flags().BoolVar(¶llel, "parallel", true, "Run calls concurrently (default)") + cmd.Flags().BoolVar(&sequential, "sequential", false, "Run calls one at a time") + cmd.Flags().IntVar(&maxConcurrent, "max-concurrent", 0, "Max concurrent calls (0 = NumCPU)") + cmd.Flags().BoolVar(&stopOnError, "stop-on-error", false, "Abort the batch on first error") + return cmd +} + +// readBatch parses NDJSON entries from r. Blank lines are skipped. IDs are +// auto-assigned when missing so the output stream stays addressable. +func readBatch(r io.Reader) ([]BatchEntry, error) { + scanner := bufio.NewScanner(r) + scanner.Buffer(make([]byte, 64*1024), 4*1024*1024) + var entries []BatchEntry + idx := 0 + for scanner.Scan() { + line := scanner.Bytes() + if len(line) == 0 { + continue + } + var e BatchEntry + if err := json.Unmarshal(line, &e); err != nil { + return nil, fmt.Errorf("batch: line %d: %w", idx+1, err) + } + if e.Server == "" || e.Tool == "" { + return nil, fmt.Errorf("batch: line %d: server and tool are required", idx+1) + } + if e.ID == "" { + e.ID = strconv.Itoa(idx) + } + entries = append(entries, e) + idx++ + } + return entries, scanner.Err() +} + +// executeBatch runs entries against per-server clients. Per-server clients are +// created lazily and closed at the end. Parallel mode bounds concurrency by +// maxConcurrent. Returns results in the original input order. +func executeBatch(ctx context.Context, entries []BatchEntry, cfg *config.Config, parallel bool, maxConcurrent int, stopOnError bool) []BatchResult { + results := make([]BatchResult, len(entries)) + clients := newClientPool(cfg) + defer clients.closeAll() + + type job struct{ idx int } + + exec := func(j job) { + e := entries[j.idx] + results[j.idx] = runBatchEntry(ctx, e, clients) + } + + if !parallel { + for i := range entries { + exec(job{i}) + if stopOnError && !results[i].OK { + // Mark remaining as skipped. + for k := i + 1; k < len(entries); k++ { + results[k] = BatchResult{ID: entries[k].ID, OK: false, Error: "skipped (stop-on-error)"} + } + return results + } + } + return results + } + + sem := make(chan struct{}, maxConcurrent) + var wg sync.WaitGroup + for i := range entries { + wg.Add(1) + sem <- struct{}{} + go func(i int) { + defer wg.Done() + defer func() { <-sem }() + exec(job{i}) + }(i) + } + wg.Wait() + return results +} + +func runBatchEntry(ctx context.Context, e BatchEntry, pool *clientPool) BatchResult { + start := time.Now() + res := BatchResult{ID: e.ID} + + defer func() { + preview, truncated := "", false + if res.Result != nil { + preview, truncated = stats.TruncateForPreview(extractText(res.Result)) + } + recordStats(stats.Record{ + ID: stats.NewID(), + Server: e.Server, + Tool: e.Tool, + Args: e.Args, + ArgsBytes: jsonBytes(e.Args), + ResponseBytes: jsonBytes(res.Result), + ResultPreview: preview, + ResultTruncated: truncated, + LatencyMS: time.Since(start).Milliseconds(), + ExitCode: res.ExitCode, + Error: res.Error, + Daemon: pool.daemon(e.Server), + Transport: pool.transport(e.Server), + PolicyAction: "allowed", + }) + }() + + if _, err := pool.findTool(ctx, e.Server, e.Tool); err != nil { + res.Error = err.Error() + res.ExitCode = exitToolNotFound + res.LatencyMS = time.Since(start).Milliseconds() + return res + } + + client, err := pool.client(ctx, e.Server) + if err != nil { + res.Error = err.Error() + res.ExitCode = exitConnectErr + res.LatencyMS = time.Since(start).Milliseconds() + return res + } + + result, err := client.CallTool(ctx, e.Tool, e.Args) + res.LatencyMS = time.Since(start).Milliseconds() + if err != nil { + res.Error = err.Error() + res.ExitCode = exitToolError + return res + } + res.OK = true + res.Result = result + return res +} + +// printBatchSummary writes an end-of-batch human summary to stderr. +func printBatchSummary(results []BatchResult) { + ok, fail := 0, 0 + var totalMS int64 + for _, r := range results { + if r.OK { + ok++ + } else { + fail++ + } + totalMS += r.LatencyMS + } + fmt.Fprintf(os.Stderr, "\nbatch: %d ok, %d fail, total %dms (sum)\n", ok, fail, totalMS) +} + +// clientPool manages one MCP client per server name across the batch lifetime. +// Lazy connect on first use, reused for subsequent calls, closed on closeAll. +type clientPool struct { + cfg *config.Config + mu sync.Mutex + clients map[string]*pooledClient +} + +type pooledClient struct { + c *mcp.Client + clean func() + tools []mcp.Tool + toolMu sync.Mutex +} + +func newClientPool(cfg *config.Config) *clientPool { + return &clientPool{cfg: cfg, clients: map[string]*pooledClient{}} +} + +func (p *clientPool) client(ctx context.Context, server string) (*mcp.Client, error) { + p.mu.Lock() + pc, ok := p.clients[server] + p.mu.Unlock() + if ok { + return pc.c, nil + } + + sc, ok := p.cfg.Servers[server] + if !ok { + return nil, fmt.Errorf("server %q not in config", server) + } + c, clean, err := connectServer(ctx, server, sc) + if err != nil { + return nil, err + } + p.mu.Lock() + p.clients[server] = &pooledClient{c: c, clean: clean} + p.mu.Unlock() + return c, nil +} + +func (p *clientPool) findTool(ctx context.Context, server, name string) (*mcp.Tool, error) { + c, err := p.client(ctx, server) + if err != nil { + return nil, err + } + p.mu.Lock() + pc := p.clients[server] + p.mu.Unlock() + + pc.toolMu.Lock() + defer pc.toolMu.Unlock() + if pc.tools == nil { + sc := p.cfg.Servers[server] + tools, _, _, terr := listToolsCached(ctx, server, sc, c) + if terr != nil { + return nil, terr + } + pc.tools = tools + } + for i := range pc.tools { + if pc.tools[i].Name == name { + return &pc.tools[i], nil + } + } + return nil, fmt.Errorf("tool %q not in server %q", name, server) +} + +func (p *clientPool) daemon(server string) bool { + if sc, ok := p.cfg.Servers[server]; ok { + return sc.Daemon + } + return false +} + +func (p *clientPool) transport(server string) string { + if sc, ok := p.cfg.Servers[server]; ok { + return sc.Transport + } + return "" +} + +func (p *clientPool) closeAll() { + p.mu.Lock() + defer p.mu.Unlock() + for _, pc := range p.clients { + if pc.clean != nil { + pc.clean() + } + } +} + diff --git a/internal/cli/cache_helpers.go b/internal/cli/cache_helpers.go new file mode 100644 index 0000000..8d4ed3a --- /dev/null +++ b/internal/cli/cache_helpers.go @@ -0,0 +1,93 @@ +package cli + +import ( + "context" + "time" + + "github.com/codestz/mcpx/internal/config" + "github.com/codestz/mcpx/internal/mcp" + "github.com/codestz/mcpx/internal/schemacache" +) + +// listToolsCached returns a server's tools from the schema cache when fresh, +// otherwise calls the live MCP client and persists the response. +// +// Returns the tools, whether the result came from cache, the precomputed +// native_baseline_tokens for this server, and any error from the live call. +func listToolsCached(ctx context.Context, serverName string, sc *config.ServerConfig, client *mcp.Client) ([]mcp.Tool, bool, int, error) { + if schemacache.Bypass() { + return fetchAndStore(ctx, serverName, sc, client) + } + + resolvedArgs, resolvedEnv, _ := resolveServerConfig(sc) + key := schemacache.Key(sc.Command, resolvedArgs, resolvedEnv, sc.URL) + + entry, hit, _ := schemacache.Load(key) + if hit { + return entry.Tools, true, entry.NativeBaselineToks, nil + } + + tools, err := client.ListTools(ctx) + if err != nil { + return nil, false, 0, err + } + storeEntry(key, sc, client, tools) + if e, _, _ := schemacache.Load(key); e != nil { + return tools, false, e.NativeBaselineToks, nil + } + return tools, false, 0, nil +} + +func fetchAndStore(ctx context.Context, serverName string, sc *config.ServerConfig, client *mcp.Client) ([]mcp.Tool, bool, int, error) { + tools, err := client.ListTools(ctx) + if err != nil { + return nil, false, 0, err + } + resolvedArgs, resolvedEnv, _ := resolveServerConfig(sc) + key := schemacache.Key(sc.Command, resolvedArgs, resolvedEnv, sc.URL) + storeEntry(key, sc, client, tools) + if e, _, _ := schemacache.Load(key); e != nil { + return tools, false, e.NativeBaselineToks, nil + } + return tools, false, 0, nil +} + +func storeEntry(key string, sc *config.ServerConfig, client *mcp.Client, tools []mcp.Tool) { + ttl := schemaTTL(sc) + caps := client.ServerCapabilities() + entry := &schemacache.Entry{ + TTL: ttl, + Initialize: mcp.InitializeResult{ + ProtocolVersion: client.ProtocolVersion(), + Capabilities: caps, + ServerInfo: client.ServerInfo(), + }, + Tools: tools, + } + // Best-effort prompts/resources fetch — failures don't block. + if caps.Prompts != nil { + if p, err := client.ListPrompts(context.Background()); err == nil { + entry.Prompts = p + } + } + if caps.Resources != nil { + if r, err := client.ListResources(context.Background()); err == nil { + entry.Resources = r + } + } + _ = schemacache.Save(key, entry) +} + +// schemaTTL resolves the schema cache TTL for a server. Defaults to 5m. +func schemaTTL(_ *config.ServerConfig) time.Duration { + cfg, _, err := config.Load() + if err != nil || cfg == nil { + return 5 * time.Minute + } + cc := cfg.Cache.Default() + d, err := time.ParseDuration(cc.SchemaTTL) + if err != nil { + return 5 * time.Minute + } + return d +} diff --git a/internal/cli/commands.go b/internal/cli/commands.go index 3c26472..92a562e 100644 --- a/internal/cli/commands.go +++ b/internal/cli/commands.go @@ -17,6 +17,7 @@ import ( "github.com/codestz/mcpx/internal/resolver" "github.com/codestz/mcpx/internal/secret" "github.com/codestz/mcpx/internal/security" + "github.com/codestz/mcpx/internal/stats" "github.com/spf13/cobra" ) @@ -53,6 +54,8 @@ func buildServerCommand(name string, sc *config.ServerConfig, globalSec *config. i++ opts.timeout = args[i] } + case "--full": + opts.full = true case "--help", "-h": hasHelp = true default: @@ -165,16 +168,54 @@ func showServerHelp(ctx context.Context, serverName string, sc *config.ServerCon resources, _ = client.ListResources(ctx) } - return out.printServerHelpFull(serverName, sc, tools, prompts, resources) + if opts.full { + return out.printServerHelpFull(serverName, sc, tools, prompts, resources) + } + return out.printServerHelpCompact(serverName, sc, tools, prompts, resources) } // runTool connects to a server, finds the named tool, parses flags, and executes. -func runTool(ctx context.Context, serverName string, sc *config.ServerConfig, globalSec *config.SecurityConfig, toolName string, rawArgs []string, opts *globalOpts) error { +func runTool(ctx context.Context, serverName string, sc *config.ServerConfig, globalSec *config.SecurityConfig, toolName string, rawArgs []string, opts *globalOpts) (retErr error) { out := newOutput(opts.outputMode()) - // Check for help or stdin mode. + projectRoot, _ := findProjectRoot() + rec := stats.Record{ + ID: stats.NewID(), + TS: time.Now().UTC(), + Project: projectRoot, + Server: serverName, + Tool: toolName, + Transport: sc.Transport, + Daemon: sc.Daemon, + } + start := time.Now() + defer func() { + rec.LatencyMS = time.Since(start).Milliseconds() + if retErr != nil { + rec.Error = retErr.Error() + if rec.ExitCode == 0 { + rec.ExitCode = 1 + } + } + // Tokens saved = native_baseline - args - response. baseline is filled + // in once schema cache (F2) is wired; left at 0 until then. + rec.ArgsTokensEst = (rec.ArgsBytes + 3) / 4 + rec.ResponseTokensEst = (rec.ResponseBytes + 3) / 4 + if rec.NativeBaselineToks > 0 { + saved := rec.NativeBaselineToks - rec.ArgsTokensEst - rec.ResponseTokensEst + if saved < 0 { + saved = 0 + } + rec.TokensSaved = saved + } + recordStats(rec) + }() + + // Check for help, stdin, --example, and --validate-args modes. wantHelp := false useStdin := false + wantExample := false + wantValidate := false var filteredArgs []string for _, a := range rawArgs { switch a { @@ -182,6 +223,10 @@ func runTool(ctx context.Context, serverName string, sc *config.ServerConfig, gl wantHelp = true case "--stdin": useStdin = true + case "--example": + wantExample = true + case "--validate-args": + wantValidate = true default: filteredArgs = append(filteredArgs, a) } @@ -193,10 +238,12 @@ func runTool(ctx context.Context, serverName string, sc *config.ServerConfig, gl } defer cleanup() - tools, err := client.ListTools(ctx) + tools, cacheHit, baseline, err := listToolsCached(ctx, serverName, sc, client) if err != nil { return fmt.Errorf("list tools: %w", err) } + rec.SchemaCacheHit = cacheHit + rec.NativeBaselineToks = baseline var tool *mcp.Tool for i := range tools { @@ -210,8 +257,15 @@ func runTool(ctx context.Context, serverName string, sc *config.ServerConfig, gl for _, t := range tools { names = append(names, t.Name) } - return fmt.Errorf("tool %q not found in server %q\nAvailable tools: %s\nRun: mcpx %s --help", - toolName, serverName, strings.Join(names, ", "), serverName) + rec.ExitCode = exitToolNotFound + hint := nearestTool(toolName, tools) + msg := fmt.Sprintf("tool %q not found in server %q", toolName, serverName) + if hint != "" { + msg += "\n " + hint + } + msg += fmt.Sprintf("\n Available: %s\n Run: mcpx %s --help", + strings.Join(names, ", "), serverName) + return fmt.Errorf("%s", msg) } if wantHelp { @@ -219,6 +273,11 @@ func runTool(ctx context.Context, serverName string, sc *config.ServerConfig, gl return nil } + if wantExample { + printExample(tool, opts.outputMode() == outputJSON) + return nil + } + // Parse arguments: either from stdin JSON (with optional flag merge) or from flags. var toolArgs map[string]any if useStdin { @@ -226,7 +285,6 @@ func runTool(ctx context.Context, serverName string, sc *config.ServerConfig, gl if err != nil { return fmt.Errorf("--stdin: %w", err) } - // Merge CLI flags on top (flags win). if len(filteredArgs) > 0 { flagArgs, flagErr := parseToolFlagsPartial(tool, filteredArgs) if flagErr != nil { @@ -236,7 +294,6 @@ func runTool(ctx context.Context, serverName string, sc *config.ServerConfig, gl toolArgs[k] = v } } - // Validate required fields against merged result. for _, req := range tool.InputSchema.Required { if _, ok := toolArgs[req]; !ok { return fmt.Errorf("required field %q not provided (via --stdin or flags)", req) @@ -248,6 +305,21 @@ func runTool(ctx context.Context, serverName string, sc *config.ServerConfig, gl return enhanceParseError(err, serverName, tool) } } + rec.ArgsBytes = jsonBytes(toolArgs) + rec.Args = toolArgs + + if wantValidate { + issues := validateArgs(tool, toolArgs) + if len(issues) == 0 { + fmt.Fprintln(os.Stdout, "ok") + return nil + } + for _, iss := range issues { + fmt.Fprintln(os.Stderr, iss.String()) + } + rec.ExitCode = exitToolError + return fmt.Errorf("%d validation issue(s)", len(issues)) + } if opts.dryRun { resolvedArgs, resolvedEnv, err := resolveServerConfig(sc) @@ -258,7 +330,6 @@ func runTool(ctx context.Context, serverName string, sc *config.ServerConfig, gl return nil } - // Apply per-call timeout if specified. callCtx := ctx if opts.timeout != "" { d, err := time.ParseDuration(opts.timeout) @@ -270,15 +341,16 @@ func runTool(ctx context.Context, serverName string, sc *config.ServerConfig, gl defer cancel() } - // Evaluate security policies before calling the tool. if globalSec != nil && globalSec.Enabled || sc.Security != nil { eval := security.NewEvaluator(serverName, globalSec, sc.Security) secResult := eval.Evaluate(toolName, toolArgs) + rec.PolicyAction = security.ActionString(secResult.Action) + rec.PolicyName = secResult.PolicyName switch secResult.Action { case security.ActionDeny: - // Log denial to audit if configured. logAudit(globalSec, serverName, toolName, toolArgs, secResult) + rec.ExitCode = exitPolicyDenied msg := fmt.Sprintf("server %q: policy %q denied tool %q\n Reason: %s", serverName, secResult.PolicyName, toolName, secResult.Message) if secResult.Details != "" { @@ -293,12 +365,26 @@ func runTool(ctx context.Context, serverName string, sc *config.ServerConfig, gl } } + preCallEditGuard(serverName, toolName, toolArgs) + result, err := client.CallTool(callCtx, toolName, toolArgs) if err != nil { return err } + rec.ResponseBytes = jsonBytes(result) + if result != nil { + preview, truncated := stats.TruncateForPreview(extractText(result)) + rec.ResultPreview = preview + rec.ResultTruncated = truncated + } + + if err := postCallEditGuard(serverName, toolName, toolArgs, projectRoot); err != nil { + // Surface as warning + non-zero exit. The edit went through (we already + // have a result); the file is broken so the agent should know now. + fmt.Fprintf(os.Stderr, "mcpx: %v\n", err) + rec.ExitCode = exitToolError + } - // Extract a specific field if --pick was specified. if opts.pick != "" { val, err := pickField(result, opts.pick) if err != nil { @@ -512,10 +598,24 @@ func parseStdinJSON() (map[string]any, error) { } // enhanceParseError adds flag hints to parse errors (e.g. missing required flags). +// Also surfaces a "did you mean" suggestion when the unknown flag is close to a known one. func enhanceParseError(err error, serverName string, tool *mcp.Tool) error { msg := err.Error() - // Build flag summary for the hint. + // Try to extract typo from "unknown flag: --xxx" or similar. + suggestion := "" + if i := strings.Index(msg, "--"); i >= 0 { + typo := msg[i+2:] + if j := strings.IndexAny(typo, " \n\t"); j >= 0 { + typo = typo[:j] + } + if typo != "" { + if hint := nearestProperty(typo, tool.InputSchema.Properties); hint != "" { + suggestion = " " + hint + } + } + } + required := make(map[string]bool) for _, r := range tool.InputSchema.Required { required[r] = true @@ -532,8 +632,13 @@ func enhanceParseError(err error, serverName string, tool *mcp.Tool) error { flags = append(flags, entry) } - return fmt.Errorf("%s\n\nAvailable flags for %s:\n %s\n\nRun: mcpx %s %s --help", - msg, tool.Name, strings.Join(flags, "\n "), serverName, tool.Name) + out := msg + if suggestion != "" { + out += "\n" + suggestion + } + out += fmt.Sprintf("\n\nAvailable flags for %s:\n %s\n\nRun: mcpx %s %s --help", + tool.Name, strings.Join(flags, "\n "), serverName, tool.Name) + return fmt.Errorf("%s", out) } // parseToolFlags builds flags from a tool's JSON schema and parses rawArgs. @@ -935,3 +1040,23 @@ func findProjectRoot() (string, error) { dir = parent } } +func extractText(r *mcp.CallResult) string { + if r == nil { + return "" + } + var b strings.Builder + for _, c := range r.Content { + switch c.Type { + case "text", "": + b.WriteString(c.Text) + case "image", "audio": + b.WriteString("[" + c.Type + "]") + case "resource": + if c.Resource != nil && c.Resource.Text != "" { + b.WriteString(c.Resource.Text) + } + } + b.WriteByte('\n') + } + return b.String() +} diff --git a/internal/cli/configure.go b/internal/cli/configure.go index ffe0c13..4341761 100644 --- a/internal/cli/configure.go +++ b/internal/cli/configure.go @@ -13,23 +13,22 @@ import ( func configureCmd() *cobra.Command { var global bool - var format string + var format string // accepted for backward compatibility; ignored. cmd := &cobra.Command{ Use: "configure", - Short: "Set up mcpx for Claude Code (creates MCPX.md + per-server docs)", - Long: `Configure mcpx for Claude Code integration. + Short: "Set up mcpx for Claude Code (writes MCPX.md + per-server docs)", + Long: `Generate the agent-facing reference docs that teach Claude Code (and any +mcpx-aware coding agent) how to use the configured MCP servers. -Creates MCPX.md (quick reference), connects to each configured server, -generates per-server tool docs (SERVER.md), and updates CLAUDE.md references. +Writes: + - MCPX.md composition + discover + edit-safety + exit codes + - .md tool-selector table + compact reference, per server + - CLAUDE.md adds @MCPX.md and @.md references -Formats: - default — tables with flag/type/required/description columns - compact — one line per tool, flags inline (smaller, ~50% fewer tokens)`, +Idempotent — re-run after adding/removing servers.`, RunE: func(cmd *cobra.Command, args []string) error { - if format != "default" && format != "compact" { - return fmt.Errorf("invalid format %q: must be 'default' or 'compact'", format) - } + _ = format // legacy flag cfg, _, err := config.Load() if err != nil { @@ -78,93 +77,75 @@ Formats: if global { scope = "global" } - fmt.Printf("\nDone! Claude Code will load mcpx references at %s scope (format: %s).\n", scope, format) + fmt.Printf("\nDone! Claude Code will load mcpx references at %s scope.\n", scope) return nil }, } cmd.Flags().BoolVar(&global, "global", false, "Configure globally (~/.claude/) instead of project (.claude/)") - cmd.Flags().StringVar(&format, "format", "default", "Doc format: 'default' (tables) or 'compact' (one-line per tool)") + cmd.Flags().StringVar(&format, "format", "default", "Deprecated; format is now agent-optimized by default") + _ = cmd.Flags().MarkHidden("format") return cmd } -// generateMCPXMD creates the content for MCPX.md. +// generateMCPXMD creates the content for MCPX.md — the always-loaded reference +// an AI agent uses to call mcpx-wrapped MCP servers. +// +// Designed for a coding agent reading the file at session start. Tabular, +// composition-first, no human-ops content (caching internals, observability, +// dashboard, doctor — all excluded; they're available via mcpx commands when +// the human needs them, but they don't help the agent pick a tool). func generateMCPXMD(cfg *config.Config) string { var b strings.Builder - b.WriteString("# mcpx — MCP Server CLI Proxy\n\n") - b.WriteString("mcpx wraps MCP servers into CLI tools. Call them via Bash instead of loading schemas into context.\n\n") - - b.WriteString("## Quick Reference\n\n") - b.WriteString("```bash\n") - b.WriteString("mcpx list # List configured servers\n") - b.WriteString("mcpx list -v # List all tools with flags\n") - b.WriteString("mcpx --help # Show server tools\n") - b.WriteString("mcpx --help # Show tool flags\n") - b.WriteString("mcpx --flags # Call a tool\n") - b.WriteString("mcpx --stdin # Read args from stdin JSON\n") - b.WriteString("mcpx --json # Output raw JSON\n") - b.WriteString("mcpx daemon status # Show running daemons\n") - b.WriteString("mcpx info # Show server capabilities\n") - b.WriteString("mcpx prompt list # List available prompts\n") - b.WriteString("mcpx prompt --args # Get a prompt\n") - b.WriteString("mcpx resource list # List available resources\n") - b.WriteString("mcpx resource read # Read a resource\n") - b.WriteString("```\n\n") - - b.WriteString("## Configured Servers\n\n") + b.WriteString("# mcpx\n\n") + b.WriteString("Call MCP tools through `mcpx --flags`. ") + b.WriteString("Don't load native MCP — use the CLI commands below.\n\n") + + b.WriteString("## Servers\n\n") if len(cfg.Servers) == 0 { - b.WriteString("No servers configured. Run `mcpx init` to import from `.mcp.json`.\n") + b.WriteString("(none — run `mcpx init`)\n\n") } else { for name, sc := range cfg.Servers { - b.WriteString(fmt.Sprintf("- **%s** — `%s`", name, sc.Command)) + b.WriteString(fmt.Sprintf("- **%s**", name)) if sc.Daemon { - b.WriteString(" (daemon)") + b.WriteString(" *(daemon)*") + } + if sc.Transport != "" && sc.Transport != "stdio" { + b.WriteString(fmt.Sprintf(" *(%s)*", sc.Transport)) } b.WriteString("\n") } + b.WriteString("\n") } - b.WriteString("\n## Usage Pattern\n\n") - b.WriteString("1. Discover: `mcpx --help` to see available tools\n") - b.WriteString("2. Inspect: `mcpx --help` to see flags\n") - b.WriteString("3. Call: `mcpx --flag value`\n") - b.WriteString("4. For long args: `printf '{\"key\":\"value\"}' | mcpx --stdin`\n") - - b.WriteString("\n## Large Content: @file syntax\n\n") - b.WriteString("Any string flag accepts `@/path` to read from a file or `@-`/`-` to read from stdin:\n") - b.WriteString("```bash\n") - b.WriteString("mcpx --body @/tmp/code.go # Read file into --body\n") - b.WriteString("mcpx --body @- # Read stdin into --body\n") - b.WriteString("mcpx --body - # Same (backward compat)\n") - b.WriteString("```\n") - - b.WriteString("\n## Output Extraction: --pick\n\n") - b.WriteString("Extract a JSON field from the result without jq:\n") - b.WriteString("```bash\n") - b.WriteString("mcpx --pick field.path # Dot-separated path\n") - b.WriteString("mcpx --pick items.0.name # Array index access\n") - b.WriteString("```\n") - - b.WriteString("\n## Timeout Override: --timeout\n\n") - b.WriteString("Override the default call timeout for a single invocation:\n") - b.WriteString("```bash\n") - b.WriteString("mcpx --timeout 60s # Go duration format\n") - b.WriteString("```\n") - - b.WriteString("\n## Stdin Merge\n\n") - b.WriteString("`--stdin` can be combined with CLI flags. Flags win on conflict:\n") - b.WriteString("```bash\n") - b.WriteString("echo '{\"body\":\"content\"}' | mcpx --stdin --name_path Foo\n") - b.WriteString("```\n") - - b.WriteString("\n## Tips for AI Agents\n\n") - b.WriteString("- Use `--body @/tmp/file` for large content to avoid shell escaping\n") - b.WriteString("- Use `--pick field` instead of piping through jq for single fields\n") - b.WriteString("- Combine `--stdin` with flags for mixed large+small arguments\n") - b.WriteString("- Use `--timeout 120s` for long-running operations\n") + b.WriteString("## Compose\n\n") + b.WriteString("| Need | How |\n") + b.WriteString("|---|---|\n") + b.WriteString("| Standard call | `mcpx --flag value` |\n") + b.WriteString("| Large arg from file | `--body @/path/to/file` |\n") + b.WriteString("| Read body from stdin | `--body @-` or `--body -` |\n") + b.WriteString("| Pass full args as JSON | `printf '{...}' \\| mcpx --stdin` |\n") + b.WriteString("| Mix stdin + flags | `--stdin --flag value` (flags win) |\n") + b.WriteString("| Extract one JSON field | `--pick path.to.field` |\n") + b.WriteString("| Raw JSON output | `--json` |\n") + b.WriteString("| Per-call timeout | `--timeout 60s` (Go duration) |\n") + b.WriteString("| Show resolved command | `--dry-run` |\n") + b.WriteString("| Args skeleton | `mcpx --example` |\n") + b.WriteString("| Type-check args | `mcpx --validate-args ...` |\n\n") + + b.WriteString("## Discover\n\n") + b.WriteString("| Need | How |\n") + b.WriteString("|---|---|\n") + b.WriteString("| Find the right tool by intent | `mcpx find \"\"` |\n") + b.WriteString("| One-line list of a server's tools | `mcpx --help` |\n") + b.WriteString("| Full schema for one tool | `mcpx --help` |\n") + b.WriteString("| Run many tool calls in parallel | `mcpx batch < calls.jsonl` |\n\n") + + b.WriteString("## Exit codes\n\n") + b.WriteString("`0` ok · `1` tool error · `2` config · `3` connection · `4` timeout · `5` policy denied · `6` tool not found.\n") return b.String() } diff --git a/internal/cli/doctor.go b/internal/cli/doctor.go new file mode 100644 index 0000000..e8d0021 --- /dev/null +++ b/internal/cli/doctor.go @@ -0,0 +1,294 @@ +package cli + +import ( + "context" + "fmt" + "net/http" + "os" + "os/exec" + "strings" + "time" + + "github.com/codestz/mcpx/internal/config" + "github.com/codestz/mcpx/internal/daemon" + "github.com/codestz/mcpx/internal/resolver" + "github.com/codestz/mcpx/internal/secret" + "github.com/fatih/color" + "github.com/spf13/cobra" +) + +type checkLevel int + +const ( + checkOK checkLevel = iota + checkWarn + checkFail +) + +type checkResult struct { + Server string + Name string + Level checkLevel + Detail string +} + +func doctorCmd(opts *globalOpts) *cobra.Command { + cmd := &cobra.Command{ + Use: "doctor", + Short: "Diagnose mcpx configuration and connectivity", + Long: `Runs a series of checks against the mcpx config and every configured server. +Verifies syntax, command paths, transports, secret resolution, and daemons. + +Examples: + mcpx doctor # human table + mcpx doctor --json # machine-readable`, + RunE: func(cmd *cobra.Command, args []string) error { + results := runDoctor(cmd.Context()) + if opts.outputMode() == outputJSON { + out := newOutput(outputJSON) + return out.printJSON(results) + } + return printDoctor(results) + }, + } + return cmd +} + +// runDoctor performs every check and returns the result list. +func runDoctor(ctx context.Context) []checkResult { + var results []checkResult + + cfg, projectRoot, err := config.Load() + if err != nil { + return []checkResult{{Name: "config", Level: checkFail, Detail: err.Error()}} + } + results = append(results, checkResult{ + Name: "config", Level: checkOK, + Detail: fmt.Sprintf("loaded %d server(s)%s", len(cfg.Servers), projectScope(projectRoot)), + }) + + if cfg.Servers == nil || len(cfg.Servers) == 0 { + results = append(results, checkResult{ + Name: "servers", Level: checkWarn, + Detail: "no servers configured — add to .mcpx/config.yml or ~/.mcpx/config.yml", + }) + return results + } + + res := resolver.New(projectRoot, secret.NewKeyringStore()) + + for name, sc := range cfg.Servers { + results = append(results, checkServer(ctx, name, sc, res)...) + } + + return results +} + +func checkServer(ctx context.Context, name string, sc *config.ServerConfig, res *resolver.Resolver) []checkResult { + var out []checkResult + + switch sc.Transport { + case "stdio", "": + if sc.Command == "" { + out = append(out, checkResult{ + Server: name, Name: "command", Level: checkFail, + Detail: "missing required `command` field", + }) + return out + } + path, err := exec.LookPath(sc.Command) + if err != nil { + out = append(out, checkResult{ + Server: name, Name: "command", Level: checkFail, + Detail: fmt.Sprintf("%q not on $PATH", sc.Command), + }) + } else { + out = append(out, checkResult{ + Server: name, Name: "command", Level: checkOK, + Detail: path, + }) + } + case "http", "sse": + url, err := res.Resolve(sc.URL) + if err != nil { + out = append(out, checkResult{ + Server: name, Name: "url", Level: checkFail, + Detail: fmt.Sprintf("could not resolve url: %v", err), + }) + return out + } + client := &http.Client{Timeout: 3 * time.Second} + req, _ := http.NewRequestWithContext(ctx, "HEAD", url, nil) + resp, err := client.Do(req) + if err != nil { + out = append(out, checkResult{ + Server: name, Name: "url", Level: checkFail, + Detail: fmt.Sprintf("HEAD %s: %v", url, err), + }) + } else { + resp.Body.Close() + out = append(out, checkResult{ + Server: name, Name: "url", Level: checkOK, + Detail: fmt.Sprintf("%s — %d", url, resp.StatusCode), + }) + } + } + + // Check secret resolution for auth + headers (without leaking values). + for _, ref := range collectSecretRefs(sc) { + if _, err := res.Resolve(ref); err != nil { + out = append(out, checkResult{ + Server: name, Name: "secret", Level: checkFail, + Detail: fmt.Sprintf("%s: %v", ref, err), + }) + } else { + out = append(out, checkResult{ + Server: name, Name: "secret", Level: checkOK, + Detail: ref + " resolved", + }) + } + } + + // Daemon liveness when daemon=true. + if sc.Daemon { + scope := daemonScope() + if daemon.IsRunning(name, scope) { + out = append(out, checkResult{ + Server: name, Name: "daemon", Level: checkOK, + Detail: "running, socket reachable", + }) + } else { + out = append(out, checkResult{ + Server: name, Name: "daemon", Level: checkWarn, + Detail: "no daemon yet — first call will spawn one", + }) + } + } + + // Live initialize check (3s budget). + cctx, cancel := context.WithTimeout(ctx, 3*time.Second) + defer cancel() + client, cleanup, err := connectServer(cctx, name, sc) + if err != nil { + out = append(out, checkResult{ + Server: name, Name: "initialize", Level: checkFail, + Detail: shortErr(err), + }) + return out + } + defer cleanup() + out = append(out, checkResult{ + Server: name, Name: "initialize", Level: checkOK, + Detail: client.ServerInfo().Name + " " + client.ServerInfo().Version, + }) + + tools, err := client.ListTools(cctx) + if err != nil { + out = append(out, checkResult{ + Server: name, Name: "tools/list", Level: checkWarn, + Detail: shortErr(err), + }) + } else { + out = append(out, checkResult{ + Server: name, Name: "tools/list", Level: checkOK, + Detail: fmt.Sprintf("%d tool(s)", len(tools)), + }) + } + + return out +} + +// collectSecretRefs walks a server config for $(secret.*) references. +func collectSecretRefs(sc *config.ServerConfig) []string { + var refs []string + scan := func(s string) { + if strings.Contains(s, "$(secret.") { + refs = append(refs, s) + } + } + if sc.Auth != nil { + scan(sc.Auth.Token) + } + for _, v := range sc.Headers { + scan(v) + } + return refs +} + +func projectScope(root string) string { + if root == "" { + return ", global only" + } + return ", project = " + root +} + +func shortErr(err error) string { + s := err.Error() + if len(s) > 80 { + return s[:77] + "…" + } + return s +} + +func printDoctor(results []checkResult) error { + bold := color.New(color.Bold) + dim := color.New(color.FgHiBlack) + green := color.New(color.FgGreen) + yellow := color.New(color.FgYellow) + red := color.New(color.FgRed) + + fmt.Println() + bold.Println(" mcpx doctor") + dim.Println(" ─────────────────────────────────────────────────────") + + currServer := "(global)" + for _, r := range results { + srv := r.Server + if srv == "" { + srv = "(global)" + } + if srv != currServer { + fmt.Println() + bold.Print(" " + srv + "\n") + currServer = srv + } + switch r.Level { + case checkOK: + green.Print(" [ok] ") + case checkWarn: + yellow.Print(" [warn] ") + case checkFail: + red.Print(" [fail] ") + } + fmt.Printf("%-14s ", r.Name) + dim.Println(r.Detail) + } + + fails := 0 + warns := 0 + for _, r := range results { + if r.Level == checkFail { + fails++ + } else if r.Level == checkWarn { + warns++ + } + } + fmt.Println() + dim.Println(" ─────────────────────────────────────────────────────") + if fails > 0 { + red.Printf(" %d failure(s)", fails) + if warns > 0 { + fmt.Printf(", ") + yellow.Printf("%d warning(s)", warns) + } + fmt.Println() + os.Exit(2) + } else if warns > 0 { + yellow.Printf(" %d warning(s)\n", warns) + } else { + green.Println(" all checks passed") + } + fmt.Println() + return nil +} + diff --git a/internal/cli/edit_guard.go b/internal/cli/edit_guard.go new file mode 100644 index 0000000..b539057 --- /dev/null +++ b/internal/cli/edit_guard.go @@ -0,0 +1,131 @@ +package cli + +import ( + "fmt" + "go/parser" + "go/token" + "os" + "path/filepath" + "strings" +) + +// bodyMutatingTools are MCP tools that overwrite a file region with caller-supplied +// body text. Adding new entries is safe: the guards no-op when arguments don't +// match what they expect. +var bodyMutatingTools = map[string]bool{ + "replace_symbol_body": true, + "insert_after_symbol": true, + "insert_before_symbol": true, +} + +// langKeywords maps file extensions to the set of declaration keywords whose +// presence at the start of a `replace_symbol_body` payload almost certainly +// indicates duplication of the existing symbol's keyword. +// +// Logic for each entry: serena preserves the symbol's ` ` prefix +// and only swaps in the body's content after it; if the body itself starts +// with one of these keywords the file ends up with ` Name`. +var langKeywords = map[string][]string{ + ".go": {"type ", "func "}, + ".py": {"def ", "async def ", "class "}, + ".pyi": {"def ", "async def ", "class "}, + ".ts": {"function ", "class ", "interface ", "type ", "enum "}, + ".tsx": {"function ", "class ", "interface ", "type ", "enum "}, + ".js": {"function ", "class "}, + ".jsx": {"function ", "class "}, + ".mjs": {"function ", "class "}, + ".rs": {"fn ", "struct ", "enum ", "trait ", "impl "}, + ".rb": {"def ", "class ", "module "}, + ".java": {"class ", "interface ", "enum ", "void ", "public ", "private ", "protected "}, + ".kt": {"fun ", "class ", "interface ", "object "}, + ".swift": {"func ", "class ", "struct ", "enum ", "protocol "}, +} + +// preCallEditGuard warns when arguments to a body-mutating tool match a known +// failure mode: body starts with the same declaration keyword that the +// symbol-aware MCP server is going to preserve, producing duplicated keywords +// and a syntax error. +// +// Warns to stderr; never blocks the call. Insert variants are skipped because +// they legitimately accept whole new declarations. +func preCallEditGuard(serverName, toolName string, args map[string]any) { + if toolName != "replace_symbol_body" { + return + } + body, _ := args["body"].(string) + rel, _ := args["relative_path"].(string) + if body == "" || rel == "" { + return + } + keywords, ok := langKeywords[strings.ToLower(filepath.Ext(rel))] + if !ok { + return + } + trimmed := strings.TrimLeft(body, " \t\n") + for _, kw := range keywords { + if strings.HasPrefix(trimmed, kw) { + fmt.Fprintf(os.Stderr, + "mcpx: warning: body for %s.%s begins with %q. The replacement "+ + "region starts AFTER the symbol's keyword — your %q will be "+ + "duplicated and the file will not parse. Strip the leading keyword.\n", + serverName, toolName, strings.TrimSpace(kw), strings.TrimSpace(kw)) + return + } + } +} + +// postCallEditGuard validates the file targeted by a body-mutating tool. For Go +// (.go) we have `go/parser` in stdlib; other languages would require shelling +// out to language-native compilers (python -m py_compile, node --check, etc.) +// which adds external-dependency risk — we keep this in-process for now. +// +// Returns nil for non-edit tools, missing paths, non-Go files, and successful +// parses. Returns a wrapped parse error otherwise so the caller can surface +// it as a tool failure. +func postCallEditGuard(serverName, toolName string, args map[string]any, projectRoot string) error { + if !bodyMutatingTools[toolName] { + return nil + } + rel, _ := args["relative_path"].(string) + if rel == "" { + return nil + } + ext := strings.ToLower(filepath.Ext(rel)) + if ext != ".go" { + // Other languages get the pre-call warning but no post-call check. + return nil + } + + full := rel + if !filepath.IsAbs(rel) { + root := projectRoot + if root == "" { + cwd, _ := os.Getwd() + root = cwd + } + full = filepath.Join(root, rel) + } + + if _, err := os.Stat(full); err != nil { + return nil + } + + fset := token.NewFileSet() + if _, err := parser.ParseFile(fset, full, nil, parser.AllErrors); err != nil { + return fmt.Errorf("post-edit parse check failed for %s\n %v\n "+ + "the body sent to %s may have duplicated the symbol's declaration "+ + "keyword (e.g. `type type Foo`); strip the leading `type`/`func`", + rel, summarizeParseErr(err), toolName) + } + return nil +} + +// summarizeParseErr trims Go's verbose multi-line parse error to the most +// useful single line for an end-user message. +func summarizeParseErr(err error) string { + s := err.Error() + if i := strings.Index(s, "\n"); i > 0 { + return s[:i] + } + return s +} diff --git a/internal/cli/edit_guard_test.go b/internal/cli/edit_guard_test.go new file mode 100644 index 0000000..a454e8e --- /dev/null +++ b/internal/cli/edit_guard_test.go @@ -0,0 +1,91 @@ +package cli + +import ( + "os" + "path/filepath" + "testing" +) + +func TestPreCallEditGuard_NoOpOnIrrelevantTools(t *testing.T) { + // preCallEditGuard writes to stderr; assert it does not panic or fire on + // tools that aren't body-mutating, on missing args, or on insert variants. + preCallEditGuard("serena", "find_symbol", map[string]any{"body": "type X struct{}", "relative_path": "x.go"}) + preCallEditGuard("serena", "replace_symbol_body", map[string]any{}) + preCallEditGuard("serena", "replace_symbol_body", map[string]any{"body": "type X struct{}", "relative_path": "x.unknown"}) + preCallEditGuard("serena", "insert_after_symbol", map[string]any{"body": "type X struct{}", "relative_path": "x.go"}) +} + +func TestPreCallEditGuard_DetectsAcrossLanguages(t *testing.T) { + cases := []struct { + ext string + body string + }{ + {".go", "type Foo struct{}"}, + {".go", "func Foo() {}"}, + {".py", "def foo():"}, + {".py", "class Foo:"}, + {".py", "async def foo():"}, + {".ts", "function foo() {}"}, + {".ts", "class Foo {}"}, + {".ts", "interface Foo {}"}, + {".rs", "fn foo() {}"}, + {".rs", "struct Foo {}"}, + {".rb", "def foo end"}, + {".java", "class Foo {}"}, + } + for _, c := range cases { + args := map[string]any{"body": c.body, "relative_path": "x" + c.ext} + // We can't capture stderr without plumbing — just confirm no panic. + preCallEditGuard("serena", "replace_symbol_body", args) + } +} + +func TestPostCallEditGuard_ReportsBrokenGo(t *testing.T) { + dir := t.TempDir() + bad := filepath.Join(dir, "broken.go") + if err := os.WriteFile(bad, []byte("package x\nfunc func Foo() {}\n"), 0o644); err != nil { + t.Fatal(err) + } + err := postCallEditGuard("serena", "replace_symbol_body", + map[string]any{"relative_path": "broken.go"}, dir) + if err == nil { + t.Fatal("expected parse error, got nil") + } +} + +func TestPostCallEditGuard_AllowsValidGo(t *testing.T) { + dir := t.TempDir() + good := filepath.Join(dir, "good.go") + if err := os.WriteFile(good, []byte("package x\nfunc Foo() {}\n"), 0o644); err != nil { + t.Fatal(err) + } + err := postCallEditGuard("serena", "replace_symbol_body", + map[string]any{"relative_path": "good.go"}, dir) + if err != nil { + t.Fatalf("expected nil, got %v", err) + } +} + +func TestPostCallEditGuard_SkipsNonGo(t *testing.T) { + err := postCallEditGuard("serena", "replace_symbol_body", + map[string]any{"relative_path": "anything.py"}, t.TempDir()) + if err != nil { + t.Errorf("non-Go should pass through: %v", err) + } +} + +func TestPostCallEditGuard_SkipsNonEditTools(t *testing.T) { + err := postCallEditGuard("serena", "find_symbol", + map[string]any{"relative_path": "broken.go"}, t.TempDir()) + if err != nil { + t.Errorf("non-edit tool should pass through: %v", err) + } +} + +func TestPostCallEditGuard_HandlesMissingFile(t *testing.T) { + err := postCallEditGuard("serena", "replace_symbol_body", + map[string]any{"relative_path": "does/not/exist.go"}, t.TempDir()) + if err != nil { + t.Errorf("missing file should not error: %v", err) + } +} diff --git a/internal/cli/example.go b/internal/cli/example.go new file mode 100644 index 0000000..69c578e --- /dev/null +++ b/internal/cli/example.go @@ -0,0 +1,222 @@ +package cli + +import ( + "encoding/json" + "fmt" + "sort" + "strings" + + "github.com/codestz/mcpx/internal/mcp" +) + +// generateExample builds a JSON skeleton object showing every flag a tool +// accepts. Required flags get a real placeholder ("" / 0 / false / +// matching enum); optional flags use the schema default when present. +// +// Designed for AI agents that need to construct a valid `--stdin` payload +// quickly without scanning prose descriptions. +func generateExample(tool *mcp.Tool) map[string]any { + required := map[string]bool{} + for _, r := range tool.InputSchema.Required { + required[r] = true + } + + out := map[string]any{} + for name, prop := range tool.InputSchema.Properties { + if required[name] { + out[name] = placeholderFor(prop) + } else if prop.Default != nil { + out[name] = prop.Default + } else if len(prop.Enum) > 0 { + out[name] = prop.Enum[0] + } else { + out[name] = placeholderFor(prop) + } + } + return out +} + +func placeholderFor(p mcp.PropertySchema) any { + if len(p.Enum) > 0 { + return p.Enum[0] + } + switch p.Type { + case "string": + return "" + case "integer": + return 0 + case "number": + return 0.0 + case "boolean": + return false + case "array": + if p.Items != nil { + return []any{placeholderFor(*p.Items)} + } + return []any{} + case "object": + return map[string]any{} + default: + return nil + } +} + +// printExample prints the example JSON. Pretty-printed by default; --json mode +// strips indentation for piping into other tools. +func printExample(tool *mcp.Tool, jsonMode bool) { + ex := generateExample(tool) + enc := json.NewEncoder(stdoutWriter()) + enc.SetEscapeHTML(false) + if !jsonMode { + enc.SetIndent("", " ") + } + _ = enc.Encode(ex) +} + +func stdoutWriter() *stdoutWriterT { return &stdoutWriterT{} } + +type stdoutWriterT struct{} + +func (*stdoutWriterT) Write(p []byte) (int, error) { + return fmt.Print(string(p)) +} + +// validationIssue describes one problem with a tool-call arguments object. +type validationIssue struct { + Field string + Got string + Expected string + Hint string +} + +func (v validationIssue) String() string { + parts := []string{fmt.Sprintf("--%s", v.Field)} + if v.Expected != "" { + parts = append(parts, "expected "+v.Expected) + } + if v.Got != "" { + parts = append(parts, "got "+v.Got) + } + s := strings.Join(parts, ": ") + if v.Hint != "" { + s += " — " + v.Hint + } + return s +} + +// validateArgs checks args against a tool's normalized schema. +// Returns nil when everything passes, otherwise a list of problems. +func validateArgs(tool *mcp.Tool, args map[string]any) []validationIssue { + var issues []validationIssue + + // Required check. + for _, req := range tool.InputSchema.Required { + if _, ok := args[req]; !ok { + issues = append(issues, validationIssue{ + Field: req, Expected: "required field", Got: "missing", + }) + } + } + + // Type + enum check on supplied values. + names := make([]string, 0, len(args)) + for k := range args { + names = append(names, k) + } + sort.Strings(names) + + for _, name := range names { + prop, defined := tool.InputSchema.Properties[name] + if !defined { + issues = append(issues, validationIssue{ + Field: name, Expected: "known flag", Got: "unknown", + Hint: nearestProperty(name, tool.InputSchema.Properties), + }) + continue + } + if !typeMatches(prop, args[name]) { + issues = append(issues, validationIssue{ + Field: name, + Expected: prop.Type, + Got: describeValue(args[name]), + }) + } + if len(prop.Enum) > 0 && !inEnum(args[name], prop.Enum) { + vals := make([]string, len(prop.Enum)) + for i, v := range prop.Enum { + vals[i] = fmt.Sprintf("%v", v) + } + issues = append(issues, validationIssue{ + Field: name, + Expected: "one of [" + strings.Join(vals, ", ") + "]", + Got: describeValue(args[name]), + }) + } + } + return issues +} + +func typeMatches(p mcp.PropertySchema, v any) bool { + if v == nil { + return p.Nullable + } + switch p.Type { + case "string": + _, ok := v.(string) + return ok + case "integer": + switch v.(type) { + case int, int32, int64, float64: + return true + } + return false + case "number": + switch v.(type) { + case float32, float64, int, int32, int64: + return true + } + return false + case "boolean": + _, ok := v.(bool) + return ok + case "array": + _, ok := v.([]any) + return ok + case "object": + _, ok := v.(map[string]any) + return ok + case "any", "": + return true + } + return true +} + +func inEnum(v any, enum []any) bool { + for _, e := range enum { + if fmt.Sprintf("%v", e) == fmt.Sprintf("%v", v) { + return true + } + } + return false +} + +func describeValue(v any) string { + if v == nil { + return "null" + } + switch v.(type) { + case string: + return "string" + case bool: + return "boolean" + case int, int32, int64: + return "integer" + case float32, float64: + return "number" + case []any: + return "array" + case map[string]any: + return "object" + } + return fmt.Sprintf("%T", v) +} diff --git a/internal/cli/find.go b/internal/cli/find.go new file mode 100644 index 0000000..ce86725 --- /dev/null +++ b/internal/cli/find.go @@ -0,0 +1,211 @@ +package cli + +import ( + "context" + "fmt" + "os" + "strings" + "sync" + "time" + + "github.com/codestz/mcpx/internal/config" + "github.com/codestz/mcpx/internal/find" + "github.com/codestz/mcpx/internal/stats" + "github.com/fatih/color" + "github.com/spf13/cobra" +) + +func findCmd(opts *globalOpts) *cobra.Command { + var topK int + var serverFilter string + + cmd := &cobra.Command{ + Use: "find ", + Short: "Rank MCP tools across all configured servers by relevance", + Long: `Find ranks tools across every configured MCP server by BM25 relevance to a free-text query. +Built for AI agents at discovery time: instead of scanning 'mcpx list -v' (5–15K tokens), +'mcpx find "search code"' returns 3 ranked candidates in ~80 tokens. + +Examples: + mcpx find "search code by regex" + mcpx find "github issue" --top 3 + mcpx find "..." --json + mcpx find "..." --server serena`, + Args: cobra.MinimumNArgs(1), + RunE: func(cmd *cobra.Command, args []string) error { + query := strings.Join(args, " ") + start := time.Now() + + cfg, _, err := config.Load() + if err != nil { + return fmt.Errorf("config: %w", err) + } + + servers := cfg.Servers + if serverFilter != "" { + if _, ok := cfg.Servers[serverFilter]; !ok { + return fmt.Errorf("server %q not in config", serverFilter) + } + servers = map[string]*config.ServerConfig{serverFilter: cfg.Servers[serverFilter]} + } + + corpus, perServerHits := gatherCorpus(cmd.Context(), servers) + results := find.Rank(query, corpus, topK) + + recordStats(stats.Record{ + ID: stats.NewID(), + Server: "mcpx", + Tool: "find", + Args: map[string]any{"query": query, "top": topK}, + ArgsBytes: jsonBytes(map[string]any{"query": query, "top": topK}), + ResponseBytes: jsonBytes(results), + LatencyMS: time.Since(start).Milliseconds(), + ExitCode: 0, + PolicyAction: "allowed", + }) + + out := newOutput(opts.outputMode()) + if opts.outputMode() == outputJSON { + return out.printJSON(results) + } + printFindResults(query, results, perServerHits) + return nil + }, + } + cmd.Flags().IntVar(&topK, "top", 5, "Max number of results to show") + cmd.Flags().StringVar(&serverFilter, "server", "", "Restrict search to one server") + return cmd +} + +// gatherCorpus connects to every server, lists its tools (using the schema +// cache), and returns a flat corpus suitable for the BM25 ranker. Errors are +// swallowed per server — a broken server should not break find for others. +// Returns a map of server → tool count for the result-footer. +func gatherCorpus(ctx context.Context, servers map[string]*config.ServerConfig) ([]find.Tool, map[string]int) { + type result struct { + name string + tools []find.Tool + } + + var ( + wg sync.WaitGroup + mu sync.Mutex + results []result + hits = map[string]int{} + ) + + for name, sc := range servers { + wg.Add(1) + go func(name string, sc *config.ServerConfig) { + defer wg.Done() + cctx, cancel := context.WithTimeout(ctx, 5*time.Second) + defer cancel() + + client, cleanup, err := connectServer(cctx, name, sc) + if err != nil { + return + } + defer cleanup() + + tools, _, _, err := listToolsCached(cctx, name, sc, client) + if err != nil { + return + } + + out := make([]find.Tool, len(tools)) + for i, t := range tools { + out[i] = find.Tool{Server: name, Name: t.Name, Description: t.Description} + } + + mu.Lock() + results = append(results, result{name: name, tools: out}) + hits[name] = len(tools) + mu.Unlock() + }(name, sc) + } + wg.Wait() + + var corpus []find.Tool + for _, r := range results { + corpus = append(corpus, r.tools...) + } + return corpus, hits +} + +func printFindResults(query string, results []find.Result, hits map[string]int) { + bold := color.New(color.Bold) + dim := color.New(color.FgHiBlack) + cyan := color.New(color.FgCyan, color.Bold) + green := color.New(color.FgGreen) + + if len(results) == 0 { + fmt.Fprintln(os.Stderr, "No tools matched.") + fmt.Fprintf(os.Stderr, "Searched %d server(s): %s\n", len(hits), summarizeServers(hits)) + return + } + + dim.Fprintf(os.Stdout, "query ") + bold.Fprintf(os.Stdout, "%q\n", query) + dim.Fprintf(os.Stdout, "found %d result(s) across %d server(s)\n\n", len(results), len(hits)) + + maxName := 0 + for _, r := range results { + full := r.Server + "." + r.Name + if len(full) > maxName { + maxName = len(full) + } + } + if maxName > 40 { + maxName = 40 + } + + for i, r := range results { + full := r.Server + "." + r.Name + if len(full) > maxName { + full = full[:maxName-1] + "…" + } + score := fmt.Sprintf("%.2f", normalizeScore(r.Score, results[0].Score)) + + switch i { + case 0: + cyan.Fprintf(os.Stdout, " %-*s", maxName, full) + default: + bold.Fprintf(os.Stdout, " %-*s", maxName, full) + } + green.Fprintf(os.Stdout, " %s", score) + desc := r.Description + if desc != "" { + dim.Fprintf(os.Stdout, " %s", truncate(desc, 60)) + } + fmt.Fprintln(os.Stdout) + } + + fmt.Fprintln(os.Stdout) + dim.Fprintf(os.Stdout, "Run: mcpx --help for any of the above.\n") +} + +func summarizeServers(hits map[string]int) string { + if len(hits) == 0 { + return "(none)" + } + names := make([]string, 0, len(hits)) + for n := range hits { + names = append(names, n) + } + return strings.Join(names, ", ") +} + +// normalizeScore rescales scores to [0,1] relative to the top result. +func normalizeScore(s, top float64) float64 { + if top == 0 { + return 0 + } + v := s / top + if v > 1 { + return 1 + } + if v < 0 { + return 0 + } + return v +} diff --git a/internal/cli/gain.go b/internal/cli/gain.go new file mode 100644 index 0000000..25beb10 --- /dev/null +++ b/internal/cli/gain.go @@ -0,0 +1,561 @@ +package cli + +import ( + "fmt" + "net" + "os" + "strings" + "time" + + "github.com/codestz/mcpx/internal/config" + "github.com/codestz/mcpx/internal/render" + "github.com/codestz/mcpx/internal/stats" + "github.com/fatih/color" + "github.com/spf13/cobra" +) + +func gainCmd(opts *globalOpts) *cobra.Command { + var ( + allProjects bool + project string + since time.Duration + byMode string + history int + watch bool + suggest bool + ) + + cmd := &cobra.Command{ + Use: "gain", + Short: "Show mcpx token-savings dashboard", + Long: `Renders a premium terminal dashboard summarizing every mcpx call: +tokens saved vs native MCP loading, top tools, cache hit rate, recent calls. + +Examples: + mcpx gain # current project, last 7 days + mcpx gain --all # every project + mcpx gain --since 24h + mcpx gain --by tool # ranked tools only + mcpx gain --history 20 + mcpx gain --suggest # mined recommendations + mcpx gain --watch # live refresh + mcpx gain --json`, + RunE: func(cmd *cobra.Command, args []string) error { + if since == 0 { + since = 7 * 24 * time.Hour + } + runGain := func() error { + return renderGainOnce(opts, runGainArgs{ + all: allProjects, project: project, since: since, + byMode: byMode, history: history, suggest: suggest, + }) + } + + if watch { + return watchLoop(runGain) + } + return runGain() + }, + } + cmd.Flags().BoolVar(&allProjects, "all", false, "Aggregate across every project") + cmd.Flags().StringVar(&project, "project", "", "Filter to a specific project root") + cmd.Flags().DurationVar(&since, "since", 0, "Time window (default 7 days)") + cmd.Flags().StringVar(&byMode, "by", "", "Group by tool|server|day") + cmd.Flags().IntVar(&history, "history", 0, "Show last N calls only") + cmd.Flags().BoolVar(&watch, "watch", false, "Live refresh every second") + cmd.Flags().BoolVar(&suggest, "suggest", false, "Mine the data for recommendations") + return cmd +} + +type runGainArgs struct { + all bool + project string + since time.Duration + byMode string + history int + suggest bool +} + +func renderGainOnce(opts *globalOpts, a runGainArgs) error { + cfg, projectRoot, err := config.Load() + if err != nil { + return fmt.Errorf("config: %w", err) + } + + gainCfg := (*config.GainConfig)(nil).Default() + if cfg != nil && cfg.Gain != nil { + gainCfg = cfg.Gain.Default() + } + path := gainCfg.StatsPath + if path == "" { + path = stats.DefaultPath() + } + + filter := stats.Filter{Since: time.Now().Add(-a.since)} + if !a.all { + if a.project != "" { + filter.Project = a.project + } else if projectRoot != "" { + filter.Project = projectRoot + } + } + + summary, err := stats.Aggregate(path, filter, max(a.history, 5)) + if err != nil { + return err + } + + if opts.outputMode() == outputJSON { + out := newOutput(outputJSON) + return out.printJSON(summary) + } + + switch a.byMode { + case "tool": + renderToolTable(summary) + return nil + case "server": + renderServerTable(summary) + return nil + case "day": + renderDayTable(summary) + return nil + } + if a.history > 0 { + renderHistory(summary, a.history) + return nil + } + if a.suggest { + renderSuggestions(summary) + return nil + } + + dashURL := readDashURL() + scope := "this project · " + humanizeDuration(a.since) + if a.all { + scope = "all projects · " + humanizeDuration(a.since) + } else if a.project != "" { + scope = "project " + truncProject(a.project) + " · " + humanizeDuration(a.since) + } + renderGainDashboard(summary, scope, dashURL) + return nil +} + +func renderGainDashboard(s *stats.Summary, scope, dashURL string) { + w := render.TermWidth() + if w > 100 { + w = 100 + } + if w < 70 { + w = 70 + } + + bold := color.New(color.Bold) + cyan := color.New(color.FgCyan, color.Bold) + green := color.New(color.FgGreen, color.Bold) + yellow := color.New(color.FgYellow) + red := color.New(color.FgRed) + dim := color.New(color.FgHiBlack) + + hr := func() { + fmt.Println(" " + dim.Sprint(strings.Repeat(render.BoxH, w-4))) + } + + // Header + fmt.Println() + cyan.Print(" mcpx ") + dim.Printf(" %s", scope) + saved := "saved: " + render.FormatNumber(s.TokensSaved) + pad := w - 8 - len(scope) - len(saved) + if pad < 1 { + pad = 1 + } + fmt.Print(strings.Repeat(" ", pad)) + green.Println(saved) + hr() + + // Hero metric + fmt.Println() + bold.Print(" Tokens saved ") + green.Printf("%s", render.FormatNumber(s.TokensSaved)) + dim.Printf(" (vs %s native)\n", render.FormatNumber(s.NativeBaseline)) + + if len(s.Daily) > 0 { + var vals []int64 + for _, d := range s.Daily { + vals = append(vals, d.TokensSaved) + } + // Pad to constant width when fewer days exist than the window. + fmt.Print(" ") + dim.Println(render.Sparkline(vals) + " daily") + } + fmt.Println() + + // Stat grid (2 rows x 3 cols) + col1 := 24 + col2 := 24 + col3 := 24 + row := func(k1, v1, k2, v2, k3, v3 string) { + fmt.Print(" ") + dim.Printf("%-12s", k1) + bold.Printf("%-*s", col1-12, v1) + dim.Printf("%-12s", k2) + bold.Printf("%-*s", col2-12, v2) + dim.Printf("%-12s", k3) + bold.Printf("%-*s", col3-12, v3) + fmt.Println() + } + row( + "Calls", render.FormatNumber(int64(s.Calls)), + "Cache hit", render.FormatPercent(s.CacheHitRate), + "Errors", colorErr(red, s.ErrorRate), + ) + row( + "Avg latency", render.FormatDuration(int64(s.AvgLatencyMS)), + "p95", render.FormatDuration(s.P95LatencyMS), + "Servers", fmt.Sprintf("%d", len(s.Servers)), + ) + fmt.Println() + + // Top tools (by calls) + if len(s.TopTools) > 0 { + hr() + bold.Println(" Top tools (by calls)") + fmt.Println() + topTools := s.TopTools + if len(topTools) > 5 { + topTools = topTools[:5] + } + maxCalls := topTools[0].Calls + nameW := tableNameWidth(topTools, w-30) + for _, t := range topTools { + full := t.Server + "." + t.Tool + full = render.Truncate(full, nameW) + pct := float64(t.Calls) / float64(maxCalls) + bar := render.Bar(pct, 18) + fmt.Print(" ") + yellow.Printf("%-*s", nameW, full) + fmt.Printf(" %5d ", t.Calls) + green.Println(bar) + } + fmt.Println() + } + + // Top savings + if hasSavings(s.TopSavers) { + hr() + bold.Println(" Top savings (vs native MCP)") + fmt.Println() + topSavers := s.TopSavers + if len(topSavers) > 5 { + topSavers = topSavers[:5] + } + nameW := tableNameWidth(topSavers, w-30) + for _, t := range topSavers { + if t.TokensSaved <= 0 { + continue + } + full := render.Truncate(t.Server+"."+t.Tool, nameW) + fmt.Print(" ") + yellow.Printf("%-*s", nameW, full) + green.Printf(" %8s", render.FormatNumber(t.TokensSaved)) + dim.Printf(" saved\n") + } + fmt.Println() + } + + // Recent + if len(s.Recent) > 0 { + hr() + bold.Println(" Last calls") + fmt.Println() + for i := len(s.Recent) - 1; i >= 0; i-- { + r := s.Recent[i] + tstamp := r.TS.Local().Format("15:04") + full := render.Truncate(r.Server+"."+r.Tool, 36) + fmt.Print(" ") + dim.Printf("%s ", tstamp) + fmt.Printf("%-36s ", full) + latency := render.FormatDuration(r.LatencyMS) + if r.SchemaCacheHit { + yellow.Printf("%6s warm ", latency) + } else { + fmt.Printf("%6s ", latency) + } + if r.ExitCode != 0 { + red.Println(" ✗") + } else { + green.Println(" ✓") + } + } + fmt.Println() + } + + // Footer + hr() + if dashURL != "" { + dim.Print(" dashboard ") + fmt.Println(dashURL) + } else { + dim.Println(" dashboard inactive (start mcpx-ui or run any tool call)") + } + dim.Println(" token counts are estimates (bytes ÷ 4 — close approximation of Claude tokenization)") + fmt.Println() +} + +func renderToolTable(s *stats.Summary) { + bold := color.New(color.Bold) + dim := color.New(color.FgHiBlack) + yellow := color.New(color.FgYellow) + green := color.New(color.FgGreen) + + bold.Println("\n Tools") + fmt.Println() + if len(s.TopTools) == 0 { + dim.Println(" (no calls in window)") + return + } + max := s.TopTools[0].Calls + for _, t := range s.TopTools { + full := render.Truncate(t.Server+"."+t.Tool, 36) + bar := render.Bar(float64(t.Calls)/float64(max), 18) + fmt.Print(" ") + yellow.Printf("%-36s", full) + fmt.Printf(" %5d ", t.Calls) + green.Print(bar) + dim.Printf(" %s saved %s avg\n", + render.FormatNumber(t.TokensSaved), + render.FormatDuration(int64(t.AvgLatencyMS)), + ) + } + fmt.Println() +} + +func renderServerTable(s *stats.Summary) { + bold := color.New(color.Bold) + dim := color.New(color.FgHiBlack) + yellow := color.New(color.FgYellow) + green := color.New(color.FgGreen) + + bold.Println("\n Servers") + fmt.Println() + for _, ss := range s.TopServers { + fmt.Print(" ") + yellow.Printf("%-24s", ss.Server) + fmt.Printf(" %5d calls ", ss.Calls) + green.Printf("%s saved ", render.FormatNumber(ss.TokensSaved)) + dim.Printf("%s avg\n", render.FormatDuration(int64(ss.AvgLatencyMS))) + } + fmt.Println() +} + +func renderDayTable(s *stats.Summary) { + bold := color.New(color.Bold) + dim := color.New(color.FgHiBlack) + green := color.New(color.FgGreen) + + bold.Println("\n Daily") + fmt.Println() + if len(s.Daily) == 0 { + dim.Println(" (no calls)") + return + } + var max int64 + for _, d := range s.Daily { + if d.TokensSaved > max { + max = d.TokensSaved + } + } + for _, d := range s.Daily { + bar := render.Bar(float64(d.TokensSaved)/float64(maxOr1(max)), 24) + fmt.Print(" ") + dim.Print(d.Day.Format("2006-01-02")) + fmt.Printf(" %5d ", d.Calls) + green.Print(bar) + dim.Printf(" %s saved\n", render.FormatNumber(d.TokensSaved)) + } + fmt.Println() +} + +func renderHistory(s *stats.Summary, n int) { + bold := color.New(color.Bold) + dim := color.New(color.FgHiBlack) + yellow := color.New(color.FgYellow) + green := color.New(color.FgGreen) + red := color.New(color.FgRed) + + bold.Printf("\n Last %d calls\n", n) + fmt.Println() + if len(s.Recent) == 0 { + dim.Println(" (no calls)") + return + } + for i := len(s.Recent) - 1; i >= 0; i-- { + r := s.Recent[i] + fmt.Print(" ") + dim.Print(r.TS.Local().Format("Mon 15:04:05")) + fmt.Print(" ") + yellow.Printf("%-36s", render.Truncate(r.Server+"."+r.Tool, 36)) + latency := render.FormatDuration(r.LatencyMS) + fmt.Printf(" %6s", latency) + if r.SchemaCacheHit { + yellow.Print(" warm") + } + if r.ExitCode != 0 { + red.Println(" ✗") + } else { + green.Println(" ✓") + } + } + fmt.Println() +} + +func renderSuggestions(s *stats.Summary) { + bold := color.New(color.Bold) + dim := color.New(color.FgHiBlack) + yellow := color.New(color.FgYellow) + + bold.Println("\n Recommendations") + fmt.Println() + suggestions := mineSuggestions(s) + if len(suggestions) == 0 { + dim.Println(" No actionable patterns yet — run more calls and try again.") + return + } + for _, sg := range suggestions { + fmt.Print(" ") + yellow.Print("• ") + fmt.Println(sg) + } + fmt.Println() +} + +// mineSuggestions returns terse, actionable recommendations from a Summary. +// Heuristics deliberately conservative — false positives erode trust. +func mineSuggestions(s *stats.Summary) []string { + var out []string + if s.Calls < 5 { + return nil + } + if s.SchemaHitRate < 0.4 { + out = append(out, fmt.Sprintf("Schema cache hit rate %s — verify cache.schema_ttl is reasonable.", + render.FormatPercent(s.SchemaHitRate))) + } + if s.ErrorRate > 0.1 { + out = append(out, fmt.Sprintf("Error rate %s — top failing tools may need argument fixes.", + render.FormatPercent(s.ErrorRate))) + } + if s.P95LatencyMS > 2000 { + out = append(out, fmt.Sprintf("p95 latency %s — slow tools dominate; consider mcpx batch for parallel execution.", + render.FormatDuration(s.P95LatencyMS))) + } + if len(s.TopTools) > 0 { + t := s.TopTools[0] + share := float64(t.Calls) / float64(s.Calls) + if share > 0.5 { + out = append(out, fmt.Sprintf("%s.%s accounts for %s of all calls — strong candidate for an alias or shortcut.", + t.Server, t.Tool, render.FormatPercent(share))) + } + } + return out +} + +func tableNameWidth(items any, max int) int { + w := 24 + switch v := items.(type) { + case []stats.ToolStat: + for _, t := range v { + full := t.Server + "." + t.Tool + if len(full) > w { + w = len(full) + } + } + } + if w > max { + w = max + } + return w +} + +func hasSavings(ts []stats.ToolStat) bool { + for _, t := range ts { + if t.TokensSaved > 0 { + return true + } + } + return false +} + +func maxOr1(n int64) int64 { + if n <= 0 { + return 1 + } + return n +} + +func colorErr(red *color.Color, rate float64) string { + if rate > 0.05 { + return red.Sprint(render.FormatPercent(rate)) + } + return render.FormatPercent(rate) +} + +func humanizeDuration(d time.Duration) string { + if d == 0 { + return "all time" + } + if d >= 24*time.Hour { + days := int(d / (24 * time.Hour)) + return fmt.Sprintf("%dd", days) + } + if d >= time.Hour { + return fmt.Sprintf("%dh", int(d/time.Hour)) + } + return fmt.Sprintf("%dm", int(d/time.Minute)) +} + +func truncProject(p string) string { + if len(p) <= 32 { + return p + } + return "…" + p[len(p)-30:] +} + +func max(a, b int) int { + if a > b { + return a + } + return b +} + +// readDashURL returns the dashboard URL from ~/.mcpx/ui.json if the daemon is alive. +func readDashURL() string { + home, err := os.UserHomeDir() + if err != nil { + return "" + } + uiInfo, err := readUIHandshake(home) + if err != nil || uiInfo.Port == 0 { + return "" + } + addr := net.JoinHostPort(uiInfo.Bind, fmt.Sprintf("%d", uiInfo.Port)) + conn, err := net.DialTimeout("tcp", addr, 200*time.Millisecond) + if err != nil { + return "" + } + conn.Close() + return fmt.Sprintf("http://%s/?t=%s", addr, uiInfo.Token) +} + +// watchLoop redraws the dashboard once per second until interrupted. +func watchLoop(run func() error) error { + ticker := time.NewTicker(1 * time.Second) + defer ticker.Stop() + for { + fmt.Print("\033[H\033[2J") // clear screen + if err := run(); err != nil { + return err + } + <-ticker.C + } +} diff --git a/internal/cli/generate.go b/internal/cli/generate.go index c0b4bb1..0ca44f0 100644 --- a/internal/cli/generate.go +++ b/internal/cli/generate.go @@ -14,6 +14,9 @@ import ( // runGenerate connects to a server, fetches tools (and prompts/resources if supported), // and generates a concise Markdown reference file for Claude Code. +// +// Uses the cached tools-list path so a configure run populates the schema cache +// for any server we hadn't yet touched — agent's first real call lands warm. func runGenerate(ctx context.Context, serverName string, sc *config.ServerConfig, global bool, format string) error { client, cleanup, err := connectServer(ctx, serverName, sc) if err != nil { @@ -21,7 +24,7 @@ func runGenerate(ctx context.Context, serverName string, sc *config.ServerConfig } defer cleanup() - tools, err := client.ListTools(ctx) + tools, _, _, err := listToolsCached(ctx, serverName, sc, client) if err != nil { return fmt.Errorf("list tools: %w", err) } @@ -81,112 +84,164 @@ func runGenerate(ctx context.Context, serverName string, sc *config.ServerConfig return nil } -// generateServerMDWithFormat dispatches to the right format generator. +// generateServerMDWithFormat is kept for the configureCmd flag surface but +// now collapses to a single agent-grounded format. The `format` parameter is +// accepted for backward compatibility and ignored. func generateServerMDWithFormat(serverName string, tools []mcp.Tool, prompts []mcp.Prompt, resources []mcp.Resource, format string) string { - if format == "compact" { - return generateServerMDCompact(serverName, tools, prompts, resources) - } + _ = format return generateServerMD(serverName, tools, prompts, resources) } -// generateServerMD creates a concise reference for a server's tools, prompts, and resources. -// Optimized for AI consumption: compact, all flags visible, ~100 lines max. +// generateServerMD produces an agent-optimized reference for one MCP server. +// +// Design priorities (in order): +// 1. A "tool selector" table at the top — one row per tool, three columns +// (tool, what it does, required flags). This is the primary surface the +// agent reads when picking a tool. +// 2. A compact reference at the bottom — every tool's name + required flags +// on one line, for fast scanning when the selector matches multiple. +// 3. Prompts and resources as terse bullet lists; agents don't reach for +// them often, so they don't earn full sections. +// +// No per-tool full schema tables. The agent runs `mcpx --help` +// for the full schema or `--example` for a JSON skeleton when actually calling. func generateServerMD(serverName string, tools []mcp.Tool, prompts []mcp.Prompt, resources []mcp.Resource) string { var b strings.Builder - b.WriteString(fmt.Sprintf("# %s — mcpx tool reference\n\n", serverName)) - b.WriteString(fmt.Sprintf("Server with %d tools. Call via: `mcpx %s --flags`\n\n", len(tools), serverName)) + b.WriteString(fmt.Sprintf("# %s\n\n", serverName)) + b.WriteString(fmt.Sprintf("`mcpx %s --flags` — %d tools. Use `--help` / `--example` / `--validate-args` per tool.\n\n", + serverName, len(tools))) + b.WriteString("## Tool selector\n\n") + b.WriteString("| Tool | What it does | Required |\n") + b.WriteString("|---|---|---|\n") for _, t := range tools { - b.WriteString(fmt.Sprintf("## %s\n", t.Name)) - - if t.Description != "" { - // First sentence only for brevity. - desc := firstSentence(t.Description) - b.WriteString(fmt.Sprintf("%s\n", desc)) + desc := firstSentence(t.Description) + if desc == "" { + desc = "(no description)" } + desc = strings.ReplaceAll(desc, "|", "\\|") + req := requiredFlagList(t) + b.WriteString(fmt.Sprintf("| `%s` | %s | %s |\n", t.Name, desc, req)) + } + b.WriteString("\n") + + if hasEditTools(tools) { + b.WriteString("## Edit-tool safety\n\n") + b.WriteString("For `replace_symbol_body`, body must start AFTER the declaration keyword — the keyword is preserved by the server. mcpx warns on duplicates and (for `.go`) re-parses the file post-edit.\n\n") + b.WriteString("| Language | ✓ correct | ✗ wrong |\n") + b.WriteString("|---|---|---|\n") + b.WriteString("| Go | `Foo struct{...}` / `Foo() error {...}` | `type Foo...` / `func Foo...` |\n") + b.WriteString("| Python | `foo():\\n ...` | `def foo():` |\n") + b.WriteString("| TS / JS | `bar() { ... }` | `function bar()` / `class Bar` |\n") + b.WriteString("| Rust | `foo() { ... }` | `fn foo()` / `struct Foo` |\n\n") + } - if len(t.InputSchema.Properties) == 0 { - b.WriteString("```\nmcpx " + serverName + " " + t.Name + "\n```\n\n") - continue - } + if hasSymbolSearch(tools) { + b.WriteString("## Notes\n\n") + b.WriteString("- Symbol-aware lookups (`find_symbol`) are faster and more accurate than text search (`search_for_pattern`) for code symbols — prefer them.\n\n") + } - required := make(map[string]bool) + b.WriteString("## Compact reference\n\n") + for _, t := range tools { + propNames := sortedPropNames(t) + req := map[string]bool{} for _, r := range t.InputSchema.Required { - required[r] = true + req[r] = true } - - propNames := make([]string, 0, len(t.InputSchema.Properties)) - for name := range t.InputSchema.Properties { - propNames = append(propNames, name) - } - sort.Strings(propNames) - - // Build example command with required flags. - var exampleParts []string - exampleParts = append(exampleParts, "mcpx", serverName, t.Name) - - b.WriteString("| Flag | Type | Req | Description |\n") - b.WriteString("|---|---|---|---|\n") - + var parts []string for _, name := range propNames { prop := t.InputSchema.Properties[name] - req := "" - if required[name] { - req = "yes" - exampleParts = append(exampleParts, fmt.Sprintf("--%s ", name)) - } - desc := compactDesc(prop.Description) - if prop.Default != nil { - desc += fmt.Sprintf(" (default: %v)", prop.Default) + entry := fmt.Sprintf("--%s <%s>", name, flagTypeLabel(prop.Type)) + if req[name] { + entry += "*" } - b.WriteString(fmt.Sprintf("| `--%s` | %s | %s | %s |\n", - name, flagTypeLabel(prop.Type), req, desc)) + parts = append(parts, entry) + } + if len(parts) == 0 { + b.WriteString(fmt.Sprintf("- `%s`\n", t.Name)) + } else { + b.WriteString(fmt.Sprintf("- `%s` %s\n", t.Name, strings.Join(parts, " "))) } - - b.WriteString(fmt.Sprintf("```\n%s\n```\n\n", strings.Join(exampleParts, " "))) } + b.WriteString("\n`*` = required\n") if len(prompts) > 0 { - b.WriteString(fmt.Sprintf("## Prompts (%d)\n\n", len(prompts))) - b.WriteString(fmt.Sprintf("Usage: `mcpx %s prompt [--arg value ...]`\n\n", serverName)) + b.WriteString(fmt.Sprintf("\n## Prompts (%d)\n\n", len(prompts))) + b.WriteString(fmt.Sprintf("`mcpx %s prompt [--arg value ...]`\n\n", serverName)) for _, p := range prompts { - b.WriteString(fmt.Sprintf("**%s**", p.Name)) + line := fmt.Sprintf("- `%s`", p.Name) if p.Description != "" { - b.WriteString(" — " + firstSentence(p.Description)) - } - b.WriteString("\n") - if len(p.Arguments) > 0 { - var args []string - for _, a := range p.Arguments { - entry := fmt.Sprintf("--%s", a.Name) - if a.Required { - entry += " *" - } - args = append(args, entry) - } - b.WriteString(" " + strings.Join(args, ", ") + "\n") + line += " — " + firstSentence(p.Description) } - b.WriteString("\n") + b.WriteString(line + "\n") } } if len(resources) > 0 { - b.WriteString(fmt.Sprintf("## Resources (%d)\n\n", len(resources))) - b.WriteString(fmt.Sprintf("Usage: `mcpx %s resource read `\n\n", serverName)) + b.WriteString(fmt.Sprintf("\n## Resources (%d)\n\n", len(resources))) + b.WriteString(fmt.Sprintf("`mcpx %s resource read `\n\n", serverName)) for _, r := range resources { - b.WriteString(fmt.Sprintf("- `%s`", r.URI)) + line := fmt.Sprintf("- `%s`", r.URI) if r.Name != "" { - b.WriteString(fmt.Sprintf(" — %s", r.Name)) + line += " — " + r.Name } - b.WriteString("\n") + b.WriteString(line + "\n") } - b.WriteString("\n") } return b.String() } +// requiredFlagList renders a tool's required flags as a compact comma string +// for the selector table. Empty when no flags are required. +func requiredFlagList(t mcp.Tool) string { + if len(t.InputSchema.Required) == 0 { + return "—" + } + out := make([]string, 0, len(t.InputSchema.Required)) + for _, r := range t.InputSchema.Required { + out = append(out, "`--"+r+"`") + } + return strings.Join(out, " ") +} + +// sortedPropNames returns a tool's input property names in stable order. +func sortedPropNames(t mcp.Tool) []string { + out := make([]string, 0, len(t.InputSchema.Properties)) + for name := range t.InputSchema.Properties { + out = append(out, name) + } + sort.Strings(out) + return out +} + +// hasEditTools detects servers that expose body-mutation tools so we can +// emit the keyword-trap reminder in the per-server doc. +func hasEditTools(tools []mcp.Tool) bool { + for _, t := range tools { + if bodyMutatingTools[t.Name] { + return true + } + } + return false +} + +// hasSymbolSearch detects servers that expose both symbol-aware and text-based +// search tools, so the doc can advise the agent to prefer the structured one. +func hasSymbolSearch(tools []mcp.Tool) bool { + hasSym, hasText := false, false + for _, t := range tools { + switch t.Name { + case "find_symbol": + hasSym = true + case "search_for_pattern": + hasText = true + } + } + return hasSym && hasText +} + // firstSentence returns the first sentence of a string (up to first period+space or newline). func firstSentence(s string) string { // Cut at first newline. @@ -204,98 +259,4 @@ func firstSentence(s string) string { return s } -// compactDesc shortens a description for table display. -func compactDesc(s string) string { - // Remove newlines. - s = strings.ReplaceAll(s, "\n", " ") - // Collapse spaces. - for strings.Contains(s, " ") { - s = strings.ReplaceAll(s, " ", " ") - } - s = strings.TrimSpace(s) - if len(s) > 80 { - s = s[:77] + "..." - } - return s -} - -// generateServerMDCompact creates a minimal one-line-per-tool reference. -// Optimized for maximum token efficiency: ~50% smaller than table format. -func generateServerMDCompact(serverName string, tools []mcp.Tool, prompts []mcp.Prompt, resources []mcp.Resource) string { - var b strings.Builder - - b.WriteString(fmt.Sprintf("# %s (%d tools)\n\n", serverName, len(tools))) - b.WriteString(fmt.Sprintf("Usage: `mcpx %s --flags`\n\n", serverName)) - - for _, t := range tools { - required := make(map[string]bool) - for _, r := range t.InputSchema.Required { - required[r] = true - } - - // Tool name + short description - desc := "" - if t.Description != "" { - desc = " — " + firstSentence(t.Description) - } - b.WriteString(fmt.Sprintf("**%s**%s\n", t.Name, desc)) - - if len(t.InputSchema.Properties) > 0 { - propNames := make([]string, 0, len(t.InputSchema.Properties)) - for name := range t.InputSchema.Properties { - propNames = append(propNames, name) - } - sort.Strings(propNames) - - var flags []string - for _, name := range propNames { - prop := t.InputSchema.Properties[name] - entry := fmt.Sprintf("--%s <%s>", name, flagTypeLabel(prop.Type)) - if required[name] { - entry += " *" - } - flags = append(flags, entry) - } - b.WriteString(" " + strings.Join(flags, ", ") + "\n") - } - b.WriteString("\n") - } - - b.WriteString("`*` = required\n") - - if len(prompts) > 0 { - b.WriteString(fmt.Sprintf("\n## Prompts (%d)\n\n", len(prompts))) - for _, p := range prompts { - desc := "" - if p.Description != "" { - desc = " — " + firstSentence(p.Description) - } - b.WriteString(fmt.Sprintf("**%s**%s\n", p.Name, desc)) - if len(p.Arguments) > 0 { - var args []string - for _, a := range p.Arguments { - entry := fmt.Sprintf("--%s", a.Name) - if a.Required { - entry += " *" - } - args = append(args, entry) - } - b.WriteString(" " + strings.Join(args, ", ") + "\n") - } - b.WriteString("\n") - } - } - - if len(resources) > 0 { - b.WriteString(fmt.Sprintf("\n## Resources (%d)\n\n", len(resources))) - for _, r := range resources { - b.WriteString(fmt.Sprintf("- `%s`", r.URI)) - if r.Name != "" { - b.WriteString(" — " + r.Name) - } - b.WriteString("\n") - } - } - return b.String() -} diff --git a/internal/cli/output.go b/internal/cli/output.go index eecd225..2c67526 100644 --- a/internal/cli/output.go +++ b/internal/cli/output.go @@ -180,6 +180,92 @@ type serverInfo struct { } +// printServerHelpCompact is the default help view: one line per tool, name + +// required-flag summary, no descriptions. Built for AI-agent discovery — keeps +// the entire help under a few hundred tokens for servers with many tools. +// +// Use --full to get the verbose layout (printServerHelpFull). +func (o *output) printServerHelpCompact(serverName string, sc *config.ServerConfig, tools []mcp.Tool, prompts []mcp.Prompt, resources []mcp.Resource) error { + if o.mode == outputJSON { + return o.printJSON(map[string]any{ + "tools": tools, "prompts": prompts, "resources": resources, + }) + } + + bold := color.New(color.Bold) + dim := color.New(color.FgHiBlack) + yellow := color.New(color.FgYellow) + + bold.Fprintf(o.stdout, "%s", serverName) + dim.Fprintf(o.stdout, " %d tool", len(tools)) + if len(tools) != 1 { + dim.Fprint(o.stdout, "s") + } + if len(prompts) > 0 { + dim.Fprintf(o.stdout, ", %d prompt", len(prompts)) + if len(prompts) != 1 { + dim.Fprint(o.stdout, "s") + } + } + if len(resources) > 0 { + dim.Fprintf(o.stdout, ", %d resource", len(resources)) + if len(resources) != 1 { + dim.Fprint(o.stdout, "s") + } + } + fmt.Fprintln(o.stdout) + dim.Fprintln(o.stdout, "—") + + maxName := 0 + for _, t := range tools { + if len(t.Name) > maxName { + maxName = len(t.Name) + } + } + if maxName > 36 { + maxName = 36 + } + + for _, t := range tools { + name := t.Name + if len(name) > maxName { + name = name[:maxName-1] + "…" + } + bold.Fprintf(o.stdout, " %-*s", maxName, name) + req := requiredFlagSummary(&t) + if req != "" { + yellow.Fprintf(o.stdout, " %s", req) + } + fmt.Fprintln(o.stdout) + } + + if len(prompts) > 0 { + dim.Fprintln(o.stdout, "—") + for _, p := range prompts { + bold.Fprintf(o.stdout, " prompt %-*s", maxName-7, p.Name) + fmt.Fprintln(o.stdout) + } + } + + fmt.Fprintln(o.stdout) + dim.Fprintf(o.stdout, "→ mcpx %s --help — flags + descriptions for one tool\n", serverName) + dim.Fprintf(o.stdout, "→ mcpx %s --help --full — full descriptions for all tools\n", serverName) + dim.Fprintf(o.stdout, "→ mcpx find — rank tools across all servers\n") + return nil +} + +// requiredFlagSummary returns "--a --b" for a tool's required flags, "" if none. +func requiredFlagSummary(t *mcp.Tool) string { + if len(t.InputSchema.Required) == 0 { + return "" + } + parts := make([]string, 0, len(t.InputSchema.Required)) + for _, r := range t.InputSchema.Required { + parts = append(parts, "--"+r) + } + return strings.Join(parts, " ") +} + // printServerHelpFull displays a dynamic help page showing tools, prompts, and resources. func (o *output) printServerHelpFull(serverName string, sc *config.ServerConfig, tools []mcp.Tool, prompts []mcp.Prompt, resources []mcp.Resource) error { if o.mode == outputJSON { diff --git a/internal/cli/root.go b/internal/cli/root.go index 3a5b489..77c0bf3 100644 --- a/internal/cli/root.go +++ b/internal/cli/root.go @@ -5,6 +5,7 @@ import ( "os" "runtime/debug" "sort" + "strings" "github.com/codestz/mcpx/internal/config" "github.com/codestz/mcpx/internal/daemon" @@ -27,6 +28,7 @@ type globalOpts struct { dryRun bool pick string timeout string + full bool // verbose mode for server/tool help (default is compact) } func (o *globalOpts) outputMode() outputMode { @@ -39,12 +41,15 @@ func (o *globalOpts) outputMode() outputMode { return outputPretty } -// Exit codes. +// Exit codes. Documented in README + --help. const ( - exitOK = 0 - exitToolError = 1 - exitConfigErr = 2 - exitConnectErr = 3 + exitOK = 0 + exitToolError = 1 + exitConfigErr = 2 + exitConnectErr = 3 + exitTimeout = 4 + exitPolicyDenied = 5 + exitToolNotFound = 6 ) // Execute is the main entry point for the CLI. @@ -62,6 +67,11 @@ func Execute() { root.PersistentFlags().BoolVar(&opts.quiet, "quiet", false, "Suppress output") root.PersistentFlags().BoolVar(&opts.dryRun, "dry-run", false, "Show what would execute without running") + // Load config once — used for stats init and dynamic server commands. + cfg, _, cfgErr := config.Load() + initStats(cfg) + defer closeStats() + // Static commands. root.AddCommand(versionCmd()) root.AddCommand(listCmd(opts)) @@ -70,16 +80,26 @@ func Execute() { root.AddCommand(completionCmd()) root.AddCommand(pingCmd(opts)) root.AddCommand(secretCmd(opts)) + root.AddCommand(findCmd(opts)) + root.AddCommand(batchCmd(opts)) + root.AddCommand(gainCmd(opts)) + root.AddCommand(doctorCmd(opts)) // Hidden daemon runner command. root.AddCommand(daemon.NewDaemonRunCommand()) + root.AddCommand(uiRunCmd()) + root.AddCommand(uiManageCmd()) + + // Always-on dashboard daemon — lazy spawn, opt-out via config or MCPX_UI=off. + startUIIfEnabled(cfg) // Dynamic server commands from config. - if err := addServerCommands(root, opts); err != nil { + if cfgErr != nil { out := newOutput(opts.outputMode()) - out.errorf("config: %v", err) + out.errorf("config: %v", cfgErr) os.Exit(exitConfigErr) } + addServerCommandsFromCfg(root, cfg, opts) if err := root.Execute(); err != nil { out := newOutput(opts.outputMode()) @@ -95,6 +115,11 @@ func versionCmd() *cobra.Command { Short: "Print mcpx version", Run: func(cmd *cobra.Command, args []string) { fmt.Printf("mcpx %s\n", Version) + if strings.Contains(Version, "+dirty") { + fmt.Fprintln(os.Stderr, + "warning: this build was made from a dirty working tree. "+ + "Behavior may differ from the published release. Run 'make install' on a clean tree before reporting bugs.") + } }, } } @@ -190,13 +215,10 @@ func completionCmd() *cobra.Command { } // addServerCommands reads the config and adds a subcommand for each server. -func addServerCommands(root *cobra.Command, opts *globalOpts) error { - cfg, _, err := config.Load() - if err != nil { - return err +func addServerCommandsFromCfg(root *cobra.Command, cfg *config.Config, opts *globalOpts) { + if cfg == nil { + return } - - // Sort server names for deterministic command order. names := make([]string, 0, len(cfg.Servers)) for name := range cfg.Servers { names = append(names, name) @@ -207,10 +229,7 @@ func addServerCommands(root *cobra.Command, opts *globalOpts) error { root.AddCommand(buildServerCommand(name, cfg.Servers[name], cfg.Security, opts)) } - // Daemon management commands. root.AddCommand(daemon.NewDaemonManageCommand(names)) - - return nil } func joinOr(ss []string) string { diff --git a/internal/cli/stats_init.go b/internal/cli/stats_init.go new file mode 100644 index 0000000..43fc82c --- /dev/null +++ b/internal/cli/stats_init.go @@ -0,0 +1,80 @@ +package cli + +import ( + "encoding/json" + "os" + "strconv" + "sync" + + "github.com/codestz/mcpx/internal/config" + "github.com/codestz/mcpx/internal/stats" +) + +var ( + statsWriter *stats.Writer + statsOnce sync.Once + statsAgent string + statsSession string +) + +// initStats lazily creates the package-level stats writer using the merged config. +// Safe to call from multiple goroutines; only the first call has effect. +func initStats(cfg *config.Config) { + statsOnce.Do(func() { + gainCfg := (*config.GainConfig)(nil).Default() + if cfg != nil && cfg.Gain != nil { + gainCfg = cfg.Gain.Default() + } + enabled := gainCfg.Enabled != nil && *gainCfg.Enabled + path := gainCfg.StatsPath + if path == "" { + path = stats.DefaultPath() + } + w, err := stats.NewWriter(path, 1024, enabled) + if err == nil { + statsWriter = w + } + + // Resolve agent identity once. + statsAgent = os.Getenv("MCPX_AGENT") + if statsAgent == "" { + statsAgent = "unknown" + } + + // Session = parent PID. Cheap, stable per-shell, no PII. + statsSession = strconv.Itoa(os.Getppid()) + }) +} + +// closeStats drains the writer. Called at the end of cli.Execute. +func closeStats() { + if statsWriter != nil { + _ = statsWriter.Close() + } +} + +// recordStats emits a stats record. Safe to call when stats are disabled. +func recordStats(r stats.Record) { + if statsWriter == nil { + return + } + if r.Agent == "" { + r.Agent = statsAgent + } + if r.Session == "" { + r.Session = statsSession + } + statsWriter.Write(r) +} + +// jsonBytes returns the byte length of v marshalled to JSON. 0 on error or nil. +func jsonBytes(v any) int { + if v == nil { + return 0 + } + b, err := json.Marshal(v) + if err != nil { + return 0 + } + return len(b) +} diff --git a/internal/cli/suggest.go b/internal/cli/suggest.go new file mode 100644 index 0000000..30cf29d --- /dev/null +++ b/internal/cli/suggest.go @@ -0,0 +1,82 @@ +package cli + +import ( + "fmt" + + "github.com/codestz/mcpx/internal/mcp" +) + +// nearestTool returns "Did you mean 'X'?" if any tool name is within Levenshtein +// distance 2 of the typo. Empty string otherwise. +func nearestTool(typo string, tools []mcp.Tool) string { + best, dist := "", 99 + for _, t := range tools { + d := levenshtein(typo, t.Name) + if d < dist { + best, dist = t.Name, d + } + } + if dist <= 2 && best != "" { + return fmt.Sprintf("Did you mean %q?", best) + } + return "" +} + +// nearestProperty suggests a flag name when an unknown one is given. +func nearestProperty(typo string, props map[string]mcp.PropertySchema) string { + best, dist := "", 99 + for name := range props { + d := levenshtein(typo, name) + if d < dist { + best, dist = name, d + } + } + if dist <= 2 && best != "" { + return fmt.Sprintf("did you mean --%s?", best) + } + return "" +} + +// levenshtein returns the edit distance between a and b. +// Iterative two-row implementation; O(len(a)*len(b)) time, O(len(b)) space. +func levenshtein(a, b string) int { + if a == b { + return 0 + } + if len(a) == 0 { + return len(b) + } + if len(b) == 0 { + return len(a) + } + prev := make([]int, len(b)+1) + curr := make([]int, len(b)+1) + for j := range prev { + prev[j] = j + } + for i := 1; i <= len(a); i++ { + curr[0] = i + for j := 1; j <= len(b); j++ { + cost := 1 + if a[i-1] == b[j-1] { + cost = 0 + } + del := prev[j] + 1 + ins := curr[j-1] + 1 + sub := prev[j-1] + cost + curr[j] = min3(del, ins, sub) + } + prev, curr = curr, prev + } + return prev[len(b)] +} + +func min3(a, b, c int) int { + if a <= b && a <= c { + return a + } + if b <= c { + return b + } + return c +} diff --git a/internal/cli/ui_cmd.go b/internal/cli/ui_cmd.go new file mode 100644 index 0000000..653efb9 --- /dev/null +++ b/internal/cli/ui_cmd.go @@ -0,0 +1,126 @@ +package cli + +import ( + "context" + "fmt" + "os" + "os/signal" + "syscall" + "time" + + "github.com/codestz/mcpx/internal/config" + "github.com/codestz/mcpx/internal/ui" + "github.com/spf13/cobra" +) + +// uiRunCmd is the hidden background entrypoint spawned by ui.EnsureRunningAsync. +// Equivalent in spirit to the existing __daemon command. +func uiRunCmd() *cobra.Command { + var port int + var bind string + var idle time.Duration + + cmd := &cobra.Command{ + Use: "__ui-run", + Hidden: true, + RunE: func(cmd *cobra.Command, args []string) error { + ctx, cancel := context.WithCancel(context.Background()) + defer cancel() + sigCh := make(chan os.Signal, 1) + signal.Notify(sigCh, syscall.SIGTERM, syscall.SIGINT) + go func() { + <-sigCh + cancel() + }() + return ui.Run(ctx, port, bind, idle) + }, + } + cmd.Flags().IntVar(&port, "port", 7878, "TCP port (0=ephemeral)") + cmd.Flags().StringVar(&bind, "bind", "127.0.0.1", "Bind address") + cmd.Flags().DurationVar(&idle, "idle-timeout", 1*time.Hour, "Idle shutdown") + return cmd +} + +// uiManageCmd exposes user-facing dashboard controls: status, stop, open, disable. +func uiManageCmd() *cobra.Command { + cmd := &cobra.Command{ + Use: "ui", + Short: "Manage the always-on dashboard daemon", + } + cmd.AddCommand(&cobra.Command{ + Use: "status", + Short: "Show dashboard URL or 'inactive'", + RunE: func(cmd *cobra.Command, args []string) error { + h, err := ui.LoadHandshake() + if err != nil { + fmt.Println("inactive") + return nil + } + fmt.Printf("http://%s:%d/?t=%s (pid %d)\n", h.Bind, h.Port, h.Token, h.PID) + return nil + }, + }) + cmd.AddCommand(&cobra.Command{ + Use: "stop", + Short: "Stop the dashboard daemon", + RunE: func(cmd *cobra.Command, args []string) error { + h, err := ui.LoadHandshake() + if err != nil { + fmt.Println("not running") + return nil + } + if h.PID > 0 { + if proc, err := os.FindProcess(h.PID); err == nil { + _ = proc.Signal(syscall.SIGTERM) + } + } + _ = ui.RemoveHandshake() + fmt.Println("stopped") + return nil + }, + }) + cmd.AddCommand(&cobra.Command{ + Use: "open", + Short: "Print the dashboard URL (suitable for `open $(...)`)", + RunE: func(cmd *cobra.Command, args []string) error { + h, err := ui.LoadHandshake() + if err != nil { + return fmt.Errorf("dashboard not running") + } + fmt.Printf("http://%s:%d/?t=%s\n", h.Bind, h.Port, h.Token) + return nil + }, + }) + cmd.AddCommand(&cobra.Command{ + Use: "disable", + Short: "Hint how to disable the dashboard permanently", + Run: func(cmd *cobra.Command, args []string) { + fmt.Println("Add to ~/.mcpx/config.yml:") + fmt.Println() + fmt.Println(" ui:") + fmt.Println(" enabled: false") + fmt.Println() + fmt.Println("Or set MCPX_UI=off in your environment.") + }, + }) + return cmd +} + +// startUIIfEnabled spawns the dashboard daemon according to config + env. +// +// The dashboard runs silently — its URL is surfaced only when the user asks: +// - `mcpx gain` prints the URL in its footer when the daemon is up +// - `mcpx ui status` prints the URL or "inactive" +// - `mcpx ui open` prints the URL alone (use with `open $(...)`) +// +// No CLI invocation prints the banner unsolicited. Opt out of spawning entirely +// with `MCPX_UI=off` or `ui.enabled: false` in config. +func startUIIfEnabled(cfg *config.Config) { + uiCfg := (*config.UIConfig)(nil).Default() + if cfg != nil && cfg.UI != nil { + uiCfg = cfg.UI.Default() + } + enabled := uiCfg.Enabled != nil && *uiCfg.Enabled + ui.EnsureRunningAsync(enabled, uiCfg.Port, uiCfg.Bind) +} + diff --git a/internal/cli/ui_handshake.go b/internal/cli/ui_handshake.go new file mode 100644 index 0000000..afea9ce --- /dev/null +++ b/internal/cli/ui_handshake.go @@ -0,0 +1,32 @@ +package cli + +import ( + "encoding/json" + "os" + "path/filepath" +) + +// uiHandshake mirrors the file written by the UI daemon at ~/.mcpx/ui.json. +// Loaded by the gain command (and any other CLI surface that wants to surface +// the dashboard URL) without depending on the internal/ui package. +type uiHandshake struct { + Port int `json:"port"` + Token string `json:"token"` + PID int `json:"pid"` + Bind string `json:"bind"` +} + +func readUIHandshake(home string) (uiHandshake, error) { + data, err := os.ReadFile(filepath.Join(home, ".mcpx", "ui.json")) + if err != nil { + return uiHandshake{}, err + } + var h uiHandshake + if err := json.Unmarshal(data, &h); err != nil { + return uiHandshake{}, err + } + if h.Bind == "" { + h.Bind = "127.0.0.1" + } + return h, nil +} diff --git a/internal/config/config.go b/internal/config/config.go index ce64582..77fea1e 100644 --- a/internal/config/config.go +++ b/internal/config/config.go @@ -12,6 +12,9 @@ import ( type Config struct { Servers map[string]*ServerConfig `yaml:"servers"` Security *SecurityConfig `yaml:"security"` + Gain *GainConfig `yaml:"gain"` + Cache *CacheConfig `yaml:"cache"` + UI *UIConfig `yaml:"ui"` } // ServerConfig describes a single MCP server. @@ -98,6 +101,90 @@ type ServerSecurity struct { Policies []Policy `yaml:"policies"` } +// GainConfig controls token-savings analytics and JSONL stats. +type GainConfig struct { + Enabled *bool `yaml:"enabled"` + Tokenizer string `yaml:"tokenizer"` // "estimate" (default) | "tiktoken" + RetainDays int `yaml:"retain_days"` // 0 = no retention pruning + StatsPath string `yaml:"stats_path"` // override default ~/.mcpx/stats.jsonl +} + +// CacheConfig controls the schema cache (tools/list, prompts/list, resources/list). +// +// mcpx deliberately ships without a result cache: caching tool RESPONSES is a +// correctness footgun (stale data after edits, mutations, or external state +// changes) that we couldn't make correct without server cooperation +// (`readOnlyHint`). The schema cache is correctness-safe — schemas change +// only on server upgrades and the worst case is missing a freshly-added tool +// for one TTL window. +type CacheConfig struct { + SchemaTTL string `yaml:"schema_ttl"` // duration, e.g. "5m" +} + +// UIConfig controls the always-on dashboard. +type UIConfig struct { + Enabled *bool `yaml:"enabled"` + Port int `yaml:"port"` // 0 = ephemeral + Bind string `yaml:"bind"` // default "127.0.0.1" + IdleTimeout string `yaml:"idle_timeout"` // duration, default "1h" +} + + +// Default returns the GainConfig with defaults applied. +func (g *GainConfig) Default() GainConfig { + out := GainConfig{Tokenizer: "estimate", RetainDays: 30} + if g != nil { + if g.Enabled != nil { + out.Enabled = g.Enabled + } + if g.Tokenizer != "" { + out.Tokenizer = g.Tokenizer + } + if g.RetainDays != 0 { + out.RetainDays = g.RetainDays + } + out.StatsPath = g.StatsPath + } + if out.Enabled == nil { + t := true + out.Enabled = &t + } + return out +} + +// Default returns the CacheConfig with defaults applied. +func (c *CacheConfig) Default() CacheConfig { + out := CacheConfig{SchemaTTL: "5m"} + if c != nil && c.SchemaTTL != "" { + out.SchemaTTL = c.SchemaTTL + } + return out +} + +// Default returns the UIConfig with defaults applied. +func (u *UIConfig) Default() UIConfig { + out := UIConfig{Port: 7878, Bind: "127.0.0.1", IdleTimeout: "1h"} + if u != nil { + if u.Enabled != nil { + out.Enabled = u.Enabled + } + if u.Port != 0 { + out.Port = u.Port + } + if u.Bind != "" { + out.Bind = u.Bind + } + if u.IdleTimeout != "" { + out.IdleTimeout = u.IdleTimeout + } + } + if out.Enabled == nil { + t := true + out.Enabled = &t + } + return out +} + // Load reads the global (~/.mcpx/config.yml) and project (.mcpx/config.yml) // configs, merges them, and validates the result. // Returns the merged config, the project root (empty if no project config), and any error. @@ -201,6 +288,20 @@ func Merge(global, project *Config) *Config { merged.Security = project.Security } + // Gain/Cache/UI: project overrides global at the section level. + merged.Gain = global.Gain + if project.Gain != nil { + merged.Gain = project.Gain + } + merged.Cache = global.Cache + if project.Cache != nil { + merged.Cache = project.Cache + } + merged.UI = global.UI + if project.UI != nil { + merged.UI = project.UI + } + return merged } diff --git a/internal/find/find.go b/internal/find/find.go new file mode 100644 index 0000000..93794c9 --- /dev/null +++ b/internal/find/find.go @@ -0,0 +1,222 @@ +// Package find ranks MCP tools across all configured servers by relevance to +// a free-text query. Built for the AI-agent discovery moment: instead of +// scanning `mcpx list -v` (5–15K tokens), `mcpx find "search code"` returns +// 3 ranked candidates in ~80 tokens. +// +// Algorithm: BM25 over (tool name + description), with a name-match bonus and +// snake_case/camelCase splitting so "find symbol" matches "find_symbol". +package find + +import ( + "math" + "regexp" + "sort" + "strings" +) + +// Tool is the input record for ranking. Server distinguishes tools with the +// same name across multiple servers. +type Tool struct { + Server string + Name string + Description string +} + +// Result is a ranked match. +type Result struct { + Server string + Name string + Description string + Score float64 +} + +// Rank scores tools against query and returns the top-K matches by score. +// topK <= 0 returns all matches with score > 0. +func Rank(query string, tools []Tool, topK int) []Result { + qTokens := tokenize(query) + if len(qTokens) == 0 || len(tools) == 0 { + return nil + } + + // Pre-tokenize each tool's name and description. + type doc struct { + idx int + nameToks []string + descToks []string + totalLen int + } + docs := make([]doc, len(tools)) + totalLen := 0 + for i, t := range tools { + nt := tokenize(t.Name) + dt := tokenize(t.Description) + docs[i] = doc{idx: i, nameToks: nt, descToks: dt, totalLen: len(nt) + len(dt)} + totalLen += len(nt) + len(dt) + } + avgLen := float64(totalLen) / float64(len(docs)) + if avgLen == 0 { + avgLen = 1 + } + + // Document frequency per query token. + df := map[string]int{} + for _, q := range qTokens { + for i := range docs { + if contains(docs[i].nameToks, q) || contains(docs[i].descToks, q) { + df[q]++ + } + } + } + + qSet := map[string]bool{} + for _, q := range qTokens { + qSet[q] = true + } + + const k1, b float64 = 1.5, 0.75 + results := make([]Result, 0, len(docs)) + for _, d := range docs { + score := 0.0 + nameMatches := 0 + for _, q := range qTokens { + n := df[q] + if n == 0 { + continue + } + idf := math.Log(1 + (float64(len(docs))-float64(n)+0.5)/(float64(n)+0.5)) + + nameTF := count(d.nameToks, q) + descTF := count(d.descToks, q) + tf := float64(descTF) + 5.0*float64(nameTF) // name tokens weigh much more + + if tf == 0 { + continue + } + if nameTF > 0 { + nameMatches++ + } + norm := tf * (k1 + 1) / (tf + k1*(1-b+b*float64(d.totalLen)/avgLen)) + score += idf * norm + } + + // Tight-match bonus: tools whose name covers the entire query rank above + // tools that merely contain the query tokens among many extras. + if nameMatches == len(qTokens) { + extras := 0 + for _, t := range d.nameToks { + if !qSet[t] { + extras++ + } + } + score *= 1.0 + 0.5/float64(1+extras) // 1.5x at extras=0, 1.25x at extras=1, ... + } + + if score > 0 { + t := tools[d.idx] + results = append(results, Result{ + Server: t.Server, Name: t.Name, Description: t.Description, Score: score, + }) + } + } + + sort.Slice(results, func(i, j int) bool { + if results[i].Score != results[j].Score { + return results[i].Score > results[j].Score + } + // Stable tiebreak: server.tool ascending. + return results[i].Server+"."+results[i].Name < results[j].Server+"."+results[j].Name + }) + + if topK > 0 && len(results) > topK { + results = results[:topK] + } + return results +} + +// tokenize lowercases input and splits on non-alphanumeric chars and case +// boundaries. "find_symbol" → ["find","symbol"]; "FindSymbol" → ["find","symbol"]. +// Trailing-s plurals collapse so "issue" matches "issues". +func tokenize(s string) []string { + if s == "" { + return nil + } + s = camelSplit(s) + parts := splitWords(strings.ToLower(s)) + out := make([]string, 0, len(parts)) + for _, p := range parts { + if p == "" || stopwords[p] { + continue + } + out = append(out, stem(p)) + } + return out +} + +// stem collapses trivial English plurals: "issues"→"issue", "files"→"file", +// but preserves "class"→"class" (double s) and "is"→"is" (too short). +func stem(s string) string { + if len(s) <= 3 { + return s + } + if !strings.HasSuffix(s, "s") { + return s + } + if strings.HasSuffix(s, "ss") || strings.HasSuffix(s, "us") || strings.HasSuffix(s, "is") { + return s + } + return s[:len(s)-1] +} + +var camelRE = regexp.MustCompile(`([a-z0-9])([A-Z])`) + +func camelSplit(s string) string { + return camelRE.ReplaceAllString(s, "$1 $2") +} + +func splitWords(s string) []string { + var out []string + start := -1 + for i, r := range s { + isWord := (r >= 'a' && r <= 'z') || (r >= '0' && r <= '9') + if isWord { + if start < 0 { + start = i + } + } else { + if start >= 0 { + out = append(out, s[start:i]) + start = -1 + } + } + } + if start >= 0 { + out = append(out, s[start:]) + } + return out +} + +func contains(toks []string, q string) bool { + for _, t := range toks { + if t == q { + return true + } + } + return false +} + +func count(toks []string, q string) int { + n := 0 + for _, t := range toks { + if t == q { + n++ + } + } + return n +} + +// stopwords are skipped in queries and corpora — they don't help discrimination. +var stopwords = map[string]bool{ + "the": true, "a": true, "an": true, "and": true, "or": true, "to": true, + "of": true, "in": true, "on": true, "for": true, "with": true, "by": true, + "is": true, "be": true, "are": true, "this": true, "that": true, +} diff --git a/internal/find/find_test.go b/internal/find/find_test.go new file mode 100644 index 0000000..d3231a1 --- /dev/null +++ b/internal/find/find_test.go @@ -0,0 +1,118 @@ +package find + +import "testing" + +func corpus() []Tool { + return []Tool{ + {Server: "serena", Name: "find_symbol", Description: "Retrieves info on symbols by name path"}, + {Server: "serena", Name: "find_referencing_symbols", Description: "Find references to a symbol"}, + {Server: "serena", Name: "search_for_pattern", Description: "Flexible regex search across files"}, + {Server: "serena", Name: "list_dir", Description: "List directory contents"}, + {Server: "github", Name: "search_issues", Description: "Search GitHub issues by query"}, + {Server: "github", Name: "get_issue", Description: "Fetch a single GitHub issue by number"}, + {Server: "postgres", Name: "query", Description: "Execute SQL queries"}, + } +} + +func TestRankByName(t *testing.T) { + out := Rank("find symbol", corpus(), 3) + if len(out) == 0 { + t.Fatal("no results") + } + if out[0].Name != "find_symbol" { + t.Errorf("top result: got %s want find_symbol", out[0].Name) + } +} + +func TestRankByDescription(t *testing.T) { + out := Rank("regex pattern", corpus(), 3) + if len(out) == 0 { + t.Fatal("no results") + } + if out[0].Name != "search_for_pattern" { + t.Errorf("top: got %s want search_for_pattern", out[0].Name) + } +} + +func TestRankCrossServer(t *testing.T) { + out := Rank("issue", corpus(), 5) + if len(out) < 2 { + t.Fatal("expected >=2 results") + } + servers := map[string]bool{} + for _, r := range out { + servers[r.Server] = true + } + if !servers["github"] { + t.Error("github tools missing from issue ranking") + } +} + +func TestRankNoMatchReturnsEmpty(t *testing.T) { + out := Rank("xyzzy", corpus(), 5) + if len(out) != 0 { + t.Errorf("expected zero results, got %d", len(out)) + } +} + +func TestRankTopKLimit(t *testing.T) { + out := Rank("symbol", corpus(), 1) + if len(out) != 1 { + t.Errorf("topK=1: got %d results", len(out)) + } +} + +func TestRankIgnoresStopwords(t *testing.T) { + a := Rank("the symbol", corpus(), 3) + b := Rank("symbol", corpus(), 3) + if len(a) == 0 || len(b) == 0 { + t.Fatal("no results") + } + if a[0].Name != b[0].Name { + t.Errorf("stopword affected top match: a=%s b=%s", a[0].Name, b[0].Name) + } +} + +func TestCamelSplit(t *testing.T) { + if got := camelSplit("FindSymbol"); got != "Find Symbol" { + t.Errorf("got %q", got) + } + if got := camelSplit("find_symbol"); got != "find_symbol" { + t.Errorf("got %q", got) + } + if got := camelSplit("HTTPRequest"); got != "HTTPRequest" { + t.Errorf("HTTPRequest acronym (no boundary detected): got %q", got) + } +} + +func TestTokenize(t *testing.T) { + cases := map[string][]string{ + "find_symbol": {"find", "symbol"}, + "FindSymbol": {"find", "symbol"}, + "search-for-pattern": {"search", "for", "pattern"}, + "the symbol of": {"symbol"}, + "": nil, + } + // "for" is in our stopwords, but I want it to remain because it's domain-relevant. + // Actually "for" is in the stopwords list. So "search-for-pattern" should be ["search","pattern"]. + cases["search-for-pattern"] = []string{"search", "pattern"} + + for in, want := range cases { + got := tokenize(in) + if !sliceEq(got, want) { + t.Errorf("tokenize(%q): got %v want %v", in, got, want) + } + } +} + +func sliceEq(a, b []string) bool { + if len(a) != len(b) { + return false + } + for i := range a { + if a[i] != b[i] { + return false + } + } + return true +} diff --git a/internal/mcp/client.go b/internal/mcp/client.go index 7bdb462..11dad9f 100644 --- a/internal/mcp/client.go +++ b/internal/mcp/client.go @@ -27,7 +27,7 @@ func (c *Client) Initialize(ctx context.Context) error { "capabilities": map[string]any{}, "clientInfo": map[string]any{ "name": "mcpx", - "version": "1.5.0", + "version": "1.6.0", }, } diff --git a/internal/mcp/schema.go b/internal/mcp/schema.go new file mode 100644 index 0000000..7f12fc0 --- /dev/null +++ b/internal/mcp/schema.go @@ -0,0 +1,252 @@ +package mcp + +import ( + "encoding/json" + "fmt" + "os" +) + +// JSON Schema normalization for MCP tool inputs. +// +// MCP servers report tool schemas as JSON Schema documents. Real-world servers +// use draft-07 features that do not map 1:1 to mcpx's flat PropertySchema: +// +// - "type": ["string", "null"] ← Sentry, GitHub (issue #14) +// - "oneOf"/"anyOf" ← discriminated unions +// - "allOf" ← schema composition +// - "$ref" ← cross-references +// +// This file gives PropertySchema and InputSchema custom UnmarshalJSON +// implementations that flatten all of the above into the canonical mcpx shape: +// +// * type chosen as the first non-null primitive +// * Nullable=true if "null" appeared in a union +// * unknown keywords preserved in Ext for round-tripping / inspection +// +// We never error out — unrecognized shapes degrade to type="any" with the +// original payload kept in Ext. MCPX_VERBOSE=1 logs warnings to stderr. + +// Known JSON Schema keywords mcpx maps to PropertySchema fields. Anything else +// goes into Ext. +var knownKeys = map[string]bool{ + "type": true, + "description": true, + "default": true, + "enum": true, + "items": true, + "nullable": true, + "properties": true, + "required": true, +} + +// UnmarshalJSON normalizes a JSON Schema property into a PropertySchema. +// Handles type union arrays, oneOf/anyOf branches, and allOf composition. +func (p *PropertySchema) UnmarshalJSON(data []byte) error { + var raw map[string]json.RawMessage + if err := json.Unmarshal(data, &raw); err != nil { + // Not an object — treat as untyped. + p.Type = "any" + return nil + } + + // allOf: merge all branches into raw before any other processing. + if a, ok := raw["allOf"]; ok { + var branches []map[string]json.RawMessage + if err := json.Unmarshal(a, &branches); err == nil { + for _, br := range branches { + for k, v := range br { + if _, exists := raw[k]; !exists { + raw[k] = v + } + } + } + } + delete(raw, "allOf") + } + + // oneOf / anyOf: pick the first non-null branch's "type", remember all in Ext. + if branches, ok := pickUnion(raw, "oneOf"); ok { + applyFirstNonNullType(&raw, branches, p) + } + if branches, ok := pickUnion(raw, "anyOf"); ok { + applyFirstNonNullType(&raw, branches, p) + } + + // type: string OR [string,...] + if t, ok := raw["type"]; ok { + if typ, nullable, err := normalizeType(t); err == nil { + p.Type = typ + if nullable { + p.Nullable = true + } + } else { + verboseLog("schema: type keyword not parseable: %v", err) + p.Type = "any" + } + } else if p.Type == "" { + p.Type = "any" + } + + // description, default, enum, items. + if d, ok := raw["description"]; ok { + _ = json.Unmarshal(d, &p.Description) + } + if d, ok := raw["default"]; ok { + _ = json.Unmarshal(d, &p.Default) + } + if d, ok := raw["enum"]; ok { + _ = json.Unmarshal(d, &p.Enum) + } + if d, ok := raw["items"]; ok { + var items PropertySchema + if err := json.Unmarshal(d, &items); err == nil { + p.Items = &items + } + } + + // Everything else goes into Ext. + for k, v := range raw { + if knownKeys[k] { + continue + } + if p.Ext == nil { + p.Ext = map[string]json.RawMessage{} + } + p.Ext[k] = v + } + + return nil +} + +// UnmarshalJSON normalizes the top-level tool input schema. Same union/composition +// handling as PropertySchema but only the relevant subset. +func (s *InputSchema) UnmarshalJSON(data []byte) error { + var raw map[string]json.RawMessage + if err := json.Unmarshal(data, &raw); err != nil { + s.Type = "object" + return nil + } + + if t, ok := raw["type"]; ok { + typ, _, err := normalizeType(t) + if err == nil { + s.Type = typ + } else { + s.Type = "object" + } + } else { + s.Type = "object" + } + + if p, ok := raw["properties"]; ok { + _ = json.Unmarshal(p, &s.Properties) + } + if r, ok := raw["required"]; ok { + _ = json.Unmarshal(r, &s.Required) + } + + for k, v := range raw { + if knownKeys[k] { + continue + } + if s.Ext == nil { + s.Ext = map[string]json.RawMessage{} + } + s.Ext[k] = v + } + + return nil +} + +// normalizeType handles JSON Schema's `type` keyword which may be a string or +// a string array. For arrays, returns the first non-null entry plus nullable=true +// if "null" was present. +func normalizeType(raw json.RawMessage) (string, bool, error) { + // Try string first. + var s string + if err := json.Unmarshal(raw, &s); err == nil { + if s == "null" { + return "any", true, nil + } + return s, false, nil + } + // Try array. + var arr []string + if err := json.Unmarshal(raw, &arr); err == nil { + nullable := false + first := "" + for _, t := range arr { + if t == "null" { + nullable = true + continue + } + if first == "" { + first = t + } + } + if first == "" { + first = "any" + } + return first, nullable, nil + } + return "", false, fmt.Errorf("type is neither string nor []string: %s", raw) +} + +// pickUnion extracts a oneOf/anyOf array if present; deletes the key from raw +// and returns the parsed branches. +func pickUnion(raw map[string]json.RawMessage, key string) ([]map[string]json.RawMessage, bool) { + v, ok := raw[key] + if !ok { + return nil, false + } + delete(raw, key) + var branches []map[string]json.RawMessage + if err := json.Unmarshal(v, &branches); err != nil { + return nil, false + } + return branches, true +} + +// applyFirstNonNullType walks union branches; for the first branch with a +// non-null `type`, copies it into raw if raw has none. Sets nullable=true if +// any branch was {"type":"null"}. +func applyFirstNonNullType(raw *map[string]json.RawMessage, branches []map[string]json.RawMessage, p *PropertySchema) { + for _, br := range branches { + t, ok := br["type"] + if !ok { + continue + } + typ, nullable, err := normalizeType(t) + if err != nil { + continue + } + if typ == "null" || typ == "any" && nullable { + p.Nullable = true + continue + } + if _, has := (*raw)["type"]; !has { + (*raw)["type"] = t + } + if nullable { + p.Nullable = true + } + // Copy other primitive keywords from the chosen branch if not already set. + for k, v := range br { + if k == "type" { + continue + } + if _, has := (*raw)[k]; !has { + (*raw)[k] = v + } + } + return + } +} + +// verboseLog emits a warning to stderr only when MCPX_VERBOSE=1. +func verboseLog(format string, args ...any) { + if os.Getenv("MCPX_VERBOSE") != "1" { + return + } + fmt.Fprintf(os.Stderr, "mcpx: "+format+"\n", args...) +} diff --git a/internal/mcp/schema_test.go b/internal/mcp/schema_test.go new file mode 100644 index 0000000..9e9d9f3 --- /dev/null +++ b/internal/mcp/schema_test.go @@ -0,0 +1,186 @@ +package mcp + +import ( + "encoding/json" + "testing" +) + +// TestSentryUnionType reproduces issue #14: type as ["string", "null"]. +func TestSentryUnionType(t *testing.T) { + raw := []byte(`{"type": ["string", "null"], "description": "An optional id"}`) + var p PropertySchema + if err := json.Unmarshal(raw, &p); err != nil { + t.Fatalf("unmarshal failed: %v", err) + } + if p.Type != "string" { + t.Errorf("type: got %q want %q", p.Type, "string") + } + if !p.Nullable { + t.Error("expected Nullable=true") + } + if p.Description != "An optional id" { + t.Errorf("description: got %q", p.Description) + } +} + +func TestPlainStringType(t *testing.T) { + raw := []byte(`{"type": "string", "description": "x"}`) + var p PropertySchema + if err := json.Unmarshal(raw, &p); err != nil { + t.Fatal(err) + } + if p.Type != "string" || p.Nullable { + t.Errorf("got type=%q nullable=%v", p.Type, p.Nullable) + } +} + +func TestArrayItems(t *testing.T) { + raw := []byte(`{"type": "array", "items": {"type": "string"}}`) + var p PropertySchema + if err := json.Unmarshal(raw, &p); err != nil { + t.Fatal(err) + } + if p.Type != "array" { + t.Fatalf("type: got %q", p.Type) + } + if p.Items == nil || p.Items.Type != "string" { + t.Errorf("items: %+v", p.Items) + } +} + +func TestOneOfFirstNonNull(t *testing.T) { + raw := []byte(`{"oneOf": [{"type": "null"}, {"type": "string"}]}`) + var p PropertySchema + if err := json.Unmarshal(raw, &p); err != nil { + t.Fatal(err) + } + if p.Type != "string" { + t.Errorf("type: got %q want string", p.Type) + } + if !p.Nullable { + t.Error("expected Nullable=true") + } +} + +func TestAllOfMerges(t *testing.T) { + raw := []byte(`{"allOf": [{"type": "string"}, {"description": "merged"}]}`) + var p PropertySchema + if err := json.Unmarshal(raw, &p); err != nil { + t.Fatal(err) + } + if p.Type != "string" { + t.Errorf("type: got %q want string", p.Type) + } + if p.Description != "merged" { + t.Errorf("description: got %q want merged", p.Description) + } +} + +func TestUnknownKeywordsPreservedInExt(t *testing.T) { + raw := []byte(`{"type": "string", "format": "uuid", "pattern": "^[0-9a-f-]+$"}`) + var p PropertySchema + if err := json.Unmarshal(raw, &p); err != nil { + t.Fatal(err) + } + if p.Ext["format"] == nil { + t.Error("format not in Ext") + } + if p.Ext["pattern"] == nil { + t.Error("pattern not in Ext") + } +} + +func TestInputSchemaUnknownKeywords(t *testing.T) { + raw := []byte(`{ + "type": "object", + "properties": {"id": {"type": ["string","null"]}}, + "required": ["id"], + "additionalProperties": false + }`) + var s InputSchema + if err := json.Unmarshal(raw, &s); err != nil { + t.Fatal(err) + } + if s.Type != "object" { + t.Errorf("type: got %q", s.Type) + } + if s.Ext["additionalProperties"] == nil { + t.Error("additionalProperties not in Ext") + } + prop, ok := s.Properties["id"] + if !ok { + t.Fatal("id property missing") + } + if prop.Type != "string" || !prop.Nullable { + t.Errorf("id: type=%q nullable=%v", prop.Type, prop.Nullable) + } +} + +// Real-world Sentry-style schema fragment, reproducing #14 in full context. +func TestSentryFullToolSchema(t *testing.T) { + raw := []byte(`{ + "type": "object", + "properties": { + "organization_slug": {"type": "string"}, + "project_slug": {"type": ["string", "null"], "description": "Optional project filter"}, + "environment": {"type": ["string", "null"]}, + "limit": {"type": ["integer", "null"], "default": 10} + }, + "required": ["organization_slug"] + }`) + var s InputSchema + if err := json.Unmarshal(raw, &s); err != nil { + t.Fatalf("Sentry schema failed: %v", err) + } + if len(s.Properties) != 4 { + t.Fatalf("properties: got %d want 4", len(s.Properties)) + } + for name, want := range map[string]struct { + typ string + nullable bool + }{ + "organization_slug": {"string", false}, + "project_slug": {"string", true}, + "environment": {"string", true}, + "limit": {"integer", true}, + } { + got := s.Properties[name] + if got.Type != want.typ || got.Nullable != want.nullable { + t.Errorf("%s: got type=%q nullable=%v want %q nullable=%v", + name, got.Type, got.Nullable, want.typ, want.nullable) + } + } +} + +// Nested oneOf inside properties (GitHub-style discriminated union). +func TestNestedOneOf(t *testing.T) { + raw := []byte(`{ + "type": "object", + "properties": { + "action": {"oneOf": [{"type": "string", "enum": ["create"]}, {"type": "string", "enum": ["delete"]}]} + } + }`) + var s InputSchema + if err := json.Unmarshal(raw, &s); err != nil { + t.Fatal(err) + } + got := s.Properties["action"] + if got.Type != "string" { + t.Errorf("type: got %q", got.Type) + } +} + +// Degenerate / unknown shape — must not panic, must not error, falls back to "any". +func TestDegenerateSchema(t *testing.T) { + raw := []byte(`{"$ref": "#/definitions/Whatever"}`) + var p PropertySchema + if err := json.Unmarshal(raw, &p); err != nil { + t.Fatal(err) + } + if p.Type != "any" { + t.Errorf("type: got %q want any", p.Type) + } + if p.Ext["$ref"] == nil { + t.Error("$ref should be preserved in Ext") + } +} diff --git a/internal/mcp/types.go b/internal/mcp/types.go index f8004d1..032eedd 100644 --- a/internal/mcp/types.go +++ b/internal/mcp/types.go @@ -46,9 +46,10 @@ type Tool struct { // InputSchema describes the JSON Schema for a tool's input. type InputSchema struct { - Type string `json:"type"` - Properties map[string]PropertySchema `json:"properties"` - Required []string `json:"required"` + Type string `json:"type"` + Properties map[string]PropertySchema `json:"properties"` + Required []string `json:"required"` + Ext map[string]json.RawMessage `json:"-"` } // PropertySchema describes a single property in a tool's input schema. @@ -58,6 +59,15 @@ type PropertySchema struct { Default any `json:"default,omitempty"` Enum []any `json:"enum,omitempty"` Items *PropertySchema `json:"items,omitempty"` + + // Nullable is true when the source schema specified ["T", "null"] or had a + // "null" branch in oneOf/anyOf. Allows tools to know a value may legitimately + // be missing or null. + Nullable bool `json:"nullable,omitempty"` + + // Ext preserves JSON Schema keywords mcpx does not natively model + // (oneOf, anyOf, allOf, $ref, format, pattern, etc). Round-trippable. + Ext map[string]json.RawMessage `json:"-"` } // Prompt describes an MCP prompt exposed by a server. diff --git a/internal/render/render.go b/internal/render/render.go new file mode 100644 index 0000000..10c5117 --- /dev/null +++ b/internal/render/render.go @@ -0,0 +1,181 @@ +// Package render provides terminal-rendering primitives for mcpx's premium TUI: +// boxes, bars, sparklines, formatted numbers/durations, and width detection. +// +// All helpers are color-agnostic — pass already-colored strings if needed. +package render + +import ( + "fmt" + "os" + "strings" + "time" + + "golang.org/x/term" +) + +// TermWidth returns the current terminal width in columns. Falls back to 80 +// for non-tty / piped output. +func TermWidth() int { + if !term.IsTerminal(int(os.Stdout.Fd())) { + return 100 + } + w, _, err := term.GetSize(int(os.Stdout.Fd())) + if err != nil || w <= 0 { + return 80 + } + return w +} + +// FormatNumber renders an integer with K/M/B suffix at one decimal place. +// 1234 → "1.2K" +// 487231 → "487K" +// 4800000 → "4.8M" +func FormatNumber(n int64) string { + abs := n + if abs < 0 { + abs = -abs + } + switch { + case abs >= 1_000_000_000: + return fmt.Sprintf("%.1fB", float64(n)/1_000_000_000) + case abs >= 1_000_000: + v := float64(n) / 1_000_000 + if v >= 100 { + return fmt.Sprintf("%.0fM", v) + } + return fmt.Sprintf("%.1fM", v) + case abs >= 1_000: + v := float64(n) / 1_000 + if v >= 100 { + return fmt.Sprintf("%.0fK", v) + } + return fmt.Sprintf("%.1fK", v) + default: + return fmt.Sprintf("%d", n) + } +} + +// FormatPercent renders a [0,1] float as a percent string with no decimals. +func FormatPercent(p float64) string { + return fmt.Sprintf("%d%%", int(p*100+0.5)) +} + +// FormatDuration renders ms as the most natural unit: +// <1000 → "412ms" +// 1000..59999 → "12s" +// ≥60000 → "1m23s" +func FormatDuration(ms int64) string { + if ms < 1000 { + return fmt.Sprintf("%dms", ms) + } + d := time.Duration(ms) * time.Millisecond + if d < time.Minute { + s := int(d.Seconds()) + return fmt.Sprintf("%ds", s) + } + m := int(d.Minutes()) + s := int(d.Seconds()) - m*60 + return fmt.Sprintf("%dm%02ds", m, s) +} + +// Bar renders a horizontal block-bar of width chars representing pct ∈ [0,1]. +// Uses 8 fractional levels for sub-cell precision. +// +// Width 10, pct 0.42 → "▓▓▓▓▎ " +func Bar(pct float64, width int) string { + if width <= 0 { + return "" + } + if pct < 0 { + pct = 0 + } + if pct > 1 { + pct = 1 + } + full := int(pct * float64(width)) + rem := pct*float64(width) - float64(full) + out := strings.Repeat("█", full) + if full < width { + // 8-level partials + idx := int(rem * 8) + if idx > 0 { + parts := []rune{' ', '▏', '▎', '▍', '▌', '▋', '▊', '▉'} + out += string(parts[idx]) + full++ + } + out += strings.Repeat(" ", width-full) + } + return out +} + +// Sparkline renders values into a single-row sparkline using 8 vertical block +// levels. Empty input returns an empty string. +// +// [1,3,2,5,4] → "▁▄▃▇▅" +func Sparkline(values []int64) string { + if len(values) == 0 { + return "" + } + var max int64 + for _, v := range values { + if v > max { + max = v + } + } + if max == 0 { + return strings.Repeat("▁", len(values)) + } + levels := []rune{'▁', '▂', '▃', '▄', '▅', '▆', '▇', '█'} + var b strings.Builder + for _, v := range values { + pct := float64(v) / float64(max) + idx := int(pct*7 + 0.0001) + if idx > 7 { + idx = 7 + } + if idx < 0 { + idx = 0 + } + b.WriteRune(levels[idx]) + } + return b.String() +} + +// Truncate clips s to max chars adding an ellipsis if needed. +func Truncate(s string, max int) string { + if max <= 0 { + return "" + } + if len([]rune(s)) <= max { + return s + } + r := []rune(s) + if max <= 1 { + return string(r[:max]) + } + return string(r[:max-1]) + "…" +} + +// PadRight pads s with spaces on the right to fit width chars. +func PadRight(s string, width int) string { + n := len([]rune(s)) + if n >= width { + return s + } + return s + strings.Repeat(" ", width-n) +} + +// Box-drawing constants for the Premium TUI. +const ( + BoxTL = "╭" + BoxTR = "╮" + BoxBL = "╰" + BoxBR = "╯" + BoxH = "─" + BoxV = "│" + BoxT = "┬" + BoxB = "┴" + BoxX = "┼" + BoxL = "├" + BoxR = "┤" +) diff --git a/internal/render/render_test.go b/internal/render/render_test.go new file mode 100644 index 0000000..a1f02d5 --- /dev/null +++ b/internal/render/render_test.go @@ -0,0 +1,95 @@ +package render + +import "testing" + +func TestFormatNumber(t *testing.T) { + cases := map[int64]string{ + 0: "0", + 42: "42", + 999: "999", + 1234: "1.2K", + 487231: "487K", + 4_800_000: "4.8M", + 120_000_000: "120M", + 1_500_000_000: "1.5B", + } + for in, want := range cases { + if got := FormatNumber(in); got != want { + t.Errorf("FormatNumber(%d): got %s want %s", in, got, want) + } + } +} + +func TestFormatPercent(t *testing.T) { + if FormatPercent(0.5) != "50%" { + t.Error("0.5 → 50%") + } + if FormatPercent(1) != "100%" { + t.Error("1 → 100%") + } + if FormatPercent(0) != "0%" { + t.Error("0 → 0%") + } +} + +func TestFormatDuration(t *testing.T) { + cases := map[int64]string{ + 412: "412ms", + 1500: "1s", + 59000: "59s", + 60000: "1m00s", + 61500: "1m01s", + 125000: "2m05s", + } + for in, want := range cases { + if got := FormatDuration(in); got != want { + t.Errorf("FormatDuration(%d): got %s want %s", in, got, want) + } + } +} + +func TestBar(t *testing.T) { + if got := Bar(0, 10); got != " " { + t.Errorf("0%%: got %q", got) + } + if got := Bar(1, 10); got != "██████████" { + t.Errorf("100%%: got %q", got) + } + if got := Bar(0.5, 10); got != "█████ " { + t.Errorf("50%%: got %q", got) + } +} + +func TestSparkline(t *testing.T) { + if got := Sparkline(nil); got != "" { + t.Errorf("empty: got %q", got) + } + if got := Sparkline([]int64{0, 0, 0}); got != "▁▁▁" { + t.Errorf("all zero: got %q", got) + } + got := Sparkline([]int64{1, 2, 4, 8}) + if len([]rune(got)) != 4 { + t.Errorf("4 values → %d runes", len([]rune(got))) + } +} + +func TestTruncate(t *testing.T) { + if Truncate("hello", 10) != "hello" { + t.Error("under width") + } + if Truncate("hello world", 5) != "hell…" { + t.Errorf("got %q", Truncate("hello world", 5)) + } + if Truncate("", 5) != "" { + t.Error("empty") + } +} + +func TestPadRight(t *testing.T) { + if PadRight("hi", 5) != "hi " { + t.Errorf("got %q", PadRight("hi", 5)) + } + if PadRight("hello", 3) != "hello" { + t.Error("over width preserved") + } +} diff --git a/internal/schemacache/cache.go b/internal/schemacache/cache.go new file mode 100644 index 0000000..0252ace --- /dev/null +++ b/internal/schemacache/cache.go @@ -0,0 +1,171 @@ +// Package schemacache stores MCP server schemas (initialize result + tools/ +// prompts/resources lists) in ~/.mcpx/cache/.json with a TTL. +// +// Hits skip the full MCP handshake on repeat invocations. The native_baseline_tokens +// field — what loading this server natively would cost an agent — is computed +// once at populate time and consumed by the stats writer. +package schemacache + +import ( + "crypto/sha256" + "encoding/hex" + "encoding/json" + "errors" + "os" + "path/filepath" + "sort" + "strings" + "time" + + "github.com/codestz/mcpx/internal/mcp" +) + +// Entry is the on-disk schema cache record for one server. +type Entry struct { + Version int `json:"version"` + CapturedAt time.Time `json:"captured_at"` + TTL time.Duration `json:"ttl"` + ServerHash string `json:"server_hash"` + Initialize mcp.InitializeResult `json:"initialize"` + Tools []mcp.Tool `json:"tools"` + Prompts []mcp.Prompt `json:"prompts,omitempty"` + Resources []mcp.Resource `json:"resources,omitempty"` + NativeBaselineToks int `json:"native_baseline_tokens"` +} + +// Key derives a stable cache key from server-defining attributes. Different +// command/args/env/url → different cache entry, so config changes are never +// served stale data. +func Key(command string, args []string, env map[string]string, url string) string { + h := sha256.New() + _, _ = h.Write([]byte(command)) + _, _ = h.Write([]byte{0}) + for _, a := range args { + _, _ = h.Write([]byte(a)) + _, _ = h.Write([]byte{0}) + } + keys := make([]string, 0, len(env)) + for k := range env { + keys = append(keys, k) + } + sort.Strings(keys) + for _, k := range keys { + _, _ = h.Write([]byte(k)) + _, _ = h.Write([]byte("=")) + _, _ = h.Write([]byte(env[k])) + _, _ = h.Write([]byte{0}) + } + _, _ = h.Write([]byte(url)) + return hex.EncodeToString(h.Sum(nil))[:16] +} + +// Path returns the on-disk path for a cache entry. Caller must ensure the +// parent directory exists; Save() handles creation. +func Path(key string) string { + home, err := os.UserHomeDir() + if err != nil { + return "" + } + return filepath.Join(home, ".mcpx", "cache", "schemas", key+".json") +} + +// Load reads an entry from disk. Returns (entry, true, nil) on a fresh hit, +// (nil, false, nil) on a miss or expired entry, and an error only on I/O +// failures other than "not exist". +func Load(key string) (*Entry, bool, error) { + path := Path(key) + if path == "" { + return nil, false, nil + } + data, err := os.ReadFile(path) + if err != nil { + if os.IsNotExist(err) { + return nil, false, nil + } + return nil, false, err + } + var e Entry + if err := json.Unmarshal(data, &e); err != nil { + // Corrupt cache entry — treat as miss so the caller refetches. + return nil, false, nil + } + if e.TTL > 0 && time.Since(e.CapturedAt) > e.TTL { + return nil, false, nil + } + return &e, true, nil +} + +// Save writes the entry to disk. Creates parent directories as needed. +func Save(key string, e *Entry) error { + path := Path(key) + if path == "" { + return errors.New("schemacache: no home dir") + } + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + return err + } + if e.CapturedAt.IsZero() { + e.CapturedAt = time.Now().UTC() + } + if e.Version == 0 { + e.Version = 1 + } + if e.ServerHash == "" { + e.ServerHash = key + } + if e.NativeBaselineToks == 0 { + e.NativeBaselineToks = ComputeBaseline(e) + } + data, err := json.MarshalIndent(e, "", " ") + if err != nil { + return err + } + tmp := path + ".tmp" + if err := os.WriteFile(tmp, data, 0o644); err != nil { + return err + } + return os.Rename(tmp, path) +} + +// Clear removes a single cache entry. No-op if the file doesn't exist. +func Clear(key string) error { + path := Path(key) + if path == "" { + return nil + } + err := os.Remove(path) + if err != nil && os.IsNotExist(err) { + return nil + } + return err +} + +// ClearAll wipes the entire schema cache. +func ClearAll() error { + home, err := os.UserHomeDir() + if err != nil { + return err + } + return os.RemoveAll(filepath.Join(home, ".mcpx", "cache", "schemas")) +} + +// ComputeBaseline approximates how many tokens a native MCP client would burn +// loading this server: stringify init result + tools list + prompts list + +// resources list, then divide by 4 (char heuristic). +func ComputeBaseline(e *Entry) int { + var b strings.Builder + enc := json.NewEncoder(&b) + enc.SetIndent("", " ") + _ = enc.Encode(e.Initialize) + _ = enc.Encode(e.Tools) + _ = enc.Encode(e.Prompts) + _ = enc.Encode(e.Resources) + return (b.Len() + 3) / 4 +} + +// Bypass returns true when callers should skip the cache entirely. Honors the +// MCPX_CACHE=off env var. +func Bypass() bool { + v := strings.ToLower(os.Getenv("MCPX_CACHE")) + return v == "off" || v == "0" || v == "false" +} diff --git a/internal/schemacache/cache_test.go b/internal/schemacache/cache_test.go new file mode 100644 index 0000000..d916e36 --- /dev/null +++ b/internal/schemacache/cache_test.go @@ -0,0 +1,153 @@ +package schemacache + +import ( + "os" + "path/filepath" + "testing" + "time" + + "github.com/codestz/mcpx/internal/mcp" +) + +func withTempHome(t *testing.T) func() { + t.Helper() + tmp := t.TempDir() + old := os.Getenv("HOME") + os.Setenv("HOME", tmp) + return func() { os.Setenv("HOME", old) } +} + +func TestKeyStable(t *testing.T) { + k1 := Key("serena", []string{"start"}, map[string]string{"FOO": "bar"}, "") + k2 := Key("serena", []string{"start"}, map[string]string{"FOO": "bar"}, "") + if k1 != k2 { + t.Errorf("expected stable keys, got %s vs %s", k1, k2) + } + k3 := Key("serena", []string{"start"}, map[string]string{"FOO": "baz"}, "") + if k1 == k3 { + t.Errorf("expected different keys for different env, got same: %s", k1) + } +} + +func TestSaveAndLoad(t *testing.T) { + defer withTempHome(t)() + + key := "abc123" + e := &Entry{ + TTL: 5 * time.Minute, + Tools: []mcp.Tool{ + {Name: "find_symbol", Description: "find a symbol"}, + }, + } + if err := Save(key, e); err != nil { + t.Fatal(err) + } + + loaded, hit, err := Load(key) + if err != nil { + t.Fatal(err) + } + if !hit { + t.Fatal("expected cache hit") + } + if loaded.Tools[0].Name != "find_symbol" { + t.Errorf("tool name: got %s", loaded.Tools[0].Name) + } + if loaded.NativeBaselineToks <= 0 { + t.Errorf("baseline tokens should be > 0, got %d", loaded.NativeBaselineToks) + } + if loaded.Version != 1 { + t.Errorf("version: got %d", loaded.Version) + } +} + +func TestExpiredEntryMisses(t *testing.T) { + defer withTempHome(t)() + key := "expired" + e := &Entry{ + CapturedAt: time.Now().Add(-1 * time.Hour), + TTL: 1 * time.Minute, + } + if err := Save(key, e); err != nil { + t.Fatal(err) + } + _, hit, _ := Load(key) + if hit { + t.Error("expected miss on expired entry") + } +} + +func TestMissingEntryMisses(t *testing.T) { + defer withTempHome(t)() + _, hit, err := Load("never-saved") + if err != nil { + t.Fatal(err) + } + if hit { + t.Error("expected miss") + } +} + +func TestClear(t *testing.T) { + defer withTempHome(t)() + key := "to-clear" + Save(key, &Entry{TTL: time.Hour}) + if _, hit, _ := Load(key); !hit { + t.Fatal("expected initial hit") + } + if err := Clear(key); err != nil { + t.Fatal(err) + } + if _, hit, _ := Load(key); hit { + t.Error("expected miss after clear") + } +} + +func TestClearAll(t *testing.T) { + defer withTempHome(t)() + Save("a", &Entry{TTL: time.Hour}) + Save("b", &Entry{TTL: time.Hour}) + if err := ClearAll(); err != nil { + t.Fatal(err) + } + if _, hit, _ := Load("a"); hit { + t.Error("expected miss after ClearAll") + } +} + +func TestBypass(t *testing.T) { + old := os.Getenv("MCPX_CACHE") + defer os.Setenv("MCPX_CACHE", old) + + for _, v := range []string{"off", "0", "false", "OFF"} { + os.Setenv("MCPX_CACHE", v) + if !Bypass() { + t.Errorf("MCPX_CACHE=%s should bypass", v) + } + } + for _, v := range []string{"", "on", "1"} { + os.Setenv("MCPX_CACHE", v) + if Bypass() { + t.Errorf("MCPX_CACHE=%s should not bypass", v) + } + } +} + +func TestComputeBaselineGrowsWithSize(t *testing.T) { + small := &Entry{Tools: []mcp.Tool{{Name: "x"}}} + big := &Entry{} + for i := 0; i < 50; i++ { + big.Tools = append(big.Tools, mcp.Tool{Name: "tool", Description: "lengthy description x"}) + } + if ComputeBaseline(big) <= ComputeBaseline(small) { + t.Errorf("big baseline should exceed small") + } +} + +func TestCachePathWithHome(t *testing.T) { + defer withTempHome(t)() + want := filepath.Join(os.Getenv("HOME"), ".mcpx", "cache", "schemas", "abc.json") + if got := Path("abc"); got != want { + t.Errorf("path: got %s want %s", got, want) + } +} diff --git a/internal/stats/agg.go b/internal/stats/agg.go new file mode 100644 index 0000000..cea378e --- /dev/null +++ b/internal/stats/agg.go @@ -0,0 +1,263 @@ +package stats + +import ( + "sort" + "time" +) + +// Summary is the headline aggregation: totals, top-K, hit rates, daily buckets. +type Summary struct { + Calls int + TokensSaved int64 + ArgsTokens int64 + ResponseTokens int64 + NativeBaseline int64 + CacheHitRate float64 // schema cache hits / calls, 0..1 + SchemaHitRate float64 + ErrorRate float64 + AvgLatencyMS float64 + P50LatencyMS int64 + P95LatencyMS int64 + + TopTools []ToolStat + TopSavers []ToolStat + TopServers []ServerStat + Daily []DailyBucket + Recent []Record // last N entries + + Projects []string // distinct project roots seen in window + Servers []string // distinct server names seen in window + Agents []string // distinct agent names seen in window +} + +// ToolStat aggregates per-tool metrics. +type ToolStat struct { + Server string + Tool string + Calls int + TokensSaved int64 + AvgLatencyMS float64 + Errors int +} + +// ServerStat aggregates per-server metrics. +type ServerStat struct { + Server string + Calls int + TokensSaved int64 + AvgLatencyMS float64 + P95LatencyMS int64 + Errors int +} + +// DailyBucket aggregates by calendar day (UTC). +type DailyBucket struct { + Day time.Time // truncated to day + Calls int + TokensSaved int64 + Errors int +} + +// Aggregate computes a Summary over records matching f. recentN controls how +// many tail records to retain in Summary.Recent. +func Aggregate(path string, f Filter, recentN int) (*Summary, error) { + if recentN < 0 { + recentN = 0 + } + + s := &Summary{} + type tk struct{ srv, tool string } + toolBy := map[tk]*ToolStat{} + srvBy := map[string]*ServerStat{} + srvLatencies := map[string][]int64{} + dayBy := map[time.Time]*DailyBucket{} + projects := map[string]struct{}{} + servers := map[string]struct{}{} + agents := map[string]struct{}{} + + var schemaHits, errors int + var latencies []int64 + var latencySum int64 + recent := newRing(recentN) + + err := Iter(path, f, func(r Record) error { + s.Calls++ + s.TokensSaved += int64(r.TokensSaved) + s.ArgsTokens += int64(r.ArgsTokensEst) + s.ResponseTokens += int64(r.ResponseTokensEst) + s.NativeBaseline += int64(r.NativeBaselineToks) + latencySum += r.LatencyMS + latencies = append(latencies, r.LatencyMS) + if r.SchemaCacheHit { + schemaHits++ + } + if r.ExitCode != 0 { + errors++ + } + + key := tk{r.Server, r.Tool} + ts, ok := toolBy[key] + if !ok { + ts = &ToolStat{Server: r.Server, Tool: r.Tool} + toolBy[key] = ts + } + ts.Calls++ + ts.TokensSaved += int64(r.TokensSaved) + ts.AvgLatencyMS += float64(r.LatencyMS) + if r.ExitCode != 0 { + ts.Errors++ + } + + ss, ok := srvBy[r.Server] + if !ok { + ss = &ServerStat{Server: r.Server} + srvBy[r.Server] = ss + } + ss.Calls++ + ss.TokensSaved += int64(r.TokensSaved) + ss.AvgLatencyMS += float64(r.LatencyMS) + srvLatencies[r.Server] = append(srvLatencies[r.Server], r.LatencyMS) + if r.ExitCode != 0 { + ss.Errors++ + } + + day := r.TS.UTC().Truncate(24 * time.Hour) + db, ok := dayBy[day] + if !ok { + db = &DailyBucket{Day: day} + dayBy[day] = db + } + db.Calls++ + db.TokensSaved += int64(r.TokensSaved) + if r.ExitCode != 0 { + db.Errors++ + } + + if r.Project != "" { + projects[r.Project] = struct{}{} + } + if r.Server != "" { + servers[r.Server] = struct{}{} + } + if r.Agent != "" { + agents[r.Agent] = struct{}{} + } + + recent.push(r) + return nil + }) + if err != nil { + return nil, err + } + + if s.Calls > 0 { + s.AvgLatencyMS = float64(latencySum) / float64(s.Calls) + s.SchemaHitRate = float64(schemaHits) / float64(s.Calls) + s.CacheHitRate = s.SchemaHitRate + s.ErrorRate = float64(errors) / float64(s.Calls) + s.P50LatencyMS = percentile(latencies, 0.5) + s.P95LatencyMS = percentile(latencies, 0.95) + } + + for _, ts := range toolBy { + if ts.Calls > 0 { + ts.AvgLatencyMS /= float64(ts.Calls) + } + s.TopTools = append(s.TopTools, *ts) + } + sort.Slice(s.TopTools, func(i, j int) bool { + return s.TopTools[i].Calls > s.TopTools[j].Calls + }) + s.TopSavers = append([]ToolStat(nil), s.TopTools...) + sort.Slice(s.TopSavers, func(i, j int) bool { + return s.TopSavers[i].TokensSaved > s.TopSavers[j].TokensSaved + }) + + for _, ss := range srvBy { + if ss.Calls > 0 { + ss.AvgLatencyMS /= float64(ss.Calls) + ss.P95LatencyMS = percentile(srvLatencies[ss.Server], 0.95) + } + s.TopServers = append(s.TopServers, *ss) + } + sort.Slice(s.TopServers, func(i, j int) bool { + return s.TopServers[i].Calls > s.TopServers[j].Calls + }) + + for _, db := range dayBy { + s.Daily = append(s.Daily, *db) + } + sort.Slice(s.Daily, func(i, j int) bool { + return s.Daily[i].Day.Before(s.Daily[j].Day) + }) + + s.Recent = recent.snapshot() + s.Projects = sortedKeys(projects) + s.Servers = sortedKeys(servers) + s.Agents = sortedKeys(agents) + + return s, nil +} + +// percentile returns the requested percentile (0..1) of the slice. +// Mutates the slice (sorts it). +func percentile(v []int64, p float64) int64 { + if len(v) == 0 { + return 0 + } + sort.Slice(v, func(i, j int) bool { return v[i] < v[j] }) + idx := int(float64(len(v)-1) * p) + if idx < 0 { + idx = 0 + } + if idx >= len(v) { + idx = len(v) - 1 + } + return v[idx] +} + +func sortedKeys(m map[string]struct{}) []string { + out := make([]string, 0, len(m)) + for k := range m { + out = append(out, k) + } + sort.Strings(out) + return out +} + +// ring is a tiny fixed-size ring buffer for "last N" record retention. +type ring struct { + buf []Record + idx int + size int +} + +func newRing(n int) *ring { + if n <= 0 { + return &ring{} + } + return &ring{buf: make([]Record, n)} +} + +func (r *ring) push(rec Record) { + if r.buf == nil { + return + } + r.buf[r.idx] = rec + r.idx = (r.idx + 1) % len(r.buf) + if r.size < len(r.buf) { + r.size++ + } +} + +func (r *ring) snapshot() []Record { + if r.size == 0 { + return nil + } + out := make([]Record, r.size) + start := (r.idx - r.size + len(r.buf)) % len(r.buf) + for i := 0; i < r.size; i++ { + out[i] = r.buf[(start+i)%len(r.buf)] + } + return out +} diff --git a/internal/stats/read.go b/internal/stats/read.go new file mode 100644 index 0000000..16598ad --- /dev/null +++ b/internal/stats/read.go @@ -0,0 +1,95 @@ +package stats + +import ( + "bufio" + "encoding/json" + "errors" + "io" + "os" + "time" +) + +// Filter narrows which records are returned by Iter and Aggregate. +// Zero values mean "no filter". Time bounds are inclusive of Since, exclusive of Until. +type Filter struct { + Since time.Time + Until time.Time + Project string // exact match; "" = any + Server string // exact match; "" = any + Tool string // exact match; "" = any + Session string // exact match; "" = any + Agent string // exact match; "" = any +} + +// Iter walks records in the file, calling fn for each match. +// Stop iteration by returning io.EOF from fn. Other errors propagate. +// A missing file yields zero records and no error. +func Iter(path string, f Filter, fn func(Record) error) error { + file, err := os.Open(path) + if err != nil { + if os.IsNotExist(err) { + return nil + } + return err + } + defer file.Close() + + scanner := bufio.NewScanner(file) + scanner.Buffer(make([]byte, 64*1024), 1024*1024) + for scanner.Scan() { + line := scanner.Bytes() + if len(line) == 0 { + continue + } + var r Record + if err := json.Unmarshal(line, &r); err != nil { + continue // skip corrupt lines silently + } + if !f.matches(r) { + continue + } + if err := fn(r); err != nil { + if errors.Is(err, io.EOF) { + return nil + } + return err + } + } + return scanner.Err() +} + +func (f Filter) matches(r Record) bool { + if !f.Since.IsZero() && r.TS.Before(f.Since) { + return false + } + if !f.Until.IsZero() && !r.TS.Before(f.Until) { + return false + } + if f.Project != "" && r.Project != f.Project { + return false + } + if f.Server != "" && r.Server != f.Server { + return false + } + if f.Tool != "" && r.Tool != f.Tool { + return false + } + if f.Session != "" && r.Session != f.Session { + return false + } + if f.Agent != "" && r.Agent != f.Agent { + return false + } + return true +} + +// Collect is a convenience wrapper that returns all matching records as a slice. +// For large files prefer Iter to keep memory bounded. +func Collect(path string, f Filter) ([]Record, error) { + var out []Record + err := Iter(path, f, func(r Record) error { + out = append(out, r) + return nil + }) + return out, err +} diff --git a/internal/stats/stats.go b/internal/stats/stats.go new file mode 100644 index 0000000..7151ef3 --- /dev/null +++ b/internal/stats/stats.go @@ -0,0 +1,213 @@ +// Package stats records every mcpx tool invocation as a JSONL line on disk. +// Reads, aggregations, and dashboard queries all consume the same file. +// +// Writes are async and never block or error out the caller. If the buffer +// overflows we drop entries and increment a counter (see DroppedCount). +package stats + +import ( + "crypto/rand" + "encoding/json" + "errors" + "os" + "path/filepath" + "sync" + "sync/atomic" + "time" +) + +// Record is one mcpx tool invocation. One JSONL line per record. +type Record struct { + ID string `json:"id,omitempty"` + TS time.Time `json:"ts"` + Session string `json:"session,omitempty"` + Project string `json:"project,omitempty"` + Agent string `json:"agent,omitempty"` + Server string `json:"server"` + Tool string `json:"tool"` + Args map[string]any `json:"args,omitempty"` + ArgsBytes int `json:"args_bytes"` + ArgsTokensEst int `json:"args_tokens_est"` + ResponseBytes int `json:"response_bytes"` + ResponseTokensEst int `json:"response_tokens_est"` + ResultPreview string `json:"result_preview,omitempty"` + ResultTruncated bool `json:"result_truncated,omitempty"` + LatencyMS int64 `json:"latency_ms"` + Transport string `json:"transport,omitempty"` + Daemon bool `json:"daemon,omitempty"` + SchemaCacheHit bool `json:"schema_cache_hit,omitempty"` + ExitCode int `json:"exit_code"` + Error string `json:"error,omitempty"` + PolicyAction string `json:"policy_action,omitempty"` + PolicyName string `json:"policy_name,omitempty"` + NativeBaselineToks int `json:"native_baseline_tokens,omitempty"` + TokensSaved int `json:"tokens_saved"` +} + +// MaxResultPreviewBytes caps the inlined result preview to keep JSONL rows bounded. +const MaxResultPreviewBytes = 2 * 1024 + +// NewID returns a short stable identifier for a record. Combines unix-nano with +// a small random suffix; readable in logs and short enough for URLs. +func NewID() string { + const alpha = "0123456789abcdefghijklmnopqrstuvwxyz" + b := make([]byte, 6) + _, _ = rand.Read(b) + for i := range b { + b[i] = alpha[int(b[i])%len(alpha)] + } + return time.Now().UTC().Format("20060102T150405") + "-" + string(b) +} + +// TruncateForPreview returns at most MaxResultPreviewBytes of s and a flag. +func TruncateForPreview(s string) (string, bool) { + if len(s) <= MaxResultPreviewBytes { + return s, false + } + return s[:MaxResultPreviewBytes], true +} + +// Writer appends Records to a JSONL file. Writes are buffered and flushed by +// a background goroutine; the caller never blocks on disk I/O. +type Writer struct { + path string + ch chan Record + dropped atomic.Int64 + closeMu sync.Mutex + closed bool + doneCh chan struct{} + enabled atomic.Bool +} + +// NewWriter creates a Writer that appends to path. Buffer size caps in-flight +// records; on overflow Write() drops the record and bumps DroppedCount. +// +// Pass enabled=false to make every Write a no-op (config: gain.enabled=false). +func NewWriter(path string, bufSize int, enabled bool) (*Writer, error) { + if bufSize <= 0 { + bufSize = 1024 + } + w := &Writer{ + path: path, + ch: make(chan Record, bufSize), + doneCh: make(chan struct{}), + } + w.enabled.Store(enabled) + if enabled { + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + return nil, err + } + go w.run() + } else { + close(w.doneCh) + } + return w, nil +} + +// Write enqueues a record. Never blocks; never errors. If the buffer is full +// the record is silently dropped and DroppedCount() increments. +func (w *Writer) Write(r Record) { + if w == nil || !w.enabled.Load() { + return + } + if r.TS.IsZero() { + r.TS = time.Now().UTC() + } + select { + case w.ch <- r: + default: + w.dropped.Add(1) + } +} + +// DroppedCount returns the number of records dropped due to buffer overflow. +func (w *Writer) DroppedCount() int64 { + if w == nil { + return 0 + } + return w.dropped.Load() +} + +// Close drains the buffer and stops the writer goroutine. Safe to call once. +func (w *Writer) Close() error { + if w == nil { + return nil + } + w.closeMu.Lock() + if w.closed { + w.closeMu.Unlock() + return nil + } + w.closed = true + close(w.ch) + w.closeMu.Unlock() + <-w.doneCh + return nil +} + +// Flush waits for all currently buffered records to be written. Convenience +// wrapper for tests; prefer Close in production paths. +func (w *Writer) Flush(timeout time.Duration) error { + if w == nil || !w.enabled.Load() { + return nil + } + deadline := time.After(timeout) + for { + if len(w.ch) == 0 { + return nil + } + select { + case <-deadline: + return errors.New("stats: flush timeout") + case <-time.After(2 * time.Millisecond): + } + } +} + +func (w *Writer) run() { + defer close(w.doneCh) + for r := range w.ch { + w.append(r) + } +} + +func (w *Writer) append(r Record) { + data, err := json.Marshal(r) + if err != nil { + return + } + data = append(data, '\n') + f, err := os.OpenFile(w.path, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0o644) + if err != nil { + return + } + _, _ = f.Write(data) + _ = f.Close() +} + +// EstimateTokens returns a char-based token estimate (bytes / 4). +// Honest approximation; real tokenizer is opt-in via build tag in v1.6.1. +func EstimateTokens(b []byte) int { + if len(b) == 0 { + return 0 + } + return (len(b) + 3) / 4 +} + +// EstimateTokensString is EstimateTokens for strings. +func EstimateTokensString(s string) int { + if s == "" { + return 0 + } + return (len(s) + 3) / 4 +} + +// DefaultPath returns the conventional stats file path under the user home dir. +// Empty string on failure. +func DefaultPath() string { + home, err := os.UserHomeDir() + if err != nil { + return "" + } + return filepath.Join(home, ".mcpx", "stats.jsonl") +} diff --git a/internal/stats/stats_test.go b/internal/stats/stats_test.go new file mode 100644 index 0000000..dd5c9e2 --- /dev/null +++ b/internal/stats/stats_test.go @@ -0,0 +1,128 @@ +package stats + +import ( + "path/filepath" + "testing" + "time" +) + +func TestWriterAppendsAndReads(t *testing.T) { + dir := t.TempDir() + path := filepath.Join(dir, "stats.jsonl") + w, err := NewWriter(path, 16, true) + if err != nil { + t.Fatal(err) + } + for i := 0; i < 5; i++ { + w.Write(Record{ + Server: "serena", Tool: "find_symbol", + ArgsBytes: 100, ResponseBytes: 1000, + LatencyMS: int64(10 + i), TokensSaved: 200, + }) + } + if err := w.Close(); err != nil { + t.Fatal(err) + } + + recs, err := Collect(path, Filter{}) + if err != nil { + t.Fatal(err) + } + if len(recs) != 5 { + t.Fatalf("want 5 records, got %d", len(recs)) + } +} + +func TestWriterDisabledIsNoop(t *testing.T) { + dir := t.TempDir() + path := filepath.Join(dir, "stats.jsonl") + w, err := NewWriter(path, 16, false) + if err != nil { + t.Fatal(err) + } + w.Write(Record{Server: "x", Tool: "y"}) + w.Close() + + recs, _ := Collect(path, Filter{}) + if len(recs) != 0 { + t.Fatalf("disabled writer wrote %d records", len(recs)) + } +} + +func TestFilterMatching(t *testing.T) { + now := time.Now().UTC() + r := Record{TS: now, Project: "/p", Server: "s", Tool: "t", Agent: "a"} + cases := []struct { + f Filter + want bool + }{ + {Filter{}, true}, + {Filter{Server: "s"}, true}, + {Filter{Server: "x"}, false}, + {Filter{Project: "/p"}, true}, + {Filter{Project: "/q"}, false}, + {Filter{Tool: "t"}, true}, + {Filter{Tool: "u"}, false}, + {Filter{Since: now.Add(-1 * time.Hour)}, true}, + {Filter{Since: now.Add(1 * time.Hour)}, false}, + {Filter{Until: now.Add(1 * time.Hour)}, true}, + {Filter{Until: now.Add(-1 * time.Hour)}, false}, + {Filter{Agent: "a"}, true}, + {Filter{Agent: "b"}, false}, + } + for i, c := range cases { + if got := c.f.matches(r); got != c.want { + t.Errorf("case %d: filter %+v: got %v want %v", i, c.f, got, c.want) + } + } +} + +func TestAggregate(t *testing.T) { + dir := t.TempDir() + path := filepath.Join(dir, "stats.jsonl") + w, _ := NewWriter(path, 16, true) + now := time.Now().UTC() + + w.Write(Record{TS: now, Server: "s", Tool: "find_symbol", LatencyMS: 10, TokensSaved: 100, ArgsTokensEst: 5, ResponseTokensEst: 50, SchemaCacheHit: true}) + w.Write(Record{TS: now, Server: "s", Tool: "find_symbol", LatencyMS: 20, TokensSaved: 200, ArgsTokensEst: 5, ResponseTokensEst: 60, SchemaCacheHit: true}) + w.Write(Record{TS: now, Server: "s", Tool: "search", LatencyMS: 100, TokensSaved: 50, ArgsTokensEst: 5, ResponseTokensEst: 200, ExitCode: 1}) + w.Close() + + s, err := Aggregate(path, Filter{}, 2) + if err != nil { + t.Fatal(err) + } + if s.Calls != 3 { + t.Errorf("calls: got %d want 3", s.Calls) + } + if s.TokensSaved != 350 { + t.Errorf("tokens saved: got %d want 350", s.TokensSaved) + } + if len(s.TopTools) != 2 { + t.Errorf("top tools: got %d want 2", len(s.TopTools)) + } + if s.TopTools[0].Tool != "find_symbol" { + t.Errorf("top tool: got %s want find_symbol", s.TopTools[0].Tool) + } + if s.ErrorRate < 0.33 || s.ErrorRate > 0.34 { + t.Errorf("error rate: got %f want ~0.333", s.ErrorRate) + } + if s.SchemaHitRate < 0.66 || s.SchemaHitRate > 0.67 { + t.Errorf("schema hit rate: got %f want ~0.667", s.SchemaHitRate) + } + if len(s.Recent) != 2 { + t.Errorf("recent: got %d want 2", len(s.Recent)) + } +} + +func TestEstimateTokens(t *testing.T) { + if got := EstimateTokens([]byte("12345678")); got != 2 { + t.Errorf("8 bytes: got %d want 2", got) + } + if got := EstimateTokens([]byte("123")); got != 1 { + t.Errorf("3 bytes: got %d want 1", got) + } + if got := EstimateTokens([]byte("")); got != 0 { + t.Errorf("empty: got %d want 0", got) + } +} diff --git a/internal/ui/assets/app.js b/internal/ui/assets/app.js new file mode 100644 index 0000000..7322c2e --- /dev/null +++ b/internal/ui/assets/app.js @@ -0,0 +1,410 @@ +// mcpx dashboard — vanilla JS, zero npm +(function () { + const T = window.MCPX_TOKEN; + const auth = (path) => path + (path.includes("?") ? "&" : "?") + "t=" + T; + + const state = { + scope: { kind: "this" }, // {kind:"this"|"all"|"project", value?} + since: "168h", + sinceSeconds: 7 * 24 * 3600, + summary: null, + selectedRowEl: null, + selectedRecord: null, + sseAlive: false, + lastEventAt: Date.now(), + filterRegex: null, + }; + + // ---------- formatters ---------- + function fmtNumber(n) { + if (n == null) return "—"; + const a = Math.abs(n); + if (a >= 1e9) return (n / 1e9).toFixed(1) + "B"; + if (a >= 1e6) return (n / 1e6).toFixed(a >= 1e8 ? 0 : 1) + "M"; + if (a >= 1e3) return (n / 1e3).toFixed(a >= 1e5 ? 0 : 1) + "K"; + return String(n); + } + function fmtPct(p) { return p == null ? "—" : Math.round(p * 100) + "%"; } + function fmtMs(ms) { + if (ms == null) return "—"; + if (ms < 1000) return ms + "ms"; + if (ms < 60_000) return Math.round(ms / 1000) + "s"; + return Math.floor(ms / 60_000) + "m" + String(Math.round((ms % 60_000) / 1000)).padStart(2, "0") + "s"; + } + function fmtTime(iso) { + const d = new Date(iso); + return d.toLocaleTimeString([], { hour: "2-digit", minute: "2-digit", second: "2-digit", hour12: false }); + } + function fmtAgo(ts) { + const s = Math.floor((Date.now() - ts) / 1000); + if (s < 5) return "just now"; + if (s < 60) return s + "s ago"; + if (s < 3600) return Math.floor(s / 60) + "m ago"; + return Math.floor(s / 3600) + "h ago"; + } + function shortName(server, tool) { return server + "." + tool; } + + // ---------- API ---------- + async function fetchSummary() { + const params = new URLSearchParams({ since: state.since }); + if (state.scope.kind === "project") params.set("project", state.scope.value); + if (state.scope.kind === "all") params.set("all", "1"); + const r = await fetch(auth("/api/summary?" + params.toString())); + if (!r.ok) throw new Error("summary fetch failed"); + return r.json(); + } + async function fetchProjects() { + const r = await fetch(auth("/api/projects")); + if (!r.ok) return []; + return r.json(); + } + async function fetchRecord(id) { + if (state.recentById?.[id]) return state.recentById[id]; + const r = await fetch(auth("/api/call/" + encodeURIComponent(id))); + if (!r.ok) return null; + return r.json(); + } + async function fetchToolDetail(server, tool) { + const r = await fetch(auth("/api/tool/" + encodeURIComponent(server) + "/" + encodeURIComponent(tool))); + if (!r.ok) return null; + return r.json(); + } + async function fetchAudit() { + const r = await fetch(auth("/api/audit")); + if (!r.ok) return []; + return r.json(); + } + + // ---------- render: hero + tiles ---------- + function render(s) { + state.summary = s; + + // hero + document.getElementById("hero-saved").textContent = fmtNumber(s.TokensSaved); + document.getElementById("hero-foot").textContent = "vs native MCP loading · " + windowLabel(); + + const eff = s.NativeBaseline > 0 ? s.TokensSaved / s.NativeBaseline : 0; + const effPct = Math.max(0, Math.min(1, eff)); + document.getElementById("eff-pct").textContent = fmtPct(effPct); + document.getElementById("eff-fill").style.width = (effPct * 100).toFixed(1) + "%"; + document.getElementById("eff-saved").textContent = fmtNumber(s.TokensSaved); + document.getElementById("eff-native").textContent = fmtNumber(s.NativeBaseline); + + // tiles + setTile("t-calls", fmtNumber(s.Calls)); + setTile("t-cache", fmtPct(s.CacheHitRate)); + document.getElementById("t-cache-foot").textContent = "schema cache"; + setTile("t-errors", fmtPct(s.ErrorRate), s.ErrorRate > 0.05 ? "bad" : (s.ErrorRate > 0 ? "warn" : "")); + setTile("t-avg", fmtMs(Math.round(s.AvgLatencyMS))); + document.getElementById("t-avg-foot").textContent = "p95 " + fmtMs(s.P95LatencyMS); + + renderTopTools(s); + renderTopSavers(s); + renderServerHealth(s); + renderRecentList(s.Recent || []); + renderServerSidebar(s); + + document.getElementById("scope-window").textContent = windowLabel(); + } + function setTile(id, text, cls) { + const el = document.getElementById(id); + el.textContent = text; + el.classList.remove("warn", "bad"); + if (cls) el.classList.add(cls); + } + function windowLabel() { + if (state.since === "all") return "all time"; + return ({ + "1h": "last hour", "24h": "last 24h", "168h": "last 7 days", "720h": "last 30 days", + }[state.since]) || state.since; + } + + function renderTopTools(s) { + const tbl = document.getElementById("tool-table"); + tbl.innerHTML = ""; + const items = (s.TopTools || []).slice(0, 6); + if (!items.length) { tbl.innerHTML = 'no calls in window'; return; } + const max = items[0].Calls; + items.forEach((t) => { + const tr = document.createElement("tr"); + tr.style.cursor = "pointer"; + tr.title = "click to drill into " + shortName(t.Server, t.Tool); + tr.innerHTML = + '' + escapeHtml(shortName(t.Server, t.Tool)) + "" + + '
' + + '' + t.Calls + ""; + tr.onclick = () => openToolDetail(t.Server, t.Tool); + tbl.appendChild(tr); + }); + } + + async function openToolDetail(server, tool) { + const detail = await fetchToolDetail(server, tool); + if (!detail) return; + document.getElementById("drawer-title").textContent = shortName(server, tool); + document.getElementById("drawer-sub").textContent = detail.calls + " calls in window"; + const kv = document.getElementById("drawer-kv"); + kv.innerHTML = ""; + const cell = (k, v) => { kv.innerHTML += '
' + k + '
' + v + '
'; }; + cell("calls", detail.calls); + cell("errors", detail.errors); + cell("avg latency", fmtMs(Math.round(detail.avg_latency_ms))); + cell("tokens saved", fmtNumber(detail.tokens_saved)); + + const recent = (detail.recent || []).slice(-15).reverse(); + document.getElementById("drawer-args").textContent = "(tool detail view — recent calls listed below)"; + document.getElementById("drawer-result").textContent = recent + .map((r) => fmtTime(r.ts) + " " + fmtMs(r.latency_ms).padStart(6) + + (r.exit_code ? " [error: " + r.error + "]" : "")) + .join("\n"); + document.getElementById("drawer").classList.remove("hidden"); + } + function renderTopSavers(s) { + const tbl = document.getElementById("save-table"); + tbl.innerHTML = ""; + const items = (s.TopSavers || []).filter((t) => t.TokensSaved > 0).slice(0, 6); + if (!items.length) { tbl.innerHTML = 'no savings yet'; return; } + const max = items[0].TokensSaved; + items.forEach((t) => { + const tr = document.createElement("tr"); + tr.innerHTML = + '' + shortName(t.Server, t.Tool) + "" + + '
' + + '' + fmtNumber(t.TokensSaved) + ""; + tbl.appendChild(tr); + }); + } + function renderServerHealth(s) { + const el = document.getElementById("server-health"); + el.innerHTML = ""; + const servers = s.TopServers || []; + if (!servers.length) { + el.innerHTML = '
no servers active in window
'; + return; + } + servers.forEach((sv) => { + const errRate = sv.Calls > 0 ? sv.Errors / sv.Calls : 0; + const statusCls = errRate > 0.1 ? "bad" : (errRate > 0 ? "warn" : ""); + const statusText = errRate > 0.1 ? "degraded" : (errRate > 0 ? "warn" : "ok"); + const card = document.createElement("div"); + card.className = "health-card"; + card.innerHTML = + '
' + sv.Server + "" + + '' + statusText + "
" + + '
' + + '' + sv.Calls + " calls" + + 'p95 ' + fmtMs(sv.P95LatencyMS || 0) + "" + + '' + fmtPct(errRate) + " err" + + "
"; + el.appendChild(card); + }); + } + + function renderProjects(projects) { + const ul = document.getElementById("project-list"); + ul.innerHTML = ""; + const liAll = document.createElement("li"); + liAll.innerHTML = "all projects"; + if (state.scope.kind === "all") liAll.classList.add("active"); + liAll.onclick = () => { state.scope = { kind: "all" }; bootstrap(); }; + ul.appendChild(liAll); + projects.forEach((p) => { + const li = document.createElement("li"); + const name = p.Name || "(no name)"; + li.innerHTML = "" + escapeHtml(name) + "" + p.Calls + ""; + const isCurrent = (state.scope.kind === "this" && p.Current) || + (state.scope.kind === "project" && state.scope.value === p.Path); + if (isCurrent) li.classList.add("active"); + li.onclick = () => { state.scope = { kind: "project", value: p.Path }; bootstrap(); }; + ul.appendChild(li); + }); + } + function renderServerSidebar(s) { + const ul = document.getElementById("server-list"); + ul.innerHTML = ""; + (s.Servers || []).forEach((name) => { + const li = document.createElement("li"); + const calls = (s.PerServer && s.PerServer[name]) || 0; + li.innerHTML = "" + escapeHtml(name) + "" + calls + ""; + ul.appendChild(li); + }); + } + + // ---------- live tail ---------- + function renderRecentList(rows) { + state.recentById = {}; + const el = document.getElementById("live-tail"); + el.innerHTML = ""; + if (!rows.length) { + el.innerHTML = '
no calls yet — run any mcpx command to start filling this view
'; + return; + } + // newest first + [...rows].reverse().forEach((r, i) => prependTail(el, r, false, i === 0)); + } + function prependTail(el, r, animate, isNewest) { + if (!r) return; + if (state.filterRegex && !state.filterRegex.test(shortName(r.server, r.tool))) { + return; + } + const id = r.ts + "|" + r.server + "|" + r.tool + "|" + (r.session || ""); + state.recentById[id] = r; + + const row = document.createElement("div"); + row.className = "tail-row"; + row.dataset.id = id; + let badge = ""; + if (r.schema_cache_hit) badge = 'warm'; + row.innerHTML = + '' + fmtTime(r.ts) + "" + + '' + escapeHtml(shortName(r.server, r.tool)) + "" + + '' + fmtMs(r.latency_ms) + "" + + "" + badge + "" + + '' + (r.exit_code ? "✗" : "✓") + ""; + row.onclick = () => openDrawer(row, r); + el.prepend(row); + while (el.children.length > 80) el.removeChild(el.lastChild); + + if (animate) { + row.style.opacity = 0; + requestAnimationFrame(() => { + row.style.transition = "opacity 0.25s, background 0.25s"; + row.style.opacity = 1; + }); + } + } + + // ---------- drawer ---------- + async function openDrawer(rowEl, recordOrId) { + if (state.selectedRowEl) state.selectedRowEl.classList.remove("selected"); + if (rowEl) { + rowEl.classList.add("selected"); + state.selectedRowEl = rowEl; + } + + const r = typeof recordOrId === "string" ? await fetchRecord(recordOrId) : recordOrId; + if (!r) { closeDrawer(); return; } + state.selectedRecord = r; + + document.getElementById("drawer-title").textContent = shortName(r.server, r.tool); + document.getElementById("drawer-sub").textContent = + new Date(r.ts).toLocaleString() + (r.id ? " · " + r.id : ""); + + const kv = document.getElementById("drawer-kv"); + kv.innerHTML = ""; + const cell = (k, v) => { kv.innerHTML += '
' + k + '
' + v + '
'; }; + cell("latency", fmtMs(r.latency_ms)); + cell("exit", String(r.exit_code || 0)); + cell("project", r.project ? '' + escapeHtml(r.project) + '' : '—'); + cell("session", r.session || "—"); + cell("agent", r.agent || "—"); + cell("transport", (r.transport || "—") + (r.daemon ? " (daemon)" : "")); + cell("schema cache", r.schema_cache_hit ? "hit" : "miss"); + cell("baseline tokens", fmtNumber(r.native_baseline_tokens || 0)); + cell("tokens saved", fmtNumber(r.tokens_saved || 0)); + if (r.policy_action) cell("policy", r.policy_action + (r.policy_name ? " (" + r.policy_name + ")" : "")); + if (r.error) cell("error", '' + escapeHtml(r.error) + ''); + + document.getElementById("drawer-args").textContent = + r.args ? JSON.stringify(r.args, null, 2) : "(no args recorded)"; + + let resultDisplay = "(no result preview captured)"; + if (r.result_preview) { + resultDisplay = r.result_preview; + if (r.result_truncated) resultDisplay += "\n\n… (truncated)"; + } + document.getElementById("drawer-result").textContent = resultDisplay; + + document.getElementById("drawer").classList.remove("hidden"); + } + function closeDrawer() { + document.getElementById("drawer").classList.add("hidden"); + if (state.selectedRowEl) state.selectedRowEl.classList.remove("selected"); + state.selectedRowEl = null; + } + + // ---------- SSE ---------- + function startSSE() { + const ev = new EventSource(auth("/api/events")); + ev.onopen = () => { setConn(true); }; + ev.addEventListener("call", (e) => { + try { + const r = JSON.parse(e.data); + state.lastEventAt = Date.now(); + prependTail(document.getElementById("live-tail"), r, true, true); + // refresh aggregates lazily + fetchSummary().then(render).catch(() => {}); + } catch {} + }); + ev.onerror = () => { + setConn(false); + ev.close(); + setTimeout(startSSE, 3000); + }; + } + function setConn(alive) { + state.sseAlive = alive; + const dot = document.getElementById("conn-dot"); + dot.classList.remove("live", "dead"); + dot.classList.add(alive ? "live" : "dead"); + } + + // ---------- bootstrap ---------- + async function bootstrap() { + try { + const [s, projects] = await Promise.all([fetchSummary(), fetchProjects()]); + render(s); + renderProjects(projects); + const label = + state.scope.kind === "all" ? "All projects" : + state.scope.kind === "project" ? short(state.scope.value) : + "This project"; + document.getElementById("scope-label").textContent = label; + tickFreshness(); + } catch (e) { console.error(e); } + } + function short(p) { + if (!p) return "—"; + if (p.length <= 36) return p; + return "…" + p.slice(-32); + } + function escapeHtml(s) { + return String(s).replace(/[&<>"']/g, (c) => ({"&":"&","<":"<",">":">",'"':""","'":"'"}[c])); + } + function tickFreshness() { + const el = document.getElementById("freshness"); + el.textContent = fmtAgo(state.lastEventAt); + setTimeout(tickFreshness, 1000); + } + + // ---------- wire interactions ---------- + document.getElementById("time-tabs").addEventListener("click", (e) => { + const btn = e.target.closest("button[data-since]"); + if (!btn) return; + document.querySelectorAll("#time-tabs button").forEach((b) => b.classList.remove("active")); + btn.classList.add("active"); + state.since = btn.dataset.since; + bootstrap(); + }); + document.getElementById("drawer-close").onclick = closeDrawer; + document.getElementById("drawer-copy").onclick = () => { + if (!state.selectedRecord) return; + navigator.clipboard.writeText(JSON.stringify(state.selectedRecord, null, 2)); + }; + document.addEventListener("keydown", (e) => { + if (e.key === "Escape") closeDrawer(); + if (e.key === "/" && document.activeElement.tagName !== "INPUT") { + e.preventDefault(); + document.getElementById("tail-filter").focus(); + } + }); + document.getElementById("tail-filter").addEventListener("input", (e) => { + const v = e.target.value.trim(); + try { + state.filterRegex = v ? new RegExp(v, "i") : null; + } catch { state.filterRegex = null; } + if (state.summary) renderRecentList(state.summary.Recent || []); + }); + + bootstrap().then(startSSE).catch(console.error); +})(); diff --git a/internal/ui/assets/index.html b/internal/ui/assets/index.html new file mode 100644 index 0000000..0e316a6 --- /dev/null +++ b/internal/ui/assets/index.html @@ -0,0 +1,169 @@ + + + + + mcpx + + + + +
+ + + +
+ +
+
+ … + · + 7 days +
+ +
+ just now +
+
+ +
+
+
+ tokens saved + + ? + +
+
—
+
vs native MCP loading · estimate
+
+
+
+ efficiency + — +
+
+
+
+
+ — + / + — + native +
+
+
+ +
+
+
calls
+
—
+
+
+
cache hit
+
—
+
+
+
+
errors
+
—
+
+
+
avg latency
+
—
+
p95 —
+
+
+ +
+
+
+ top tools + by calls +
+
+
+
+
+ top savings + vs native MCP +
+
+
+
+ +
+
+
+ server health +
+
+
+
+ +
+
+ live tail + click row to inspect + +
+
+
+
+ + +
+ + + + + diff --git a/internal/ui/assets/style.css b/internal/ui/assets/style.css new file mode 100644 index 0000000..cb13be1 --- /dev/null +++ b/internal/ui/assets/style.css @@ -0,0 +1,616 @@ +:root { + --bg: #0a0d12; + --bg-1: #0f131a; + --bg-2: #151b24; + --bg-3: #1d2530; + --border: #232b36; + --border-strong: #2d3744; + --text: #c5cdd6; + --text-bright: #f5f7fa; + --dim: #6b7480; + --accent: #6ea8ff; + --green: #45c46a; + --green-2: #2d8a48; + --yellow: #d4a142; + --red: #e85a5a; + --purple: #a98bff; + --shadow-1: 0 1px 2px rgba(0, 0, 0, 0.4); + --radius: 7px; +} + +* { box-sizing: border-box; } + +html, body { + margin: 0; + padding: 0; + font-family: + -apple-system, BlinkMacSystemFont, "Inter", "SF Pro Text", + "Segoe UI", Helvetica, Arial, sans-serif; + background: var(--bg); + color: var(--text); + font-size: 13.5px; + line-height: 1.45; + -webkit-font-smoothing: antialiased; + -moz-osx-font-smoothing: grayscale; +} + +code, pre, .mono { + font-family: "SF Mono", "JetBrains Mono", Menlo, Consolas, monospace; + font-feature-settings: "tnum" 1, "ss01" 1; +} + +button { + font-family: inherit; + background: transparent; + border: 1px solid var(--border); + color: var(--text); + padding: 5px 10px; + border-radius: 5px; + cursor: pointer; + font-size: 12.5px; + transition: background 0.12s, border-color 0.12s, color 0.12s; +} +button:hover { background: var(--bg-2); border-color: var(--border-strong); } +button.primary { + background: var(--accent); + color: #0a0d12; + border-color: var(--accent); + font-weight: 600; +} +button.primary:hover { background: #82b6ff; } +button.ghost { color: var(--dim); } + +.dim { color: var(--dim); } +.small { font-size: 11px; } +.sep { color: var(--border-strong); margin: 0 4px; } + +.app { + display: grid; + grid-template-columns: 220px 1fr; + height: 100vh; + overflow: hidden; +} + +/* === SIDEBAR === */ +.sidebar { + background: var(--bg-1); + border-right: 1px solid var(--border); + padding: 18px 14px; + overflow-y: auto; + display: flex; + flex-direction: column; +} + +.brand { + display: flex; + align-items: center; + justify-content: space-between; + margin-bottom: 22px; + padding-bottom: 14px; + border-bottom: 1px solid var(--border); +} + +.logo { + font-weight: 700; + font-size: 17px; + letter-spacing: -0.02em; + color: var(--text-bright); +} + +.conn { + width: 8px; + height: 8px; + border-radius: 50%; + background: var(--dim); + transition: background 0.2s, box-shadow 0.2s; +} +.conn.live { + background: var(--green); + box-shadow: 0 0 6px var(--green); + animation: pulse 1.4s ease-in-out infinite; +} +.conn.dead { + background: var(--red); +} +@keyframes pulse { 0%, 100% { opacity: 1; } 50% { opacity: 0.45; } } + +.section { margin-bottom: 20px; } +.spacer { flex: 1; } + +.section-title { + font-size: 10.5px; + font-weight: 600; + text-transform: uppercase; + letter-spacing: 0.08em; + color: var(--dim); + margin-bottom: 6px; +} + +.vlist { list-style: none; margin: 0; padding: 0; } +.vlist li { + padding: 5px 8px; + border-radius: 5px; + cursor: pointer; + display: flex; + justify-content: space-between; + align-items: center; + font-size: 12.5px; + transition: background 0.1s, color 0.1s; +} +.vlist li:hover { background: var(--bg-2); } +.vlist li.active { + background: rgba(110, 168, 255, 0.12); + color: var(--text-bright); + font-weight: 500; + border: 1px solid rgba(110, 168, 255, 0.3); + padding: 4px 7px; +} +.vlist li .count { + font-size: 10.5px; + color: var(--dim); + font-variant-numeric: tabular-nums; +} + +.help pre { + background: var(--bg-2); + padding: 8px 10px; + border-radius: 6px; + font-size: 10.5px; + color: var(--dim); + margin: 0; + overflow: hidden; + border: 1px solid var(--border); +} + +/* === MAIN === */ +.content { + padding: 16px 22px 22px; + overflow-y: auto; + display: flex; + flex-direction: column; + gap: 14px; +} + +.topbar { + display: flex; + align-items: center; + justify-content: space-between; + gap: 16px; + padding-bottom: 12px; + border-bottom: 1px solid var(--border); + margin-bottom: 4px; +} + +.title { + font-size: 16px; + color: var(--text-bright); + font-weight: 600; + display: flex; + align-items: baseline; + gap: 4px; +} + +.tabs { + display: flex; + gap: 2px; + background: var(--bg-2); + padding: 3px; + border-radius: 6px; + border: 1px solid var(--border); +} +.tabs button { + border: none; + padding: 4px 11px; + font-size: 11.5px; + border-radius: 4px; + color: var(--dim); + background: transparent; +} +.tabs button:hover { color: var(--text); background: transparent; } +.tabs button.active { + background: var(--bg-3); + color: var(--text-bright); + font-weight: 500; +} + +.freshness { color: var(--dim); font-size: 11px; } + +/* === HERO === */ +.hero { + display: grid; + grid-template-columns: 1.2fr 1fr; + gap: 14px; + background: var(--bg-1); + border: 1px solid var(--border); + border-radius: var(--radius); + padding: 22px 24px; +} + +.hero-left { display: flex; flex-direction: column; } +.hero-label { + font-size: 11px; + text-transform: uppercase; + letter-spacing: 0.1em; + color: var(--dim); + font-weight: 600; + margin-bottom: 4px; + display: flex; + align-items: center; + gap: 6px; +} + +.info-badge { + display: inline-flex; + align-items: center; + justify-content: center; + width: 14px; + height: 14px; + border-radius: 50%; + background: var(--bg-3); + color: var(--dim); + font-size: 10px; + font-weight: 700; + cursor: help; + position: relative; + border: 1px solid var(--border-strong); + transition: background 0.12s, color 0.12s; +} +.info-badge:hover, .info-badge:focus { + background: var(--accent); + color: var(--bg); + outline: none; +} +.info-badge::after { + content: attr(data-tooltip); + position: absolute; + top: 22px; + left: -8px; + background: var(--bg-3); + border: 1px solid var(--border-strong); + border-radius: 6px; + padding: 8px 11px; + font-size: 11px; + font-weight: 400; + color: var(--text); + text-transform: none; + letter-spacing: normal; + width: 280px; + max-width: 280px; + white-space: normal; + line-height: 1.45; + opacity: 0; + pointer-events: none; + transform: translateY(-4px); + transition: opacity 0.12s, transform 0.12s; + z-index: 10; + box-shadow: 0 4px 14px rgba(0, 0, 0, 0.5); +} +.info-badge:hover::after, .info-badge:focus::after { + opacity: 1; + transform: translateY(0); +} +.hero-value { + font-size: 64px; + font-weight: 700; + color: var(--green); + line-height: 1; + letter-spacing: -0.03em; + font-variant-numeric: tabular-nums; + margin-bottom: 6px; +} +.hero-foot { font-size: 11.5px; } + +.hero-right { + display: flex; + flex-direction: column; + justify-content: center; + gap: 8px; +} +.eff-label { + font-size: 11px; + text-transform: uppercase; + letter-spacing: 0.1em; + color: var(--dim); + font-weight: 600; + display: flex; + justify-content: space-between; + align-items: baseline; +} +#eff-pct { + text-transform: none; + letter-spacing: 0; + font-weight: 700; + font-size: 13px; + color: var(--text-bright); +} +.eff-bar { + height: 14px; + background: var(--bg-3); + border-radius: 7px; + overflow: hidden; + position: relative; +} +.eff-bar-fill { + height: 100%; + background: linear-gradient(90deg, var(--green-2), var(--green)); + transition: width 0.4s ease; +} +.eff-foot { + display: flex; + justify-content: flex-start; + font-variant-numeric: tabular-nums; +} +.eff-foot .sep { color: var(--border-strong); } + +/* === TILE ROW === */ +.tile-row { + display: grid; + grid-template-columns: repeat(4, 1fr); + gap: 14px; +} + +.tile { + background: var(--bg-1); + border: 1px solid var(--border); + border-radius: var(--radius); + padding: 14px 18px; + min-height: 78px; + display: flex; + flex-direction: column; + justify-content: center; +} +.tile-label { + font-size: 10.5px; + text-transform: uppercase; + letter-spacing: 0.08em; + color: var(--dim); + margin-bottom: 4px; +} +.tile-value { + font-size: 26px; + font-weight: 700; + color: var(--text-bright); + letter-spacing: -0.02em; + font-variant-numeric: tabular-nums; + line-height: 1.1; +} +.tile-value.warn { color: var(--yellow); } +.tile-value.bad { color: var(--red); } +.tile-foot { font-size: 11px; margin-top: 4px; } + +/* === ROW LAYOUT === */ +.row { display: grid; grid-template-columns: 1fr 1fr; gap: 14px; } +.row.grow { display: block; } + +.card { + background: var(--bg-1); + border: 1px solid var(--border); + border-radius: var(--radius); + padding: 14px 16px 12px; +} +.card.grow { display: block; } + +.card-head { + display: flex; + align-items: baseline; + gap: 8px; + margin-bottom: 10px; + flex-wrap: wrap; +} +.card-title { + font-size: 11px; + font-weight: 600; + text-transform: uppercase; + letter-spacing: 0.08em; + color: var(--text-bright); +} +.card-sub { font-size: 10.5px; } + +.bar-table { + width: 100%; + border-collapse: collapse; + font-size: 12.5px; + font-variant-numeric: tabular-nums; +} +.bar-table td { padding: 5px 0; vertical-align: middle; } +.bar-table td.name { + color: var(--text-bright); + font-weight: 500; + white-space: nowrap; + max-width: 240px; + overflow: hidden; + text-overflow: ellipsis; +} +.bar-table td.bar { width: 100%; padding: 0 12px; } +.bar-table td.bar .fill { + height: 6px; + background: linear-gradient(90deg, var(--green-2), var(--green)); + border-radius: 3px; + transition: width 0.3s; +} +.bar-table td.value { + color: var(--dim); + text-align: right; + font-weight: 500; + min-width: 60px; + white-space: nowrap; +} + +/* === SERVER HEALTH === */ +.health-grid { + display: grid; + grid-template-columns: repeat(auto-fill, minmax(220px, 1fr)); + gap: 10px; +} +.health-card { + background: var(--bg-2); + border: 1px solid var(--border); + border-radius: 6px; + padding: 10px 12px; + display: flex; + flex-direction: column; + gap: 4px; +} +.health-card .top { + display: flex; + justify-content: space-between; + align-items: center; +} +.health-card .name { color: var(--text-bright); font-weight: 500; } +.health-card .status { + font-size: 10.5px; + text-transform: uppercase; + letter-spacing: 0.08em; + font-weight: 600; + color: var(--green); +} +.health-card .status.warn { color: var(--yellow); } +.health-card .status.bad { color: var(--red); } +.health-card .stats { + display: flex; + justify-content: space-between; + font-size: 11.5px; + color: var(--dim); + font-variant-numeric: tabular-nums; +} +.health-card .stats b { color: var(--text); font-weight: 500; } + +/* === LIVE TAIL === */ +.tail-filter { + margin-left: auto; + background: var(--bg-2); + border: 1px solid var(--border); + border-radius: 5px; + padding: 4px 9px; + color: var(--text); + font-family: inherit; + font-size: 12px; + width: 240px; +} +.tail-filter:focus { + outline: none; + border-color: var(--accent); +} + +.tail { + font-family: "SF Mono", "JetBrains Mono", Menlo, Consolas, monospace; + font-size: 12px; + max-height: 360px; + overflow-y: auto; +} +.tail-row { + display: grid; + grid-template-columns: 80px 1fr 80px 75px 18px; + align-items: center; + padding: 6px 8px; + border-bottom: 1px solid var(--border); + cursor: pointer; + gap: 8px; + transition: background 0.08s; +} +.tail-row:hover { background: var(--bg-2); } +.tail-row.selected { + background: rgba(110, 168, 255, 0.08); + border-left: 2px solid var(--accent); + padding-left: 6px; +} +.tail-row .ts { color: var(--dim); } +.tail-row .tool { + color: var(--text-bright); + white-space: nowrap; + overflow: hidden; + text-overflow: ellipsis; +} +.tail-row .lat { + color: var(--text); + text-align: right; + font-variant-numeric: tabular-nums; +} +.tail-row .badge { + font-size: 10px; + text-align: center; + padding: 1px 6px; + border-radius: 4px; + white-space: nowrap; + text-transform: lowercase; +} +.tail-row .badge.warm { background: rgba(110, 168, 255, 0.16); color: var(--accent); } +.tail-row .ok { color: var(--green); text-align: center; font-weight: 600; } +.tail-row .err { color: var(--red); text-align: center; font-weight: 600; } + +.tail-empty { + padding: 24px 8px; + text-align: center; + color: var(--dim); + font-size: 12px; +} + +/* === DRAWER === */ +.drawer { + position: fixed; + top: 0; + right: 0; + width: 460px; + height: 100vh; + background: var(--bg-1); + border-left: 1px solid var(--border-strong); + box-shadow: -4px 0 16px rgba(0,0,0,0.4); + display: flex; + flex-direction: column; + z-index: 50; + transform: translateX(0); + transition: transform 0.18s ease; +} +.drawer.hidden { + transform: translateX(100%); + pointer-events: none; +} +.drawer-head { + padding: 14px 18px; + border-bottom: 1px solid var(--border); + display: flex; + align-items: center; + justify-content: space-between; +} +.drawer-title { color: var(--text-bright); font-weight: 600; font-size: 14px; } +.drawer-body { + padding: 14px 18px 24px; + overflow-y: auto; + flex: 1; +} +.kv-grid { + display: grid; + grid-template-columns: max-content 1fr; + column-gap: 16px; + row-gap: 4px; + font-size: 12px; + margin-bottom: 14px; +} +.kv-grid .k { color: var(--dim); } +.kv-grid .v { color: var(--text-bright); font-variant-numeric: tabular-nums; } +.json { + background: var(--bg-2); + border: 1px solid var(--border); + border-radius: 5px; + padding: 9px 11px; + font-size: 11px; + margin: 6px 0 14px; + white-space: pre-wrap; + word-break: break-word; + max-height: 220px; + overflow-y: auto; + color: var(--text); +} +.drawer-actions { display: flex; gap: 8px; margin-top: 6px; } + +/* === RESPONSIVE === */ +@media (max-width: 1100px) { + .hero { grid-template-columns: 1fr; } + .row { grid-template-columns: 1fr; } + .tile-row { grid-template-columns: repeat(2, 1fr); } +} +@media (max-width: 720px) { + .app { grid-template-columns: 1fr; } + .sidebar { display: none; } + .drawer { width: 100vw; } +} diff --git a/internal/ui/detach_unix.go b/internal/ui/detach_unix.go new file mode 100644 index 0000000..8a85440 --- /dev/null +++ b/internal/ui/detach_unix.go @@ -0,0 +1,9 @@ +//go:build !windows + +package ui + +import "syscall" + +func detachAttr() *syscall.SysProcAttr { + return &syscall.SysProcAttr{Setsid: true} +} diff --git a/internal/ui/detach_windows.go b/internal/ui/detach_windows.go new file mode 100644 index 0000000..ab0177a --- /dev/null +++ b/internal/ui/detach_windows.go @@ -0,0 +1,9 @@ +//go:build windows + +package ui + +import "syscall" + +func detachAttr() *syscall.SysProcAttr { + return &syscall.SysProcAttr{} +} diff --git a/internal/ui/handshake.go b/internal/ui/handshake.go new file mode 100644 index 0000000..40c0c26 --- /dev/null +++ b/internal/ui/handshake.go @@ -0,0 +1,91 @@ +// Package ui hosts the always-on dashboard for mcpx. +// +// Lifecycle: any `mcpx ` invocation calls EnsureRunningAsync which +// lazily spawns a supervisor daemon. The daemon writes ~/.mcpx/ui.json with +// {port, token, pid, bind} so the CLI (and other tools) can discover the URL. +// Idle 1h with no HTTP traffic → daemon self-exits and removes the file. +package ui + +import ( + "crypto/rand" + "encoding/hex" + "encoding/json" + "errors" + "os" + "path/filepath" +) + +// Handshake is what the UI daemon writes to ~/.mcpx/ui.json so other processes +// can discover the dashboard. +type Handshake struct { + Port int `json:"port"` + Token string `json:"token"` + PID int `json:"pid"` + Bind string `json:"bind"` +} + +// HandshakePath returns the on-disk handshake file path. +func HandshakePath() string { + home, err := os.UserHomeDir() + if err != nil { + return "" + } + return filepath.Join(home, ".mcpx", "ui.json") +} + +// LoadHandshake reads the handshake. Returns (zero, error) if missing. +func LoadHandshake() (Handshake, error) { + path := HandshakePath() + if path == "" { + return Handshake{}, errors.New("ui: no home dir") + } + data, err := os.ReadFile(path) + if err != nil { + return Handshake{}, err + } + var h Handshake + if err := json.Unmarshal(data, &h); err != nil { + return Handshake{}, err + } + return h, nil +} + +// SaveHandshake writes the handshake atomically with mode 0600. +func SaveHandshake(h Handshake) error { + path := HandshakePath() + if path == "" { + return errors.New("ui: no home dir") + } + if err := os.MkdirAll(filepath.Dir(path), 0o755); err != nil { + return err + } + data, err := json.MarshalIndent(h, "", " ") + if err != nil { + return err + } + tmp := path + ".tmp" + if err := os.WriteFile(tmp, data, 0o600); err != nil { + return err + } + return os.Rename(tmp, path) +} + +// RemoveHandshake removes the handshake file. No-op if missing. +func RemoveHandshake() error { + path := HandshakePath() + if path == "" { + return nil + } + err := os.Remove(path) + if err != nil && os.IsNotExist(err) { + return nil + } + return err +} + +// NewToken generates a 32-char hex token (128 bits). +func NewToken() string { + b := make([]byte, 16) + _, _ = rand.Read(b) + return hex.EncodeToString(b) +} diff --git a/internal/ui/server.go b/internal/ui/server.go new file mode 100644 index 0000000..bbfc926 --- /dev/null +++ b/internal/ui/server.go @@ -0,0 +1,505 @@ +package ui + +import ( + "context" + "embed" + "encoding/json" + "errors" + "fmt" + "html/template" + "io" + "io/fs" + "net" + "net/http" + "os" + "path/filepath" + "strconv" + "strings" + "sync" + "sync/atomic" + "time" + + "github.com/codestz/mcpx/internal/stats" +) + +//go:embed assets/* +var assetsFS embed.FS + +// Run starts the dashboard HTTP server. Blocks until ctx is cancelled or the +// idle timeout fires. +func Run(ctx context.Context, port int, bind string, idleTimeout time.Duration) error { + if bind == "" { + bind = "127.0.0.1" + } + addr := bind + ":" + strconv.Itoa(port) + listener, err := net.Listen("tcp", addr) + if err != nil { + // If the requested port is taken, try an ephemeral one. + listener, err = net.Listen("tcp", bind+":0") + if err != nil { + return fmt.Errorf("ui: listen: %w", err) + } + } + defer listener.Close() + + resolved := listener.Addr().(*net.TCPAddr) + token := NewToken() + + if err := SaveHandshake(Handshake{ + Port: resolved.Port, Token: token, PID: os.Getpid(), Bind: bind, + }); err != nil { + return fmt.Errorf("ui: handshake: %w", err) + } + defer RemoveHandshake() + + srv := newServer(token, idleTimeout) + httpSrv := &http.Server{ + Handler: srv, + ReadHeaderTimeout: 5 * time.Second, + } + + // Idle watchdog: when no requests in idleTimeout, shutdown gracefully. + idleCtx, cancelIdle := context.WithCancel(ctx) + defer cancelIdle() + go srv.idleLoop(idleCtx, idleTimeout, func() { + shutdownCtx, cancel := context.WithTimeout(context.Background(), 3*time.Second) + defer cancel() + _ = httpSrv.Shutdown(shutdownCtx) + }) + + errCh := make(chan error, 1) + go func() { + errCh <- httpSrv.Serve(listener) + }() + + select { + case <-ctx.Done(): + shutdownCtx, cancel := context.WithTimeout(context.Background(), 3*time.Second) + defer cancel() + _ = httpSrv.Shutdown(shutdownCtx) + return nil + case err := <-errCh: + if errors.Is(err, http.ErrServerClosed) { + return nil + } + return err + } +} + +type server struct { + token string + mux *http.ServeMux + tmpl *template.Template + lastHit atomic.Int64 + idleTimeout time.Duration + + subsMu sync.Mutex + subs map[chan stats.Record]struct{} + + tailMu sync.Mutex + statsPath string + tailOffset int64 +} + +func newServer(token string, idleTimeout time.Duration) *server { + s := &server{ + token: token, + idleTimeout: idleTimeout, + subs: map[chan stats.Record]struct{}{}, + statsPath: stats.DefaultPath(), + } + s.lastHit.Store(time.Now().UnixNano()) + + tmpl, err := template.ParseFS(assetsFS, "assets/index.html") + if err == nil { + s.tmpl = tmpl + } + + mux := http.NewServeMux() + mux.HandleFunc("/", s.handleIndex) + mux.HandleFunc("/assets/", s.handleAssets) + mux.HandleFunc("/api/summary", s.tokenGuard(s.handleSummary)) + mux.HandleFunc("/api/projects", s.tokenGuard(s.handleProjects)) + mux.HandleFunc("/api/events", s.tokenGuard(s.handleEvents)) + mux.HandleFunc("/api/call/", s.tokenGuard(s.handleCall)) + mux.HandleFunc("/api/audit", s.tokenGuard(s.handleAudit)) + mux.HandleFunc("/api/tool/", s.tokenGuard(s.handleToolDetail)) + s.mux = mux + + go s.tailLoop() + return s +} + +func (s *server) ServeHTTP(w http.ResponseWriter, r *http.Request) { + s.lastHit.Store(time.Now().UnixNano()) + s.mux.ServeHTTP(w, r) +} + +func (s *server) tokenGuard(h http.HandlerFunc) http.HandlerFunc { + return func(w http.ResponseWriter, r *http.Request) { + if r.URL.Query().Get("t") != s.token { + http.Error(w, "forbidden", http.StatusForbidden) + return + } + h(w, r) + } +} + +func (s *server) handleIndex(w http.ResponseWriter, r *http.Request) { + // Browser hits "/" without token, then loads JS that includes ?t=. + // Anyone with the URL+token can access — that's the entire auth model. + if r.URL.Path != "/" { + http.NotFound(w, r) + return + } + tok := r.URL.Query().Get("t") + if tok != s.token { + w.Header().Set("Content-Type", "text/html; charset=utf-8") + fmt.Fprintf(w, ` +mcpx + +

mcpx dashboard

+

Open the URL printed by the CLI on first run, or run:
+mcpx ui open

+`) + return + } + if s.tmpl == nil { + http.Error(w, "template missing", 500) + return + } + w.Header().Set("Content-Type", "text/html; charset=utf-8") + _ = s.tmpl.Execute(w, map[string]string{"Token": tok}) +} + +func (s *server) handleAssets(w http.ResponseWriter, r *http.Request) { + tok := r.URL.Query().Get("t") + if tok != s.token { + http.Error(w, "forbidden", 403) + return + } + rel := strings.TrimPrefix(r.URL.Path, "/assets/") + data, err := fs.ReadFile(assetsFS, "assets/"+rel) + if err != nil { + http.NotFound(w, r) + return + } + switch filepath.Ext(rel) { + case ".js": + w.Header().Set("Content-Type", "application/javascript") + case ".css": + w.Header().Set("Content-Type", "text/css") + case ".html": + w.Header().Set("Content-Type", "text/html; charset=utf-8") + } + _, _ = w.Write(data) +} + +func (s *server) handleSummary(w http.ResponseWriter, r *http.Request) { + since, _ := time.ParseDuration(r.URL.Query().Get("since")) + if since == 0 { + since = 7 * 24 * time.Hour + } + filter := stats.Filter{Since: time.Now().Add(-since)} + + if r.URL.Query().Get("all") == "" { + if p := r.URL.Query().Get("project"); p != "" { + filter.Project = p + } + } + + summary, err := stats.Aggregate(s.statsPath, filter, 30) + if err != nil { + http.Error(w, err.Error(), 500) + return + } + + // Per-server call counts for the sidebar. + perServer := map[string]int{} + for _, st := range summary.TopServers { + perServer[st.Server] = st.Calls + } + resp := map[string]any{ + "Calls": summary.Calls, + "TokensSaved": summary.TokensSaved, + "NativeBaseline": summary.NativeBaseline, + "AvgLatencyMS": summary.AvgLatencyMS, + "P50LatencyMS": summary.P50LatencyMS, + "P95LatencyMS": summary.P95LatencyMS, + "CacheHitRate": summary.CacheHitRate, + "SchemaHitRate": summary.SchemaHitRate, + "ErrorRate": summary.ErrorRate, + "TopTools": summary.TopTools, + "TopSavers": summary.TopSavers, + "Daily": summary.Daily, + "Recent": summary.Recent, + "Servers": summary.Servers, + "Projects": summary.Projects, + "PerServer": perServer, + } + w.Header().Set("Content-Type", "application/json") + _ = json.NewEncoder(w).Encode(resp) +} + +// handleProjects lists distinct projects observed in stats with per-project totals. +func (s *server) handleProjects(w http.ResponseWriter, r *http.Request) { + type proj struct { + Name string + Path string + Calls int + TokensSaved int64 + Current bool + } + cwd, _ := os.Getwd() + + // Load broad summary across all time. + summary, err := stats.Aggregate(s.statsPath, stats.Filter{}, 0) + if err != nil { + http.Error(w, err.Error(), 500) + return + } + + // Walk records once to build per-project counts. + type bucket struct { + Calls int + TokensSaved int64 + } + by := map[string]*bucket{} + _ = stats.Iter(s.statsPath, stats.Filter{}, func(rec stats.Record) error { + if rec.Project == "" { + return nil + } + b, ok := by[rec.Project] + if !ok { + b = &bucket{} + by[rec.Project] = b + } + b.Calls++ + b.TokensSaved += int64(rec.TokensSaved) + return nil + }) + _ = summary + + out := make([]proj, 0, len(by)) + for path, b := range by { + out = append(out, proj{ + Name: filepath.Base(path), + Path: path, + Calls: b.Calls, + TokensSaved: b.TokensSaved, + Current: path == cwd, + }) + } + w.Header().Set("Content-Type", "application/json") + _ = json.NewEncoder(w).Encode(out) +} + +// handleCall returns the full record for a given call ID. +func (s *server) handleCall(w http.ResponseWriter, r *http.Request) { + id := strings.TrimPrefix(r.URL.Path, "/api/call/") + if id == "" { + http.Error(w, "missing id", http.StatusBadRequest) + return + } + var found *stats.Record + _ = stats.Iter(s.statsPath, stats.Filter{}, func(rec stats.Record) error { + if rec.ID == id { + cp := rec + found = &cp + } + return nil + }) + if found == nil { + http.NotFound(w, r) + return + } + w.Header().Set("Content-Type", "application/json") + _ = json.NewEncoder(w).Encode(found) +} + +// handleAudit returns recent denied/warn policy decisions. +func (s *server) handleAudit(w http.ResponseWriter, r *http.Request) { + limit := 100 + var rows []stats.Record + _ = stats.Iter(s.statsPath, stats.Filter{}, func(rec stats.Record) error { + if rec.PolicyAction == "denied" || rec.PolicyAction == "warn" { + rows = append(rows, rec) + if len(rows) > limit { + rows = rows[1:] + } + } + return nil + }) + w.Header().Set("Content-Type", "application/json") + _ = json.NewEncoder(w).Encode(rows) +} + +// handleToolDetail returns aggregated stats and recent calls for one tool. +func (s *server) handleToolDetail(w http.ResponseWriter, r *http.Request) { + rest := strings.TrimPrefix(r.URL.Path, "/api/tool/") + parts := strings.SplitN(rest, "/", 2) + if len(parts) != 2 { + http.Error(w, "expected /api/tool//", http.StatusBadRequest) + return + } + server, tool := parts[0], parts[1] + + var recent []stats.Record + calls := 0 + var totalLat, totalSaved int64 + var errors int + latencies := []int64{} + + _ = stats.Iter(s.statsPath, stats.Filter{Server: server, Tool: tool}, func(rec stats.Record) error { + calls++ + totalLat += rec.LatencyMS + totalSaved += int64(rec.TokensSaved) + latencies = append(latencies, rec.LatencyMS) + if rec.ExitCode != 0 { + errors++ + } + recent = append(recent, rec) + if len(recent) > 50 { + recent = recent[1:] + } + return nil + }) + + avg := 0.0 + if calls > 0 { + avg = float64(totalLat) / float64(calls) + } + resp := map[string]any{ + "server": server, + "tool": tool, + "calls": calls, + "errors": errors, + "tokens_saved": totalSaved, + "avg_latency_ms": avg, + "recent": recent, + } + w.Header().Set("Content-Type", "application/json") + _ = json.NewEncoder(w).Encode(resp) +} + +// handleEvents streams new stats records over Server-Sent Events. +func (s *server) handleEvents(w http.ResponseWriter, r *http.Request) { + flusher, ok := w.(http.Flusher) + if !ok { + http.Error(w, "streaming unsupported", 500) + return + } + w.Header().Set("Content-Type", "text/event-stream") + w.Header().Set("Cache-Control", "no-cache") + w.Header().Set("Connection", "keep-alive") + + ch := make(chan stats.Record, 64) + s.subsMu.Lock() + s.subs[ch] = struct{}{} + s.subsMu.Unlock() + + defer func() { + s.subsMu.Lock() + delete(s.subs, ch) + s.subsMu.Unlock() + }() + + enc := json.NewEncoder(w) + for { + select { + case <-r.Context().Done(): + return + case rec := <-ch: + fmt.Fprintf(w, "event: call\ndata: ") + _ = enc.Encode(rec) + fmt.Fprintf(w, "\n") + flusher.Flush() + } + } +} + +// tailLoop watches the stats JSONL file and broadcasts new records to +// SSE subscribers. +func (s *server) tailLoop() { + if s.statsPath == "" { + return + } + tick := time.NewTicker(500 * time.Millisecond) + defer tick.Stop() + for range tick.C { + s.tailOnce() + } +} + +func (s *server) tailOnce() { + s.tailMu.Lock() + defer s.tailMu.Unlock() + + f, err := os.Open(s.statsPath) + if err != nil { + return + } + defer f.Close() + stat, _ := f.Stat() + if stat == nil { + return + } + if s.tailOffset == 0 { + // First poll: skip to end so we only stream NEW records. + s.tailOffset = stat.Size() + return + } + if stat.Size() < s.tailOffset { + // File rotated/truncated. + s.tailOffset = stat.Size() + return + } + if stat.Size() == s.tailOffset { + return + } + if _, err := f.Seek(s.tailOffset, io.SeekStart); err != nil { + return + } + dec := json.NewDecoder(f) + for dec.More() { + var rec stats.Record + if err := dec.Decode(&rec); err != nil { + break + } + s.broadcast(rec) + } + s.tailOffset = stat.Size() +} + +func (s *server) broadcast(rec stats.Record) { + s.subsMu.Lock() + defer s.subsMu.Unlock() + for ch := range s.subs { + select { + case ch <- rec: + default: + // Slow subscriber — drop rather than block. + } + } +} + +// idleLoop shuts the server down after idleTimeout of no activity. +func (s *server) idleLoop(ctx context.Context, idleTimeout time.Duration, shutdown func()) { + if idleTimeout <= 0 { + return + } + tick := time.NewTicker(30 * time.Second) + defer tick.Stop() + for { + select { + case <-ctx.Done(): + return + case <-tick.C: + last := s.lastHit.Load() + if time.Since(time.Unix(0, last)) > idleTimeout { + shutdown() + return + } + } + } +} diff --git a/internal/ui/supervisor.go b/internal/ui/supervisor.go new file mode 100644 index 0000000..f66b768 --- /dev/null +++ b/internal/ui/supervisor.go @@ -0,0 +1,81 @@ +package ui + +import ( + "net" + "os" + "os/exec" + "strconv" + "sync" + "syscall" + "time" +) + +var ensureOnce sync.Once + +// EnsureRunningAsync spawns the dashboard daemon if it isn't already running. +// Non-blocking: returns immediately and lets the daemon come up in background. +// +// Honors `MCPX_UI=off` to opt out. +func EnsureRunningAsync(enabled bool, configuredPort int, bind string) { + if !enabled { + return + } + if v := os.Getenv("MCPX_UI"); v == "off" || v == "0" || v == "false" { + return + } + ensureOnce.Do(func() { + if isAlive() { + return + } + go spawn(configuredPort, bind) + }) +} + +// isAlive returns true if the recorded daemon is reachable. +func isAlive() bool { + h, err := LoadHandshake() + if err != nil { + return false + } + if h.PID > 0 { + if proc, err := os.FindProcess(h.PID); err == nil { + if err := proc.Signal(syscall.Signal(0)); err != nil { + _ = RemoveHandshake() + return false + } + } + } + addr := h.Bind + ":" + strconv.Itoa(h.Port) + conn, err := net.DialTimeout("tcp", addr, 200*time.Millisecond) + if err != nil { + _ = RemoveHandshake() + return false + } + conn.Close() + return true +} + +// spawn launches the UI daemon as a detached child process. +func spawn(port int, bind string) { + self, err := os.Executable() + if err != nil { + return + } + args := []string{"__ui-run"} + if port > 0 { + args = append(args, "--port", strconv.Itoa(port)) + } + if bind != "" { + args = append(args, "--bind", bind) + } + + cmd := exec.Command(self, args...) + cmd.Stdout = nil + cmd.Stderr = nil + cmd.Stdin = nil + cmd.SysProcAttr = detachAttr() + _ = cmd.Start() + if cmd.Process != nil { + _ = cmd.Process.Release() + } +} diff --git a/website/.vitepress/config.mts b/website/.vitepress/config.mts index 91354e8..3a86d32 100644 --- a/website/.vitepress/config.mts +++ b/website/.vitepress/config.mts @@ -20,7 +20,7 @@ export default defineConfig({ { text: 'Reference', link: '/reference/cli' }, { text: 'Integrations', link: '/integrations/serena' }, { - text: 'v1.5.0', + text: 'v1.6.0', items: [ { text: 'Changelog', link: '/about/changelog' }, { text: 'GitHub', link: 'https://github.com/codestz/mcpx' }, diff --git a/website/about/changelog.md b/website/about/changelog.md index 70bfc41..b9556f8 100644 --- a/website/about/changelog.md +++ b/website/about/changelog.md @@ -5,6 +5,70 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [1.6.0] - 2026-05-03 + +The "Agentic Supremacy" release. mcpx becomes a measurable, observable, and intelligent control plane: every call is recorded, every schema is cached, every server is one `mcpx find` away, and an always-on dashboard makes ROI visible. + +### Added — Foundation + +- **JSONL stats** (`internal/stats/`) — every tool call writes one line to `~/.mcpx/stats.jsonl`: timestamp, args, latency, cache hits, exit code, tokens saved vs native MCP loading. Async writer; never blocks the caller. +- **Schema cache** (`internal/schemacache/`) — `tools/list` (and prompts/resources) cached at `~/.mcpx/cache/schemas/.json` with TTL. First call per server is slow; everything after is instant. Computes `native_baseline_tokens` once at populate time so the stats writer can quote real savings. +- **Result cache: deliberately not shipped.** Caching tool RESPONSES is a correctness footgun — recursive searches, multi-file reads, network-backed tools, and any non-filesystem state can all return stale data inside a TTL. mcpx is the trustworthy intermediary; we'd rather make the round-trip than risk acting on stale snapshots. If a future MCP server adopts `readOnlyHint: true` annotations widely, server-driven result caching becomes safe to add back. +- **Schema normalizer** — handles JSON Schema union types (`type: ["string","null"]`), `oneOf`/`anyOf`/`allOf`, `$ref`; unknown keywords preserved in `Ext`. Fixes #14 (Sentry MCP server initialization). + +### Added — Agent superpowers + +- **`mcpx find `** — BM25-ranked tool search across every configured server. ~80 tokens for top candidates instead of 5–15K tokens for `mcpx list -v`. +- **`mcpx batch`** — NDJSON in/out, parallel by default, single client per server reused across the entire batch. +- **`mcpx --example`** — JSON skeleton with placeholder values from the normalized schema. +- **`mcpx --validate-args ...`** — type/required check without invoking the tool. +- **Default-compact help** — `mcpx --help` is one line per tool with required flags surfaced. Add `--full` for descriptions. +- **Typo remediation** — Levenshtein-2 "did you mean…" suggestions on tool-not-found and unknown-flag errors. +- **Pre/post edit guards** — `replace_symbol_body` calls are warned when the body starts with a duplicate declaration keyword (Go, Python, TS, JS, Rust, Ruby, Java, Kotlin, Swift). For `.go` files the modified file is re-parsed and `file:line:col` is surfaced when the edit broke syntax. + +### Added — Operator surfaces + +- **`mcpx gain`** — premium terminal dashboard: hero metric (tokens saved), 7-day sparkline, top-tools bars, top-savers, server health, recent calls. Subcommands: `--by tool|server|day`, `--history N`, `--suggest`, `--watch`, `--all`, `--project`, `--since`, `--json`. +- **Always-on web dashboard** — auto-spawned on first call. Token-protected, 127.0.0.1-only, idle-shutdown after 1h, opt-out via `MCPX_UI=off` or `ui.enabled: false`. Single-page UI: project sidebar, time range tabs (1h/24h/7d/30d/all), token efficiency bar, top tools, server health, click-to-inspect drawer, SSE live tail, regex filter (`/`). +- **`mcpx ui status|stop|open|disable`** — dashboard daemon controls. +- **`mcpx doctor`** — config + command path + secret resolution + daemon liveness + initialize + tools/list checks. `--json` for machine-readable output. + +### Changed + +- **Structured exit codes** — `0` ok · `1` tool error · `2` config · `3` connection · `4` timeout · `5` policy denied · `6` tool not found. +- **`mcpx version`** warns loudly on `+dirty` builds (so leaked debug code from local builds doesn't silently propagate). +- **`mcpx configure`** rewritten — generates an agent-grounded `MCPX.md` (composition + discover + exit codes) and per-server `.md` files (tool-selector table + compact reference). The `--format compact` flag is now hidden; output is agent-optimized by default. Edit-tool safety guidance only emitted for servers that expose body-mutation tools. +- Client version bumped to `1.6.0` in the MCP handshake. + +### New packages + +- `internal/stats/` — JSONL writer (async, drop-on-overflow), reader, aggregator (top-K, p95, daily buckets, hit rates). +- `internal/schemacache/` — schema cache, result cache, idempotence detection. +- `internal/find/` — BM25 ranker (snake/camel splitting, name-coverage bonus, plural stem). +- `internal/render/` — terminal primitives (Box, Bar, Sparkline, FormatNumber/Duration/Percent, term width). +- `internal/ui/` — dashboard daemon (lazy supervisor, HTTP, SSE, embedded HTML/CSS/JS). + +### Config additions + +```yaml +gain: + enabled: true + tokenizer: estimate # bytes ÷ 4; honest approximation of Claude tokenization + retain_days: 30 + stats_path: "" # default: ~/.mcpx/stats.jsonl + +cache: + schema_ttl: 5m # only the schema cache is shipped — see notes above + +ui: + enabled: true + port: 7878 # 0 = ephemeral + bind: 127.0.0.1 + idle_timeout: 1h +``` + +Environment overrides: `MCPX_AGENT`, `MCPX_CACHE`, `MCPX_UI`, `MCPX_VERBOSE`. + ## [1.5.0] - 2026-03-30 ### Added diff --git a/website/reference/cli.md b/website/reference/cli.md index 8732163..f01068a 100644 --- a/website/reference/cli.md +++ b/website/reference/cli.md @@ -60,12 +60,38 @@ Override the default call timeout for a single invocation. Uses Go duration form mcpx serena search_for_pattern --substring_pattern "TODO" --timeout 60s ``` +### `mcpx --example` + +Print a JSON skeleton for the tool's arguments based on the normalized schema. Required fields get placeholder values; optional fields use schema defaults or enum head. + +```bash +mcpx serena find_symbol --example +# { +# "name_path_pattern": "", +# "depth": 0, +# "include_body": false, +# ... +# } +``` + +Pair with `--json` for piping into other tools. + +### `mcpx --validate-args` + +Type-check arguments without invoking the tool. Reports missing required fields, wrong types, and enum mismatches. Exits `0` on success, `1` on issues. + +```bash +mcpx serena find_symbol --name_path_pattern Auth --validate-args +# ok +``` + ### `mcpx --help` -Show all tools available on a server. If the server supports prompts or resources, they are listed too. +List tools for a server. Default is one line per tool with required flags surfaced — designed to fit a server with 20+ tools in ~30 lines. ```bash -mcpx serena --help +mcpx serena --help # compact (default) +mcpx serena --help --full # verbose with descriptions ``` --- @@ -192,6 +218,112 @@ mcpx ping serena --json Exit code 3 on failure. +### `mcpx find ` + +BM25-ranked tool search across every configured server. Use this when you don't know which tool you need — returns the top candidates in ~80 tokens instead of 5–15K from `list -v`. + +```bash +mcpx find "search code by regex" +# query "search code by regex" +# found 4 result(s) across 1 server(s) +# +# serena.search_for_pattern 1.00 Offers a flexible search for arbitrary patterns... +# serena.find_symbol 0.34 Retrieves info on symbols/code entities... +# ... + +mcpx find "github issue" --top 3 --json +mcpx find "..." --server serena # restrict to one server +``` + +Indexed from the schema cache; first run per server populates the cache. + +--- + +## Batching + +### `mcpx batch` + +Run many tool calls in parallel from NDJSON on stdin. One mcpx process, one MCP client per server reused across the entire batch — no handshake-per-call overhead. + +```bash +printf '%s\n%s\n' \ + '{"id":"a","server":"serena","tool":"find_symbol","args":{"name_path_pattern":"Auth"}}' \ + '{"id":"b","server":"serena","tool":"find_symbol","args":{"name_path_pattern":"Token"}}' | mcpx batch +``` + +Output is NDJSON in input order with `id`, `ok`, `latency_ms`, `cached`, `result` or `error`. + +Flags: +- `--parallel` (default) / `--sequential` +- `--max-concurrent N` (default = NumCPU) +- `--stop-on-error` — abort on first failure +- `--cache=false` — bypass result cache + +The result cache deduplicates identical calls within the batch when enabled. + +--- + +## Observability + +### `mcpx gain` + +Premium terminal dashboard for token-savings analytics. Reads `~/.mcpx/stats.jsonl` (every call writes one line transparently). + +```bash +mcpx gain # current project, last 7 days +mcpx gain --all # every project +mcpx gain --since 24h # narrow window +mcpx gain --by tool # ranked tool table +mcpx gain --by server # ranked server table +mcpx gain --by day # daily breakdown +mcpx gain --history 20 # last N calls +mcpx gain --suggest # mined recommendations +mcpx gain --watch # live refresh +mcpx gain --json # machine-readable +``` + +The dashboard shows the hero "tokens saved" metric, sparkline, top tools by calls, top tools by savings, and the most recent calls with their latency + cache state. + +### `mcpx ui` + +Always-on web dashboard daemon. Auto-spawned on the first tool call (token-protected, 127.0.0.1 only, idle-shutdown after 1h). The URL is printed once per shell session. + +```bash +mcpx ui status # show URL or "inactive" +mcpx ui open # print URL (use with `open $(mcpx ui open)`) +mcpx ui stop # stop the daemon +mcpx ui disable # how to disable permanently +``` + +Opt out via `MCPX_UI=off` env var or `ui.enabled: false` in config. + +--- + +## Diagnostics + +### `mcpx doctor` + +Run a series of checks against config and connectivity: YAML validity, command paths, secret resolution, daemon liveness, MCP `initialize` handshake, `tools/list` reachability. + +```bash +mcpx doctor +# mcpx doctor +# ───────────────────────────────────── +# [ok] config loaded 1 server(s), project = /path +# +# serena +# [ok] command /path/to/serena +# [ok] daemon running, socket reachable +# [ok] initialize FastMCP 1.23.0 +# [ok] tools/list 21 tool(s) +# +# all checks passed + +mcpx doctor --json # machine-readable +``` + +Exits `0` on all-ok, `2` on any failure. + --- ## Configuration @@ -206,17 +338,20 @@ mcpx init ### `mcpx configure` -Auto-generate tool documentation for CLAUDE.md from MCP server schemas. Scans configured servers and writes per-server reference files. +Generate the agent-facing reference docs that teach Claude Code (and any mcpx-aware coding agent) how to use the configured MCP servers. Idempotent — re-run after adding/removing servers. + +Writes: +- `.claude/MCPX.md` — composition primitives, discovery, exit codes +- `.claude/.md` — per-server tool selector table + compact reference +- `.claude/CLAUDE.md` — adds `@MCPX.md` and `@.md` references so the agent loads them automatically ```bash -mcpx configure -# Scanning MCP servers... -# → serena: 21 tools found -# Generating documentation... -# ✓ SERENA.md written (21 tools) -# ✓ MCPX.md updated +mcpx configure # project (.claude/) +mcpx configure --global # user-level (~/.claude/) ``` +The format is now agent-optimized by default: tabular, decision-oriented, no human-ops content (caching internals, observability, dashboard live in the CLI commands above when a human needs them). + --- ## Secrets diff --git a/website/reference/exit-codes.md b/website/reference/exit-codes.md index 8860d2d..5252ad6 100644 --- a/website/reference/exit-codes.md +++ b/website/reference/exit-codes.md @@ -7,9 +7,14 @@ mcpx uses specific exit codes to indicate the type of failure. | Code | Name | Meaning | |------|------|---------| | `0` | Success | Command completed successfully | -| `1` | Tool Error | The MCP tool returned an error (`isError: true`) or Cobra command failed | +| `1` | Tool Error | The MCP tool returned an error (`isError: true`), Cobra command failed, or `--validate-args` reported issues | | `2` | Config Error | Configuration problem: bad YAML, missing server, invalid variable | -| `3` | Connection Error | Cannot reach the server: spawn failure, timeout, transport death | +| `3` | Connection Error | Cannot reach the server: spawn failure, transport death, daemon socket unreachable | +| `4` | Timeout | `--timeout` exceeded for the call | +| `5` | Policy Denied | A security policy denied the call (see `mcpx ` audit log) | +| `6` | Tool Not Found | Tool name not in the server's `tools/list` | + +Codes 4–6 were introduced in v1.6.0 to give agents and scripts finer-grained branching. ## When Each Code Fires @@ -19,18 +24,19 @@ mcpx uses specific exit codes to indicate the type of failure. - `mcpx list` displayed servers - `mcpx ping` got a response - `mcpx secret set` stored the secret +- `mcpx find` returned results +- `mcpx --validate-args` succeeded ### Exit 1 — Tool Error - MCP tool returned `isError: true` in its response -- Unknown tool name: `mcpx serena nonexistent_tool` -- Flag parsing error: missing required flag +- Flag parsing error: missing required flag, type mismatch +- `--validate-args` reported one or more issues - General Cobra command errors ### Exit 2 — Config Error - YAML parse error in config file -- Server name not found in config: `mcpx nonexistent --help` - Variable resolution failed: `$(secret.missing_key)` - Invalid variable namespace: `$(invalid.var)` @@ -38,9 +44,25 @@ mcpx uses specific exit codes to indicate the type of failure. - Server command not found in PATH - Server process exited during startup -- Startup timeout exceeded - Daemon socket unreachable - `mcpx ping` failure +- HTTP/SSE transport unreachable + +### Exit 4 — Timeout + +- The call exceeded `--timeout` +- Surfaces as `context deadline exceeded` in stderr + +### Exit 5 — Policy Denied + +- A `deny` policy matched the tool name, arguments, or content +- The call is logged to the audit log if audit is enabled +- `mcpx doctor` can verify policy configuration + +### Exit 6 — Tool Not Found + +- Tool name doesn't appear in the server's `tools/list` +- mcpx prints a "did you mean…" suggestion when the typo is within Levenshtein distance 2 ## Usage in Scripts @@ -49,6 +71,7 @@ mcpx ping serena --quiet case $? in 0) echo "Server is healthy" ;; 3) echo "Server unreachable" ;; + 4) echo "Timed out" ;; *) echo "Unexpected error" ;; esac ``` @@ -56,7 +79,7 @@ esac ```bash # Fail fast if server is down mcpx ping serena --quiet || exit 1 -mcpx serena search_symbol --name "Auth" +mcpx serena find_symbol --name_path_pattern "Auth" ``` ## Usage by AI Agents @@ -64,6 +87,9 @@ mcpx serena search_symbol --name "Auth" AI agents can use exit codes to decide next steps: - **Exit 0**: parse stdout for results -- **Exit 1**: read stderr for tool error details, possibly retry with different arguments -- **Exit 2**: config problem — check `.mcpx/config.yml` -- **Exit 3**: server unreachable — try `mcpx daemon stop ` and retry, or report the issue +- **Exit 1**: read stderr for tool error details; possibly retry with different arguments (consider `--example` to learn the correct shape) +- **Exit 2**: config problem — check `.mcpx/config.yml`, run `mcpx doctor` +- **Exit 3**: server unreachable — try `mcpx daemon stop ` and retry, or run `mcpx doctor` +- **Exit 4**: extend `--timeout` (e.g., `--timeout 120s`) and retry +- **Exit 5**: do not retry — a policy explicitly denied the call. The user must adjust policies if the call is legitimate. +- **Exit 6**: typo in the tool name — read the suggestion from stderr or run `mcpx find ` to discover the right tool