feat(websearch): add provider-native hosted search tools and local search providers

_docs/config/index.mdx

@@ -57,7 +57,11 @@ If `XDG_CONFIG_HOME` is set, replace `~/.config` with `$XDG_CONFIG_HOME` in the
   },
   "websearch": {
     "enabled": true,
-    "provider": "exa-hosted-mcp"
+    "provider": "exa-hosted-mcp",
+    "native": {
+      "web": false,
+      "x": true
+    }
   },
   "mcp": {
     "gh_grep": {

_docs/config/websearch.mdx

@@ -1,51 +1,96 @@
 ---
-title: Websearch
-description: Configure the built-in websearch tool and search provider.
+title: websearch
+summary: Local search backends and provider-native search tools
 ---
 
-# Web Search
+Configure web search for agents. Crabcode supports two complementary paths:
 
-crabcode exposes a built-in `websearch` tool by default. Agents use `websearch` for discovery and `webfetch` when they already know the URL. The default provider is Exa's hosted MCP service, which does not require an API key.
+1. **Local adapters** (`websearch.provider`) — crabcode runs a `websearch` tool against Exa, Tavily, Brave, Ollama Cloud, etc.
+2. **Native tools** (`websearch.native`) — the active LLM provider executes search server-side (`web_search`, `x_search`, …). These are normal aisdk tools composed into the tools list.
 
-```jsonc title="crabcode.jsonc"
+```jsonc
 {
   "websearch": {
     "enabled": true,
-    "provider": "exa-hosted-mcp"
-  },
-  "permission": {
-    "websearch": "allow"
+    "provider": "exa-hosted-mcp",
+    "apiKey": null,
+    "endpoint": null,
+    "native": {
+      "web": false,
+      "x": true
+    }
   }
 }
 ```
 
-Use `websearch: false` to disable the tool.
+`websearch: false` (or `"enabled": false`) disables both local and native search.
 
-## Providers
+## Local vs native
 
-| Provider | API key config |
-| --- | --- |
-| `exa-hosted-mcp` | Optional `apiKey`; works without one |
-| `firecrawl-hosted-mcp` | Optional `apiKey`; works without one (Firecrawl Keyless) |
-| `exa` | `apiKey: "{env:EXA_API_KEY}"` |
-| `tavily` | `apiKey: "{env:TAVILY_API_KEY}"` |
-| `perplexity` | `apiKey: "{env:PERPLEXITY_API_KEY}"` |
-| `brave` | `apiKey: "{env:BRAVE_SEARCH_API_KEY}"` |
-| `ollama-cloud` | `apiKey: "{env:OLLAMA_API_KEY}"` |
-| `serpapi` | `apiKey: "{env:SERPAPI_API_KEY}"` |
-| `keiro` | `apiKey: "{env:KEIRO_API_KEY}"` |
+| Concern | Config | Who runs it |
+|---|---|---|
+| General web search backend | `provider` / `apiKey` / `endpoint` | Local `websearch` tool (crabcode) |
+| Provider web search | `native.web` | LLM provider (`web_search`, Anthropic hosted web, OpenRouter web plugin) |
+| X/Twitter search | `native.x` | xAI only (`x_search`) |
 
-Only the provider names above are accepted.
+`native.web` and the local `websearch` tool are **substitutes**: when `native.web` is on and the active provider supports hosted web search, local websearch is skipped.
 
-## Keyed provider example
+`native.x` is a **complement**: just to make crabcode as capable as grok-build cli.
 
-```jsonc title="crabcode.jsonc"
+```jsonc
+// Local Ollama Cloud web search + xAI x_search
 {
   "websearch": {
-    "provider": "keiro",
-    "apiKey": "{env:KEIRO_API_KEY}"
+    "provider": "ollama-cloud",
+    "apiKey": "{env:OLLAMA_API_KEY}",
+    "native": {
+      "web": false,
+      "x": true
+    }
   }
 }
 ```
 
-`endpoint` can override the default provider URL for testing or proxying.
+## Defaults
+
+| Field | Default |
+|---|---|
+| `enabled` | `true` when unset |
+| `provider` | `exa-hosted-mcp` (keyless, rate-limited) |
+| `native.web` | `false` |
+| `native.x` | `true` |
+
+Unsupported native flags are ignored (e.g. `native.x` on OpenAI). If `native.web` is true but the active provider has no hosted web tool (Ollama, Groq, …), crabcode falls back to the local adapter.
+
+## Native tool costs
+
+| Provider | Tool | API tool fee | Source |
+|---|---|---|---|
+| OpenAI | `web_search` | **$10 / 1k calls** + search-content tokens | [OpenAI](https://developers.openai.com/api/docs/pricing) |
+| Anthropic | hosted web search | **$10 / 1k searches** + tokens | [Anthropic](https://platform.claude.com/docs/en/about-claude/pricing#web-search-tool) |
+| xAI | `web_search` | **$5 / 1k successful calls** + tokens | [xAI](https://docs.x.ai/developers/pricing#tools-pricing) |
+| xAI | `x_search` | **$5 / 1k successful calls** + tokens | [xAI](https://docs.x.ai/developers/pricing#tools-pricing) |
+
+These fees apply on **API-key** auth. Models often invoke search multiple times per turn.
+
+**Subscription / OAuth** logins (ChatGPT, Claude, SuperGrok, …) use the provider’s subscription transport instead — native search is included in plan quota / rate limits, not billed as the API tool fees above.
+
+**Local adapters** bill (or rate-limit) through their own APIs — `exa-hosted-mcp` and `firecrawl-hosted-mcp` are keyless/rate-limited; Exa/Tavily/Brave/Ollama Cloud/etc. use your key.
+
+Default is local websearch + `native.x` on xAI. Set `native.web: true` to use provider-hosted web search instead (API tool fees may apply).
+
+## Providers (local adapters)
+
+| Provider | Auth | Notes |
+|---|---|---|
+| `exa-hosted-mcp` | None | Default. Hosted MCP bridge; rate-limited |
+| `firecrawl-hosted-mcp` | None (optional key) | Firecrawl hosted MCP bridge; rate-limited without a key |
+| `exa` | `EXA_API_KEY` | Direct Exa API |
+| `tavily` | `TAVILY_API_KEY` | Tavily search API |
+| `perplexity` | `PERPLEXITY_API_KEY` | Perplexity Search API |
+| `brave` | `BRAVE_API_KEY` | Brave Search API |
+| `ollama-cloud` | `OLLAMA_API_KEY` (required) | Ollama Cloud `POST /api/web_search` — separate HTTP API, **not** a chat-native tool |
+| `serpapi` | `SERPAPI_API_KEY` | SerpAPI results from Google, DuckDuckGo, etc. |
+| `keiro` | `KEIRO_API_KEY` | Keiro search API |
+| `parallel` | `PARALLEL_API_KEY` | Parallel search API |
+| `tako` | `TAKO_API_KEY` | Tako search API |

_plans/__TODOS.md

@@ -484,3 +484,11 @@ I think this is how the TUI works already anyway right?
 - [x] I want to add 'g e' to scroll down.
 
 - [ ] I wanna be able to cancel queued messages if needed.
+
+- [x] Hosted search
+
+- [ ] Asking crabcode to run some tui like `lazygitrs` is causing it to crash the agent.
+
+- [ ] Extra padding in non compact mode. Or idk. controllable in tui? field? Right now it's close to the edge and it only looks good in some terminals.
+
+- [ ] /btw command

src/agent/subagent.rs

@@ -68,7 +68,7 @@ pub async fn run_subagent(
 
     let scoped_registry = build_scoped_registry(full_registry, &agent).await;
 
-    let aisdk_tools = crate::tools::aisdk_bridge::convert_to_aisdk_tools(
+    let mut aisdk_tools = crate::tools::aisdk_bridge::convert_to_aisdk_tools(
         &scoped_registry,
         sender.clone(),
         agent.name.clone(),
@@ -79,6 +79,30 @@ pub async fn run_subagent(
         cancel_token.clone(),
     )
     .await;
+    let hosted_selection = match crate::config::ConfigLoader::load() {
+        Ok(loaded) => {
+            let ws = &loaded.merged_config.websearch;
+            if ws.enabled.unwrap_or(true) {
+                Some(
+                    crate::aisdk::providers::hosted_search::HostedSearchSelection {
+                        web: ws.native.web_enabled(),
+                        x: ws.native.x_enabled(),
+                    },
+                )
+            } else {
+                None
+            }
+        }
+        Err(_) => Some(crate::aisdk::providers::hosted_search::HostedSearchSelection::DEFAULT),
+    };
+    if let Some(selection) = hosted_selection {
+        if selection.web || selection.x {
+            aisdk_tools.extend(crate::aisdk::providers::hosted_search::tools_for(
+                &session.provider_name,
+                selection,
+            ));
+        }
+    }
 
     let system_prompt = agent
         .instructions
@@ -154,6 +178,19 @@ pub async fn run_subagent(
                     .unwrap_or(1);
                 tool_call_count = tool_call_count.saturating_add(calls);
             }
+            ChunkType::ProviderToolCall(payload) => {
+                tool_call_count = tool_call_count.saturating_add(1);
+                if let Some(sender) = sender.as_ref() {
+                    let (calls, result) =
+                        crate::llm::client::provider_tool_call_ui_events(&payload);
+                    if !calls.is_empty() {
+                        let _ = sender.send(crate::llm::ChunkMessage::ToolCalls(calls));
+                    }
+                    if let Some(result) = result {
+                        let _ = sender.send(crate::llm::ChunkMessage::ToolResult(result));
+                    }
+                }
+            }
             ChunkType::Failed(err) => {
                 crate::emit_log!(
                     "[SUBAGENT] stream_failed session_id={} subagent_type={} duration_ms={} error={}",

src/aisdk/chunk.rs

@@ -4,13 +4,28 @@ pub enum ChunkType {
     Text(String),
     Reasoning(String),
     ToolCall(String),
-    AssistantMessagePhase { phase: Option<MessagePhase> },
-    ResponseCompleted { end_turn: Option<bool> },
+    /// Provider-executed tool lifecycle (hosted search, etc.).
+    ///
+    /// Display / observability only — must never enter the client tool-execute
+    /// loop. Payload JSON:
+    /// `{ "id", "name", "status": "running"|"completed"|"failed", "arguments"?, "output"? }`.
+    ProviderToolCall(String),
+    AssistantMessagePhase {
+        phase: Option<MessagePhase>,
+    },
+    ResponseCompleted {
+        end_turn: Option<bool>,
+    },
     Retry(crate::retry::RetryStatus),
-    StreamRollback { text: String, reasoning: String },
+    StreamRollback {
+        text: String,
+        reasoning: String,
+    },
     Warning(String),
     Metadata(String),
-    End { reason: Option<FinishReason> },
+    End {
+        reason: Option<FinishReason>,
+    },
     RetryableFailure(crate::retry::RetryError),
     Failed(String),
     Incomplete(String),

src/aisdk/providers/anthropic.rs

@@ -111,17 +111,25 @@ impl Provider for Anthropic {
         // immediately followed by tool_result blocks in one user message.
         let user_messages = anthropic_messages(messages);
 
-        let tool_params: Vec<serde_json::Value> = tools
-            .iter()
-            .map(|t| {
-                let schema = serde_json::to_value(&t.input_schema).unwrap_or_default();
-                serde_json::json!({
-                    "name": t.name,
-                    "description": t.description,
-                    "input_schema": schema,
-                })
-            })
-            .collect();
+        let mut tool_params: Vec<serde_json::Value> = Vec::new();
+        let mut has_hosted_search = false;
+        for t in tools {
+            match &t.transport {
+                crate::aisdk::tool::ToolTransport::ProviderNative(value) => {
+                    has_hosted_search = true;
+                    tool_params.push(value.clone());
+                }
+                crate::aisdk::tool::ToolTransport::OpenRouterPlugin(_) => {}
+                crate::aisdk::tool::ToolTransport::ClientFunction => {
+                    let schema = serde_json::to_value(&t.input_schema).unwrap_or_default();
+                    tool_params.push(serde_json::json!({
+                        "name": t.name,
+                        "description": t.description,
+                        "input_schema": schema,
+                    }));
+                }
+            }
+        }
 
         let mut body = serde_json::json!({
             "model": self.model_name,
@@ -156,6 +164,10 @@ impl Provider for Anthropic {
             request_headers.insert("x-api-key", self.api_key.parse().unwrap());
         }
         request_headers.insert("anthropic-version", "2023-06-01".parse().unwrap());
+        if has_hosted_search {
+            // Hosted web_search tool requires the anthropic-beta header.
+            request_headers.insert("anthropic-beta", "web-search-2025-03-05".parse().unwrap());
+        }
 
         let client = reqwest::Client::builder()
             .connect_timeout(std::time::Duration::from_secs(
@@ -228,9 +240,17 @@ fn anthropic_stream_chunk(
             }
             None
         }
-        "content_block_start" => anthropic_tool_call_start(value)
-            .map(ChunkType::ToolCall)
-            .map(Ok),
+        "content_block_start" => {
+            if let Some(payload) = anthropic_hosted_search_start(value) {
+                Some(Ok(ChunkType::ProviderToolCall(payload)))
+            } else if let Some(payload) = anthropic_hosted_search_result(value) {
+                Some(Ok(ChunkType::ProviderToolCall(payload)))
+            } else {
+                anthropic_tool_call_start(value)
+                    .map(ChunkType::ToolCall)
+                    .map(Ok)
+            }
+        }
         "content_block_delta" => anthropic_content_block_delta(value).map(Ok),
         "message_delta" => {
             // Final usage wins for cache_read / cache_creation.
@@ -326,13 +346,79 @@ fn anthropic_message_delta(value: &serde_json::Value) -> Option<ChunkType> {
     }
 }
 
+fn anthropic_hosted_search_start(value: &serde_json::Value) -> Option<String> {
+    let content_block = value.get("content_block")?;
+    if content_block.get("type").and_then(|v| v.as_str()) != Some("server_tool_use") {
+        return None;
+    }
+    let id = content_block
+        .get("id")
+        .and_then(|v| v.as_str())
+        .unwrap_or("hosted_search");
+    let name = content_block
+        .get("name")
+        .and_then(|v| v.as_str())
+        .unwrap_or("web_search");
+    let args = content_block
+        .get("input")
+        .cloned()
+        .unwrap_or_else(|| serde_json::json!({}));
+    Some(
+        serde_json::json!({
+            "id": id,
+            "name": name,
+            "status": "running",
+            "provider_executed": true,
+            "arguments": args,
+        })
+        .to_string(),
+    )
+}
+
+fn anthropic_hosted_search_result(value: &serde_json::Value) -> Option<String> {
+    let content_block = value.get("content_block")?;
+    let block_type = content_block.get("type").and_then(|v| v.as_str())?;
+    let failed = block_type == "web_search_tool_result_error";
+    if block_type != "web_search_tool_result" && !failed {
+        return None;
+    }
+    let id = content_block
+        .get("tool_use_id")
+        .or_else(|| content_block.get("id"))
+        .and_then(|v| v.as_str())
+        .unwrap_or("hosted_search");
+    let output = content_block
+        .get("content")
+        .cloned()
+        .unwrap_or_else(|| content_block.clone());
+    Some(
+        serde_json::json!({
+            "id": id,
+            "name": "web_search",
+            "status": if failed { "failed" } else { "completed" },
+            "provider_executed": true,
+            "output": output,
+        })
+        .to_string(),
+    )
+}
+
 fn anthropic_tool_call_start(value: &serde_json::Value) -> Option<String> {
     let content_block = value.get("content_block")?;
-    if content_block
+    let block_type = content_block
         .get("type")
-        .and_then(|block_type| block_type.as_str())
-        != Some("tool_use")
-    {
+        .and_then(|block_type| block_type.as_str())?;
+
+    // Hosted web_search runs server-side; ignore those content blocks for the
+    // client tool loop (results come back as text / citations).
+    if matches!(
+        block_type,
+        "server_tool_use" | "web_search_tool_result" | "web_search_tool_result_error"
+    ) {
+        return None;
+    }
+
+    if block_type != "tool_use" {
         return None;
     }