Skip to content

Design: provider-native web search integration #37

Description

@zjshen14

Background

Follows from #27 (tool set gaps). Web search is the remaining high-value tool but cannot be implemented as a provider-agnostic Tool in ToolRegistry — every provider executes search server-side using their own declaration format. This issue captures the design and tracks implementation.

How each provider does it

Provider Native Search? Declaration Models Caveats
Anthropic { type: "web_search_20250305" } in tools[] Claude 3.5+ Server-side, no extra key
Gemini { googleSearch: {} } in tools[] Gemini 1.5/2.x+ May require billing enabled
OpenAI ⚠️ partial Responses API: { type: "web_search" }; Chat Completions: dedicated search models only (gpt-5-search-api) gpt-4.1, gpt-5-search-api Standard gpt-4o/o1/o3/o4 via Chat Completions get no native search
Kimi { type: "builtin_function", function: { name: "$web_search" } } in tools[] kimi-k2.6+ $0.005/call; thinking mode must be disabled
Qwen enable_search: true via extra_body, or Responses API { type: "web_search" } qwen3-max, qwen3.5-* 1000 free calls/day on DashScope
Grok (xAI) { type: "web_search" } in Responses API grok-4.3+ Also has x_search for X/Twitter; Chat Completions Live Search deprecated
Mistral ⚠️ partial { type: "web_search" } in Agents/Conversations API mistral-large-latest+ Not available in standard /v1/chat/completions
DeepSeek None UI-only toggle; API has no built-in search
Gemma None Open weights / local — no server to execute it

Key design constraints

  1. Not a ToolRegistry tool. Provider-native search is server-side — the provider resolves it before the response reaches the client. There is no function_call for the executor to handle. It must be injected at the provider client level, not through ToolRegistry.

  2. Provider clients own it. Each client (gemini.ts, anthropic.ts, openai.ts, future kimi.ts etc.) would conditionally append the provider-specific search declaration to the tools[] sent to the API, gated by a supportsNativeSearch(model) check — similar to the existing hasNativeThinking(model) in factory.ts.

  3. Three providers have no native search (OpenAI standard models, DeepSeek, Gemma). For those, a client-side fallback using a third-party search API (Brave, Tavily) + ToolRegistry is the only option if search is needed.

  4. Gemma is a permanent exception. Open-weights models running locally (Ollama/vLLM) will never have server-side search. A client-side web_search tool is the only path.

  5. Plan mode compatibility. Native search is read-only and should be available in plan mode. Since it bypasses ToolRegistry, the PLAN_MODE_TOOLS filter in core.ts doesn't apply — it needs to be conditionally excluded when readOnly is set on ExecutorDeps, or the provider clients need to accept a readOnly flag.

Proposed implementation

Phase 1 — Anthropic + Gemini (highest user overlap)

  • Add supportsNativeSearch(model): boolean to factory.ts
  • GeminiClient.stream(): append { googleSearch: {} } to tools when supported
  • AnthropicClient.stream(): append { type: "web_search_20250305", ... } to tools when supported
  • Handle search result parts in the streaming response (grounding metadata for Gemini, search result blocks for Anthropic)

Phase 2 — Kimi, Qwen, Grok

  • Add provider clients for these models
  • Inject their respective search tool declarations

Phase 3 — Fallback for unsupported providers

  • Implement a client-side web_search tool in src/tools/web/search.ts
  • Opt-in via OPENCLI_SEARCH_API_KEY + OPENCLI_SEARCH_PROVIDER (brave | tavily)
  • Used for OpenAI standard models, DeepSeek, Gemma, and any provider without native search

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions