docs: proposal for toolbox-level image embedding pipeline

[Deeven-Seru]

Summary

Adds a design proposal for toolbox-level image embedding (image ingestion + embeddings + storage adapters) to address #2948.

Details

• New doc:
• Outlines architecture, interfaces, config, rollout, and risks

Closes #2948

[Deeven-Seru]

Design proposal (short)

Goal: first‑class image ingestion + embedding, centralized at Toolbox level,
not per‑connector.

Architecture

• New Toolbox embedding module
• Ingestion: local | HTTP | GCS
• Embedder backend (pluggable): gemini | vertexai | openai | hf | local
• Storage adapters: BigQuery + Postgres/pgvector (MVP), then Elastic/Mongo
• Tools: embed_image, search_image, search_multimodal

Draft API surface

• Embedder.EmbedImage(ctx, input) -> (vector, metadata, error)
• StorageAdapter.Store(ctx, vector, metadata) -> error
• StorageAdapter.Search(ctx, query_vector, filters, top_k) -> (results, error)
• index_images(source, target, model, batch_size)

Config sketch

  embeddings:                                                                  
    backend: gemini|vertexai|openai|hf                                         
    model: <model-name>                                                        
    batch_size: 64                                                             
    image_sources:                                                             
      - type: gcs|local|http                                                   
        path: gs://bucket/path                                                 
  storage:                                                                     
    target: bigquery|postgres                                                  
    index_table: <name>                                                        

Rollout

Phase 1: embedding module + gemini/vertex/hf + BigQuery/pgvector + CLI batch
ingest
Phase 2: OpenAI backend + Elastic/Mongo
Phase 3: optional external vector stores

This preserves Toolbox consistency, avoids connector bloat, and unlocks
image‑RAG workflows with minimal infra.

[gemini-code-assist]

Code Review

This pull request introduces a proposal for a toolbox-level image embedding pipeline, outlining the architecture for pluggable embedding backends and storage adapters. The review feedback highlights the need to adhere to documentation style guides by removing redundant H1 headers, suggests aligning conceptual interfaces with Go idioms through the inclusion of context and error handling, and recommends maintaining naming consistency with existing internal embedding implementations.

[Deeven-Seru]

@duwenxin99 PTAL at the design proposal I will start working on it if its feasible

[Yuan325]

Hi @Deeven-Seru For design proposal, please add that as a comment in the issue instead of adding a docs~ I've copied over your proposal to the issue. Will close this PR for now, Thank you! :)