Skip to content
Draft
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
169 changes: 167 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,167 @@
# visual_search
SDK for visual search on Images, Videos,
<div align="center">

# 🔍 Visual Search SDK

**AI-powered visual search for images and videos — find similar content at scale.**

[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE)
[![GitHub Stars](https://img.shields.io/github/stars/v-modal/visual_search?style=social)](https://github.com/v-modal/visual_search/stargazers)
[![GitHub Issues](https://img.shields.io/github/issues/v-modal/visual_search)](https://github.com/v-modal/visual_search/issues)

</div>

---

## 📖 Overview

**Visual Search SDK** is a powerful, developer-friendly toolkit that enables semantic visual search across images and videos. Whether you need to find visually similar product images in an e-commerce catalogue, detect duplicate video clips, or power a reverse-image search engine, this SDK provides the building blocks you need.

| Capability | Description |
|---|---|
| 🖼️ **Image Search** | Find visually similar images using deep-learning embeddings |
| 🎬 **Video Search** | Query across video frames and extract scene-level matches |
| ⚡ **Fast Indexing** | Efficiently index millions of assets with vector-based retrieval |
| 🔌 **Easy Integration** | Drop-in SDK for Python projects with a clean, intuitive API |

---

## ✨ Features

- **Cross-modal search** — query by image or video clip and retrieve ranked results from a mixed index
- **Frame-level video search** — automatically sample or seek frames to locate exact scenes
- **Pluggable backends** — swap in any vector store (FAISS, Milvus, Qdrant, …) via a single interface
- **Pre-built models** — ships with state-of-the-art vision encoders (CLIP, ViT, ResNet); bring-your-own-model also supported
- **Batch & streaming ingestion** — ingest assets one-by-one or in large parallel batches
- **REST-ready** — optional HTTP server mode to expose the SDK as a microservice

---

## 🚀 Quick Start

### Installation

```bash
pip install visual-search-sdk
```

Or install directly from source:

```bash
git clone https://github.com/v-modal/visual_search.git
cd visual_search
pip install -e .
```

### Index & Search Images

```python
from visual_search import VisualSearchClient

client = VisualSearchClient()

# Index a folder of images
client.index_images("./my_image_folder")

# Search for similar images
results = client.search_image("query.jpg", top_k=5)

for result in results:
print(f"{result.path} (score: {result.score:.4f})")
```

### Index & Search Videos

```python
from visual_search import VisualSearchClient

client = VisualSearchClient()

# Index a video — frames are sampled automatically
client.index_video("product_demo.mp4")

# Search using a reference frame or image
results = client.search_image("reference_frame.png", top_k=10)

for result in results:
print(f"{result.video_path} @ {result.timestamp:.2f}s (score: {result.score:.4f})")
```

---

## 🗂️ Project Structure

```
visual_search/
├── visual_search/ # Core SDK source
│ ├── client.py # Main VisualSearchClient API
│ ├── encoder.py # Vision model wrappers (CLIP, ViT, …)
│ ├── indexer.py # Asset ingestion & vector indexing
│ ├── searcher.py # Query & ranking logic
│ └── video.py # Video frame extraction utilities
├── examples/ # Runnable example scripts
├── tests/ # Unit & integration tests
├── LICENSE
└── README.md
```

---

## 📚 API Reference

### `VisualSearchClient`

| Method | Description |
|---|---|
| `index_images(path, *, recursive=True)` | Index all images under `path` |
| `index_video(path, *, fps=1)` | Index a video file by sampling frames at `fps` |
| `search_image(query, top_k=10)` | Return the top-k most similar assets for an image query |
| `search_video(query, top_k=10)` | Return the top-k most similar assets for a video query |
| `delete(asset_id)` | Remove an asset from the index |
| `save(directory)` | Persist the index to disk |
| `load(directory)` | Load a previously persisted index |

Full API documentation is available in the [`docs/`](docs/) directory.

---

## ⚙️ Configuration

```python
from visual_search import VisualSearchClient, Config

config = Config(
model="openai/clip-vit-base-patch32", # encoder model
backend="faiss", # vector store backend
device="cuda", # "cpu" | "cuda" | "mps"
video_fps=2, # frames per second for video indexing
batch_size=64, # ingestion batch size
)

client = VisualSearchClient(config=config)
```

---

## 🤝 Contributing

Contributions are welcome! Please follow these steps:

1. Fork the repository
2. Create a feature branch: `git checkout -b feature/amazing-feature`
3. Commit your changes: `git commit -m 'Add amazing feature'`
4. Push to the branch: `git push origin feature/amazing-feature`
5. Open a Pull Request

Please read [CONTRIBUTING.md](CONTRIBUTING.md) for details on our code of conduct and development guidelines.

---

## 📄 License

This project is licensed under the **Apache License 2.0** — see the [LICENSE](LICENSE) file for details.

---

<div align="center">
Made with ❤️ by the Visual Search team
</div>