Skip to content

kailas-cloud/vecdex

BranchesTags

Repository files navigation

vecdex

Lightweight vector search engine on top of Valkey & Redis

Go Report Card Codacy Grade Codacy Coverage CI License

Why vecdex?

Most vector databases are heavy, cloud-locked, or expensive. vecdex takes a different approach:

  • Zero new infrastructure — runs on Valkey or Redis you already have
  • Automatic embeddings — send text, get vectors (Nebius AI, OpenAI-compatible providers)
  • Three search modes — hybrid (RRF), semantic (KNN), keyword (BM25) via one endpoint
  • Swap the backend — Valkey or Redis 8, same API, same results
  • Budget controls — daily/monthly token limits with automatic tracking
  • 300+ E2E tests — battle-tested across both backends

Examples

Semantic code search

Define a struct, tag the fields, search with one line:

package main

import (
    "context"
    "fmt"

    vecdex "github.com/kailas-cloud/vecdex/pkg/sdk"
)

type CodeChunk struct {
    ID       string `vecdex:"id"`
    Content  string `vecdex:"content,content"`
    Language string `vecdex:"language,tag"`
    Repo     string `vecdex:"repo,tag"`
}

func main() {
    ctx := context.Background()
    client, _ := vecdex.New(ctx,
        vecdex.WithValkey("localhost:6379", ""),
        vecdex.WithEmbedder(myEmbedder),       // any OpenAI-compatible provider
    )
    defer client.Close()

    idx, _ := vecdex.NewIndex[CodeChunk](client, "code-chunks")
    _ = idx.Ensure(ctx)

    // Index some code
    _ = idx.UpsertBatch(ctx, []CodeChunk{
        {ID: "1", Content: "CreateCollection validates the name and builds an FT index", Language: "go", Repo: "vecdex"},
        {ID: "2", Content: "SearchBuilder chains Near, Km, Where, Limit into a query", Language: "go", Repo: "vecdex"},
        {ID: "3", Content: "BudgetTracker enforces daily and monthly token limits", Language: "go", Repo: "vecdex"},
    })

    // Semantic search — one line
    hits, _ := idx.Search().
        Query("how does collection creation work").
        Mode(vecdex.ModeSemantic).
        Where("language", "go").
        Limit(5).
        Do(ctx)

    for _, h := range hits {
        fmt.Printf("%.2f  %s\n", h.Score, h.Item.Content)
    }
}

Find places nearby

Geo collections need no embedder — distance is computed from coordinates:

type Cafe struct {
    ID   string  `vecdex:"id"`
    Name string  `vecdex:"name,content"`
    City string  `vecdex:"city,tag"`
    Lat  float64 `vecdex:"lat,geo_lat"`
    Lon  float64 `vecdex:"lon,geo_lon"`
}

idx, _ := vecdex.NewIndex[Cafe](client, "cafes")
_ = idx.Ensure(ctx)

_ = idx.UpsertBatch(ctx, []Cafe{
    {ID: "1", Name: "Skuratov",  City: "moscow", Lat: 55.7558, Lon: 37.6173},
    {ID: "2", Name: "Surf Coffee", City: "moscow", Lat: 55.7601, Lon: 37.6186},
})

// Find cafes within 2 km of Red Square
hits, _ := idx.Search().
    Near(55.7539, 37.6208).
    Km(2).
    Where("city", "moscow").
    Limit(10).
    Do(ctx)

for _, h := range hits {
    fmt.Printf("%s — %.0f m away\n", h.Item.Name, h.Distance)
}

Low-level API

For full control without struct tags:

client, _ := vecdex.New(ctx,
    vecdex.WithRedis("localhost:6379", ""),
    vecdex.WithEmbedder(myEmbedder),
)
defer client.Close()

// Create collection with filterable fields
client.Collections().Create(ctx, "articles",
    vecdex.WithField("author", vecdex.FieldTag),
    vecdex.WithField("year", vecdex.FieldNumeric),
)

// Upsert a document — embedding happens automatically
client.Documents("articles").Upsert(ctx, vecdex.Document{
    ID:       "article-1",
    Content:  "Vector search with HNSW indexes in Redis",
    Tags:     map[string]string{"author": "alice"},
    Numerics: map[string]float64{"year": 2025},
})

// Hybrid search (vector KNN + BM25 fused via RRF)
resp, _ := client.Search("articles").Query(ctx, "HNSW performance", &vecdex.SearchOptions{
    Mode:  vecdex.ModeHybrid,
    Limit: 10,
})

for _, r := range resp.Results {
    fmt.Printf("%.2f  %s\n", r.Score, r.Content)
}

How it compares

vecdex Pinecone Qdrant Weaviate pgvector
Self-hosted Yes No Yes Yes Yes
Managed cloud No Yes Yes Yes Yes
Underlying storage Valkey / Redis Proprietary Custom Custom PostgreSQL
Auto-embedding Yes No No Yes No
Hybrid search (RRF) Yes Yes Yes Yes No
Token budget tracking Yes No No No No
Go SDK with generics Yes No Yes No No
Setup complexity Low None Medium High Low
License Apache 2.0 Proprietary Apache 2.0 BSD-3 PostgreSQL

Search modes

Mode How it works Embedding cost Backend support
hybrid (default) Vector KNN + BM25 fused via Reciprocal Rank Fusion 1 call Redis 8
semantic Pure cosine-similarity KNN 1 call Redis 8, Valkey 9
keyword BM25 full-text search 0 calls Redis 8
geo ECEF-based geographic proximity 0 calls Redis 8, Valkey 9

Backend support

Backend Status Notes
Valkey 9+ (valkey-search) Supported Semantic + geo search. Keyword/hybrid when valkey-search adds BM25
Redis 8+ (Redis Search) Supported Full hybrid search (semantic + keyword + RRF + geo)
AWS ElastiCache Planned
PostgreSQL + pgvector Planned

Key features

Feature Description
Hybrid search Reciprocal Rank Fusion combining vector KNN and BM25 keyword search
Semantic search Pure cosine-similarity KNN over HNSW vectors
Keyword search BM25 full-text search — zero embedding tokens consumed
Geo search ECEF-based geographic proximity with radius filtering
Structured filters must / should / must_not with tag match and numeric range operators
Auto-embedding Send text, get vectors via any OpenAI-compatible provider
Typed Go SDK Schema-first generics with TypedIndex[T] and fluent search builder
Batch operations Upsert/delete up to 100 items per call with per-item status
Token budget Daily/monthly limits with warn or reject policies
Cursor pagination Stable, opaque-cursor pagination for collections and documents
Embedding cache SHA256-keyed cache in Valkey — identical content is never re-embedded
Prometheus metrics Request latency, embedding tokens, budget, cache hit/miss

Architecture

┌─────────────────────────────────────────────────────┐
│              Go SDK  (pkg/sdk)                       │
│  TypedIndex[T] · SearchBuilder · Fluent API          │
├─────────────────────────────────────────────────────┤
│              HTTP API  (Chi)                          │
│  Auth · Metrics · Wide-event logging · Recovery       │
├─────────────────────────────────────────────────────┤
│                 Use Cases                             │
│  Collection · Document · Search · Batch · Embedding   │
├─────────────────────────────────────────────────────┤
│                Repositories                           │
│  Consumer interfaces (ISP) over Store facade          │
├──────────────────────┬──────────────────────────────┤
│    Redis backend     │     Valkey backend            │
│  (rueidis, RESP2)    │   (rueidis, RESP2)            │
└──────────────────────┴──────────────────────────────┘
          │                        │
    Redis 8 + Search         Valkey 9 + valkey-search

Embedding pipeline (decorator chain):

OpenAIProvider → CachedProvider → InstrumentedProvider → StringVectorizer
       ↓               ↓                  ↓
  Nebius API     SHA256 cache       Prometheus +
                  in Valkey        BudgetTracker

API reference

Go SDK

go get github.com/kailas-cloud/vecdex
Type Description
vecdex.Client Connection to Valkey/Redis, entry point for all operations
vecdex.TypedIndex[T] Generic index with schema inferred from struct tags
vecdex.SearchBuilder[T] Fluent search: .Query(), .Near(), .Where(), .Limit(), .Do()
vecdex.Hit[T] Search result with .Item, .Score, .Distance
vecdex.Document Untyped document for low-level API
vecdex.Embedder Interface for text-to-vector providers

Struct tag format: vecdex:"name,modifier" — modifiers: id, content, tag, numeric, geo_lat, geo_lon.

REST API

Full OpenAPI 3.0 specification: api/openapi.yaml

Method Path Description
POST /collections Create collection
GET /collections List collections (cursor pagination)
GET /collections/{name} Get collection details
DELETE /collections/{name} Delete collection
PUT /collections/{name}/documents/{id} Upsert document (auto-embeds)
GET /collections/{name}/documents/{id} Get document
PATCH /collections/{name}/documents/{id} Partial update (no re-vectorization)
DELETE /collections/{name}/documents/{id} Delete document
GET /collections/{name}/documents List documents (cursor pagination)
POST /collections/{name}/documents/search Search documents
POST /collections/{name}/documents/batch-upsert Batch upsert (up to 100)
POST /collections/{name}/documents/batch-delete Batch delete (up to 100)
GET /usage Embedding usage & budget info
GET /health Health check
GET /metrics Prometheus metrics

Quick start

Docker Compose (recommended)

git clone https://github.com/kailas-cloud/vecdex.git
cd vecdex
cp .env.example .env
# Edit .env — set NEBIUS_API_KEY and optionally VECDEX_API_KEY

docker compose up vecdex
# API is running at http://localhost:8080

From source

# Prerequisites: Go 1.25+, just
git clone https://github.com/kailas-cloud/vecdex.git
cd vecdex

just build
# Binary at build/vecdex

# Start Valkey locally
just valkey-up

# Run the server
ENV=local ./build/vecdex

Configuration

vecdex uses YAML config files from config/ selected by the ENV environment variable (default: local). Supports ${VAR_NAME} interpolation from environment.

Variable Description Default
ENV Config file to load (local, dev, docker, prod) local
CACHE_ADDR Valkey/Redis address localhost:6379
DB_PASSWORD Database password —
HTTP_PORT HTTP server port 8080
NEBIUS_API_KEY Nebius AI embedding API key —
VECDEX_API_KEY API authentication key —

Testing

just test-unit              # Unit tests (no Valkey needed)
just test-pytest-valkey     # E2E — Valkey backend (300+ pytest tests)
just test-pytest-redis      # E2E — Redis backend
just test-pytest            # E2E — both backends sequentially
just pre-commit             # build + lint + unit tests

The pytest E2E suite runs in Docker Compose with a mock embedding server — no API keys required for CI.

Roadmap

  • vecdex-cli — command-line client
  • Claude Code agent skills integration
  • Performance & latency benchmarks
  • Demo use cases (RAG, code search, etc.)
  • Valkey full-text search (when valkey-search adds BM25)
  • AWS ElastiCache backend
  • PostgreSQL + pgvector backend
  • Multi-tenancy / namespaces
  • Reranking support
  • Additional embedding providers
  • Admin UI
  • Helm chart for Kubernetes

License

Apache License 2.0

Contributing

Contributions are welcome! Please open an issue first to discuss what you'd like to change.

just build          # Build + fmt + vet
just lint           # Run linter
just test-unit      # Fast unit tests
just pre-commit     # Full pre-commit check

About

Lightweight vector search engine on top of Valkey & Redis

Topics

go redis ai pytest knn embedding bm25 rag vector-search valkey

Resources

Readme

License

Apache-2.0 license
Activity
Custom properties

Stars

2 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases 8

v1.0.7 Latest
Feb 25, 2026
+ 7 releases

Packages

 
 
 

Contributors

Languages