VectorCore is an embedded, high-performance vector search engine designed to demonstrate advanced C++ optimization techniques, SIMD acceleration, and efficient foreign function interfaces (FFI) with Python.
Designed as an educational yet functional prototype, it addresses the "Two-Language Problem" in high-performance computing (HPC): using Python for its ease of use and ecosystem (NumPy, PyTorch, sentence-transformers) while leveraging C++ for the computationally intensive distance calculations required by vector search.
This project deliberately avoids high-level abstractions in the critical path, opting instead for Data-Oriented Design, flat memory layouts, and processor-specific intrinsics (AVX2 + FMA).
- Project Motivation
- High-Level Architecture
- Public API
- Detailed Technical Implementation
- Folder Structure
- Build System & Environment
- Installation & Usage
- Roadmap
- License
Modern AI applications rely heavily on vector embeddings (dense floating-point arrays representing text or images). Finding the "nearest" vector to a query is an
In pure Python this is prohibitively slow. While libraries like Faiss and hnswlib exist, understanding how to build one requires mastering several distinct domains:
- Hardware-aware C++: cache lines and SIMD registers.
- Compiler interactions: how flags like
/arch:AVX2or-mfmachange code generation. - Cross-language bindings: passing memory pointers safely between Python and C++.
VectorCore is a clean-sheet implementation of these concepts.
The system operates as a hybrid application: two languages, shared memory.
---
config:
layout: dagre
---
flowchart TB
subgraph PY["Python Land"]
User["User / Python Script"]
Numpy["NumPy Memory Allocator"]
end
subgraph CPP["C++ Land"]
Interface["pybind11 Module: vectorcore"]
BF["BruteForceIndex"]
HNSW["HnswIndex"]
Dist["AVX2 Distance Kernels"]
end
Results["Top-K ids + scores → Python"]
User -- "NumPy float32 array" --> Interface
Numpy -. "zero-copy buffer read" .-> Interface
Interface -- "const float*" --> BF
Interface -- "const float*" --> HNSW
BF --> Dist
HNSW --> Dist
Dist -- "results" --> Results
- Storage Layer: a strictly typed, contiguous, 32-byte-aligned memory block managed by C++.
- Compute Layer: AVX2-optimized distance kernels that process 8 dimensions per instruction.
- Interface Layer: a
pybind11module exposing the C++ indexes as native Python objects.
A full architecture diagram is available in VectraX_Architecture.pdf.
The compiled module is imported as vectorcore. It exposes two indexes and a Metric enum.
| Class | Status | Search complexity | Use |
|---|---|---|---|
BruteForceIndex |
✅ Complete, exact | Ground-truth baseline, exact recall | |
HnswIndex |
✅ Multi-layer ANN | approximate, ~O(log N) | Graph-based search; tunable recall/speed via ef_search
|
On SIFT1M (1M × 128, L2), HnswIndex reaches recall@10 = 0.966 at ~6,000 QPS (M=16, efConstruction=200, efSearch=64) — about 97× faster than the exact BruteForceIndex (~62 QPS) at the same recall target.
Metrics (passed as a string): "l2" (alias "l2_squared"), "ip" (alias "inner_product"), "cosine" (alias "cos").
For l2, smaller scores are closer. For ip/cosine, larger scores are closer. Cosine is implemented as inner product over L2-normalized vectors (normalization is applied automatically on add and search).
BruteForceIndex(dim: int, metric: str = "l2")
.dim -> int
.size -> int
.add(x: np.ndarray[float32, (n, dim)], ids: np.ndarray[uint64, (n,)] | None = None)
.search(q: np.ndarray[float32, (dim,) | (m, dim)], k: int) -> (ids, scores)
HnswIndex(dim: int, M: int = 16, metric: str = "l2",
ef_construction: int = 200, seed: int = 100)
.ef_search -> int # read/write; higher = better recall, slower
.add(x, ids=None) # builds the graph incrementally
.search(q: float32[dim], k: int) -> (ids, scores)dtype matters. Inputs must be float32 and C-contiguous;
idsmust be uint64. Python's nativefloatis C++double(64-bit), so always.astype(np.float32). The bridge rejects mismatches rather than silently copying.
Files: include/vectorcore/bruteforce_index.h, include/vectorcore/aligned_allocator.h
A naive implementation stores vectors as a "vector of vectors":
// BAD: cache thrashing
std::vector<std::vector<float>> data;Each inner vector is heap-allocated separately. Iterating involves pointer chasing and cache misses.
VectorCore uses a flat layout in a single contiguous, 32-byte-aligned buffer:
// GOOD: spatial locality
std::vector<float, AlignedAllocator<float, 32>> embeddings_;
// Access vector i at: embeddings_.data() + (i * dim_)A single 64-byte cache line fetch brings in 16 consecutive floats; the prefetcher streams the next vector while the SIMD kernel works on the current one. The bottleneck shifts from memory latency to memory bandwidth (a much higher ceiling).
File: src/distance.cpp
The hot kernels use Intel AVX2 to process 8 floats per instruction:
_mm256_loadu_ps— load 8 floats (unaligned-safe)._mm256_sub_ps— subtract 8 floats in parallel (L2)._mm256_fmadd_ps— fused multiply-add:(diff * diff) + accumulatorin one instruction.
A horizontal sum reduces the 8 lanes, and a scalar tail loop handles dimensions not divisible by 8. When AVX2 is unavailable, a 4×-unrolled scalar fallback is compiled in instead (#if defined(__AVX2__)).
File: src/distance.cpp
| Metric enum | Kernel | Semantics |
|---|---|---|
L2_SQUARED |
l2_squared |
squared Euclidean; smaller = closer |
INNER_PRODUCT |
inner_product |
dot product; larger = closer |
COSINE |
inner_product over normalized vectors |
cosine similarity in [-1, 1]; larger = closer |
Cosine normalizes stored vectors at add time and the query at search time (l2_normalize_inplace), so cos(a, b) == <â, b̂> and the same fast inner-product kernel is reused.
File: src/bruteforce_index.cpp (search)
To find the top-$k$ nearest neighbors over all stored vectors:
- Score every vector against the query.
- Maintain a size-k max-heap keyed by "badness" (L2:
badness = distance; IP/cosine:badness = -similarity, so "larger badness = worse" in all cases). - The heap top is the worst kept candidate, so replacing it is
O(log k). Total work isO(N log k)— no full sort of allN.
Results are then ordered best-first and the user-facing score is recovered from the badness. If k > N, the tail is padded with UINT64_MAX / +inf sentinels.
File: src/bruteforce_index.cpp
The brute-force scan is parallelized with OpenMP: each thread keeps a thread-local size-k heap over a slice of the data (#pragma omp for nowait), then the locals are merged into the global top-k inside a #pragma omp critical section. This avoids contention on a shared heap in the hot loop.
File: src/pybind_module.cpp
store.add(np_array) does not copy through an intermediate std::vector. Instead:
py::buffer_inforequests the raw pointer + shape/strides of the NumPy array.- Safety checks:
ndim(1 or 2),shape[-1] == dim,format == float32, and C-contiguous strides. (itemsizealone is insufficient —int32is also 4 bytes.) - The validated
const float*is passed straight into the C++ index.
0 bytes are copied at the API boundary. The index then copies the data exactly once into its internal flat storage (the "zero-copy" guarantee applies to the bridge, not to persistence).
Vectra-X/
├── include/vectorcore/
│ ├── aligned_allocator.h # 32-byte aligned allocator for std::vector
│ ├── distance.h # Metric enum + distance kernel declarations
│ ├── bruteforce_index.h # Exact index (flat storage)
│ └── hnsw_index.h # Graph index (partial)
├── src/
│ ├── distance.cpp # AVX2 + scalar L2 / inner-product / normalize
│ ├── bruteforce_index.cpp # Exact kNN + OpenMP top-k
│ ├── hnsw_index.cpp # Graph build + greedy search (prototype)
│ └── pybind_module.cpp # pybind11 bindings (module: vectorcore)
├── tests/
│ └── test_smoke.cpp # Minimal C++ smoke test
├── setup.py # pip build (Pybind11Extension, flag injection)
├── CMakeLists.txt # Standalone C++ build + smoke test
├── pyproject.toml # Build-system metadata
└── VectraX_Architecture.pdf # Architecture diagram
setup.py injects optimal flags per platform:
- Windows (MSVC):
/O2 /arch:AVX2 /openmp(MSVC has no-O3). - Linux/macOS (GCC/Clang):
-O3 -mavx2 -mfma -fopenmp.
The version string is injected via the VERSION_INFO macro (defined identically by setup.py and CMakeLists.txt) and stringified in pybind_module.cpp, so vectorcore.__version__ works on both build paths.
C++17 (cxx_std=17).
- OS: Windows 10/11, Linux, or macOS.
- Compiler: Windows — Visual Studio Build Tools 2022 ("Desktop development with C++"); Linux —
build-essential(GCC 7+). - Python: 3.8+, with
numpy.
git clone <repo-url>
cd Vectra-X
pip install .Windows: run from an environment where cl.exe is on PATH, or let pip locate the Build Tools.
import numpy as np
import vectorcore
print(vectorcore.__version__)
# Exact, cosine-similarity index over 128-dim vectors.
index = vectorcore.BruteForceIndex(dim=128, metric="cosine")
# Data MUST be float32 and C-contiguous.
x = np.random.rand(1000, 128).astype(np.float32)
ids = np.arange(1000, dtype=np.uint64)
index.add(x, ids) # ids optional; defaults to 0..n-1
print("size:", index.size)
# Single query (dim,) or batched (m, dim).
q = x[0]
out_ids, out_scores = index.search(q, k=5)
for vid, score in zip(out_ids, out_scores):
print(f"id={vid} score={score:.5f}") # nearest is the query itselfEvery index serializes to a single binary file and reloads with identical search behavior (same-architecture portable):
index.save("my_index.bin")
index = vectorcore.HnswIndex.load("my_index.bin") # or BruteForceIndex / PQIndexexamples/rag_demo.py shows the end-to-end retrieval
pipeline: text → sentence-transformers embeddings → HnswIndex (cosine) →
top-k semantically relevant documents. Queries use different words than the
corpus, so only a semantic (not keyword) match succeeds.
pip install .
pip install -r examples/requirements.txt # sentence-transformers (CPU torch ok)
python examples/rag_demo.pyAn interactive dashboard animates the HNSW search path — greedy descent through the upper layers then the layer-0 beam search — over a 2D projection of the real graph, with live latency/recall metrics. See viz/README.md:
python viz/export_graph.py # trace a query -> viz/public/graph_data.json
cd viz && npm install && npm run devcmake -S . -B build_cmake -DCMAKE_BUILD_TYPE=Release
cmake --build build_cmake
ctest --test-dir build_cmake # runs the GoogleTest suite + smoke testVectorCore is being built out in measured stages (each gated by recall@k / QPS on a standard dataset).
- Test + benchmark infrastructure — GoogleTest suite + reusable Python harness (recall@k / QPS), validated on SIFT1M.
- Real HNSW — multi-layer probabilistic graph,
efConstruction/efSearchbeam search, RNG heuristic neighbor pruning. (recall@10 = 0.966 @ ~97× brute force on SIFT1M.) - Semantic search demo —
sentence-transformers→HnswIndex(cosine) RAG retrieval (examples/rag_demo.py); batched(m, dim)query support. - Product Quantization (PQ) — per-subspace K-Means (k-means++) + uint8 codes + asymmetric distance computation (PQIndex). SIFT1M tradeoff: m=64 → recall@10 0.90 @ 8×, m=32 → 0.72 @ 16×, m=16 → 0.54 @ 32×. Sweep:
python -m benchmark.pq_sweep. (Re-ranking / OPQ to push recall at high compression is a future extension.) - Persistence — binary
save(path)/load(path)for all three indexes (io.h); verified byte-identical search results across a fresh process. - Visualization — React + D3 dashboard (viz/) animating the real HNSW search path (greedy descent + layer-0 beam search) over a 2D PCA projection, with live latency / recall / nodes-visited metrics and playback controls.
All roadmap stages complete. Future extensions: PQ re-ranking / OPQ for higher recall at high compression; parallel HNSW graph construction (currently single-threaded, ~21 min for SIFT1M's 1M inserts); IVF / HNSW+PQ for billion-scale.
Open-source under the MIT License.