Add int8 / byte-vector similarity support

UPGRADING.md

@@ -1,3 +1,16 @@
+# Upgrading from 4.0.x to 4.1.x
+
+## On-disk format version 7
+- Version 7 of the on-disk graph format adds one feature, inline int8 vectors (`FeatureId.INLINE_BYTE_VECTORS`):
+  each node's byte vector stored inline, one byte per dimension. Apart from that feature, a version 7 file has
+  the version 6 layout.
+- JVector 4.0.x reads formats up to 6 and rejects version 7 files. `OnDiskGraphIndex.CURRENT_VERSION`,
+  the default for `OnDiskGraphIndexWriter.Builder` and for `CompressedVectors.write(out)`, is now 7.
+- To keep a rollback to 4.0.x possible, pass the version explicitly until every reader runs 4.1:
+  `withVersion(6)` on graph writer builders and `write(out, 6)` for compressors and compressed vectors.
+  A version 6 writer rejects `INLINE_BYTE_VECTORS`.
+- `OnDiskGraphIndexCompactor` always writes `CURRENT_VERSION`; it has no option to write an older version.
+
 # Upgrading from 3.0.x to 4.0.x
 
 ## New features

docs/release notes/4.1.0/709.feature.md

@@ -0,0 +1,53 @@
+### Native Int8 (Byte) Vector Support
+
+**Description**
+JVector can now build, store and search signed int8 vectors natively, without converting them to float32.
+This is useful when an embedding model already emits int8 vectors, or when vectors have been scaled to
+int8 up front: they take 4x less space on disk and in memory than float32, and all scoring is done
+byte-by-byte.
+
+* `RandomAccessByteVectorValues` (and `ListRandomAccessByteVectorValues`) provide random access to
+  `ByteSequence` vectors. It and `RandomAccessVectorValues` now share a common `VectorValues<V>`
+  interface, which also provides `getVectorInto(node, destination, offset)` for reading a
+  vector into a caller-owned buffer.
+* `ByteVectorSimilarityFunction` scores two int8 vectors: `EUCLIDEAN` is `1 / (1 + squaredL2)`,
+  `DOT_PRODUCT` is `dot < 0 ? 1 / (1 - dot) : dot + 1`, and `COSINE` is `(1 + cosine) / 2`. The scores are
+  not normalised by dimension, so distinct integer distances stay distinct at high dimensions.
+  `DOT_PRODUCT` scores are positive but not bounded above by 1.
+* `GraphIndexBuilder.builder(RandomAccessByteVectorValues, ByteVectorSimilarityFunction, M)` builds a graph
+  from int8 vectors, and `GraphIndexBuilder.build(VectorValues<?>)` accepts either vector type.
+* The `InlineByteVectors` on-disk feature (`FeatureId.INLINE_BYTE_VECTORS`) stores each vector as `dimension`
+  raw bytes. `OnDiskGraphIndex.View.byteVectorRerankerFor(...)` scores candidates directly from disk, and
+  `getByteVectorInto(...)` reads a vector into a reusable buffer.
+* Int8 distance kernels (squared L2, dot product, cosine) are available in the default, Panama and native
+  SIMD implementations, and accumulate in integers so all execution paths return identical results.
+* `BenchYAML` and `AutoBenchYAML` support int8 datasets: mark a dataset with `is_int8: true` in
+  `dataset-metadata.yml`. A runnable example is in `Int8Example` (`-Dtutorial=int8`).
+
+**How to Enable**
+No configuration is required; the float32 APIs are unchanged. To use int8 vectors, build with the byte-vector
+builder and write the index with the `InlineByteVectors` feature:
+
+```java
+RandomAccessByteVectorValues vectors = new ListRandomAccessByteVectorValues(byteVectors, dimension);
+try (var builder = GraphIndexBuilder.builder(vectors, ByteVectorSimilarityFunction.EUCLIDEAN, 32).build()) {
+    var graph = builder.build(vectors);
+    try (var writer = GraphIndexWriter.getBuilderFor(GraphIndexWriterTypes.RANDOM_ACCESS_PARALLEL, graph, path)
+            .with(new InlineByteVectors(dimension)).build()) {
+        writer.write(Map.of(FeatureId.INLINE_BYTE_VECTORS, n -> new InlineByteVectors.State(vectors.getVector(n))));
+    }
+}
+```
+
+**Notes**
+* The new interfaces and classes are marked `@Experimental` and the API may change in later releases.
+* Only signed int8 (-128..127) is supported. Unsigned uint8 data, such as conventional SIFT1B/BIGANN `.bvecs`
+  files, must be converted first; it is not detected, and reading it as signed silently changes distances.
+* Quantization and compression (PQ, BQ, NVQ, fused graphs) are not supported for int8 vectors. Traversal
+  already scores exactly, so a separate reranking step is not needed. `rerankK` still controls how much of the
+  graph is searched.
+* Zero vectors are not supported with `COSINE` (the result is undefined); do not index them.
+* Existing float32 callers keep working: `RandomAccessVectorValues.getVector` keeps its `VectorFloat<?>`
+  signature and `GraphIndexBuilder.build(RandomAccessVectorValues)` is retained as a forwarding overload.
+* Int8 indexes require on-disk format version 7 or later. JVector 4.0.x reads formats up to 6 and rejects
+  version 7 files; a version 6 writer rejects `INLINE_BYTE_VECTORS`.

jvector-base/src/main/java/io/github/jbellis/jvector/graph/GraphIndexBuilder.java

@@ -32,7 +32,9 @@
 import io.github.jbellis.jvector.quantization.PQVectors;
 import io.github.jbellis.jvector.quantization.ProductQuantization;
 import io.github.jbellis.jvector.util.*;
+import io.github.jbellis.jvector.vector.ByteVectorSimilarityFunction;
 import io.github.jbellis.jvector.vector.VectorSimilarityFunction;
+import io.github.jbellis.jvector.vector.types.ByteSequence;
 import io.github.jbellis.jvector.vector.types.VectorFloat;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
@@ -114,6 +116,11 @@ private static BuildScoreProvider getBuildScoreProvider(RandomAccessVectorValues
         }
     }
 
+    private static BuildScoreProvider getBuildScoreProvider(RandomAccessByteVectorValues vectorValues, ByteVectorSimilarityFunction similarityFunction) {
+        // Byte vectors do not support PQ/BQ build-time compression; always use the direct provider.
+        return BuildScoreProvider.byteVectorScoreProvider(vectorValues, similarityFunction);
+    }
+
     /**
      * Reads all the vectors from vector values, builds a graph connecting them by their dense
      * ordinals, using the given hyperparameter settings, and returns the resulting graph.
@@ -407,6 +414,31 @@ public static Builder builder(RandomAccessVectorValues vectorValues, VectorSimil
         return builder(vectorValues, similarityFunction, List.of(M));
     }
 
+    /**
+     * Entry point for the fluent builder for int8 byte vectors, resolving the {@link BuildScoreProvider}
+     * from raw byte vectors via {@link BuildScoreProvider#byteVectorScoreProvider}.
+     *
+     * @param vectorValues       the int8 byte vectors whose relations are represented by the graph
+     * @param similarityFunction the byte-vector similarity function to score vectors with
+     * @param maxDegrees         the maximum number of connections a node can have in each layer; if fewer entries
+     *                           are specified than the number of layers, the last entry is used for all remaining layers.
+     */
+    public static Builder builder(RandomAccessByteVectorValues vectorValues, ByteVectorSimilarityFunction similarityFunction, List<Integer> maxDegrees) {
+        return new Builder(getBuildScoreProvider(vectorValues, similarityFunction), vectorValues.dimension(), maxDegrees);
+    }
+
+    /**
+     * Entry point for the fluent builder for int8 byte vectors, for the common case of a single
+     * (non-hierarchical) max degree.
+     *
+     * @param vectorValues       the int8 byte vectors whose relations are represented by the graph
+     * @param similarityFunction the byte-vector similarity function to score vectors with
+     * @param M                  the maximum number of connections a node can have
+     */
+    public static Builder builder(RandomAccessByteVectorValues vectorValues, ByteVectorSimilarityFunction similarityFunction, int M) {
+        return builder(vectorValues, similarityFunction, List.of(M));
+    }
+
     /**
      * Entry point for the fluent builder, building from an existing {@link MutableGraphIndex} (e.g. one just
      * loaded from disk) rather than constructing a fresh {@link OnHeapGraphIndex}. {@code addHierarchy} is not
@@ -695,15 +727,42 @@ public static GraphIndexBuilder rescore(GraphIndexBuilder other, BuildScoreProvi
         return newBuilder;
     }
 
+    /**
+     * Retained for binary compatibility with callers compiled against the float-only signature.
+     * Equivalent to {@link #build(VectorValues)}.
+     */
     public ImmutableGraphIndex build(RandomAccessVectorValues ravv) {
-        var vv = ravv.threadLocalSupplier();
+        return build((VectorValues<?>) ravv);
+    }
+
+
+    /**
+     * Builds the graph from any {@link VectorValues} source — works for both
+     * {@link RandomAccessVectorValues} (float32) and {@link RandomAccessByteVectorValues} (int8).
+     * The score provider supplied at construction time determines how vectors are compared.
+     */
+    public ImmutableGraphIndex build(VectorValues<?> ravv) {
         int size = ravv.size();
 
-        simdExecutor.submit(() -> {
-            IntStream.range(0, size).parallel().forEach(node -> {
-                addGraphNode(node, vv.get().getVector(node));
-            });
-        }).join();
+        if (ravv instanceof RandomAccessVectorValues) {
+            // Float sources may share (borrowed) vectors; read each through a per-thread view.
+            var vv = ((RandomAccessVectorValues) ravv).threadLocalSupplier();
+            simdExecutor.submit(() -> {
+                IntStream.range(0, size).parallel().forEach(node -> {
+                    addGraphNode(node, vv.get().getVector(node));
+                });
+            }).join();
+        } else if (ravv instanceof RandomAccessByteVectorValues) {
+            // Same for int8 sources: read the supplied vectors rather than assuming the score provider's match.
+            var vv = ((RandomAccessByteVectorValues) ravv).threadLocalSupplier();
+            simdExecutor.submit(() -> {
+                IntStream.range(0, size).parallel().forEach(node -> {
+                    addGraphNode(node, vv.get().getVector(node));
+                });
+            }).join();
+        } else {
+            throw new IllegalArgumentException("Unsupported VectorValues type: " + ravv.getClass().getName());
+        }
 
         cleanup();
         return graph;
@@ -852,6 +911,19 @@ public long addGraphNode(int node, VectorFloat<?> vector) {
         return addGraphNode(node, ssp);
     }
 
+    /**
+     * Inserts a node with the given int8 byte vector into the graph.
+     *
+     * @param node   the node ID to add
+     * @param vector the byte vector to add
+     * @return an estimate of the number of extra bytes used by the graph after adding the given node
+     * @throws UnsupportedOperationException if this builder was not constructed with a byte-vector score provider
+     */
+    public long addGraphNode(int node, ByteSequence<?> vector) {
+        var ssp = scoreProvider.searchProviderFor(vector);
+        return addGraphNode(node, ssp);
+    }
+
     /**
      * Inserts a node with the given vector value to the graph.
      *
@@ -1343,4 +1415,4 @@ public static ImmutableGraphIndex buildAndMergeNewNodes(RandomAccessReader in,
             return builder.getGraph();
         }
     }
-}
+}

jvector-base/src/main/java/io/github/jbellis/jvector/graph/ListRandomAccessByteVectorValues.java

@@ -0,0 +1,72 @@
+/*
+ * Copyright DataStax, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package io.github.jbellis.jvector.graph;
+
+import io.github.jbellis.jvector.annotations.Experimental;
+import io.github.jbellis.jvector.vector.types.ByteSequence;
+
+import java.util.List;
+
+/**
+ * A List-backed implementation of the {@link RandomAccessByteVectorValues} interface.
+ * <p>
+ * It is acceptable to provide this class to a GraphBuilder, and then continue
+ * to add vectors to the backing List as you add to the graph.
+ * <p>
+ * This will be as threadsafe as the provided List.
+ */
+@Experimental
+public class ListRandomAccessByteVectorValues implements RandomAccessByteVectorValues {
+    private final List<ByteSequence<?>> vectors;
+    private final int dimension;
+
+    /**
+     * Construct a new instance of {@link ListRandomAccessByteVectorValues}.
+     *
+     * @param vectors   a (potentially mutable) list of byte vectors.
+     * @param dimension the dimension of the vectors.
+     */
+    public ListRandomAccessByteVectorValues(List<ByteSequence<?>> vectors, int dimension) {
+        this.vectors = vectors;
+        this.dimension = dimension;
+    }
+
+    @Override
+    public int size() {
+        return vectors.size();
+    }
+
+    @Override
+    public int dimension() {
+        return dimension;
+    }
+
+    @Override
+    public ByteSequence<?> getVector(int nodeId) {
+        return vectors.get(nodeId);
+    }
+
+    @Override
+    public boolean isValueShared() {
+        return false;
+    }
+
+    @Override
+    public ListRandomAccessByteVectorValues copy() {
+        return this;
+    }
+}

jvector-base/src/main/java/io/github/jbellis/jvector/graph/RandomAccessByteVectorValues.java

@@ -0,0 +1,68 @@
+/*
+ * Copyright DataStax, Inc.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package io.github.jbellis.jvector.graph;
+
+import io.github.jbellis.jvector.annotations.Experimental;
+import io.github.jbellis.jvector.util.ExplicitThreadLocal;
+import io.github.jbellis.jvector.vector.types.ByteSequence;
+
+import java.util.function.Supplier;
+
+/**
+ * Provides random access to byte (int8) vectors by dense ordinal.
+ * <p>
+ * This is the byte-vector parallel to {@link RandomAccessVectorValues}.
+ * Both extend the common super-interface {@link VectorValues}.
+ * It is used by graph-based index builders and searchers that operate natively
+ * on int8 vectors without a float32 round-trip.
+ */
+@Experimental
+public interface RandomAccessByteVectorValues extends VectorValues<ByteSequence<?>> {
+
+    /**
+     * Retrieve the vector associated with a given node, and store it in the destination vector at the given offset.
+     * The default implementation copies from {@link #getVector}; implementations that can read directly into the
+     * destination (e.g. from disk) should override it.
+     */
+    @Override
+    default void getVectorInto(int node, ByteSequence<?> destinationVector, int offset) {
+        destinationVector.copyFrom(getVector(node), 0, offset, dimension());
+    }
+
+    /**
+     * Creates a new copy of this {@link RandomAccessByteVectorValues}.
+     * Un-shared implementations may simply return {@code this}.
+     */
+    @Override
+    RandomAccessByteVectorValues copy();
+
+    /**
+     * Returns a supplier of thread-local copies of the RAVV.
+     */
+    @Override
+    default Supplier<RandomAccessByteVectorValues> threadLocalSupplier() {
+        if (!isValueShared()) {
+            return () -> this;
+        }
+
+        if (this instanceof AutoCloseable) {
+            VectorValues.LOG.warning("VectorValues is shared and implements AutoCloseable; threadLocalSupplier() may lead to leaks");
+        }
+        var tl = ExplicitThreadLocal.withInitial(this::copy);
+        return tl::get;
+    }
+}