Package org.apache.lucene.codecs.lucene104

Class Lucene104HnswScalarQuantizedVectorsFormat

java.lang.Object

org.apache.lucene.codecs.KnnVectorsFormat

org.apache.lucene.codecs.lucene104.Lucene104HnswScalarQuantizedVectorsFormat

All Implemented Interfaces:

NamedSPILoader.NamedSPI

public class Lucene104HnswScalarQuantizedVectorsFormat
extends KnnVectorsFormat

A vectors format that uses HNSW graph to store and search for vectors. But vectors are binary quantized using Lucene104ScalarQuantizedVectorsFormat before being stored in the graph.

Field Summary

Fields

Modifier and Type	Field	Description
static final String	NAME

Fields inherited from class org.apache.lucene.codecs.KnnVectorsFormat

DEFAULT_MAX_DIMENSIONS, EMPTY

Constructor Summary

Constructors

Constructor	Description
Lucene104HnswScalarQuantizedVectorsFormat()	Constructs a format using default graph construction parameters
Lucene104HnswScalarQuantizedVectorsFormat(int maxConn, int beamWidth)	Constructs a format using the given graph construction parameters.
Lucene104HnswScalarQuantizedVectorsFormat(Lucene104ScalarQuantizedVectorsFormat.ScalarEncoding encoding, int maxConn, int beamWidth)	Constructs a format using the given graph construction parameters.
Lucene104HnswScalarQuantizedVectorsFormat(Lucene104ScalarQuantizedVectorsFormat.ScalarEncoding encoding, int maxConn, int beamWidth, int numMergeWorkers, ExecutorService mergeExec)	Constructs a format using the given graph construction parameters and scalar quantization.
Lucene104HnswScalarQuantizedVectorsFormat(Lucene104ScalarQuantizedVectorsFormat.ScalarEncoding encoding, int maxConn, int beamWidth, int numMergeWorkers, ExecutorService mergeExec, int tinySegmentsThreshold)	Constructs a format using the given graph construction parameters and scalar quantization.

Method Summary

Modifier and Type	Method	Description
KnnVectorsReader	fieldsReader(SegmentReadState state)	Returns a KnnVectorsReader to read the vectors from the index.
KnnVectorsWriter	fieldsWriter(SegmentWriteState state)	Returns a KnnVectorsWriter to write the vectors to the index.
int	getMaxDimensions(String fieldName)	Returns the maximum number of vector dimensions supported by this codec for the given field name
String	toString()

Methods inherited from class org.apache.lucene.codecs.KnnVectorsFormat

availableKnnVectorsFormats, forName, getName, reloadKnnVectorsFormat

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Details

NAME

public static final String NAME

See Also:

• Constant Field Values

Constructor Details

Lucene104HnswScalarQuantizedVectorsFormat

public Lucene104HnswScalarQuantizedVectorsFormat()

Constructs a format using default graph construction parameters

Lucene104HnswScalarQuantizedVectorsFormat

public Lucene104HnswScalarQuantizedVectorsFormat(int maxConn,
 int beamWidth)

Constructs a format using the given graph construction parameters.

Parameters:

maxConn - the maximum number of connections to a node in the HNSW graph

beamWidth - the size of the queue maintained during graph construction.

Lucene104HnswScalarQuantizedVectorsFormat

public Lucene104HnswScalarQuantizedVectorsFormat(Lucene104ScalarQuantizedVectorsFormat.ScalarEncoding encoding,
 int maxConn,
 int beamWidth)

Constructs a format using the given graph construction parameters.

Parameters:

encoding - the quantization encoding used to encode the vectors

maxConn - the maximum number of connections to a node in the HNSW graph

beamWidth - the size of the queue maintained during graph construction.

Lucene104HnswScalarQuantizedVectorsFormat

public Lucene104HnswScalarQuantizedVectorsFormat(Lucene104ScalarQuantizedVectorsFormat.ScalarEncoding encoding,
 int maxConn,
 int beamWidth,
 int numMergeWorkers,
 ExecutorService mergeExec)

Constructs a format using the given graph construction parameters and scalar quantization.

Parameters:

encoding - the quantization encoding used to encode the vectors

maxConn - the maximum number of connections to a node in the HNSW graph

beamWidth - the size of the queue maintained during graph construction.

numMergeWorkers - number of workers (threads) that will be used when doing merge. If larger than 1, a non-null ExecutorService must be passed as mergeExec

mergeExec - the ExecutorService that will be used by ALL vector writers that are generated by this format to do the merge

Lucene104HnswScalarQuantizedVectorsFormat

public Lucene104HnswScalarQuantizedVectorsFormat(Lucene104ScalarQuantizedVectorsFormat.ScalarEncoding encoding,
 int maxConn,
 int beamWidth,
 int numMergeWorkers,
 ExecutorService mergeExec,
 int tinySegmentsThreshold)

Constructs a format using the given graph construction parameters and scalar quantization.

Parameters:

maxConn - the maximum number of connections to a node in the HNSW graph

beamWidth - the size of the queue maintained during graph construction.

numMergeWorkers - number of workers (threads) that will be used when doing merge. If larger than 1, a non-null ExecutorService must be passed as mergeExec

mergeExec - the ExecutorService that will be used by ALL vector writers that are generated by this format to do the merge

Method Details

fieldsWriter

public KnnVectorsWriter fieldsWriter(SegmentWriteState state)
                              throws IOException

Description copied from class: KnnVectorsFormat

Returns a KnnVectorsWriter to write the vectors to the index.

Specified by:

fieldsWriter in class KnnVectorsFormat

Throws:

IOException

fieldsReader

public KnnVectorsReader fieldsReader(SegmentReadState state)
                              throws IOException

Description copied from class: KnnVectorsFormat

Returns a KnnVectorsReader to read the vectors from the index.

Specified by:

fieldsReader in class KnnVectorsFormat

Throws:

IOException

getMaxDimensions

public int getMaxDimensions(String fieldName)

Description copied from class: KnnVectorsFormat

Returns the maximum number of vector dimensions supported by this codec for the given field name

Codecs implement this method to specify the maximum number of dimensions they support.

Specified by:

getMaxDimensions in class KnnVectorsFormat

Parameters:

fieldName - the field name

Returns:

the maximum number of vector dimensions.

toString

public String toString()

Overrides:

toString in class Object