Overview
This topic describes the concept and types of dense vector indexes in seekdb. In the following description, a dense vector index is referred to as a dense index.
A dense index is an efficient type of index provided by seekdb for dense vector data (vectors with most elements as non-zero values). Dense vector indexes are primarily used to accelerate vector searches and support nearest neighbor search through distance functions (L2, inner product, cosine distance).
Index types
seekdb provides the HNSW and IVF series of dense vector indexes.
HNSW series
The HNSW series of indexes are memory-based graph structure indexes that need to be fully loaded into memory. They support DML operations and real-time search, and offer high search performance and recall.
| Index type | Description |
|---|---|
| HNSW | The HNSW index is a memory-based index. The maximum dimension of the indexed column is 4096. |
| HNSW_SQ | The HNSW_SQ index provides similar build speed, search performance, and recall as the HNSW index, but reduces the total memory usage to 1/2 to 1/3 of the original. |
| HNSW_BQ | The HNSW_BQ index has slightly lower recall than the HNSW index, but significantly reduces memory usage. The BQ quantization compression algorithm (Rabitq) can compress vectors to 1/32 of their original size. As the vector dimension increases, the memory optimization effect of the HNSW_BQ index becomes more pronounced. |
For more information, see the related topics at the end of this topic.
IVF series
The IVF series of indexes are disk-based indexes that do not require resident memory. They are suitable for scenarios with large data volumes and cost sensitivity.
| Index type | Description |
|---|---|
| IVF | The IVF index is implemented based on a database table and does not require resident memory. |
| IVF_PQ | The IVF_PQ index is implemented based on a database table and does not require resident memory. It applies the PQ quantization technique to the IVF index, resulting in slightly lower recall than the IVF index but higher performance. In general scenarios, the PQ quantization compression algorithm can compress vectors to 1/16 to 1/32 of their original size. |
For more information, see the related topics at the end of this topic.
Syntax
Dense indexes can be created during table creation or after table creation. The search and deletion syntaxes are consistent with those of general vector indexes. For the creation syntaxes, parameter descriptions, search syntaxes, and examples of different index types, see:
- HNSW series indexes, which includes the creation, search, and deletion syntaxes and examples for HNSW, HNSW_SQ, and HNSW_BQ indexes.
- IVF series indexes, which includes the creation, search, and deletion syntaxes and examples for IVF and IVF_PQ indexes.
References
- For index selection suggestions, see Select an index type.
- Vector index memory management
- Monitor and maintain vector indexes