Overview
This topic describes the vector index types supported by seekdb and their characteristics.
Index types
seekdb supports the following vector index types:
- Dense index
- Sparse index
Dense index
seekdb supports dense vector indexes, including the HNSW series and IVF series. For convenience, they are referred to as dense indexes in the following sections.
| Index type | Description |
|---|---|
| HNSW | The maximum dimension of the index column is 4096. HNSW indexes are memory-based and must be fully loaded into memory. |
| HNSW_SQ | HNSW_SQ indexes provide similar build speed, search performance, and recall rate as HNSW indexes, but they use only 1/2 to 1/3 of the memory. |
| HNSW_BQ | HNSW_BQ indexes have slightly lower recall rates than HNSW indexes but significantly reduce memory usage. The BQ quantization compression algorithm (Rabitq) can compress vectors to 1/32 of their original size. As the vector dimension increases, the memory optimization effect of HNSW_BQ indexes becomes more pronounced. |
| IVF | The IVF index is implemented based on database tables and does not occupy resident memory. |
| IVF_PQ | The IVF_PQ index is implemented based on database tables and does not occupy resident memory. It applies the PQ quantization technique to the IVF index, resulting in slightly lower recall rates but higher performance. In general scenarios, the PQ quantization compression algorithm can compress vectors to 1/16 to 1/32 of their original size. |
Sparse index
seekdb supports in-memory sparse vector indexes. For convenience, they are referred to as in-memory sparse indexes in the following sections. In-memory sparse indexes are efficient index types provided by seekdb for sparse vectors (vectors with most elements being zero). They require the complete loading of the index into memory and support DML operations and real-time search.
In-memory sparse indexes are an experimental feature in the current version and are not recommended for use in production environments.
Considerations and limitations
- Distance algorithm: Dense vector indexes support L2, inner product (IP), and cosine distance as index distance algorithms.
- Distance function: Vector index search supports calling some distance functions. For more information, see Use SQL functions.
- Filter conditions: Vector search supports filter conditions. The filter conditions can be scalar conditions or spatial relationships, such as ST_Intersects. Multi-value indexes, full-text indexes, and global indexes cannot be used as pre-filterers.
- Offline DDL: For information about the support of vector indexes for offline DDL, see Offline DDL.
- Columnstore indexes: Columnstore vector indexes cannot be created in the current version.