Skip to main content

INDEX_VECTOR_MEMORY_ADVISOR

The INDEX_VECTOR_MEMORY_ADVISOR procedure is used to estimate the memory usage of a vector index that is not yet created.

Syntax​

FUNCTION index_vector_memory_advisor (
    IN     idx_type          VARCHAR(65535),
    IN     num_vectors       BIGINT UNSIGNED,
    IN     dim_count         INT UNSIGNED,
    IN     dim_type          VARCHAR(65535) DEFAULT 'FLOAT32',
    IN     idx_parameters    LONGTEXT DEFAULT NULL,
    IN     max_tablet_vectors BIGINT UNSIGNED DEFAULT 0)

Parameters​

ParameterDescription
idx_typeThe index type. Valid values: HNSW, HNSW_SQ, HNSW_BQ, IVF_FLAT, IVF_SQ8, and IVF_PQ. The value is case-insensitive.
num_vectorsThe number of vectors.
dim_countThe dimension of the vector.
dim_typeThe data type of the vector elements. The value must be FLOAT32. The value is case-insensitive.
idx_parametersThe index parameters. This parameter specifies the vector index parameters used when creating a table, for example, distance=l2, type=hnsw, lib=vsag.
max_tablet_vectorsThe maximum number of vectors in a partition of a partitioned table. The default value is 0, which indicates that the maximum number of vectors in a partition is equal to the total number of vectors. Using a partitioned table can reduce the minimum memory requirement when the total data size is the same.

The return value is a string that describes the estimated memory usage.

note

In seekdb, when you create an HNSW_BQ index, the system automatically chooses to create the index in parallel or sequentially for a partitioned table based on the memory. The recommended value returned by the INDEX_VECTOR_MEMORY_ADVISOR function is the maximum memory requirement for sequential index creation. If the memory only meets this value, the index creation time may be extended because the index cannot be created in parallel.

To ensure the successful execution of vector index reconstruction, the recommended memory reservation value should be greater than the actual memory occupied by the vector index after the index is built. The recommended memory value is related to the maximum number of vectors in a partition:

  • For HNSW and HNSW_SQ indexes, the recommended memory value is returned.
  • For HNSW_BQ indexes, the recommended memory value and the actual memory occupied by the vector index after the index is built are returned.
  • For IVF_FLAT, IVF_SQ8, and IVF_PQ indexes, the recommended memory value and the actual memory occupied by the vector index after the index is built are returned.

Examples​

Query the recommended memory value for an HNSW index:

SELECT dbms_vector.index_vector_memory_advisor('HNSW',1000000,768,'FLOAT32','M=10,DISTANCE=L2');
+------------------------------------------------------------------------------------------+
| dbms_vector.index_vector_memory_advisor('HNSW',1000000,768,'FLOAT32','M=10,DISTANCE=L2') |
+------------------------------------------------------------------------------------------+
| Suggested minimum vector memory is 7.2 GB                                                |
+------------------------------------------------------------------------------------------+
1 row in set

Query the recommended memory value and the actual memory occupied by the vector index after the index is built for an HNSW_BQ index:

SELECT dbms_vector.index_vector_memory_advisor('HNSW_BQ',1000000,768,'FLOAT32','M=10,DISTANCE=L2');
+---------------------------------------------------------------------------------------------------------+
| dbms_vector.index_vector_memory_advisor('HNSW_BQ',1000000,768,'FLOAT32','M=10,DISTANCE=L2')             |
+---------------------------------------------------------------------------------------------------------+
| Suggested minimum vector memory is 4.1 GB, memory consumption when providing search service is 347.5 MB |
+---------------------------------------------------------------------------------------------------------+
1 row in set

Query the recommended memory value and the actual memory occupied by the vector index after the index is built for an IVF index:

SELECT dbms_vector.index_vector_memory_advisor('IVF_FLAT',1000000,768,'FLOAT32','SAMPLE_PER_NLIST=1024,DISTANCE=L2');
+---------------------------------------------------------------------------------------------------------------+
| dbms_vector.index_vector_memory_advisor('IVF_FLAT',1000000,768,'FLOAT32','SAMPLE_PER_NLIST=1024,DISTANCE=L2') |
+---------------------------------------------------------------------------------------------------------------+
| Suggested minimum vector memory is 452.7 MB, memory consumption when providing search service is 384.0 KB     |
+---------------------------------------------------------------------------------------------------------------+
1 row in set

Contents

Edit this page