Skip to main content

seekdb VectorDBBench Benchmark Report

This topic describes the test environment, test plan, and test results of the VectorDBBench benchmark.

Test environment​

The following example uses Alibaba Cloud Elastic Compute Service (ECS) as the deployment environment. The seekdb instance has a specification of 1C2G or 4C8G.

  • When the seekdb instance has a specification of 4C8G, the corresponding hardware configuration and software version are as follows:

    • Hardware configuration

      Service typeECS typeNumber of instancesNumber of CPU coresMemory
      seekdbecs.c8i.xlarge14C8G
      VectorDBBenchecs.g8i.4xlarge116C64G

    • Software version

      Service typeSoftware version
      seekdbSeekDB 1.0.0.0
      VectorDBBench1.0.14
      OSCentOS Linux 7 (Core)

  • When the seekdb instance has a specification of 1C2G, the corresponding hardware configuration and software version are as follows:

    • Hardware configuration

      Service typeECS typeNumber of instancesNumber of CPU coresMemory
      seekdbecs.c8i.large12C (with active vCPU set to 1)4G (3G available memory), memory_limit set to 2G
      VectorDBBenchecs.g8i.4xlarge116C64G

    • Software version

      Service typeSoftware version
      seekdbSeekDB 1.0.0.0
      VectorDBBench1.0.14
      OSCentOS Linux 7 (Core)

Test plan​

  • In this test, two machines are required. VectorDBBench is deployed on one machine, and seekdb is deployed on the other machine. The seekdb instance has a specification of 4C8G or 1C2G. The log disk, clog disk, and data disk are three separate disks with performance level PL1.

  • The data set used in the test is Performance1536D50K, and the index type is HNSW. The test measures the qps, latency (p99), and recall rate of seekdb using VectorDBBench.

For more information about the test steps, see seekdb VectorDBBench Benchmark Test.

Test results​

The test is performed with a recall rate of 0.98.

  • When the seekdb instance has a specification of 4C8G, the corresponding performance data is as follows:

    | ef_search | qps | serial_latency_p99 | Recall rate | |----------------------|----------------------|---------------|--------------|--------------| | 180 | 2342.3263 | 0.0022 | 0.9801 |

  • When the seekdb instance has a specification of 1C2G, the corresponding performance data is as follows:

    | ef_search | qps | serial_latency_p99 | Recall rate | |----------------------|----------------------|---------------|--------------|--------------| | 180 | 811.2516 | 0.0025 | 0.9802 |

Contents

Edit this page