diff --git a/benchmarks/DEL/dfly_bench/DEL_reproduce.md b/benchmarks/DEL/dfly_bench/DEL_reproduce.md new file mode 100644 index 0000000..0c4b535 --- /dev/null +++ b/benchmarks/DEL/dfly_bench/DEL_reproduce.md @@ -0,0 +1,37 @@ +### Stateful setup: + +``` +dfbench setup --id maxqps2 --tune-network --ubuntu-version 24.04 \ + --server-instance m7g.8xlarge --server-arch arm64 \ + --client-instance c6gn.8xlarge --client-arch arm64 \ + --engines dragonfly,redis,valkey --availability-zone us-east-1c +``` + +### Test run: + +dfbench run-commands-dfly \ + --id maxqps2 --engines dragonfly,redis,valkey \ + \ + --command-template "DEL __key__" --command-name DEL --command-drains \ + --preload-template "SET __key__ __data__" \ + --key-maximum 100000000 \ + --test-time 300 --warmup-time 10 --trials 1 \ + \ + --dfly-bench-threads 32 --dfly-bench-conns 5 --dfly-bench-pipeline 30 \ + --dfly-bench-data-size 128 --dfly-bench-key-dist U --dfly-bench-qps 0 \ + \ + --dragonfly_version s3://df-use1-pub/v2/dragonfly-aarch64.tar.gz \ + --dragonfly_num_shards 16 --dragonfly_conn_use_incoming_cpu=true \ + \ + --redis_io_threads 8 --valkey_io_threads 8 + + +### Expected results: + +**DEL** + +| Engine | Throughput (median) | p50 | p99 | p99.9 | Avg Latency | +| --------- | ------------------- | -------- | -------- | -------- | ----------- | +| dragonfly | 10.25M ops/s | 0.445 ms | 0.822 ms | 1.131 ms | 0.456 ms | +| redis | 1.96M ops/s | 2.255 ms | 6.612 ms | 7.977 ms | 2.439 ms | +| valkey | 2.38M ops/s | 1.759 ms | 3.992 ms | 4.461 ms | 2.008 ms | \ No newline at end of file diff --git a/benchmarks/HGET/dfly_bench/HGET_reproduce.md b/benchmarks/HGET/dfly_bench/HGET_reproduce.md index 07e7a2a..74aebeb 100644 --- a/benchmarks/HGET/dfly_bench/HGET_reproduce.md +++ b/benchmarks/HGET/dfly_bench/HGET_reproduce.md @@ -27,10 +27,45 @@ dfbench run-commands-dfly \ ### Expected results: - HGET - Engine Throughput (median) p50 p99 p99.9 Avg Latency - ────── ─────────────────── ─── ─── ───── ─────────── - dragonfly 9.31M ops/s 0.468 ms 0.875 ms 1.163 ms 0.489 ms - redis 744.4K ops/s 6.522 ms 9.979 ms 13.329 ms 6.429 ms - valkey 791.8K ops/s 6.483 ms 6.990 ms 6.999 ms 6.044 ms - \ No newline at end of file +**HGET** + +| Engine | Throughput (median) | p50 | p99 | p99.9 | Avg Latency | +| --------- | ------------------- | -------- | -------- | --------- | ----------- | +| dragonfly | 9.31M ops/s | 0.468 ms | 0.875 ms | 1.163 ms | 0.489 ms | +| redis | 744.4K ops/s | 6.522 ms | 9.979 ms | 13.329 ms | 6.429 ms | +| valkey | 791.8K ops/s | 6.483 ms | 6.990 ms | 6.999 ms | 6.044 ms | + +### With improvements: + +--dfly-bench-conns 7 + +--redis_io_threads 8 --valkey_io_threads 8 + +``` +dfbench run-commands-dfly \ + --id maxqps2 --engines dragonfly,redis,valkey \ + \ + --command-template "HGET __key__ field" --command-name HGET \ + --preload-template "HSET __key__ field __data__" \ + --key-maximum 100000000 \ + --test-time 60 --warmup-time 10 --trials 1 \ + \ + --dfly-bench-threads 32 --dfly-bench-conns 7 --dfly-bench-pipeline 30 \ + --dfly-bench-data-size 128 --dfly-bench-key-dist U --dfly-bench-qps 0 \ + \ + --dragonfly_version s3://df-use1-pub/v2/dragonfly-aarch64.tar.gz \ + --dragonfly_num_shards 16 \ + \ + --redis_io_threads 8 --valkey_io_threads 8 +``` + +### Expected results after improvements: + + +**HGET** + +| Engine | Throughput (median) | p50 | p99 | p99.9 | Avg Latency | +| --------- | ------------------- | -------- | --------- | --------- | ----------- | +| dragonfly | 9.62M ops/s | 0.636 ms | 1.380 ms | 2.460 ms | 0.671 ms | +| redis | 733.7K ops/s | 8.567 ms | 13.988 ms | 17.445 ms | 9.138 ms | +| valkey | 804.0K ops/s | 8.504 ms | 8.998 ms | 9.884 ms | 8.339 ms | \ No newline at end of file diff --git a/benchmarks/HSET/dfly_bench/HSET_reproduce.md b/benchmarks/HSET/dfly_bench/HSET_reproduce.md index f47d554..205287a 100644 --- a/benchmarks/HSET/dfly_bench/HSET_reproduce.md +++ b/benchmarks/HSET/dfly_bench/HSET_reproduce.md @@ -28,11 +28,12 @@ dfbench run-commands-dfly \ ### Expected results: - HSET - Engine Throughput (median) p50 p99 p99.9 Avg Latency - ────── ─────────────────── ─── ─── ───── ─────────── - dragonfly 7.99M ops/s 0.507 ms 1.985 ms 7.332 ms 0.583 ms - redis 638.5K ops/s 6.839 ms 11.613 ms 18.655 ms 7.503 ms - valkey 704.0K ops/s 6.573 ms 13.470 ms 17.760 ms 6.804 ms +**HSET** + +| Engine | Throughput (median) | p50 | p99 | p99.9 | Avg Latency | +| --------- | ------------------- | -------- | --------- | --------- | ----------- | +| dragonfly | 7.99M ops/s | 0.507 ms | 1.985 ms | 7.332 ms | 0.583 ms | +| redis | 638.5K ops/s | 6.839 ms | 11.613 ms | 18.655 ms | 7.503 ms | +| valkey | 704.0K ops/s | 6.573 ms | 13.470 ms | 17.760 ms | 6.804 ms | diff --git a/benchmarks/MGET/dfly_bench/MGET_reproduce.md b/benchmarks/MGET/dfly_bench/MGET_reproduce.md index 59b2b51..33c4dc8 100644 --- a/benchmarks/MGET/dfly_bench/MGET_reproduce.md +++ b/benchmarks/MGET/dfly_bench/MGET_reproduce.md @@ -27,9 +27,22 @@ dfbench run-commands-dfly \ ### Expected results: - MGET - Engine Throughput (median) p50 p99 p99.9 Avg Latency - ────── ─────────────────── ─── ─── ───── ─────────── - dragonfly 8.09M ops/s 0.549 ms 1.114 ms 1.432 ms 0.570 ms - redis 1.37M ops/s 3.408 ms 6.658 ms 7.939 ms 3.478 ms - valkey 1.49M ops/s 2.754 ms 8.701 ms 9.487 ms 3.211 ms \ No newline at end of file +**MGET** + +| Engine | Throughput (median) | p50 | p99 | p99.9 | Avg Latency | +| --------- | ------------------- | -------- | -------- | -------- | ----------- | +| dragonfly | 8.09M ops/s | 0.549 ms | 1.114 ms | 1.432 ms | 0.570 ms | +| redis | 1.37M ops/s | 3.408 ms | 6.658 ms | 7.939 ms | 3.478 ms | +| valkey | 1.49M ops/s | 2.754 ms | 8.701 ms | 9.487 ms | 3.211 ms | + +### Optional improvement: + +increasing --dfly-bench-conns from 5 to 7: + +increases throughput a little but, but also almost double p99.9 latency + +**MGET** + +| Engine | Throughput (median) | p50 | p99 | p99.9 | Avg Latency | +| --------- | ------------------- | -------- | -------- | -------- | ----------- | +| dragonfly | 8.58M ops/s | 0.715 ms | 1.667 ms | 2.920 ms | 0.756 ms | diff --git a/benchmarks/MSET/dfly_bench/MSET_reproduce.md b/benchmarks/MSET/dfly_bench/MSET_reproduce.md index 4a9c67c..e337890 100644 --- a/benchmarks/MSET/dfly_bench/MSET_reproduce.md +++ b/benchmarks/MSET/dfly_bench/MSET_reproduce.md @@ -28,9 +28,10 @@ dfbench run-commands-dfly \ ### Expected results: - MSET - Engine Throughput (median) p50 p99 p99.9 Avg Latency - ────── ─────────────────── ─── ─── ───── ─────────── - dragonfly 7.42M ops/s 0.519 ms 3.193 ms 10.310 ms 0.631 ms - redis 932.9K ops/s 4.390 ms 8.630 ms 15.157 ms 5.132 ms - valkey 1.24M ops/s 3.705 ms 11.262 ms 15.579 ms 3.843 ms \ No newline at end of file +**MSET** + +| Engine | Throughput (median) | p50 | p99 | p99.9 | Avg Latency | +| --------- | ------------------- | -------- | --------- | --------- | ----------- | +| dragonfly | 7.42M ops/s | 0.519 ms | 3.193 ms | 10.310 ms | 0.631 ms | +| redis | 932.9K ops/s | 4.390 ms | 8.630 ms | 15.157 ms | 5.132 ms | +| valkey | 1.24M ops/s | 3.705 ms | 11.262 ms | 15.579 ms | 3.843 ms | \ No newline at end of file diff --git a/benchmarks/ZADD/dfly_bench/ZADD_reproduce.md b/benchmarks/ZADD/dfly_bench/ZADD_reproduce.md index 07b026b..6d440bc 100644 --- a/benchmarks/ZADD/dfly_bench/ZADD_reproduce.md +++ b/benchmarks/ZADD/dfly_bench/ZADD_reproduce.md @@ -28,9 +28,10 @@ dfbench run-commands-dfly \ ### Expected results: - ZADD - Engine Throughput (median) p50 p99 p99.9 Avg Latency - ────── ─────────────────── ─── ─── ───── ─────────── - dragonfly 7.22M ops/s 0.564 ms 1.976 ms 10.717 ms 0.642 ms - redis 448.4K ops/s 9.621 ms 16.999 ms 26.752 ms 10.691 ms - valkey 526.0K ops/s 8.852 ms 17.620 ms 23.609 ms 9.111 ms \ No newline at end of file +**ZADD** + +| Engine | Throughput (median) | p50 | p99 | p99.9 | Avg Latency | +| --------- | ------------------- | -------- | --------- | --------- | ----------- | +| dragonfly | 7.22M ops/s | 0.564 ms | 1.976 ms | 10.717 ms | 0.642 ms | +| redis | 448.4K ops/s | 9.621 ms | 16.999 ms | 26.752 ms | 10.691 ms | +| valkey | 526.0K ops/s | 8.852 ms | 17.620 ms | 23.609 ms | 9.111 ms | \ No newline at end of file diff --git a/benchmarks/ZSCORE/dfly_bench/ZSCORE_reproduce.md b/benchmarks/ZSCORE/dfly_bench/ZSCORE_reproduce.md index 996f08f..d96055d 100644 --- a/benchmarks/ZSCORE/dfly_bench/ZSCORE_reproduce.md +++ b/benchmarks/ZSCORE/dfly_bench/ZSCORE_reproduce.md @@ -28,9 +28,10 @@ dfbench run-commands-dfly \ ### Expected results: - ZSCORE - Engine Throughput (median) p50 p99 p99.9 Avg Latency - ────── ─────────────────── ─── ─── ───── ─────────── - dragonfly 9.10M ops/s 0.483 ms 0.996 ms 2.325 ms 0.509 ms - redis 692.7K ops/s 6.796 ms 7.992 ms 13.916 ms 6.916 ms - valkey 910.4K ops/s 5.434 ms 6.000 ms 6.902 ms 5.259 ms \ No newline at end of file +**ZSCORE** + +| Engine | Throughput (median) | p50 | p99 | p99.9 | Avg Latency | +| --------- | ------------------- | -------- | -------- | --------- | ----------- | +| dragonfly | 9.10M ops/s | 0.483 ms | 0.996 ms | 2.325 ms | 0.509 ms | +| redis | 692.7K ops/s | 6.796 ms | 7.992 ms | 13.916 ms | 6.916 ms | +| valkey | 910.4K ops/s | 5.434 ms | 6.000 ms | 6.902 ms | 5.259 ms | \ No newline at end of file