Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions benchmarks/EXPIRE/dfly_bench/EXPIRE_reproduce.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
### Stateful setup:

```
dfbench setup --id maxqps2 --tune-network --ubuntu-version 24.04 \
--server-instance m7g.8xlarge --server-arch arm64 \
--client-instance c6gn.8xlarge --client-arch arm64 \
--engines dragonfly,redis,valkey --availability-zone us-east-1c
```

Note: pick an availability zone that has `c6gn.8xlarge` capacity (AWS suggested
`us-east-1b/c/d/f`; `us-east-1a`/`1e` were unavailable at run time). Server and
client share a cluster placement group, so both instance types must have capacity
in the chosen AZ.

### Test run:

```
dfbench run-commands-dfly \
--id maxqps2 --engines dragonfly,redis,valkey \
\
--command-template "EXPIRE __key__ 3600" --command-name EXPIRE --command-mutates \
--preload-template "SET __key__ __data__" \
--key-maximum 100000000 \
--test-time 300 --warmup-time 10 --trials 1 \
\
--dfly-bench-threads 32 --dfly-bench-conns 5 --dfly-bench-pipeline 30 \
--dfly-bench-data-size 128 --dfly-bench-key-dist U --dfly-bench-qps 0 \
\
--dragonfly_version s3://df-use1-pub/v2/dragonfly-aarch64.tar.gz \
--dragonfly_num_shards 16 --dragonfly_conn_use_incoming_cpu=true \
\
--redis_io_threads 8 --valkey_io_threads 8
```

### Expected results:

**EXPIRE**

| Engine | Throughput (median) | p50 | p99 | p99.9 | Avg Latency |
| --------- | ------------------- | -------- | --------- | --------- | ----------- |
| dragonfly | 8.51M ops/s | 0.452 ms | 2.705 ms | 9.896 ms | 0.551 ms |
| redis | 777.5K ops/s | 5.777 ms | 13.012 ms | 15.763 ms | 6.162 ms |
| valkey | 1.10M ops/s | 3.997 ms | 7.949 ms | 11.370 ms | 4.373 ms |
42 changes: 42 additions & 0 deletions benchmarks/INCR/dfly_bench/INCR_reproduce.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
### Stateful setup:

```
dfbench setup --id maxqps2 --tune-network --ubuntu-version 24.04 \
--server-instance m7g.8xlarge --server-arch arm64 \
--client-instance c6gn.8xlarge --client-arch arm64 \
--engines dragonfly,redis,valkey --availability-zone us-east-1c
```

Note: pick an availability zone that has `c6gn.8xlarge` capacity (AWS suggested
`us-east-1b/c/d/f`; `us-east-1a`/`1e` were unavailable at run time). Server and
client share a cluster placement group, so both instance types must have capacity
in the chosen AZ.

### Test run:

```
dfbench run-commands-dfly \
--id maxqps2 --engines dragonfly,redis,valkey \
\
--command-template "INCR __key__" --command-name INCR --command-mutates \
--key-maximum 100000000 \
--test-time 300 --warmup-time 10 --trials 1 \
\
--dfly-bench-threads 32 --dfly-bench-conns 5 --dfly-bench-pipeline 30 \
--dfly-bench-data-size 128 --dfly-bench-key-dist U --dfly-bench-qps 0 \
\
--dragonfly_version s3://df-use1-pub/v2/dragonfly-aarch64.tar.gz \
--dragonfly_num_shards 16 --dragonfly_conn_use_incoming_cpu=true \
\
--redis_io_threads 8 --valkey_io_threads 8
```

### Expected results:

**INCR**

| Engine | Throughput (median) | p50 | p99 | p99.9 | Avg Latency |
| --------- | ------------------- | -------- | -------- | --------- | ----------- |
| dragonfly | 10.78M ops/s | 0.413 ms | 0.831 ms | 1.188 ms | 0.433 ms |
| redis | 1.56M ops/s | 2.803 ms | 5.972 ms | 9.891 ms | 3.074 ms |
| valkey | 1.92M ops/s | 2.317 ms | 7.349 ms | 10.855 ms | 2.495 ms |
44 changes: 44 additions & 0 deletions benchmarks/LPOP/dfly_bench/LPOP_reproduce.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
### Stateful setup:

```
dfbench setup --id maxqps2 --tune-network --ubuntu-version 24.04 \
--server-instance m7g.8xlarge --server-arch arm64 \
--client-instance c6gn.8xlarge --client-arch arm64 \
--engines dragonfly,redis,valkey --availability-zone us-east-1c
```

Note: pick an availability zone that has `c6gn.8xlarge` capacity (AWS suggested
`us-east-1b/c/d/f`; `us-east-1a`/`1e` were unavailable at run time). Server and
client share a cluster placement group, so both instance types must have capacity
in the chosen AZ.

### Test run:

```
dfbench run-commands-dfly \
--id maxqps2 --engines dragonfly,redis,valkey \
\
--command-template "LPOP __key__" --command-name LPOP --command-drains \
--preload-template "LPUSH __key__ __data__" --preload-items 100 \
--command-key-maximum 1000000 \
--key-maximum 100000000 \
--test-time 300 --warmup-time 10 --trials 1 \
\
--dfly-bench-threads 32 --dfly-bench-conns 5 --dfly-bench-pipeline 30 \
--dfly-bench-data-size 128 --dfly-bench-key-dist U --dfly-bench-qps 0 \
\
--dragonfly_version s3://df-use1-pub/v2/dragonfly-aarch64.tar.gz \
--dragonfly_num_shards 16 --dragonfly_conn_use_incoming_cpu=true \
\
--redis_io_threads 8 --valkey_io_threads 8
```

### Expected results:

**LPOP**

| Engine | Throughput (median) | p50 | p99 | p99.9 | Avg Latency |
| --------- | ------------------- | -------- | --------- | --------- | ----------- |
| dragonfly | 10.25M ops/s | 0.432 ms | 0.871 ms | 3.507 ms | 0.456 ms |
| redis | 1.33M ops/s | 1.562 ms | 11.894 ms | 13.783 ms | 3.588 ms |
| valkey | 1.69M ops/s | 1.535 ms | 10.042 ms | 11.804 ms | 2.822 ms |
42 changes: 42 additions & 0 deletions benchmarks/LPUSH/dfly_bench/LPUSH_reproduce.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
### Stateful setup:

```
dfbench setup --id maxqps2 --tune-network --ubuntu-version 24.04 \
--server-instance m7g.8xlarge --server-arch arm64 \
--client-instance c6gn.8xlarge --client-arch arm64 \
--engines dragonfly,redis,valkey --availability-zone us-east-1c
```

Note: pick an availability zone that has `c6gn.8xlarge` capacity (AWS suggested
`us-east-1b/c/d/f`; `us-east-1a`/`1e` were unavailable at run time). Server and
client share a cluster placement group, so both instance types must have capacity
in the chosen AZ.

### Test run:

```
dfbench run-commands-dfly \
--id maxqps2 --engines dragonfly,redis,valkey \
\
--command-template "LPUSH __key__ __data__" --command-name LPUSH --command-mutates \
--key-maximum 100000000 \
--test-time 300 --warmup-time 10 --trials 1 \
\
--dfly-bench-threads 32 --dfly-bench-conns 5 --dfly-bench-pipeline 30 \
--dfly-bench-data-size 128 --dfly-bench-key-dist U --dfly-bench-qps 0 \
\
--dragonfly_version s3://df-use1-pub/v2/dragonfly-aarch64.tar.gz \
--dragonfly_num_shards 16 --dragonfly_conn_use_incoming_cpu=true \
\
--redis_io_threads 8 --valkey_io_threads 8
```

### Expected results:

**LPUSH**

| Engine | Throughput (median) | p50 | p99 | p99.9 | Avg Latency |
| --------- | ------------------- | -------- | --------- | --------- | ----------- |
| dragonfly | 6.02M ops/s | 0.525 ms | 5.972 ms | 11.560 ms | 0.778 ms |
| redis | 661.9K ops/s | 7.110 ms | 11.702 ms | 18.645 ms | 7.239 ms |
| valkey | 866.3K ops/s | 5.416 ms | 11.628 ms | 15.410 ms | 5.528 ms |
44 changes: 44 additions & 0 deletions benchmarks/RPOP/dfly_bench/RPOP_reproduce.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
### Stateful setup:

```
dfbench setup --id maxqps2 --tune-network --ubuntu-version 24.04 \
--server-instance m7g.8xlarge --server-arch arm64 \
--client-instance c6gn.8xlarge --client-arch arm64 \
--engines dragonfly,redis,valkey --availability-zone us-east-1c
```

Note: pick an availability zone that has `c6gn.8xlarge` capacity (AWS suggested
`us-east-1b/c/d/f`; `us-east-1a`/`1e` were unavailable at run time). Server and
client share a cluster placement group, so both instance types must have capacity
in the chosen AZ.

### Test run:

```
dfbench run-commands-dfly \
--id maxqps2 --engines dragonfly,redis,valkey \
\
--command-template "RPOP __key__" --command-name RPOP --command-mutates \
--preload-template "LPUSH __key__ __data__" --preload-items 100 \
--command-key-maximum 1000000 \
--key-maximum 100000000 \
--test-time 300 --warmup-time 10 --trials 1 \
\
--dfly-bench-threads 32 --dfly-bench-conns 5 --dfly-bench-pipeline 30 \
--dfly-bench-data-size 128 --dfly-bench-key-dist U --dfly-bench-qps 0 \
\
--dragonfly_version s3://df-use1-pub/v2/dragonfly-aarch64.tar.gz \
--dragonfly_num_shards 16 --dragonfly_conn_use_incoming_cpu=true \
\
--redis_io_threads 8 --valkey_io_threads 8
```

### Expected results:

**RPOP**

| Engine | Throughput (median) | p50 | p99 | p99.9 | Avg Latency |
| --------- | ------------------- | -------- | --------- | --------- | ----------- |
| dragonfly | 10.67M ops/s | 0.416 ms | 0.792 ms | 3.282 ms | 0.437 ms |
| redis | 1.49M ops/s | 1.550 ms | 11.361 ms | 11.985 ms | 3.217 ms |
| valkey | 1.81M ops/s | 1.532 ms | 9.645 ms | 11.570 ms | 2.639 ms |
42 changes: 42 additions & 0 deletions benchmarks/RPUSH/dfly_bench/RPUSH_reproduce.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
### Stateful setup:

```
dfbench setup --id maxqps2 --tune-network --ubuntu-version 24.04 \
--server-instance m7g.8xlarge --server-arch arm64 \
--client-instance c6gn.8xlarge --client-arch arm64 \
--engines dragonfly,redis,valkey --availability-zone us-east-1c
```

Note: pick an availability zone that has `c6gn.8xlarge` capacity (AWS suggested
`us-east-1b/c/d/f`; `us-east-1a`/`1e` were unavailable at run time). Server and
client share a cluster placement group, so both instance types must have capacity
in the chosen AZ.

### Test run:

```
dfbench run-commands-dfly \
--id maxqps2 --engines dragonfly,redis,valkey \
\
--command-template "RPUSH __key__ __data__" --command-name RPUSH --command-mutates \
--key-maximum 100000000 \
--test-time 300 --warmup-time 10 --trials 1 \
\
--dfly-bench-threads 32 --dfly-bench-conns 5 --dfly-bench-pipeline 30 \
--dfly-bench-data-size 128 --dfly-bench-key-dist U --dfly-bench-qps 0 \
\
--dragonfly_version s3://df-use1-pub/v2/dragonfly-aarch64.tar.gz \
--dragonfly_num_shards 16 --dragonfly_conn_use_incoming_cpu=true \
\
--redis_io_threads 8 --valkey_io_threads 8
```

### Expected results:

**RPUSH**

| Engine | Throughput (median) | p50 | p99 | p99.9 | Avg Latency |
| --------- | ------------------- | -------- | --------- | --------- | ----------- |
| dragonfly | 5.99M ops/s | 0.531 ms | 5.922 ms | 11.373 ms | 0.782 ms |
| redis | 698.8K ops/s | 6.573 ms | 11.298 ms | 17.824 ms | 6.856 ms |
| valkey | 893.5K ops/s | 5.197 ms | 11.570 ms | 15.476 ms | 5.359 ms |
43 changes: 43 additions & 0 deletions benchmarks/TTL/dfly_bench/TTL_reproduce.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
### Stateful setup:

```
dfbench setup --id maxqps2 --tune-network --ubuntu-version 24.04 \
--server-instance m7g.8xlarge --server-arch arm64 \
--client-instance c6gn.8xlarge --client-arch arm64 \
--engines dragonfly,redis,valkey --availability-zone us-east-1c
```

Note: pick an availability zone that has `c6gn.8xlarge` capacity (AWS suggested
`us-east-1b/c/d/f`; `us-east-1a`/`1e` were unavailable at run time). Server and
client share a cluster placement group, so both instance types must have capacity
in the chosen AZ.

### Test run:

```
dfbench run-commands-dfly \
--id maxqps2 --engines dragonfly,redis,valkey \
\
--command-template "TTL __key__" --command-name TTL \
--preload-template "SET __key__ __data__" \
--key-maximum 100000000 \
--test-time 60 --warmup-time 10 --trials 1 \
\
--dfly-bench-threads 32 --dfly-bench-conns 5 --dfly-bench-pipeline 30 \
--dfly-bench-data-size 128 --dfly-bench-key-dist U --dfly-bench-qps 0 \
\
--dragonfly_version s3://df-use1-pub/v2/dragonfly-aarch64.tar.gz \
--dragonfly_num_shards 16 --dragonfly_conn_use_incoming_cpu=true \
\
--redis_io_threads 8 --valkey_io_threads 8
```

### Expected results:

**TTL**

| Engine | Throughput (median) | p50 | p99 | p99.9 | Avg Latency |
| --------- | ------------------- | -------- | -------- | -------- | ----------- |
| dragonfly | 10.63M ops/s | 0.411 ms | 0.851 ms | 1.076 ms | 0.439 ms |
| redis | 1.94M ops/s | 2.325 ms | 2.980 ms | 3.000 ms | 2.462 ms |
| valkey | 2.33M ops/s | 2.225 ms | 2.495 ms | 2.499 ms | 2.049 ms |
Loading