Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 25 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -641,15 +641,36 @@ Because `dfly_bench` uses its own key prefix (`--key_prefix`, default
(sequential distribution, one connection per thread covering the whole
`--key-maximum` range) rather than reusing a memtier preload.

**Flag naming: client load generator vs. server engine.** The prefix tells
you what a flag tunes:

- `--memtier-*` (in `run-commands`) and `--dfly-bench-*` (in
`run-commands-dfly`) configure the **client-side load generator**
(`memtier_benchmark` and `dfly_bench` respectively). The two commands are
deliberately kept separate so it's always explicit which tool is driving
load.
- `--dragonfly_*`, `--redis_*`, `--valkey_*` configure the **server engine**
under test and behave identically in both commands. In particular,
`--dragonfly_num_shards` is Dragonfly server tuning, not a client flag.

So `--dfly-bench-threads` sets `dfly_bench`'s `--proactor_threads` on the
client, while `--dragonfly_num_shards` sets Dragonfly's shard count on the
server; they are unrelated.

> The `--dfly-bench-*` flags were previously named `--dfly-*`
> (`--dfly-threads`, `--dfly-conns`, `--dfly-pipeline`, `--dfly-data-size`).
> Those spellings still work as hidden, deprecated aliases (they print a
> deprecation notice); prefer the `--dfly-bench-*` names.

| Flag | Default | Notes |
| ---- | ------- | ----- |
| `--id` | (required) | Setup previously created by `dfbench setup`. |
| `--command` | (required) | Single command name, e.g. `SET`, `GET`, `LRANGE`. |
| `--engines` | `dragonfly` | Subset of the setup's installed engines (`dfly_bench` speaks RESP, so Redis/Valkey work too). |
| `--dfly-threads` | `32` | `dfly_bench --proactor_threads` (analogous to memtier threads). Total connections = threads * conns. |
| `--dfly-conns` | `5` | `dfly_bench -c`, connections per thread (analogous to memtier clients). |
| `--dfly-pipeline` | `20` | `dfly_bench --pipeline`: max pending requests per connection. |
| `--dfly-data-size` | `256` | `dfly_bench -d`: value size in bytes (`__data__` placeholders). |
| `--dfly-bench-threads` | `32` | `dfly_bench --proactor_threads` (analogous to memtier threads). Total connections = threads * conns. |
| `--dfly-bench-conns` | `5` | `dfly_bench -c`, connections per thread (analogous to memtier clients). |
| `--dfly-bench-pipeline` | `20` | `dfly_bench --pipeline`: max pending requests per connection. |
| `--dfly-bench-data-size` | `256` | `dfly_bench -d`: value size in bytes (`__data__` placeholders). |
| `--key-maximum` | `100000000` | Key range (`--key_maximum`). Structured preloads use a built-in 1M-key range. |
| `--key-dist` | `U` | `dfly_bench --key_dist` for the measured pass: `U` (uniform, the memtier random `R` equivalent), `N`, `Z`, `S`. |
| `--qps` | `0` | `dfly_bench --qps` per connection; `0` = closed-loop max load (coordinated omission). |
Expand Down
4 changes: 2 additions & 2 deletions benchmarks/GET/dfly_bench/GET_reproduce.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ in the chosen AZ.
| Load generator | `dfly_bench` (ref `main`) |
| Value size | 128 B |
| Keyspace | 100,000,000 keys (`--key_maximum`) |
| Connections | 160 (`--dfly-threads 32` x `--dfly-conns 5`) |
| Connections | 160 (`--dfly-bench-threads 32` x `--dfly-bench-conns 5`) |
| Key distribution | `U` (uniform, ≈ memtier random pattern `R`) |

## Test command (recommended config: pipeline 30, unthrottled)
Expand All @@ -53,7 +53,7 @@ dfbench run-commands-dfly --id maxqps2 --engines dragonfly,redis,valkey --comman
--dragonfly_version s3://df-use1-pub/v2/dragonfly-aarch64.tar.gz \
--dragonfly_num_shards 16 \
--key-maximum 100000000 --key-dist U \
--dfly-threads 32 --dfly-conns 5 --dfly-pipeline 30 --dfly-data-size 128 \
--dfly-bench-threads 32 --dfly-bench-conns 5 --dfly-bench-pipeline 30 --dfly-bench-data-size 128 \
--qps 0 --test-time 60 --warmup-time 10 --trials 1
```

Expand Down
4 changes: 2 additions & 2 deletions benchmarks/GET/memtier/GET_reproduce.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ load generator instead of memtier. dfly_bench is installed on the client at
dfbench run-commands-dfly --id maxqps2 --engines dragonfly --command GET \
--dragonfly_version s3://df-use1-pub/v2/dragonfly-aarch64.tar.gz \
--dragonfly_num_shards 16 \
--dfly-threads 32 --dfly-conns 5 --dfly-pipeline 120 --dfly-data-size 128 \
--dfly-bench-threads 32 --dfly-bench-conns 5 --dfly-bench-pipeline 120 --dfly-bench-data-size 128 \
--key-maximum 100000000 --key-dist U --qps 0 \
--test-time 60 --warmup-time 10 --trials 1
```
Expand All @@ -73,7 +73,7 @@ Notes:
SET preload is run by dfly_bench (sequential distribution, one connection per
thread covering the whole `--key_maximum` range) rather than reused from a
memtier preload.
- **Flag mapping:** `--dfly-threads` -> `--proactor_threads`, `--dfly-conns` ->
- **Flag mapping:** `--dfly-bench-threads` -> `--proactor_threads`, `--dfly-bench-conns` ->
`-c` (32x5 = 160 connections), `--key-dist U` approximates memtier's random
key pattern `R`, `--qps 0` = closed-loop max load.
- Results use the same schema (tagged `tool="dfly_bench"`), so
Expand Down
4 changes: 2 additions & 2 deletions benchmarks/SET/dfly_bench/SET_reproduce.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ instance types must have capacity in the chosen AZ.
| Load generator | `dfly_bench` (ref `main`) |
| Value size | 128 B |
| Keyspace | 100,000,000 keys (`--key_maximum`) |
| Connections | 160 (`--dfly-threads 32` x `--dfly-conns 5`) |
| Connections | 160 (`--dfly-bench-threads 32` x `--dfly-bench-conns 5`) |
| Key distribution | `U` (uniform, ≈ memtier random pattern `R`) |

## Test command
Expand All @@ -48,7 +48,7 @@ dfbench run-commands-dfly --id maxqps2 --engines dragonfly,redis,valkey --comman
--dragonfly_version s3://df-use1-pub/v2/dragonfly-aarch64.tar.gz \
--dragonfly_num_shards 16 --dragonfly_conn_use_incoming_cpu=true \
--key-maximum 100000000 --key-dist U \
--dfly-threads 32 --dfly-conns 5 --dfly-pipeline 30 --dfly-data-size 128 \
--dfly-bench-threads 32 --dfly-bench-conns 5 --dfly-bench-pipeline 30 --dfly-bench-data-size 128 \
--qps 0 --test-time 300 --warmup-time 10 --trials 1 \
--redis_io_threads 8 --valkey_io_threads 8
```
Expand Down
4 changes: 2 additions & 2 deletions benchmarks/SET/memtier/SET_reproduce.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,12 +36,12 @@ load generator instead of memtier. dfly_bench is installed on the client at
```
dfbench run-commands-dfly --id test --engines dragonfly --command SET \
--dragonfly_num_shards 16 \
--dfly-threads 32 --dfly-conns 5 --dfly-pipeline 30 --dfly-data-size 128 \
--dfly-bench-threads 32 --dfly-bench-conns 5 --dfly-bench-pipeline 30 --dfly-bench-data-size 128 \
--key-maximum 100000000 --qps 0 \
--test-time 300 --trials 1 --dragonfly_version s3://df-use1-pub/v2/dragonfly-aarch64.tar.gz
```

Flag mapping vs memtier: `--dfly-threads` -> `--proactor_threads`, `--dfly-conns`
Flag mapping vs memtier: `--dfly-bench-threads` -> `--proactor_threads`, `--dfly-bench-conns`
-> `-c` (so 32x5 = 160 connections, same as memtier threadsxclients), `--qps 0`
= closed-loop max load. Results use the same schema (tagged `tool="dfly_bench"`),
so `dfbench compare <memtier-set.json> <dflybench-set.json>` diffs the two tools.
22 changes: 18 additions & 4 deletions cmd/run_commands_dfly.go
Original file line number Diff line number Diff line change
Expand Up @@ -132,10 +132,24 @@ compare'.`,
"Comma-separated list of engines to benchmark (must be a subset of the setup's installed engines)")
f.StringVar(&commandFlag, "command", "",
"Command to benchmark (required), one of: "+strings.Join(commands.SupportedNames(), ","))
f.IntVar(&threads, "dfly-threads", 32, "dfly_bench --proactor_threads (analogous to memtier threads)")
f.IntVar(&conns, "dfly-conns", 5, "dfly_bench -c, connections per thread (analogous to memtier clients)")
f.IntVar(&pipeline, "dfly-pipeline", 20, "dfly_bench --pipeline: max pending requests per connection")
f.IntVar(&dataSize, "dfly-data-size", 256, "dfly_bench -d: value size in bytes (fills __data__ placeholders)")
// dfly_bench (client load generator) flags. Named --dfly-bench-* to make
// it explicit these tune the dfly_bench tool, not the Dragonfly server
// (--dragonfly_*), and to parallel run-commands' --memtier-* flags.
f.IntVar(&threads, "dfly-bench-threads", 32, "dfly_bench --proactor_threads (analogous to memtier threads)")
f.IntVar(&conns, "dfly-bench-conns", 5, "dfly_bench -c, connections per thread (analogous to memtier clients)")
f.IntVar(&pipeline, "dfly-bench-pipeline", 20, "dfly_bench --pipeline: max pending requests per connection")
f.IntVar(&dataSize, "dfly-bench-data-size", 256, "dfly_bench -d: value size in bytes (fills __data__ placeholders)")

// Deprecated pre-rename aliases: keep old commands working. Hidden from
// help and print a deprecation notice when used.
f.IntVar(&threads, "dfly-threads", 32, "Deprecated alias for --dfly-bench-threads")
f.IntVar(&conns, "dfly-conns", 5, "Deprecated alias for --dfly-bench-conns")
f.IntVar(&pipeline, "dfly-pipeline", 20, "Deprecated alias for --dfly-bench-pipeline")
f.IntVar(&dataSize, "dfly-data-size", 256, "Deprecated alias for --dfly-bench-data-size")
_ = f.MarkDeprecated("dfly-threads", "use --dfly-bench-threads")
_ = f.MarkDeprecated("dfly-conns", "use --dfly-bench-conns")
_ = f.MarkDeprecated("dfly-pipeline", "use --dfly-bench-pipeline")
_ = f.MarkDeprecated("dfly-data-size", "use --dfly-bench-data-size")
f.IntVar(&keyMaximum, "key-maximum", 100_000_000,
"Key range (--key_maximum). Size it to 30-90% of server memory. Commands with multi-item structured preloads use a built-in 1M-key range instead.")
f.StringVar(&keyDist, "key-dist", "U",
Expand Down
Loading