From f9eaa54cf4db792e93b557b5f30c63fd0de93fd1 Mon Sep 17 00:00:00 2001 From: Mile Druzijanic Date: Mon, 27 Jul 2026 11:30:39 +0200 Subject: [PATCH] dflybench flags update --- README.md | 29 +++++++++++++++++++--- benchmarks/GET/dfly_bench/GET_reproduce.md | 4 +-- benchmarks/GET/memtier/GET_reproduce.md | 4 +-- benchmarks/SET/dfly_bench/SET_reproduce.md | 4 +-- benchmarks/SET/memtier/SET_reproduce.md | 4 +-- cmd/run_commands_dfly.go | 22 +++++++++++++--- 6 files changed, 51 insertions(+), 16 deletions(-) diff --git a/README.md b/README.md index e395d1e..2459e98 100644 --- a/README.md +++ b/README.md @@ -641,15 +641,36 @@ Because `dfly_bench` uses its own key prefix (`--key_prefix`, default (sequential distribution, one connection per thread covering the whole `--key-maximum` range) rather than reusing a memtier preload. +**Flag naming: client load generator vs. server engine.** The prefix tells +you what a flag tunes: + +- `--memtier-*` (in `run-commands`) and `--dfly-bench-*` (in + `run-commands-dfly`) configure the **client-side load generator** + (`memtier_benchmark` and `dfly_bench` respectively). The two commands are + deliberately kept separate so it's always explicit which tool is driving + load. +- `--dragonfly_*`, `--redis_*`, `--valkey_*` configure the **server engine** + under test and behave identically in both commands. In particular, + `--dragonfly_num_shards` is Dragonfly server tuning, not a client flag. + +So `--dfly-bench-threads` sets `dfly_bench`'s `--proactor_threads` on the +client, while `--dragonfly_num_shards` sets Dragonfly's shard count on the +server; they are unrelated. + +> The `--dfly-bench-*` flags were previously named `--dfly-*` +> (`--dfly-threads`, `--dfly-conns`, `--dfly-pipeline`, `--dfly-data-size`). +> Those spellings still work as hidden, deprecated aliases (they print a +> deprecation notice); prefer the `--dfly-bench-*` names. + | Flag | Default | Notes | | ---- | ------- | ----- | | `--id` | (required) | Setup previously created by `dfbench setup`. | | `--command` | (required) | Single command name, e.g. `SET`, `GET`, `LRANGE`. | | `--engines` | `dragonfly` | Subset of the setup's installed engines (`dfly_bench` speaks RESP, so Redis/Valkey work too). | -| `--dfly-threads` | `32` | `dfly_bench --proactor_threads` (analogous to memtier threads). Total connections = threads * conns. | -| `--dfly-conns` | `5` | `dfly_bench -c`, connections per thread (analogous to memtier clients). | -| `--dfly-pipeline` | `20` | `dfly_bench --pipeline`: max pending requests per connection. | -| `--dfly-data-size` | `256` | `dfly_bench -d`: value size in bytes (`__data__` placeholders). | +| `--dfly-bench-threads` | `32` | `dfly_bench --proactor_threads` (analogous to memtier threads). Total connections = threads * conns. | +| `--dfly-bench-conns` | `5` | `dfly_bench -c`, connections per thread (analogous to memtier clients). | +| `--dfly-bench-pipeline` | `20` | `dfly_bench --pipeline`: max pending requests per connection. | +| `--dfly-bench-data-size` | `256` | `dfly_bench -d`: value size in bytes (`__data__` placeholders). | | `--key-maximum` | `100000000` | Key range (`--key_maximum`). Structured preloads use a built-in 1M-key range. | | `--key-dist` | `U` | `dfly_bench --key_dist` for the measured pass: `U` (uniform, the memtier random `R` equivalent), `N`, `Z`, `S`. | | `--qps` | `0` | `dfly_bench --qps` per connection; `0` = closed-loop max load (coordinated omission). | diff --git a/benchmarks/GET/dfly_bench/GET_reproduce.md b/benchmarks/GET/dfly_bench/GET_reproduce.md index fa23d50..2341288 100644 --- a/benchmarks/GET/dfly_bench/GET_reproduce.md +++ b/benchmarks/GET/dfly_bench/GET_reproduce.md @@ -43,7 +43,7 @@ in the chosen AZ. | Load generator | `dfly_bench` (ref `main`) | | Value size | 128 B | | Keyspace | 100,000,000 keys (`--key_maximum`) | -| Connections | 160 (`--dfly-threads 32` x `--dfly-conns 5`) | +| Connections | 160 (`--dfly-bench-threads 32` x `--dfly-bench-conns 5`) | | Key distribution | `U` (uniform, ≈ memtier random pattern `R`) | ## Test command (recommended config: pipeline 30, unthrottled) @@ -53,7 +53,7 @@ dfbench run-commands-dfly --id maxqps2 --engines dragonfly,redis,valkey --comman --dragonfly_version s3://df-use1-pub/v2/dragonfly-aarch64.tar.gz \ --dragonfly_num_shards 16 \ --key-maximum 100000000 --key-dist U \ - --dfly-threads 32 --dfly-conns 5 --dfly-pipeline 30 --dfly-data-size 128 \ + --dfly-bench-threads 32 --dfly-bench-conns 5 --dfly-bench-pipeline 30 --dfly-bench-data-size 128 \ --qps 0 --test-time 60 --warmup-time 10 --trials 1 ``` diff --git a/benchmarks/GET/memtier/GET_reproduce.md b/benchmarks/GET/memtier/GET_reproduce.md index 6319c37..6fd1b76 100644 --- a/benchmarks/GET/memtier/GET_reproduce.md +++ b/benchmarks/GET/memtier/GET_reproduce.md @@ -62,7 +62,7 @@ load generator instead of memtier. dfly_bench is installed on the client at dfbench run-commands-dfly --id maxqps2 --engines dragonfly --command GET \ --dragonfly_version s3://df-use1-pub/v2/dragonfly-aarch64.tar.gz \ --dragonfly_num_shards 16 \ - --dfly-threads 32 --dfly-conns 5 --dfly-pipeline 120 --dfly-data-size 128 \ + --dfly-bench-threads 32 --dfly-bench-conns 5 --dfly-bench-pipeline 120 --dfly-bench-data-size 128 \ --key-maximum 100000000 --key-dist U --qps 0 \ --test-time 60 --warmup-time 10 --trials 1 ``` @@ -73,7 +73,7 @@ Notes: SET preload is run by dfly_bench (sequential distribution, one connection per thread covering the whole `--key_maximum` range) rather than reused from a memtier preload. -- **Flag mapping:** `--dfly-threads` -> `--proactor_threads`, `--dfly-conns` -> +- **Flag mapping:** `--dfly-bench-threads` -> `--proactor_threads`, `--dfly-bench-conns` -> `-c` (32x5 = 160 connections), `--key-dist U` approximates memtier's random key pattern `R`, `--qps 0` = closed-loop max load. - Results use the same schema (tagged `tool="dfly_bench"`), so diff --git a/benchmarks/SET/dfly_bench/SET_reproduce.md b/benchmarks/SET/dfly_bench/SET_reproduce.md index f739a6e..c99bdd9 100644 --- a/benchmarks/SET/dfly_bench/SET_reproduce.md +++ b/benchmarks/SET/dfly_bench/SET_reproduce.md @@ -38,7 +38,7 @@ instance types must have capacity in the chosen AZ. | Load generator | `dfly_bench` (ref `main`) | | Value size | 128 B | | Keyspace | 100,000,000 keys (`--key_maximum`) | -| Connections | 160 (`--dfly-threads 32` x `--dfly-conns 5`) | +| Connections | 160 (`--dfly-bench-threads 32` x `--dfly-bench-conns 5`) | | Key distribution | `U` (uniform, ≈ memtier random pattern `R`) | ## Test command @@ -48,7 +48,7 @@ dfbench run-commands-dfly --id maxqps2 --engines dragonfly,redis,valkey --comman --dragonfly_version s3://df-use1-pub/v2/dragonfly-aarch64.tar.gz \ --dragonfly_num_shards 16 --dragonfly_conn_use_incoming_cpu=true \ --key-maximum 100000000 --key-dist U \ - --dfly-threads 32 --dfly-conns 5 --dfly-pipeline 30 --dfly-data-size 128 \ + --dfly-bench-threads 32 --dfly-bench-conns 5 --dfly-bench-pipeline 30 --dfly-bench-data-size 128 \ --qps 0 --test-time 300 --warmup-time 10 --trials 1 \ --redis_io_threads 8 --valkey_io_threads 8 ``` diff --git a/benchmarks/SET/memtier/SET_reproduce.md b/benchmarks/SET/memtier/SET_reproduce.md index 7ca3655..cc78f31 100644 --- a/benchmarks/SET/memtier/SET_reproduce.md +++ b/benchmarks/SET/memtier/SET_reproduce.md @@ -36,12 +36,12 @@ load generator instead of memtier. dfly_bench is installed on the client at ``` dfbench run-commands-dfly --id test --engines dragonfly --command SET \ --dragonfly_num_shards 16 \ - --dfly-threads 32 --dfly-conns 5 --dfly-pipeline 30 --dfly-data-size 128 \ + --dfly-bench-threads 32 --dfly-bench-conns 5 --dfly-bench-pipeline 30 --dfly-bench-data-size 128 \ --key-maximum 100000000 --qps 0 \ --test-time 300 --trials 1 --dragonfly_version s3://df-use1-pub/v2/dragonfly-aarch64.tar.gz ``` -Flag mapping vs memtier: `--dfly-threads` -> `--proactor_threads`, `--dfly-conns` +Flag mapping vs memtier: `--dfly-bench-threads` -> `--proactor_threads`, `--dfly-bench-conns` -> `-c` (so 32x5 = 160 connections, same as memtier threadsxclients), `--qps 0` = closed-loop max load. Results use the same schema (tagged `tool="dfly_bench"`), so `dfbench compare ` diffs the two tools. diff --git a/cmd/run_commands_dfly.go b/cmd/run_commands_dfly.go index bddf980..8264ff8 100644 --- a/cmd/run_commands_dfly.go +++ b/cmd/run_commands_dfly.go @@ -132,10 +132,24 @@ compare'.`, "Comma-separated list of engines to benchmark (must be a subset of the setup's installed engines)") f.StringVar(&commandFlag, "command", "", "Command to benchmark (required), one of: "+strings.Join(commands.SupportedNames(), ",")) - f.IntVar(&threads, "dfly-threads", 32, "dfly_bench --proactor_threads (analogous to memtier threads)") - f.IntVar(&conns, "dfly-conns", 5, "dfly_bench -c, connections per thread (analogous to memtier clients)") - f.IntVar(&pipeline, "dfly-pipeline", 20, "dfly_bench --pipeline: max pending requests per connection") - f.IntVar(&dataSize, "dfly-data-size", 256, "dfly_bench -d: value size in bytes (fills __data__ placeholders)") + // dfly_bench (client load generator) flags. Named --dfly-bench-* to make + // it explicit these tune the dfly_bench tool, not the Dragonfly server + // (--dragonfly_*), and to parallel run-commands' --memtier-* flags. + f.IntVar(&threads, "dfly-bench-threads", 32, "dfly_bench --proactor_threads (analogous to memtier threads)") + f.IntVar(&conns, "dfly-bench-conns", 5, "dfly_bench -c, connections per thread (analogous to memtier clients)") + f.IntVar(&pipeline, "dfly-bench-pipeline", 20, "dfly_bench --pipeline: max pending requests per connection") + f.IntVar(&dataSize, "dfly-bench-data-size", 256, "dfly_bench -d: value size in bytes (fills __data__ placeholders)") + + // Deprecated pre-rename aliases: keep old commands working. Hidden from + // help and print a deprecation notice when used. + f.IntVar(&threads, "dfly-threads", 32, "Deprecated alias for --dfly-bench-threads") + f.IntVar(&conns, "dfly-conns", 5, "Deprecated alias for --dfly-bench-conns") + f.IntVar(&pipeline, "dfly-pipeline", 20, "Deprecated alias for --dfly-bench-pipeline") + f.IntVar(&dataSize, "dfly-data-size", 256, "Deprecated alias for --dfly-bench-data-size") + _ = f.MarkDeprecated("dfly-threads", "use --dfly-bench-threads") + _ = f.MarkDeprecated("dfly-conns", "use --dfly-bench-conns") + _ = f.MarkDeprecated("dfly-pipeline", "use --dfly-bench-pipeline") + _ = f.MarkDeprecated("dfly-data-size", "use --dfly-bench-data-size") f.IntVar(&keyMaximum, "key-maximum", 100_000_000, "Key range (--key_maximum). Size it to 30-90% of server memory. Commands with multi-item structured preloads use a built-in 1M-key range instead.") f.StringVar(&keyDist, "key-dist", "U",