From b025df33d5778fdb6f6fe9635ff750771418b98b Mon Sep 17 00:00:00 2001 From: Cliff Burdick Date: Fri, 17 Jul 2026 15:03:27 +0000 Subject: [PATCH 1/2] #176 - Add ibverbs hardware packet pacing Signed-off-by: Cliff Burdick --- AGENTS.md | 4 +- docs/api-reference/configuration.md | 13 ++-- docs/benchmarks/raw_benchmarking.md | 15 ++-- docs/tutorials/configuration-walkthrough.md | 4 +- examples/daqiri_bench_raw_tx_rx_pacing.yaml | 13 ++-- include/daqiri/types.h | 4 +- src/engines/ibverbs/daqiri_ibverbs_engine.cpp | 75 ++++++++++++++++--- src/engines/ibverbs/daqiri_ibverbs_engine.h | 1 + 8 files changed, 92 insertions(+), 37 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 129f1da..018ea8a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -35,7 +35,7 @@ There is no unit test suite. Verification is done via the benchmark executables | Executable | Source | Typical config | |---|---|---| -| `daqiri_bench_raw_gpudirect` | `raw_gpudirect_bench.cpp` | `daqiri_bench_raw_tx_rx.yaml`, `daqiri_bench_raw_tx_rx_4q.yaml`, `daqiri_bench_raw_tx_rx_spark.yaml`, `daqiri_bench_raw_{tx,rx}_spark_xhost.yaml`, `daqiri_bench_raw_sw_loopback.yaml`, `daqiri_bench_raw_rx_multi_q.yaml`, `daqiri_bench_raw_tx_rx_vxlan.yaml`, `daqiri_bench_raw_tx_rx_vlan.yaml`, `daqiri_bench_raw_tx_rx_gre.yaml`, `daqiri_bench_raw_tx_rx_nvgre.yaml`, `daqiri_bench_raw_tx_rx_spark_mq.yaml` (mq base; `run_spark_mq_bench.sh` derives the 4 cells via `scripts/gen_spark_mq_config.py`), `daqiri_bench_raw_tx_rx_pacing.yaml` (per-queue `pacing_mbps`; DPDK engine only) | +| `daqiri_bench_raw_gpudirect` | `raw_gpudirect_bench.cpp` | `daqiri_bench_raw_tx_rx.yaml`, `daqiri_bench_raw_tx_rx_4q.yaml`, `daqiri_bench_raw_tx_rx_spark.yaml`, `daqiri_bench_raw_{tx,rx}_spark_xhost.yaml`, `daqiri_bench_raw_sw_loopback.yaml`, `daqiri_bench_raw_rx_multi_q.yaml`, `daqiri_bench_raw_tx_rx_vxlan.yaml`, `daqiri_bench_raw_tx_rx_vlan.yaml`, `daqiri_bench_raw_tx_rx_gre.yaml`, `daqiri_bench_raw_tx_rx_nvgre.yaml`, `daqiri_bench_raw_tx_rx_spark_mq.yaml` (mq base; `run_spark_mq_bench.sh` derives the 4 cells via `scripts/gen_spark_mq_config.py`), `daqiri_bench_raw_tx_rx_pacing.yaml` (per-queue `pacing_mbps`; DPDK or ibverbs raw engine) | | `daqiri_example_dynamic_rx_flow` | `dynamic_rx_flow_example.cpp` | `daqiri_example_dynamic_rx_flow.yaml` — `flow_isolation: true` startup followed by runtime RX queue-steering and raw-engine decap/pop flow add/delete | | `daqiri_bench_raw_hds` | `raw_hds_bench.cpp` | `daqiri_bench_raw_tx_rx_hds.yaml` | | `daqiri_bench_raw_reorder_seq` | `raw_reorder_seq_bench.cpp` | `daqiri_bench_raw_tx_rx_reorder_seq_1024*.yaml`, `daqiri_bench_raw_rx_reorder_seq_*.yaml` | @@ -81,7 +81,7 @@ The always-built socket engine implements Linux UDP/TCP streams directly. Applic The default `dpdk` raw engine (`src/engines/dpdk/`, `DpdkEngine`) programs RX steering, send-to-kernel fallbacks (`flow_isolation: true`), and `tx_eth_src` TX offloads via DPDK RTE Flow during `daqiri_init()`. Standard UDP/IP (group 3), flex-item (group 1) and eCPRI-over-Ethernet (group 2, EtherType `0xAEFE`, via `RTE_FLOW_ITEM_TYPE_ECPRI` matching message type and pc_id/rtc_id) RX flows use separate flow groups; `validate_config()` rejects mixing these flow classes per interface, duplicate `flex_item_id` values, and unknown `action.id` / `flex_item_id` values before NIC programming. The mlx5 PMD honors the native eCPRI flow item only under **firmware steering** (`dv_flow_en=1`); under HW steering (`dv_flow_en=2`, the default) the rule installs but silently never matches on ConnectX-class NICs. So `initialize()` auto-switches any interface carrying eCPRI RX flows to `dv_flow_en=1` (with a `WARN`), which means the async/template dynamic-RX-flow path is unavailable on that port. Flex-item parser handles are created per `(port, flex_item_id)` (scoped per interface). All programmed `rte_flow` rules and flex-item handles are tracked and destroyed in order on shutdown, init failure, and engine teardown (programmed flows → flex items → group-0 ETH jump rules). See `docs/benchmarks/raw_benchmarking.md` (Flow programming smoke test) for manual verification steps. -The `ibverbs` raw engine (`src/engines/ibverbs/`, `IbverbsEngine`) drives a Mellanox/mlx5 Multi-Packet (striding) Receive Queue via **DevX** (`mlx5dv_devx_obj_create` against vendored PRM structs in `mlx5_prm_min.h`): a DevX CQ + striding RQ + TIR + `mlx5dv_dr` flow steering, with manual WQE/doorbell management and worker-driven cyclic refill. RX packets DMA strided into one pre-posted MR (host or GPU via `ibv_reg_dmabuf_mr`); a queue with >1 memory region instead uses a non-striding DevX *regular* RQ with multi-segment scatter WQEs for **physical** header-data split (header → CPU MR, payload → GPU MR). TX builds mlx5 send WQEs directly on a raw-packet QP's SQ (via `mlx5dv_init_obj`, bypassing `ibv_post_send`) from a slab of registered slots tracked by cyclic index counters, with NIC checksum offload and a `tx_eth_src` offload. It uses the libdpdk-free `daqiri::Ring`/`daqiri::ObjectPool` for the worker→app burst handoff (like the rdma engine — neither links DPDK) and drives the NIC through libibverbs/mlx5dv directly. Feature set: RX (MPRQ), TX, GPUDirect, physical/logical HDS, multi-queue 5-tuple flow steering with per-packet flow IDs, flex-item arbitrary-offset, IPv4-total-length and eCPRI-over-Ethernet (EtherType `0xAEFE`, message type + pc_id/rtc_id) flow matching (mlx5 flex parser / `misc_parameters_4`), per-packet RX hardware timestamps, accurate TX send scheduling (wait-on-time WAIT WQE), and GPU reorder/quantize. Because it uses the kernel netdev directly, `ensure_port_mtus` raises the netdev MTU at init to cover the configured frame size in either direction — RX (post-decap) and TX egress (post-encap) — sizing each direction with its own transform wire overhead (jumbo frames silently drop otherwise). Queues sharing a `cpu_core` are serviced round-robin by one poller thread. +The `ibverbs` raw engine (`src/engines/ibverbs/`, `IbverbsEngine`) drives a Mellanox/mlx5 Multi-Packet (striding) Receive Queue via **DevX** (`mlx5dv_devx_obj_create` against vendored PRM structs in `mlx5_prm_min.h`): a DevX CQ + striding RQ + TIR + `mlx5dv_dr` flow steering, with manual WQE/doorbell management and worker-driven cyclic refill. RX packets DMA strided into one pre-posted MR (host or GPU via `ibv_reg_dmabuf_mr`); a queue with >1 memory region instead uses a non-striding DevX *regular* RQ with multi-segment scatter WQEs for **physical** header-data split (header → CPU MR, payload → GPU MR). TX builds mlx5 send WQEs directly on a raw-packet QP's SQ (via `mlx5dv_init_obj`, bypassing `ibv_post_send`) from a slab of registered slots tracked by cyclic index counters, with NIC checksum offload and a `tx_eth_src` offload. It uses the libdpdk-free `daqiri::Ring`/`daqiri::ObjectPool` for the worker→app burst handoff (like the rdma engine — neither links DPDK) and drives the NIC through libibverbs/mlx5dv directly. Feature set: RX (MPRQ), TX, GPUDirect, physical/logical HDS, multi-queue 5-tuple flow steering with per-packet flow IDs, flex-item arbitrary-offset, IPv4-total-length and eCPRI-over-Ethernet (EtherType `0xAEFE`, message type + pc_id/rtc_id) flow matching (mlx5 flex parser / `misc_parameters_4`), per-packet RX hardware timestamps, per-queue hardware packet pacing through the mlx5 QP rate table, accurate TX send scheduling (wait-on-time WAIT WQE), and GPU reorder/quantize. Packet pacing and accurate send are independent: `pacing_mbps` configures the QP's average rate, while `set_packet_tx_time()` emits WAIT WQEs for absolute per-packet times. Because it uses the kernel netdev directly, `ensure_port_mtus` raises the netdev MTU at init to cover the configured frame size in either direction — RX (post-decap) and TX egress (post-encap) — sizing each direction with its own transform wire overhead (jumbo frames silently drop otherwise). Queues sharing a `cpu_core` are serviced round-robin by one poller thread. ### Zero-copy / BurstParams All packet data flows through `BurstParams`, a batch of packets. Only pointers are passed between NIC, DAQIRI internals, and the application — the caller reads directly from the buffers the NIC DMA'd into. **The caller must explicitly free bursts**; a missed free drains the pool and produces `NO_FREE_BURST_BUFFERS` / `NO_FREE_PACKET_BUFFERS` errors and NIC drops. See `docs/concepts.md` (Zero-Copy Ownership) and `docs/api-reference/cpp.md` (free-function call patterns). diff --git a/docs/api-reference/configuration.md b/docs/api-reference/configuration.md index c49323b..f6ac7a9 100644 --- a/docs/api-reference/configuration.md +++ b/docs/api-reference/configuration.md @@ -400,12 +400,13 @@ daqiri::set_reorder_cuda_stream("rx_port", "rx_reorder_0", stream); - values: `tx_eth_src` (auto-fill source MAC address) - **`pacing_mbps`**: Packet-pacing rate cap for this queue, in megabits per second of L2 frame bytes (the data the application transmits, excluding preamble/IFG/FCS). The NIC meters the queue - out so its long-run average TX rate stays at or below this value. The limit is enforced on an - average basis and idle gaps do not accumulate burst credit. `0` (the default) disables pacing - and sends at line rate. Supported only by the default `dpdk` raw engine on a NIC with hardware - send scheduling (ConnectX-7 or later); on devices without it, `pacing_mbps` is ignored with a - warning and TX runs at line rate. The `ibverbs` raw engine does **not** support `pacing_mbps` - and `daqiri_init()` fails if it is set on an `ibverbs` queue. + out so its long-run average TX rate stays at or below this value. `0` (the default) disables + pacing and sends at line rate. Both raw engines support this setting, with different capability + requirements: `dpdk` uses hardware send scheduling and falls back to line rate with a warning + when that offload is unavailable; `ibverbs` assigns the QP to an mlx5 hardware packet-pacing + rate-table entry and fails initialization if RAW_PACKET pacing is unavailable or the requested + rate is outside the device's advertised range. The ibverbs engine leaves the optional burst + bound and typical-packet-size fields at their device defaults. - type: `integer` - default: `0` diff --git a/docs/benchmarks/raw_benchmarking.md b/docs/benchmarks/raw_benchmarking.md index b9faabe..2a187d5 100644 --- a/docs/benchmarks/raw_benchmarking.md +++ b/docs/benchmarks/raw_benchmarking.md @@ -260,10 +260,11 @@ flow programming test. To meter the transmit side at a fixed rate in hardware, set a per-queue `pacing_mbps` cap on the TX queue. [`daqiri_bench_raw_tx_rx_pacing.yaml`](https://github.com/nvidia/daqiri/blob/main/examples/daqiri_bench_raw_tx_rx_pacing.yaml) is the loopback config above with `pacing_mbps: 10000` (10 Gbps) on the TX queue. Pacing is -supported only on the default DPDK engine. The `ibverbs` engine does not support `pacing_mbps` -(init fails if it is set on an `ibverbs` queue). The NIC meters the queue out so its average -TX rate stays at or below the configured value. The limit is enforced on an average basis and -idle gaps do not accumulate burst credit. +supported by both raw engines. The example uses the default DPDK engine; add +`engine: "ibverbs"` beside `stream_type: "raw"` to exercise the ibverbs path. The NIC meters the +queue out so its average TX rate stays at or below the configured value. The ibverbs path uses +the mlx5 packet-pacing rate table rather than per-packet WAIT WQEs; firmware defaults determine +the allowed burst size unless configured outside DAQIRI. ```bash /opt/daqiri/bin/daqiri_bench_raw_gpudirect /opt/daqiri/bin/daqiri_bench_raw_tx_rx_pacing.yaml --seconds 10 @@ -274,8 +275,10 @@ Validate the cap from the `RX complete:` line: `Gbps = bytes * 8 / seconds / 1e9 link speed. Change `pacing_mbps` (or set it to `0` to disable pacing and send at line rate) and re-run to see the cap move. -Pacing requires a Mellanox/mlx5 NIC with hardware send scheduling (ConnectX-7 or later). On -devices without it, `pacing_mbps` is ignored with a warning at init and TX runs at line rate. +The DPDK path requires hardware send scheduling; if unavailable, it logs a warning and runs at +line rate. The ibverbs path requires packet-pacing support for RAW_PACKET QPs and a requested rate +within the range reported by the device; otherwise initialization fails instead of silently +ignoring the cap. ## Tune RDMA SEND completion signaling diff --git a/docs/tutorials/configuration-walkthrough.md b/docs/tutorials/configuration-walkthrough.md index 54c99ac..5d92a21 100644 --- a/docs/tutorials/configuration-walkthrough.md +++ b/docs/tutorials/configuration-walkthrough.md @@ -135,9 +135,9 @@ For a shorter selection guide, start with the [Benchmarking overview](../benchma ??? question "6. I need to cap (pace) the transmit rate in hardware" - [`daqiri_bench_raw_tx_rx_pacing.yaml`](https://github.com/nvidia/daqiri/blob/main/examples/daqiri_bench_raw_tx_rx_pacing.yaml) (runs on `daqiri_bench_raw_gpudirect`). - This is the base TX+RX template with a per-queue `pacing_mbps` cap added to the TX queue (it uses the default DPDK engine. Pacing is supported only on the DPDK engine. The `ibverbs` engine does not support `pacing_mbps`, and `daqiri_init()` fails if it is set on an `ibverbs` queue). The NIC meters the queue out so its average TX rate stays at or below the configured Mbps. The limit is enforced on an average basis and idle gaps do not accumulate burst credit. Set `pacing_mbps: 0` (or remove it) to send at line rate. Validate by computing the achieved rate from the benchmark's RX line (`Gbps = bytes * 8 / seconds / 1e9`) and confirming it tracks the configured cap. See the `pacing_mbps` key in the [TX queue configuration](../api-reference/configuration.md#transmit-configuration-tx). + This is the base TX+RX template with a per-queue `pacing_mbps` cap added to the TX queue. It uses the default DPDK engine; add `engine: "ibverbs"` beside `stream_type: "raw"` to use the mlx5 QP packet-pacing rate table instead. The NIC meters the queue out so its average TX rate stays at or below the configured Mbps. Set `pacing_mbps: 0` (or remove it) to send at line rate. Validate by computing the achieved rate from the benchmark's RX line (`Gbps = bytes * 8 / seconds / 1e9`) and confirming it tracks the configured cap. See the `pacing_mbps` key in the [TX queue configuration](../api-reference/configuration.md#transmit-configuration-tx). - *Requires: a Mellanox/mlx5 NIC with hardware send scheduling (ConnectX-7 or later). On devices without it, `pacing_mbps` is ignored with a warning and TX runs at line rate.* + *Requires the engine's pacing capability: DPDK warns and runs at line rate without its send-scheduling offload; ibverbs fails initialization unless the device advertises RAW_PACKET packet pacing and the rate is in range.* ## Annotated walkthrough diff --git a/examples/daqiri_bench_raw_tx_rx_pacing.yaml b/examples/daqiri_bench_raw_tx_rx_pacing.yaml index 890d449..ed11bf1 100644 --- a/examples/daqiri_bench_raw_tx_rx_pacing.yaml +++ b/examples/daqiri_bench_raw_tx_rx_pacing.yaml @@ -5,14 +5,13 @@ # This is daqiri_bench_raw_tx_rx.yaml with a per-queue pacing_mbps cap added to # the TX queue (no "engine" key, so it uses the default raw engine, DPDK). The # NIC meters the queue out so its average TX rate stays at or below the -# configured value; the limit is enforced on an average basis and idle gaps do -# not accumulate burst credit. Pacing is supported only on the default DPDK raw -# engine; the ibverbs engine does not support pacing_mbps (daqiri_init fails if -# it is set on an ibverbs queue). +# configured value. To exercise the ibverbs path, add `engine: "ibverbs"` beside +# `stream_type: "raw"`; it programs the mlx5 hardware packet-pacing rate table +# for the TX QP and uses the device defaults for burst sizing. # -# Requirements: a Mellanox/mlx5 NIC with hardware send scheduling (ConnectX-7 or -# later). On devices without it, pacing_mbps is ignored with a warning at init -# and TX runs at line rate. +# Requirements: the DPDK path needs its hardware send-scheduling offload and +# warns/runs at line rate without it. The ibverbs path requires RAW_PACKET packet +# pacing and a rate within the device's advertised range; otherwise init fails. # # Validate: run daqiri_bench_raw_gpudirect with this config and compute the # achieved rate from the RX line's bytes/seconds (Gbps = bytes * 8 / seconds / diff --git a/include/daqiri/types.h b/include/daqiri/types.h index 14b9fbb..337239e 100644 --- a/include/daqiri/types.h +++ b/include/daqiri/types.h @@ -631,8 +631,8 @@ struct RxQueueConfig { struct TxQueueConfig { CommonQueueConfig common_; // Packet pacing: average TX rate cap in megabits/sec (L2 frame bytes). 0 - // disables pacing (line-rate). Honored only by engines/devices with accurate - // send scheduling (wait-on-time + real-time clock). + // disables pacing (line-rate). Honored only by engines/devices with hardware + // packet-pacing support. uint64_t pacing_mbps_ = 0; }; diff --git a/src/engines/ibverbs/daqiri_ibverbs_engine.cpp b/src/engines/ibverbs/daqiri_ibverbs_engine.cpp index de4c6de..77d43ee 100644 --- a/src/engines/ibverbs/daqiri_ibverbs_engine.cpp +++ b/src/engines/ibverbs/daqiri_ibverbs_engine.cpp @@ -3807,6 +3807,60 @@ IbvTxQueue* IbverbsEngine::find_tx_queue(int port, int q) { return nullptr; } +Status IbverbsEngine::configure_tx_pacing(IbvTxQueue& q, uint64_t pacing_mbps) { + if (pacing_mbps == 0) { + return Status::SUCCESS; + } + + constexpr uint64_t KBPS_PER_MBPS = 1000; + if (pacing_mbps > std::numeric_limits::max() / KBPS_PER_MBPS) { + DAQIRI_LOG_CRITICAL( + "TX queue {}: pacing_mbps={} exceeds the ibverbs rate-limit maximum of {} Mbps", q.queue_id, + pacing_mbps, std::numeric_limits::max() / KBPS_PER_MBPS); + return Status::INVALID_PARAMETER; + } + const uint32_t rate_kbps = static_cast(pacing_mbps * KBPS_PER_MBPS); + + struct ibv_device_attr_ex device_attr {}; + const int query_rc = ibv_query_device_ex(q.ctx, nullptr, &device_attr); + if (query_rc != 0) { + const int err = query_rc > 0 ? query_rc : errno; + DAQIRI_LOG_CRITICAL("TX queue {}: failed to query packet-pacing capabilities: {}", q.queue_id, + strerror(err)); + return Status::GENERIC_FAILURE; + } + const auto& caps = device_attr.packet_pacing_caps; + if ((caps.supported_qpts & (1U << IBV_QPT_RAW_PACKET)) == 0) { + DAQIRI_LOG_CRITICAL( + "TX queue {}: pacing_mbps={} requested, but the ibverbs device does not support " + "hardware packet pacing for RAW_PACKET QPs", + q.queue_id, pacing_mbps); + return Status::INVALID_PARAMETER; + } + if (rate_kbps < caps.qp_rate_limit_min || rate_kbps > caps.qp_rate_limit_max) { + DAQIRI_LOG_CRITICAL( + "TX queue {}: pacing_mbps={} ({} kbps) is outside the ibverbs device range " + "[{}, {}] kbps", + q.queue_id, pacing_mbps, rate_kbps, caps.qp_rate_limit_min, caps.qp_rate_limit_max); + return Status::INVALID_PARAMETER; + } + + struct ibv_qp_rate_limit_attr rate_attr {}; + rate_attr.rate_limit = rate_kbps; + // Leave max_burst_sz and typical_pkt_sz at zero so the device uses its + // defaults. Some devices do not support programming those optional fields. + const int modify_rc = ibv_modify_qp_rate_limit(q.qp, &rate_attr); + if (modify_rc != 0) { + const int err = modify_rc > 0 ? modify_rc : errno; + DAQIRI_LOG_CRITICAL("TX queue {}: failed to apply hardware packet-pacing rate {} Mbps: {}", + q.queue_id, pacing_mbps, strerror(err)); + return Status::GENERIC_FAILURE; + } + + DAQIRI_LOG_INFO("TX queue {} hardware packet pacing enabled: {} Mbps", q.queue_id, pacing_mbps); + return Status::SUCCESS; +} + Status IbverbsEngine::create_tx_raw_qp(IbvTxQueue& q) { // A scheduled packet may consume a WAIT WQE followed by its send WQE, so // provision up to two outstanding WRs per packet slot. Check the generic @@ -3930,18 +3984,6 @@ Status IbverbsEngine::setup_tx_queue(IbvTxQueue& q, const InterfaceConfig& intf, q.insert_eth_src = true; } } - // Packet pacing (pacing_mbps) is not supported on the ibverbs engine: the - // wait-on-time WQE it would use is not honored without the mlx5 send-scheduling - // clock/rearm-queue infrastructure that this engine does not set up, so a paced - // queue would not meter and would eventually stall. Reject it up front rather - // than silently running at line rate. Use the DPDK raw engine for pacing. - if (qcfg.pacing_mbps_ > 0) { - DAQIRI_LOG_CRITICAL( - "TX queue {}: pacing_mbps is not supported by the ibverbs engine; use the default " - "DPDK raw engine (remove engine: \"ibverbs\") for packet pacing", - q.queue_id); - return Status::INVALID_PARAMETER; - } if (qcfg.common_.mrs_.empty()) { DAQIRI_LOG_CRITICAL("TX queue {} has no memory region", q.queue_id); return Status::INVALID_PARAMETER; @@ -3988,6 +4030,15 @@ Status IbverbsEngine::setup_tx_queue(IbvTxQueue& q, const InterfaceConfig& intf, if (Status s = create_tx_raw_qp(q); s != Status::SUCCESS) { return s; } + // Packet pacing is an SQ/QP rate-table setting, independent of the WAIT WQEs + // used by set_packet_tx_time for absolute per-packet scheduling. + if (Status s = configure_tx_pacing(q, qcfg.pacing_mbps_); s != Status::SUCCESS) { + ibv_destroy_qp(q.qp); + q.qp = nullptr; + ibv_destroy_cq(q.cq); + q.cq = nullptr; + return s; + } // Hand-off ring: the app fill thread enqueues filled bursts; the pinned TX // worker dequeues and does the WQE build + doorbell + completion reclaim. diff --git a/src/engines/ibverbs/daqiri_ibverbs_engine.h b/src/engines/ibverbs/daqiri_ibverbs_engine.h index 8508f59..ce77791 100644 --- a/src/engines/ibverbs/daqiri_ibverbs_engine.h +++ b/src/engines/ibverbs/daqiri_ibverbs_engine.h @@ -456,6 +456,7 @@ class IbverbsEngine : public Engine { // ---- TX path ---- Status setup_tx_queue(IbvTxQueue& q, const InterfaceConfig& intf, const TxQueueConfig& qcfg); Status create_tx_raw_qp(IbvTxQueue& q); // IBV_QPT_RAW_PACKET, RESET->RTS + Status configure_tx_pacing(IbvTxQueue& q, uint64_t pacing_mbps); void post_tx_burst(IbvTxQueue& q, BurstParams* burst); // build send WQEs + ring doorbell void post_tx_burst_empw(IbvTxQueue& q, BurstParams* burst); // Build a WAIT-on-time WQE (ctrl + wseg = 1 WQEBB, no slot) at q.sq_pi that From b92fa245f495674538c13747af6a64512c6272bd Mon Sep 17 00:00:00 2001 From: Cliff Burdick Date: Fri, 17 Jul 2026 15:19:04 +0000 Subject: [PATCH 2/2] #176 - Handle missing ibverbs pacing bounds Signed-off-by: Cliff Burdick --- docs/api-reference/configuration.md | 3 ++- docs/benchmarks/raw_benchmarking.md | 7 ++++--- docs/tutorials/configuration-walkthrough.md | 2 +- examples/daqiri_bench_raw_tx_rx_pacing.yaml | 3 ++- src/engines/ibverbs/daqiri_ibverbs_engine.cpp | 6 +++++- 5 files changed, 14 insertions(+), 7 deletions(-) diff --git a/docs/api-reference/configuration.md b/docs/api-reference/configuration.md index f6ac7a9..2931406 100644 --- a/docs/api-reference/configuration.md +++ b/docs/api-reference/configuration.md @@ -405,7 +405,8 @@ daqiri::set_reorder_cuda_stream("rx_port", "rx_reorder_0", stream); requirements: `dpdk` uses hardware send scheduling and falls back to line rate with a warning when that offload is unavailable; `ibverbs` assigns the QP to an mlx5 hardware packet-pacing rate-table entry and fails initialization if RAW_PACKET pacing is unavailable or the requested - rate is outside the device's advertised range. The ibverbs engine leaves the optional burst + rate is outside a range advertised by the device. Older drivers that omit the range defer bounds + checking to the provider when the rate is applied. The ibverbs engine leaves the optional burst bound and typical-packet-size fields at their device defaults. - type: `integer` - default: `0` diff --git a/docs/benchmarks/raw_benchmarking.md b/docs/benchmarks/raw_benchmarking.md index 2a187d5..a8bd5f0 100644 --- a/docs/benchmarks/raw_benchmarking.md +++ b/docs/benchmarks/raw_benchmarking.md @@ -276,9 +276,10 @@ link speed. Change `pacing_mbps` (or set it to `0` to disable pacing and send at and re-run to see the cap move. The DPDK path requires hardware send scheduling; if unavailable, it logs a warning and runs at -line rate. The ibverbs path requires packet-pacing support for RAW_PACKET QPs and a requested rate -within the range reported by the device; otherwise initialization fails instead of silently -ignoring the cap. +line rate. The ibverbs path requires packet-pacing support for RAW_PACKET QPs and, when the driver +reports a supported range, a rate within that range. Older drivers that omit the range defer the +check to the provider when applying the rate; unsupported requests still fail initialization +instead of silently ignoring the cap. ## Tune RDMA SEND completion signaling diff --git a/docs/tutorials/configuration-walkthrough.md b/docs/tutorials/configuration-walkthrough.md index 5d92a21..a769b8b 100644 --- a/docs/tutorials/configuration-walkthrough.md +++ b/docs/tutorials/configuration-walkthrough.md @@ -137,7 +137,7 @@ For a shorter selection guide, start with the [Benchmarking overview](../benchma This is the base TX+RX template with a per-queue `pacing_mbps` cap added to the TX queue. It uses the default DPDK engine; add `engine: "ibverbs"` beside `stream_type: "raw"` to use the mlx5 QP packet-pacing rate table instead. The NIC meters the queue out so its average TX rate stays at or below the configured Mbps. Set `pacing_mbps: 0` (or remove it) to send at line rate. Validate by computing the achieved rate from the benchmark's RX line (`Gbps = bytes * 8 / seconds / 1e9`) and confirming it tracks the configured cap. See the `pacing_mbps` key in the [TX queue configuration](../api-reference/configuration.md#transmit-configuration-tx). - *Requires the engine's pacing capability: DPDK warns and runs at line rate without its send-scheduling offload; ibverbs fails initialization unless the device advertises RAW_PACKET packet pacing and the rate is in range.* + *Requires the engine's pacing capability: DPDK warns and runs at line rate without its send-scheduling offload; ibverbs fails initialization unless the device advertises RAW_PACKET packet pacing and accepts the requested rate. If an older driver omits its supported range, the provider validates the rate when it is applied.* ## Annotated walkthrough diff --git a/examples/daqiri_bench_raw_tx_rx_pacing.yaml b/examples/daqiri_bench_raw_tx_rx_pacing.yaml index ed11bf1..e7c0cdf 100644 --- a/examples/daqiri_bench_raw_tx_rx_pacing.yaml +++ b/examples/daqiri_bench_raw_tx_rx_pacing.yaml @@ -11,7 +11,8 @@ # # Requirements: the DPDK path needs its hardware send-scheduling offload and # warns/runs at line rate without it. The ibverbs path requires RAW_PACKET packet -# pacing and a rate within the device's advertised range; otherwise init fails. +# pacing and an accepted rate; older drivers that omit their supported range +# defer validation to the provider when the rate is applied. # # Validate: run daqiri_bench_raw_gpudirect with this config and compute the # achieved rate from the RX line's bytes/seconds (Gbps = bytes * 8 / seconds / diff --git a/src/engines/ibverbs/daqiri_ibverbs_engine.cpp b/src/engines/ibverbs/daqiri_ibverbs_engine.cpp index 77d43ee..b5c92eb 100644 --- a/src/engines/ibverbs/daqiri_ibverbs_engine.cpp +++ b/src/engines/ibverbs/daqiri_ibverbs_engine.cpp @@ -3837,7 +3837,11 @@ Status IbverbsEngine::configure_tx_pacing(IbvTxQueue& q, uint64_t pacing_mbps) { q.queue_id, pacing_mbps); return Status::INVALID_PARAMETER; } - if (rate_kbps < caps.qp_rate_limit_min || rate_kbps > caps.qp_rate_limit_max) { + // Some older MLNX_OFED/rdma-core combinations advertise RAW_PACKET pacing + // but leave both range fields at zero. In that case the capability is still + // usable; defer bounds enforcement to ibv_modify_qp_rate_limit(). + if (caps.qp_rate_limit_max != 0 && + (rate_kbps < caps.qp_rate_limit_min || rate_kbps > caps.qp_rate_limit_max)) { DAQIRI_LOG_CRITICAL( "TX queue {}: pacing_mbps={} ({} kbps) is outside the ibverbs device range " "[{}, {}] kbps",