#233 - Add automatic RSS for multi-queue flows

AGENTS.md

@@ -23,7 +23,7 @@ CMake options (full table in `docs/getting-started.md`):
 - `DAQIRI_ENABLE_S3` — enable AWS SDK-backed asynchronous raw packet writes to S3 (off by default).
 - `DAQIRI_PREFER_SYSTEM_YAML_CPP` — prefer system `yaml-cpp` over the vendored `third_party/yaml-cpp` submodule (off by default; keep off when a conda/miniforge env is on `PATH`).
 
-Package versions use CalVer (`YYYY.MM.PATCH`) from the top-level `VERSION` file. CMake reads that value into `project(daqiri VERSION ...)`, generates `daqiri/version.h`, feeds pkg-config/CMake package metadata, and exposes the same value in Python. `DAQIRI_ABI_VERSION` is separate and currently `0`; do not tie ABI policy to the CalVer year.
+Package versions use CalVer (`YYYY.MM.PATCH`) from the top-level `VERSION` file. CMake reads that value into `project(daqiri VERSION ...)`, generates `daqiri/version.h`, feeds pkg-config/CMake package metadata, and exposes the same value in Python. `DAQIRI_ABI_VERSION` is separate and currently `1`; do not tie ABI policy to the CalVer year.
 
 CUDA architectures default to `80;90` (A100, H100), with `121` (GB10) added when configuring with CUDA Toolkit 13.0 or newer. Override `CMAKE_CUDA_ARCHITECTURES` when targeting other GPUs.
 
@@ -35,8 +35,8 @@ There is no unit test suite. Verification is done via the benchmark executables
 
 | Executable | Source | Typical config |
 |---|---|---|
-| `daqiri_bench_raw_gpudirect` | `raw_gpudirect_bench.cpp` | `daqiri_bench_raw_tx_rx.yaml`, `daqiri_bench_raw_tx_rx_4q.yaml`, `daqiri_bench_raw_tx_rx_spark.yaml`, `daqiri_bench_raw_{tx,rx}_spark_xhost.yaml`, `daqiri_bench_raw_sw_loopback.yaml`, `daqiri_bench_raw_rx_multi_q.yaml`, `daqiri_bench_raw_tx_rx_vxlan.yaml`, `daqiri_bench_raw_tx_rx_vlan.yaml`, `daqiri_bench_raw_tx_rx_gre.yaml`, `daqiri_bench_raw_tx_rx_nvgre.yaml`, `daqiri_bench_raw_tx_rx_spark_mq.yaml` (mq base; `run_spark_mq_bench.sh` derives the 4 cells via `scripts/gen_spark_mq_config.py`), `daqiri_bench_raw_tx_rx_pacing.yaml` (per-queue `pacing_mbps`; DPDK or ibverbs raw engine) |
-| `daqiri_example_dynamic_rx_flow` | `dynamic_rx_flow_example.cpp` | `daqiri_example_dynamic_rx_flow.yaml` — `flow_isolation: true` startup followed by runtime RX queue-steering and raw-engine decap/pop flow add/delete |
+| `daqiri_bench_raw_gpudirect` | `raw_gpudirect_bench.cpp` | `daqiri_bench_raw_tx_rx.yaml`, `daqiri_bench_raw_tx_rx_4q.yaml`, `daqiri_bench_raw_tx_rx_spark.yaml`, `daqiri_bench_raw_{tx,rx}_spark_xhost.yaml`, `daqiri_bench_raw_sw_loopback.yaml`, `daqiri_bench_raw_rx_multi_q.yaml`, `daqiri_bench_raw_tx_rx_vxlan.yaml`, `daqiri_bench_raw_tx_rx_vlan.yaml`, `daqiri_bench_raw_tx_rx_gre.yaml`, `daqiri_bench_raw_tx_rx_nvgre.yaml`, `daqiri_bench_raw_tx_rx_spark_mq.yaml` (mq base; `run_spark_mq_bench.sh` derives the 4 cells via `scripts/gen_spark_mq_config.py`), `daqiri_bench_raw_tx_rx_pacing.yaml` (per-queue `pacing_mbps`; DPDK engine only) |
+| `daqiri_example_dynamic_rx_flow` | `dynamic_rx_flow_example.cpp` | `daqiri_example_dynamic_rx_flow.yaml` — `flow_isolation: true` startup followed by runtime scalar queue steering, multi-queue RSS, and raw-engine decap/pop flow add/delete |
 | `daqiri_bench_raw_hds` | `raw_hds_bench.cpp` | `daqiri_bench_raw_tx_rx_hds.yaml` |
 | `daqiri_bench_raw_reorder_seq` | `raw_reorder_seq_bench.cpp` | `daqiri_bench_raw_tx_rx_reorder_seq_1024*.yaml`, `daqiri_bench_raw_rx_reorder_seq_*.yaml` |
 | `daqiri_bench_raw_reorder_quantize` | `raw_reorder_quantize_bench.cpp` | `daqiri_bench_raw_tx_rx_reorder_quantize_seq_batch.yaml` |
@@ -79,7 +79,7 @@ clang-format -style=file -i -fallback-style=none <files>
 
 The always-built socket engine implements Linux UDP/TCP streams directly. Applications that need kernel socket tuning call `socket_setsockopt(conn_id, level, optname, optval, optlen)` after resolving a TCP/UDP connection ID; DAQIRI passes the numeric Linux constants through without maintaining a symbolic option map. `socket_setsockopt` is not supported for `roce://` connections, which delegate to the RDMA/ibverbs path.
 
-The default `dpdk` raw engine (`src/engines/dpdk/`, `DpdkEngine`) programs RX steering, send-to-kernel fallbacks (`flow_isolation: true`), and `tx_eth_src` TX offloads via DPDK RTE Flow during `daqiri_init()`. Standard UDP/IP (group 3), flex-item (group 1) and eCPRI-over-Ethernet (group 2, EtherType `0xAEFE`, via `RTE_FLOW_ITEM_TYPE_ECPRI` matching message type and pc_id/rtc_id) RX flows use separate flow groups; `validate_config()` rejects mixing these flow classes per interface, duplicate `flex_item_id` values, and unknown `action.id` / `flex_item_id` values before NIC programming. The mlx5 PMD honors the native eCPRI flow item only under **firmware steering** (`dv_flow_en=1`); under HW steering (`dv_flow_en=2`, the default) the rule installs but silently never matches on ConnectX-class NICs. So `initialize()` auto-switches any interface carrying eCPRI RX flows to `dv_flow_en=1` (with a `WARN`), which means the async/template dynamic-RX-flow path is unavailable on that port. Flex-item parser handles are created per `(port, flex_item_id)` (scoped per interface). All programmed `rte_flow` rules and flex-item handles are tracked and destroyed in order on shutdown, init failure, and engine teardown (programmed flows → flex items → group-0 ETH jump rules). See `docs/benchmarks/raw_benchmarking.md` (Flow programming smoke test) for manual verification steps.
+The default `dpdk` raw engine (`src/engines/dpdk/`, `DpdkEngine`) programs RX steering, send-to-kernel fallbacks (`flow_isolation: true`), and `tx_eth_src` TX offloads via DPDK RTE Flow during `daqiri_init()`. Standard UDP/IP (group 3), flex-item (group 1) and eCPRI-over-Ethernet (group 2, EtherType `0xAEFE`, via `RTE_FLOW_ITEM_TYPE_ECPRI` matching message type and pc_id/rtc_id) RX flows use separate flow groups; `validate_config()` rejects mixing these flow classes per interface, duplicate `flex_item_id` values, and unknown queue targets / `flex_item_id` values before NIC programming. Queue actions with two or more IDs use mlx5 Toeplitz IPv4/UDP RSS. The mlx5 PMD honors the native eCPRI flow item only under **firmware steering** (`dv_flow_en=1`); under HW steering (`dv_flow_en=2`, the default) the rule installs but silently never matches on ConnectX-class NICs. So `initialize()` auto-switches any interface carrying eCPRI RX flows to `dv_flow_en=1` (with a `WARN`), which means the async/template dynamic-RX-flow path is unavailable on that port. Flex-item parser handles are created per `(port, flex_item_id)` (scoped per interface). All programmed `rte_flow` rules and flex-item handles are tracked and destroyed in order on shutdown, init failure, and engine teardown (programmed flows → flex items → group-0 ETH jump rules). See `docs/benchmarks/raw_benchmarking.md` (Flow programming smoke test) for manual verification steps.
 
 The `ibverbs` raw engine (`src/engines/ibverbs/`, `IbverbsEngine`) drives a Mellanox/mlx5 Multi-Packet (striding) Receive Queue via **DevX** (`mlx5dv_devx_obj_create` against vendored PRM structs in `mlx5_prm_min.h`): a DevX CQ + striding RQ + TIR + `mlx5dv_dr` flow steering, with manual WQE/doorbell management and worker-driven cyclic refill. RX packets DMA strided into one pre-posted MR (host or GPU via `ibv_reg_dmabuf_mr`); a queue with >1 memory region instead uses a non-striding DevX *regular* RQ with multi-segment scatter WQEs for **physical** header-data split (header → CPU MR, payload → GPU MR). TX builds mlx5 send WQEs directly on a raw-packet QP's SQ (via `mlx5dv_init_obj`, bypassing `ibv_post_send`) from a slab of registered slots tracked by cyclic index counters, with NIC checksum offload and a `tx_eth_src` offload. It uses the libdpdk-free `daqiri::Ring`/`daqiri::ObjectPool` for the worker→app burst handoff (like the rdma engine — neither links DPDK) and drives the NIC through libibverbs/mlx5dv directly. Feature set: RX (MPRQ), TX, GPUDirect, physical/logical HDS, multi-queue 5-tuple flow steering with per-packet flow IDs, flex-item arbitrary-offset, IPv4-total-length and eCPRI-over-Ethernet (EtherType `0xAEFE`, message type + pc_id/rtc_id) flow matching (mlx5 flex parser / `misc_parameters_4`), per-packet RX hardware timestamps, per-queue hardware packet pacing through the mlx5 QP rate table, accurate TX send scheduling (wait-on-time WAIT WQE), and GPU reorder/quantize. Packet pacing and accurate send are independent: `pacing_mbps` configures the QP's average rate, while `set_packet_tx_time()` emits WAIT WQEs for absolute per-packet times. Because it uses the kernel netdev directly, `ensure_port_mtus` raises the netdev MTU at init to cover the configured frame size in either direction — RX (post-decap) and TX egress (post-encap) — sizing each direction with its own transform wire overhead (jumbo frames silently drop otherwise). Queues sharing a `cpu_core` are serviced round-robin by one poller thread.
 
@@ -100,7 +100,7 @@ Vendored under `third_party/` as submodules (`.gitmodules`): `yaml-cpp` (config
 
 ### Current limitations
 - TX header fill currently supports UDP only (see README).
-- Raw Ethernet RX flow legacy `action.id` or final `actions:` queue action must match an `rx.queues` ID, and flex-item flows must reference a valid `flex_item_id` on the same interface; `daqiri_init()` aborts if RX flow rules, send-to-kernel fallbacks (`flow_isolation: true`), transform flow actions, or `tx_eth_src` offload rules cannot be programmed on the NIC.
+- Raw Ethernet RX flow scalar `action.id` or every entry in `action.ids` (including the final `actions:` queue action) must match an `rx.queues` ID, and flex-item flows must reference a valid `flex_item_id` on the same interface; `daqiri_init()` aborts if RX flow rules, RSS destinations, send-to-kernel fallbacks (`flow_isolation: true`), transform flow actions, or `tx_eth_src` offload rules cannot be programmed on the NIC.
 - Raw Ethernet tunnel/VLAN transform flows are hardware-only on the DPDK and ibverbs raw engines. TX flows may contain only push/encap transform actions and RX transform flows must use pop/decap actions ending in a queue; socket/RDMA engines reject these actions instead of adding a software fallback.
 - Raw Ethernet RX flow steering: a single interface cannot mix standard (UDP/IP) and
   flex-item flows, and flex-item flows cannot combine with tunnel/VLAN transform actions; `DpdkEngine::validate_config()` rejects mixed configs at init.

CMakeLists.txt

@@ -31,7 +31,7 @@ include(GNUInstallDirs)
 include(CMakePackageConfigHelpers)
 find_package(CUDAToolkit REQUIRED)
 
-set(DAQIRI_ABI_VERSION "0" CACHE STRING "DAQIRI shared library ABI version")
+set(DAQIRI_ABI_VERSION "1" CACHE STRING "DAQIRI shared library ABI version")
 if(NOT DAQIRI_ABI_VERSION MATCHES "^(0|[1-9][0-9]*)$")
   message(FATAL_ERROR "DAQIRI_ABI_VERSION must be a non-negative integer")
 endif()

README.md

@@ -38,10 +38,14 @@ DAQIRI provides direct NIC hardware access in userspace, bypassing the Linux ker
   S3-compatible object store through the AWS SDK for C++.
 - **Flow Steering** — Configure the NIC's hardware flow engine to route packets by UDP
   source/destination port or flex-item payload fields. Raw RX flows can be configured
-  statically in YAML or added/deleted dynamically after `daqiri_init()`. Per RX
-  interface, use standard UDP/IP flows or flex-item flows, not both. Raw DPDK and
-  raw ibverbs flows can also use hardware-only VLAN push/pop and VXLAN, GRE, or
-  NVGRE encap/decap actions; socket/RDMA streams reject those tunnel actions.
+  statically in YAML or added/deleted dynamically after `daqiri_init()`. A scalar
+  queue target steers directly to one queue; `ids: [0, 1, ...]` automatically enables
+  flow-affine IPv4/UDP Toeplitz RSS across the listed queues on the raw DPDK and
+  ibverbs engines. One unchanged five-tuple remains on one queue, so balanced packet
+  counts require varied tuples. Per RX interface, use standard UDP/IP flows or
+  flex-item flows, not both. Raw DPDK and raw ibverbs flows can also use hardware-only
+  VLAN push/pop and VXLAN, GRE, or NVGRE encap/decap actions; socket/RDMA streams reject
+  those tunnel actions.
 - **RDMA** — RDMA verbs (READ, WRITE, SEND) over RoCE on Ethernet NICs or InfiniBand.
 - **Linux socket control** — TCP/UDP socket streams expose connection IDs and
   `socket_setsockopt()` for native Linux `setsockopt` tuning without YAML option

docs/api-reference/configuration.md

@@ -207,11 +207,16 @@ RX flows can also perform hardware VLAN pop or tunnel decapsulation before queue
 - **`id`**: Flow ID. Retrievable at runtime via `get_packet_flow_id()`.
   - type: `integer`
 - **`action`**: Legacy single action map. Existing configs may keep using
-  `action: {type: queue, id: ...}`.
+  `action: {type: queue, id: ...}`. A multi-queue destination uses
+  `action: {type: queue, ids: [0, 1]}`.
 - **`actions`**: Ordered action list. Use this for tunnel/VLAN transforms.
   RX transform flows must end with `type: queue`.
   - **`type: queue`**: Steer matched packets to an RX queue.
     - **`id`**: Queue ID under `rx.queues` on the same interface.
+    - **`ids`**: Non-empty list of unique queue IDs under `rx.queues` on the same
+      interface. One entry is direct steering. Two or more entries automatically
+      enable flow-affine Toeplitz RSS over source/destination IPv4 address and
+      source/destination UDP port. `id` and `ids` are mutually exclusive.
   - **`type: vlan_pop`**: Pop one VLAN tag in hardware.
   - **`type: tunnel_decap`**: Decapsulate a hardware tunnel before queue delivery.
     - **`tunnel.type`**: `vxlan`, `gre`, or `nvgre`.
@@ -262,6 +267,22 @@ Each class installs its own DPDK group-0 jump rule, and these conflict when mixe
 class is reachable per interface. `daqiri_init` rejects mixed configs with a clear error.
 Flex-item flows cannot be combined with VLAN/tunnel transform actions in v1.
 
+Multi-queue RSS is supported for standard IPv4/UDP and flex-item flows. A flex
+item selects the rule, but its sampled value is not an RSS input; distribution
+still uses the packet's IPv4/UDP tuple. VLAN-pop rules hash the IPv4/UDP packet
+after the VLAN header, and tunnel-decap rules hash the inner IPv4/UDP tuple.
+eCPRI flows cannot use multi-queue RSS because they have no applicable UDP/IP
+five tuple. Queue-list order affects hash-to-queue mapping, but does not express
+weights.
+
+RSS is flow-affine: every packet with an unchanged five tuple stays on one
+queue. Roughly even packet counts require enough distinct tuples with reasonably
+balanced traffic; this is not packet striping or exact round-robin delivery.
+There is no queue-action mode field in schema v1; a future stripe mode can be
+added without changing the multi-ID RSS default. If
+the NIC rejects an RSS action, static initialization or the dynamic flow
+completion fails rather than falling back to one queue.
+
 ### Flow Isolation
 
 `rx.flow_isolation:` When `true`, only packets matching an explicit flow rule are delivered
@@ -346,7 +367,8 @@ v1 batch-size requirement:
     nibble first; `network` endianness swaps byte-multiple input types wider than 8 bits
 - **`flow_ids`**: List of RX flow IDs this reorder config applies to.
   - type: `list[integer]`
-  - notes: flow IDs cannot overlap across reorder configs on the same interface
+  - notes: flow IDs cannot overlap across reorder configs on the same interface;
+    a referenced flow must use direct steering to one queue, not multi-queue RSS
 - **`method`**: Exactly one method must be configured:
   - **`seq_batch_number`**
     - `sequence_number.bit_offset`

docs/api-reference/cpp.md

@@ -159,8 +159,11 @@ Raw Ethernet RX flows can be added and deleted after `daqiri_init()` on the
 `dpdk` and raw `ibverbs` engines. This supports queues-only startup configs,
 including `rx.flow_isolation: true` with no initial `rx.flows`. Static YAML
 flows still use explicit configured IDs and are not deletable through this API.
-The legacy `FlowRuleConfig::action_` field remains the shorthand for a single
-queue action; `FlowRuleConfig::actions_` is the ordered form used when a dynamic
+The legacy `FlowRuleConfig::action_` field remains the shorthand for a queue
+action; set `FlowAction::id_` for direct steering or `FlowAction::ids_` for a
+queue list. A list with one entry is direct steering, while two or more entries
+automatically enable flow-affine IPv4/UDP five-tuple RSS. `FlowRuleConfig::actions_`
+is the ordered form used when a dynamic
 RX rule needs hardware VLAN pop or tunnel decapsulation before queue delivery.
 Dynamic TX transform flows are not part of v1; configure TX encapsulation/push
 rules statically under `tx.flows`.
@@ -201,6 +204,21 @@ while (flow_id == 0) {
 }
 ```
 
+To distribute distinct IPv4/UDP flows across two queues, replace the scalar
+target with a queue list:
+
+```cpp
+flow.action_.ids_ = {0, 1};
+```
+
+The queue list must be non-empty, duplicate-free, and contain only configured RX
+queue IDs. Each unchanged five tuple remains on one queue; approximately even
+packet totals require many distinct tuples with reasonably balanced traffic.
+Tunnel decapsulation hashes the inner tuple. Multi-queue RSS is not available for
+eCPRI flows or flows referenced by an RX reorder configuration. Hardware creation
+failure is reported by the static initialization or dynamic operation completion;
+DAQIRI does not silently select one queue.
+
 For a dynamic VXLAN decap rule, use ordered actions and make the final action
 the target queue:
 

docs/api-reference/python.md

@@ -622,8 +622,15 @@ The workflow sections above show the common call order and ownership rules.
 | `rdma_get_port_queue(conn_id)` | Return `(Status, port, queue)`. |
 | `rdma_get_server_conn_id(server_addr, server_port)` | Return `(Status, conn_id)`. |
 
-Dynamic RX flows are RX-only in v1. The `action` attribute remains the single queue-action
-shorthand. Use ordered `actions` when a raw DPDK or raw ibverbs dynamic rule needs hardware
+Dynamic RX flows are RX-only in v1. The `action` attribute remains the queue-action
+shorthand. Set `flow.action.id = 0` for direct steering, or set
+`flow.action.ids = [0, 1]` to enable automatic flow-affine IPv4/UDP five-tuple
+RSS. A one-entry `ids` list is direct steering. Lists must be non-empty,
+duplicate-free, and contain configured RX queue IDs. An unchanged five tuple
+stays on one queue; balanced packet totals require enough distinct, reasonably
+balanced tuples. Multi-queue RSS is unsupported for eCPRI and reorder-owned
+flows, and hardware creation errors are returned rather than falling back to a
+single queue. Use ordered `actions` when a raw DPDK or raw ibverbs dynamic rule needs hardware
 VLAN pop or VXLAN/GRE/NVGRE decapsulation before the final queue action. Static TX
 encapsulation/push rules are configured in YAML under `tx.flows`.
 
@@ -696,7 +703,7 @@ names that mostly omit the trailing underscore from the C++ member name (e.g.
 | `MemoryRegionConfig` | Memory region kind, affinity, access flags, sizes, counts, and ownership. |
 | `VlanActionConfig` | VLAN push parameters: VLAN ID, priority, DEI, and ethertype. |
 | `TunnelConfig` | VXLAN, GRE, or NVGRE tunnel template fields for hardware encap/decap actions. |
-| `FlowAction` | Flow action type, queue target ID, optional VLAN config, and optional tunnel config. |
+| `FlowAction` | Flow action type, scalar queue target `id`, queue-list target `ids`, optional VLAN config, and optional tunnel config. |
 | `FlowMatch` | Flow match fields for UDP, IPv4, and flex item matching. |
 | `FlowConfig` | Static named flow rule combining legacy `action`, ordered `actions`, and match fields. |
 | `FlowRuleConfig` | Dynamic RX flow rule combining legacy `action`, ordered `actions`, and match fields. |