Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 29 additions & 29 deletions docs/api-reference/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,21 +54,21 @@ These settings apply globally to both TX and RX:

## Memory Regions

`memory_regions:` List of regions where packet buffers are stored. The number of regions
`memory_regions:` List of regions where packet buffers are stored. The number of regions
and their `kind` determines the receive mode (CPU-only, header-data split, or batched GPU).

- **`name`**: Memory region name. Referenced by queue configurations.
- type: `string`
- **`kind`**: Memory type.
- type: `string`
- values:
- `huge` CPU hugepages (recommended for CPU buffers)
- `device`GPU VRAM (discrete GPUs only; requires GPUDirect via peermem or DMA-BUF)
- `host_pinned` Pinned CPU pages allocated via `cudaHostAlloc`. **Recommended on
- `huge`: CPU hugepages (recommended for CPU buffers)
- `device`: GPU VRAM (discrete GPUs only, requires GPUDirect via peermem or DMA-BUF)
- `host_pinned`: Pinned CPU pages allocated via `cudaHostAlloc`. **Recommended on
integrated GPUs (e.g. NVIDIA GB10 / DGX Spark)**, where the NIC cannot peer-DMA
into device memory and CUDA reports DMA-BUF unsupported. On discrete-GPU systems,
prefer `device` for high-throughput RX/TX paths.
- `host` Regular CPU memory (not recommended)
- `host`: Regular CPU memory (not recommended)
- **`affinity`**: GPU ID for `device` memory, or NUMA node ID for CPU memory.
- type: `integer`
- **`access`**: Memory access permissions.
Expand All @@ -87,7 +87,7 @@ and their `kind` determines the receive mode (CPU-only, header-data split, or ba

### Example: Header-Data Split

Two regions a small CPU region for headers and a GPU region for payload:
Two regions, a small CPU region for headers and a GPU region for payload:

```yaml
memory_regions:
Expand All @@ -109,7 +109,7 @@ memory_regions:

## Interfaces

`interfaces:` List of NIC interfaces to configure.
`interfaces:` List of NIC interfaces to configure.

- **`name`**: Interface name. Used to look up port IDs at runtime via `get_port_id()`.
- type: `string`
Expand Down Expand Up @@ -158,7 +158,7 @@ engine.

### Queues

`rx.queues:` List of receive queues on the interface.
`rx.queues:` List of receive queues on the interface.

- **`name`**: Queue name.
- type: `string`
Expand All @@ -168,11 +168,11 @@ engine.
performance.
- type: `string`
- **`batch_size`**: Number of packets per batch passed from the NIC to the application. Larger
values increase throughput; smaller values reduce latency.
values increase throughput, and smaller values reduce latency.
- type: `integer`
- **`memory_regions`**: List of memory region names (defined in [Memory Regions](#memory-regions)).
The order determines segment mapping: first region = segment 0, second = segment 1, etc.
A single region means all packet data lands in one place; two regions enables header-data
A single region means all packet data lands in one place, while two regions enables header-data
split.
- type: `list`
- **`timeout_us`**: Timeout in microseconds. A partial batch is delivered if this time elapses
Expand All @@ -182,11 +182,11 @@ engine.

### Flex Items

`rx.flex_items:` Flexible parser items for custom flow matching beyond standard UDP fields.
`rx.flex_items:` Flexible parser items for custom flow matching beyond standard UDP fields.

- **`name`**: Name of the flex item.
- type: `string`
- **`id`**: ID of the flex item. Scoped per interface; the same numeric ID on two
- **`id`**: ID of the flex item. Scoped per interface. The same numeric ID on two
interfaces may refer to different parser settings.
- type: `integer`
- **`offset`**: Byte offset after the UDP header where matching begins. Must be a multiple
Expand All @@ -197,8 +197,8 @@ engine.

### Flows

`rx.flows:` Static startup flow rules that steer packets to specific queues based on
match criteria. This sequence may be omitted; a queues-only RX config can add DPDK RX
`rx.flows:` Static startup flow rules that steer packets to specific queues based on
match criteria. This sequence may be omitted, and a queues-only RX config can add DPDK RX
flows later with the dynamic flow API. For Raw Ethernet on the DPDK and ibverbs engines,
RX flows can also perform hardware VLAN pop or tunnel decapsulation before queue delivery.

Expand Down Expand Up @@ -236,7 +236,7 @@ RX flows can also perform hardware VLAN pop or tunnel decapsulation before queue
- **`mask`**: 32-bit mask applied before matching (with flex items).
- type: `integer`
- **`ecpri`**: eCPRI-over-Ethernet match (EtherType `0xAEFE`). Presence of this map selects
the eCPRI flow class; the EtherType is matched implicitly. Cannot be combined with UDP/IP
the eCPRI flow class. The EtherType is matched implicitly. Cannot be combined with UDP/IP
or flex-item matching. A flow with an empty `ecpri: {}` map matches all eCPRI frames.
- **`msg_type`**: eCPRI common-header message type (e.g. `0` = IQ data, `2` = real-time
control). Optional.
Expand All @@ -257,14 +257,14 @@ firmware steering, so any interface with eCPRI flows is automatically switched t
`dv_flow_en=1` (logged as a warning); a side effect is that the async/template dynamic-RX-flow
API is unavailable on that interface. The `ibverbs` engine has no such restriction.

A single RX interface must use exactly one flow class standard UDP/IP, flex-item, or eCPRI.
A single RX interface must use exactly one flow class: standard UDP/IP, flex-item, or eCPRI.
Each class installs its own DPDK group-0 jump rule, and these conflict when mixed, so only one
class is reachable per interface. `daqiri_init` rejects mixed configs with a clear error.
Flex-item flows cannot be combined with VLAN/tunnel transform actions in v1.

### Flow Isolation

`rx.flow_isolation:` When `true`, only packets matching an explicit flow rule are delivered
`rx.flow_isolation:` When `true`, only packets matching an explicit flow rule are delivered
to the application. Static startup flows install send-to-kernel fallback rules per flow class
(standard, flex-item, or eCPRI), so unmatched traffic in those classes is steered back to the Linux
kernel. Queues-only configs can set `flow_isolation: true` and then install dynamic RX flows
Expand All @@ -280,17 +280,17 @@ batch.

### Dynamic Flow Capacity

`rx.dynamic_flow_capacity:` DPDK template-table capacity reserved for dynamic RX flow
`rx.dynamic_flow_capacity:` DPDK template-table capacity reserved for dynamic RX flow
rules on this interface. `0` disables DPDK template/async setup on startup. Set a positive
value to opt in to the template fast path when it is available; legacy fallback paths still
value to opt in to the template fast path when it is available. Legacy fallback paths still
accept dynamic RX flow operations but do not use a template table.

- type: `integer`
- default: `0`

### Hardware Timestamps

`rx.hardware_timestamps:` Enable per-packet hardware RX timestamps for Raw Ethernet
`rx.hardware_timestamps:` Enable per-packet hardware RX timestamps for Raw Ethernet
(`stream_type: "raw"`).
When enabled, DAQIRI requires `RTE_ETH_RX_OFFLOAD_TIMESTAMP` support from the NIC/PMD and
initialization fails if DAQIRI cannot provide nanosecond timestamps for the selected PMD.
Expand All @@ -302,7 +302,7 @@ clock domain, not wall-clock time.

### RX Reorder Configs

`rx.reorder_configs:` Optional automatic packet reordering/aggregation plans. Implemented
`rx.reorder_configs:` Optional automatic packet reordering/aggregation plans. Implemented
for Raw Ethernet (`stream_type: "raw"`) only in v1. GPU reorder requires CUDA-addressable
packet buffers (`device` or `host_pinned` memory regions). CPU reorder requires CPU-addressable
packet buffers (`host`, `host_pinned`, or `huge` memory regions).
Expand Down Expand Up @@ -330,7 +330,7 @@ v1 batch-size requirement:
- values: `gpu`, `cpu`
- **`memory_region`**: Output memory region where reordered payload is written.
- type: `string`
- requirements: for `gpu`, must reference a `device` or `host_pinned` memory region; for
- requirements: for `gpu`, must reference a `device` or `host_pinned` memory region. For
`cpu`, must reference a `host`, `host_pinned`, or `huge` memory region
- **`payload_byte_offset`**: Byte offset in each packet where copied payload starts. Bytes before
this offset are skipped.
Expand Down Expand Up @@ -380,7 +380,7 @@ daqiri::set_reorder_cuda_stream("rx_port", "rx_reorder_0", stream);

### Queues

`tx.queues:` List of transmit queues on the interface.
`tx.queues:` List of transmit queues on the interface.

- **`name`**: Queue name.
- type: `string`
Expand All @@ -390,7 +390,7 @@ daqiri::set_reorder_cuda_stream("rx_port", "rx_reorder_0", stream);
performance.
- type: `string`
- **`batch_size`**: Number of packets per batch sent to the NIC. Larger values increase
throughput; smaller values reduce latency.
throughput, and smaller values reduce latency.
- type: `integer`
- **`memory_regions`**: List of memory region names. Same segment mapping rules as RX.
- type: `list`
Expand All @@ -400,7 +400,7 @@ daqiri::set_reorder_cuda_stream("rx_port", "rx_reorder_0", stream);
- values: `tx_eth_src` (auto-fill source MAC address)
- **`pacing_mbps`**: Packet-pacing rate cap for this queue, in megabits per second of L2 frame
bytes (the data the application transmits, excluding preamble/IFG/FCS). The NIC meters the queue
out so its long-run average TX rate stays at or below this value; the limit is enforced on an
out so its long-run average TX rate stays at or below this value. The limit is enforced on an
average basis and idle gaps do not accumulate burst credit. `0` (the default) disables pacing
and sends at line rate. Supported only by the default `dpdk` raw engine on a NIC with hardware
send scheduling (ConnectX-7 or later); on devices without it, `pacing_mbps` is ignored with a
Expand All @@ -411,9 +411,9 @@ daqiri::set_reorder_cuda_stream("rx_port", "rx_reorder_0", stream);

### Transmit Flows

`tx.flows:` Raw Ethernet hardware transform rules for outgoing packets. Supported on
`tx.flows:` Raw Ethernet hardware transform rules for outgoing packets. Supported on
the DPDK and ibverbs raw engines only. TX flows match the packet as supplied by the
application, then push or encapsulate headers in hardware; the application buffer remains
application, then push or encapsulate headers in hardware. The application buffer remains
the pre-encap packet.

- **`name`** / **`id`**: Flow label and ID.
Expand All @@ -435,12 +435,12 @@ the pre-encap packet.

DAQIRI validates transform overhead against the configured packet buffer size and
the supported jumbo-frame bound. For RX decap/pop and TX encap/push, MTU sizing
accounts for the outer wire frame; packet buffers hold the post-decap (RX) /
accounts for the outer wire frame. Packet buffers hold the post-decap (RX) /
pre-encap (TX) frame.

### Accurate Send

`tx.accurate_send:` Enable hardware-timed packet transmission using PTP timestamps. When
`tx.accurate_send:` Enable hardware-timed packet transmission using PTP timestamps. When
enabled, use `set_packet_tx_time()` to schedule packets. Requires ConnectX-7 or later.

- type: `boolean`
Expand Down
22 changes: 11 additions & 11 deletions docs/api-reference/cpp.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ if (st != daqiri::Status::SUCCESS) {

## Receiving Packets

### RX Step 1 Get a burst
### RX Step 1: Get a burst

```cpp
daqiri::BurstParams *burst;
Expand All @@ -92,7 +92,7 @@ daqiri::get_rx_burst(&burst, 0);
daqiri::get_rx_burst(&burst);
```

### RX Step 2 Access packet data
### RX Step 2: Access packet data

For a single-segment configuration (CPU-only or batched GPU):

Expand Down Expand Up @@ -129,7 +129,7 @@ for (int i = 0; i < daqiri::get_num_packets(burst); i++) {
}
```

### RX Step 3 Free buffers
### RX Step 3: Free buffers

When you are done processing, free the burst to return buffers to the pool:

Expand Down Expand Up @@ -232,7 +232,7 @@ Packets matching a dynamic rule are marked with the same `FlowId` returned by
the add completion, so `get_packet_flow_id()` gives the handle to pass to
`delete_flow_async()`. `poll_flow_op()` returns `Status::NOT_READY` when no flow
operation has completed yet. A dynamic flow is deletable only after its add
completion has been polled successfully; deleting a flow that is still pending returns
completion has been polled successfully. Deleting a flow that is still pending returns
`Status::INVALID_PARAMETER`.

Multiple RX flows can be added as one operation. On DPDK this maps to a single
Expand Down Expand Up @@ -312,7 +312,7 @@ if ((burst->hdr.hdr.burst_flags & daqiri::DAQIRI_BURST_FLAG_REORDERED) != 0U) {

### GPU packet processing on reordered bursts

When using batched GPU mode, packets arrive in CUDA-addressable buffers each at an
When using batched GPU mode, packets arrive in CUDA-addressable buffers, each at an
arbitrary GPU address. Launch your own CUDA work directly on the packet pointers. Packet
reordering and aggregation should be configured through `rx.reorder_configs`; see
`raw_reorder_seq_bench.cpp` and `raw_reorder_quantize_bench.cpp` for complete examples
Expand All @@ -334,15 +334,15 @@ daqiri::free_all_packets_and_burst_rx(burst);

## Transmitting Packets

### TX Step 1 Allocate a burst
### TX Step 1: Allocate a burst

```cpp
auto burst = daqiri::create_tx_burst_params();
daqiri::set_header(burst, port_id, queue_id, batch_size, num_segments);

auto status = daqiri::get_tx_packet_burst(burst);
if (status != daqiri::Status::SUCCESS) {
// No buffers available retry later
// No buffers available, retry later
}
```

Expand All @@ -363,7 +363,7 @@ daqiri::set_connection_id(burst, conn_id);
auto rx_conn_id = daqiri::get_connection_id(rx_burst);
```

### TX Step 2 Fill packets
### TX Step 2: Fill packets

Use the header helper functions for standard UDP packets:

Expand All @@ -380,7 +380,7 @@ for (int i = 0; i < daqiri::get_num_packets(burst); i++) {
Or construct raw packets by writing directly into the packet buffer returned by
`get_packet_ptr()`.

### TX Step 3 Send
### TX Step 3: Send

```cpp
daqiri::send_tx_burst(burst);
Expand Down Expand Up @@ -535,7 +535,7 @@ copies each packet's post-offset logical bytes into owned host staging memory
before submission, so the burst may be released after
`daqiri_write_raw_to_s3_objects_async()` succeeds. Header-data split and other
multi-segment packets are concatenated into one object. The first S3 version
uses one single-part `PutObject` per packet; objects larger than 5 GiB return
uses one single-part `PutObject` per packet. Objects larger than 5 GiB return
`NOT_SUPPORTED`, and multipart/burst aggregation is future work.

The Python bindings expose the same C++ writer when both
Expand Down Expand Up @@ -721,7 +721,7 @@ workflow sections above show the common call order and ownership rules.
| Function | Purpose |
| --- | --- |
| `get_mac_addr(port, mac)` | Copy a port MAC address into a six-byte buffer. |
| `format_eth_addr(dst, addr)` | Convert a `xx:xx:xx:xx:xx:xx` MAC string into a six-byte buffer; invalid input zeroes the buffer. |
| `format_eth_addr(dst, addr)` | Convert a `xx:xx:xx:xx:xx:xx` MAC string into a six-byte buffer. Invalid input zeroes the buffer. |
| `get_port_id(key)` | Resolve an interface name or PCIe address to a port ID. |
| `get_num_rx_queues(port_id)` | Return the configured or engine-reported RX queue count. |
| `drop_all_traffic(port)` | Install a high-priority drop rule on a port. |
Expand Down
14 changes: 7 additions & 7 deletions docs/api-reference/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,20 +58,20 @@ The typical DAQIRI application lifecycle has six steps:

Each step maps directly to a section of the
[C++ API Usage](cpp.md) and [Python API Usage](python.md) pages. The
buffer-release step in particular is load-bearing — see
buffer-release step in particular is load-bearing. See
[Zero-Copy Ownership](../concepts.md#zero-copy-ownership) on the
Concepts page for why missed frees cause queue drops.

## See also

- [Concepts](../concepts.md) terminology and background for everything
- [Concepts](../concepts.md): terminology and background for everything
this page references.
- [Configuration YAML Reference](configuration.md) every YAML key, its
- [Configuration YAML Reference](configuration.md): every YAML key, its
type, and its valid values.
- [C++ API Usage](cpp.md) initialization, RX/TX workflows, buffer
- [C++ API Usage](cpp.md): initialization, RX/TX workflows, buffer
lifecycle, file writing, utilities, and the full C++ function reference.
- [Python API Usage](python.md) the same workflow through the pybind11
- [Python API Usage](python.md): the same workflow through the pybind11
bindings, including GIL behavior, tuple return shapes, and the full
Python function reference.
- [Configuration walkthrough tutorial](../tutorials/configuration-walkthrough.md)
annotated YAML walkthrough with a use-case decision tree.
- [Configuration walkthrough tutorial](../tutorials/configuration-walkthrough.md):
annotated YAML walkthrough with a use-case decision tree.
Loading
Loading