From 2b942993631e55ae3d5db057fba1148a3ff75f0f Mon Sep 17 00:00:00 2001 From: mikemiles-dev Date: Sun, 11 Oct 2026 17:49:43 -0500 Subject: [PATCH] docs: document the 1.1.0 error semantics in README README is the crate's top-level rustdoc via include_str!, so it is what lands on docs.rs. None of the four 1.1.0 entries were reflected in it, and two of the gaps made the existing text actively wrong rather than merely incomplete. parse_bytes no longer stops at the first message-level error: a malformed IPFIX message is discarded on its own and parsing resumes at the next one, so packets can appear both before and after the reported error, and the reported error is the first rather than the last. README said "Error after N packets", which now misleads on both counts. iter_packets likewise continues past a recoverable error instead of ending the stream. Its example showed an Err arm and said nothing about whether iteration continued, which is precisely the question the change altered the answer to. Document when it resumes and when it still stops, and that v5, v7 and v9 always stop because none of them can locate the next message. Add a Chained messages and error recovery section covering RFC 7011 section 9.1 discard semantics and AutoScopedParser::parse_from_source re-deriving the scope per surviving message, since "chained" appeared nowhere in README despite being the subject of two 1.1.0 fixes. Document NoTemplateReason: the pending-flow section claimed NoTemplate flowsets are cached and replayed without qualification, but an UndecodableBody flowset deliberately never is, because no template arriving later changes its octets. A table now distinguishes the two reasons and points at is_replayable(). List MalformedSetFraming and DecodedOutputLimitExceeded among the error types, and describe what the former carries and why it is distinct from Partial. DecodedOutputLimitExceeded was already used in an earlier section but had never been listed. check-readme-sync.sh passed throughout and caught none of this; it compares section headers against a lib.rs that has none, since lib.rs is just the include_str! attribute. Co-Authored-By: Claude Opus 5 (1M context) --- README.md | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 53 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 0a9ae8f..fa39558 100644 --- a/README.md +++ b/README.md @@ -22,6 +22,7 @@ A Netflow Parser library for Cisco V5, V7, V9, and IPFIX written in Rust. Suppor - [Pending Flow Caching](#pending-flow-caching) - [Filtering Versions](#filtering-versions) - [Error Handling Configuration](#error-handling-configuration) + - [Chained messages and error recovery](#chained-messages-and-error-recovery) - [Custom Enterprise Fields (IPFIX)](#custom-enterprise-fields-ipfix) - [Netflow Common](#netflow-common) - [Re-Exporting Flows](#re-exporting-flows) @@ -353,6 +354,20 @@ let parser = NetflowParser::builder() When using template-based protocols (V9/IPFIX), data packets sometimes arrive before their corresponding template. By default these flows are reported as `NoTemplate` and lost. With pending flow caching enabled, these flows are automatically cached and replayed when the template arrives. +**Not every `NoTemplate` flowset is replayable.** `NoTemplateInfo::reason` +distinguishes the two cases, and `NoTemplateInfo::is_replayable()` answers the +question directly: + +| `NoTemplateReason` | Meaning | Cached for replay | +| --- | --- | --- | +| `MissingTemplate` | The template ID is not in the cache. The records may decode once it arrives. | yes | +| `UndecodableBody` | The Set is well framed but its body does not match the octets present. | no | + +An `UndecodableBody` flowset is never cached, because no template arriving later +changes its octets — caching it would retry it forever. It is still reported in +`flowsets` so the loss stays visible. Such a Set is skipped rather than ending +the walk, so the Sets behind it still decode. + **Disabled by default.** Enable via the builder pattern: ```rust @@ -469,10 +484,18 @@ for packet in result.packets { // Check for errors if let Some(e) = result.error { - eprintln!("Error after {} packets: {}", packet_count, e); + eprintln!("{} packets parsed, first error: {}", packet_count, e); } ``` +**`error` is the first error, and packets may follow it.** A malformed IPFIX +message is discarded on its own rather than ending the parse: because the IPFIX +Message Header carries its own length, the parser resumes at the next message in +the datagram (see [Chained messages](#chained-messages-and-error-recovery)). So +`result.packets` can contain packets from both before and after the reported +error, and when several messages fail, `error` holds the first — the one closest +to the root cause. + **iter_packets()** yields `Result` for per-packet error handling: ```rust,ignore @@ -485,7 +508,35 @@ for result in parser.iter_packets(&buffer) { } ``` -**Error types**: `Incomplete`, `UnsupportedVersion`, `FilteredVersion`, `ParseError`, `Partial`. All implement `Display` and `std::error::Error`. +**Iteration continues past a recoverable error.** When a discarded IPFIX message +locates the next one, the iterator yields the error and resumes, so a single bad +message does not end the stream. It stops only on an error that leaves no +trustworthy boundary to resume from — a declared length below the 16-octet +Message Header, one that leaves nothing behind it, or a filtered version. v5, v7 +and v9 cannot locate the next message after a failure and always stop. + +### Chained messages and error recovery + +A single datagram may carry several chained IPFIX messages. +[RFC 7011 section 9.1](https://www.rfc-editor.org/rfc/rfc7011.html#section-9.1) +requires discarding a malformed *message*, not the rest of the datagram, so the +parser resumes at the next message whenever the header locates one. One bad +message therefore costs only its own records. + +`AutoScopedParser::parse_from_source` resumes the same way, and re-derives the +scope from each surviving message's header, so a message is never decoded with +another observation domain's templates. + +**Error types**: `Incomplete`, `UnsupportedVersion`, `FilteredVersion`, `ParseError`, `Partial`, `MalformedSetFraming`, `DecodedOutputLimitExceeded`. All implement `Display` and `std::error::Error`. + +`MalformedSetFraming` is reported when an IPFIX Set Header declares a length +that cannot be walked past — below the 4-octet Set Header, or beyond the end of +the Set region. It carries the offending Set's `offset`, its `declared` length +and the octets `available`, plus the discarded message's `export_time`, +`sequence_number` and `observation_domain_id`, so a collector can identify the +exporter at fault and account for the records it did not receive. It is +deliberately distinct from `Partial`, which continues to mean a datagram +truncated in flight. #### Error Sample Size Configuration