feat(netflow-db): add measure-weighted MAAD estimator

[flamboh]

Note

🤖 Claude Opus 5.5 on behalf of Oliver

ELI5

Teaches the MAAD analysis to weight each address by how much traffic it sent (packets or bytes), instead of counting every address once.

Why

Address-count MAAD misses volume anomalies. In our study, one bogus flow claiming 465 G packets was invisible to address-count MAAD but drove packet-weighted D1 to 0. A source-address spike showed the opposite pattern. The two views complement each other.

Flows to exercise

1. Weighted CLI vs the upstream oracle. Setup: a release build (cargo build --release -p atlantis-netflow-db), an upstream MAAD checkout, and an address,packets CSV. Run python3 scripts/local/validate_maad.py --weighted --rust target/release/netflow-db --haskell /path/to/MAAD name=addr_packets.csv. Expected: every structure row and D0/D1/D2 match within the validator's default 1e-10 tolerance.
2. Invalid weights. Add a row with weight 0, a negative weight, or NaN and run netflow-db maad --weighted. Expected: the command exits non-zero and names the offending address.
3. Unweighted output unchanged. netflow-db maad without --weighted produces byte-identical JSON to the base branch.

Decisions and edge cases

• Matches upstream --csv --meas-col. Distinct-address counts decide prefix validity and pruning. Summed weights drive moments, D2, and D1 entropy. Weights on duplicate addresses are summed.
• Weights must be finite and > 0, and their total must be finite. There is no weighted spectrum, because upstream's spectrum collapses on weighted measures.
• Scale invariance. Multiplying every weight by a constant does not change the result. If a q's raw powers overflow or underflow, or a moment's partition sum is not finite, that q is recomputed from masses rescaled within each moment. Example: 10.0.0.0 and 10.0.0.128 with equal weights give D2 = 1/17 whether the weights are 1, 1e160, or 1e-200. Before this fix, 1e160 gave NaN and 1e-200 gave 0. Ordinary inputs keep the raw path, so unit weights still reproduce the unweighted result exactly.
• Masses that normalize to zero. A positive weight can be too small to register against the total, such as 1e-300 next to 1e50. It contributes 0 to the entropy (the 0·log 0 = 0 limit), so D1 stays finite instead of becoming NaN.
• Performance: all measures share one prefix walk, each distinct mass is raised to a power once per q, and q values run in parallel.

Follow-up

#112 covers the integration work: handling zero-packet flows, storing weighted results, and showing them in the dashboard. This PR only adds the estimator and the CLI/validator surface.

Verification

• Automated: cargo fmt --all --check, cargo clippy --workspace --all-targets --all-features --locked -- -D warnings, and cargo test -p atlantis-netflow-db maad. The tests cover:
  • unit weights reproduce the unweighted result exactly;
  • permutation and duplicate invariance;
  • a hand-computed case;
  • an independent ordered-map reference;
  • scale invariance across 1e-300…1e300;
  • finite D1 when masses normalize to zero.
• The unweighted JSON was byte-identical to main on fixtures and 20 real windows. Weighted output matched the oracle to about 1e-15.
• Remaining manual verification: none. Flow 1 is an optional spot check for reviewers who have an upstream MAAD checkout.

---

Made by Claude Opus 5.5 (with Opus 5.5 subagents) via Claude Code.