Skip to content

feat: compute and chart IPv6 MAAD - #110

Open
flamboh wants to merge 4 commits into
maad/03-pin-bumpfrom
maad/04-ipv6
Open

flamboh wants to merge 4 commits into
maad/03-pin-bumpfrom
maad/04-ipv6

Conversation

@flamboh

@flamboh flamboh commented Sep 25, 2026 •

Copy link
Copy Markdown
Owner

Note

🤖 Claude Opus 5.5 on behalf of Oliver

ELI5

Runs the MAAD address-structure analysis on IPv6 traffic too, not only IPv4, and adds an IPv4/IPv6 switch to the dashboard spectrum card and the file detail page.

Setup

This PR bumps the product contract: both family MAAD configs are part of the pipeline identity. The pipeline rejects an existing product database, so build a fresh one.

  1. Build the nfdump fork (./vendor/scripts/compile-nfdump.sh).
  2. Choose an input window that contains IPv6 flows. Any nfcapd day from a source that carries IPv6 works. To check, run target/nfdump/libexec/nfdump -r <file> 'ipv6' -c 5 and confirm it prints flows.
  3. Build a new product for one day:
    ./scripts/netflow-db.sh pipeline --dataset <id> --start-date <day> --end-date <day> \
      --database-path data/<id>-ipv6/netflow.sqlite --nfdump target/nfdump/libexec/nfdump
    Then point the dashboard at that database.
  4. CLI-only check without captures: ./scripts/netflow-db.sh maad --ipv6 tools/netflow-db/tests/fixtures/maad-conformance/ipv6-mixed/input.txt.

Flows to exercise

  • Pipeline output. address_structure_stats has ip_version = 6 rows alongside the v4 rows, with prefix lengths /23–/64. verify --require-maad-data passes, and fails when v6 traffic exists but v6 MAAD rows are missing.
  • Dashboard. On /datasets/<id>, pick IPv6 on the spectrum card:
    • The URL gains ipVersion=6 and the card refetches.
    • Reload keeps IPv6.
    • Reset view returns to IPv4, which removes the param.
  • Drilldown. At 5-minute grouping, click any dashboard chart. The file page opens with ipVersion=6, and its family switch shows IPv6.
  • File page. Reload keeps the family, and Next File keeps it too. Switching the family replaces the URL (no new history entry) and refetches MAAD.
  • Validation. ?ipVersion=5 on MAAD API routes or the file page returns 400. On the dashboard, the search-param schema rejects it and falls back to IPv4.
  • Compare. compare against an older reference with no IPv6 rows now fails with unexpected_candidate_only_rows. Compare IPv6-era products only with IPv6-era references.

Decisions and edge cases

  • IP family is a URL search param (ipVersion, 4 or 6, default 4, left out of the URL when 4). Every dashboard→file drilldown and the Next File link carry it.
  • Next File is now a link instead of a GET form. The form dropped every query param (dataset, direction, ipVersion), because GET submission replaces the action's query string.
  • compare has no exemption for a missing IP family. An earlier revision accepted candidate-only rows whenever the reference lacked that family, which also let unrelated-source rows through. That was removed as greenfield (no compat code).
  • IPv6 uses upstream's /23–/64 range, and the nearly-full test uses 128 − pl. IPv4 output is unchanged. The two families are separate series, not comparable points. On anonymized internal v6 sources the structure below ~/40 mostly reflects the anonymizer; that labeling question is out of scope.

Verification

  • Automated: bun run format, bun run lint, bun run typecheck, bun run test:db (v6 goldens, compare rejection tests), bun run test:web, bun run test:e2e (new maad-ip-version.spec.ts covers IPv6 selection → reload → drilldown → reload → next file).
  • Manual, still needed: a real-capture product build with IPv6 traffic (steps above), then a visual check of the IPv6 spectrum and the file-page charts.

Made by Claude Opus 5.5 (with Opus 5.5 subagents) via Claude Code.

@flamboh
flamboh added this pull request to stack #113 September 25, 2026 10:02
@flamboh flamboh changed the title maad/04 ipv6 feat: compute and chart IPv6 MAAD Sep 25, 2026
Port the upstream IPv6 MAAD estimator: 128-bit prefixes, the nearly-full
test against 128 - pl, and upstream's default /23-/64 prefix range. The
q grid and full threshold match IPv4.

The pipeline now writes address_structure_stats rows for ip_version 6
scopes as well as 4. The product identity records both family configs
and bumps the MAAD contract version, so existing databases must be
rebuilt. netflow-db maad and validate_maad.py gain --ipv6, and IPv6
goldens are generated from the pinned oracle.

The web structure and spectrum routes take a validated ipVersion
parameter (default 4), and the dashboard spectrum card and file detail
page get an IPv4/IPv6 selector.
…rsing

compare allows candidate-only MAAD rows for an IP version the reference has no MAAD rows for, so a pre-IPv6 reference no longer fails. verify --require-maad-data requires IPv6 MAAD rows when IPv6 traffic exists in the window, and verify-web requests every MAAD route with ipVersion=6. The web accepts only the literal ipVersion strings 4 and 6. The conformance doc notes that IPv6 nearly-full pruning is inert at the default range and drops a stale reference to the new head; a redundant route test is removed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant