Skip to content

feat(web): add weighted MAAD dashboard view - #120

Merged
flamboh merged 4 commits into
feat/cluster-shard-mergefrom
maad/08-weighted-dashboard
Sep 28, 2026
Merged

flamboh merged 4 commits into
feat/cluster-shard-mergefrom
maad/08-weighted-dashboard

Conversation

@flamboh

@flamboh flamboh commented Sep 28, 2026

Copy link
Copy Markdown
Owner

Note

🤖 Claude Opus 5.5 on behalf of Oliver

Explain Like I'm Lost

MAAD (multifractal address-structure analysis) is computed per bucket for three measures: plain addresses, and packet- and byte-weighted addresses. The pipeline already stores all three, but the dashboard only charted the address measure. This PR lets the dashboard switch between them and charts the stored D0/D1/D2 dimensions over time.

Why

Before this PR, you could only see weighted MAAD on a single file's detail page, so you couldn't watch how packet or byte weighting changes the address structure over a range. The MAAD controls were also spread across individual cards.

Implementation

Stacked on #106.

What changes

  • MAAD group in Controls. A second toolbar row labelled MAAD: holds the measure (Addresses | Packets | Bytes) and the IP address family (IPv4 (/8–/24) | IPv6 (/23–/64)). Both are stored in the URL, as ?measure=addresses|packets|bytes and the existing ?ipVersion=4|6. Defaults (addresses, 4) are left out of generated links. The family applies to both MAAD cards, so this is the only place it appears. Reset View sets both back to their defaults. Datasets built without MAAD don't show the group.
  • Tidier Controls toolbar. Reset View moves into the card header. The granularity control keeps all five options on one line. All toolbar segmented controls are horizontal and the same size. When locality is on, Direction shares the second row with MAAD.
  • New "MAAD Dimensions" card. It has two compact selectors, Source | Destination and D0 | D1 | D2, and draws one line per selected source. The default is Source D1, because D1 responds to the weighting and D0 barely does. Each source keeps a fixed colour from a colourblind-checked palette, with separate light and dark shades. The y-axis fits the data with 10% padding. The tooltip shows three decimals. Day boundaries get x-axis labels, as in Traffic Overview. Click and range drilldown work like the other line charts.
  • Spectrum card. The source and side are chosen with segmented controls in one row above the chart. With Packets or Bytes selected, the card shows a compact placeholder ("The spectrum is only computed for the Addresses measure…") and sends no spectrum-stats request.
  • Card headers show the active measure and family, for example MAAD Dimensions Bytes · IPv4 (/8–/24). That keeps the context visible when a card sits far below Controls.
  • Datasets built without MAAD (--no-maad, so no maad_q_grid rows): both MAAD cards collapse to a short "MAAD was not computed for this dataset…" placeholder and send no MAAD requests.
  • Drilldown keeps the measure and family. A 5-minute click from any dashboard chart opens /netflow/files/<slug>?…&ipVersion=6&measure=bytes. On the file page, the family and measure controls read and write those URL params, and an invalid value returns 400. The Next File link carries dataset, direction, ipVersion and measure.
  • New APIs:
    • GET /api/netflow/dimension-stats takes the same params as structure-stats: routers, granularity, startDate and endDate (a half-open [start, end) window), direction, ipVersion and measure. It returns per-router coverage timelines with {saD0, saD1, saD2, daD0, daD1, daD2}, where null means too few addresses.
    • GET /api/netflow/maad-status returns { computed }.
  • The chart-order storage key moves to v6 because a card was added, so any saved custom order resets once.

How to review

Setup: a database built with MAAD. Either a real one or the Playwright fixture works (bun run test:e2e seeds fixture-router with one 5-minute bucket for all three measures).

  1. At about 1440px wide, open /datasets/<id>?startDate=…&endDate=…&groupBy=hour. Controls should show two tidy rows (dates, granularity and sources first, then MAAD), with no wrapped or stacked segmented controls.
  2. Scroll to MAAD Dimensions. You should see one line per source for Source D1, a y-axis fitted to the data, and day labels on the x-axis. Switch to Destination and D2: the chart redraws and the source colours stay the same.
  3. In the MAAD group, click Packets, then Bytes. The URL gains measure=…, the dimensions request carries measure=, and the card header updates. The spectrum card shows the "only computed for the Addresses measure" placeholder.
  4. Click IPv6 once. The URL gains ipVersion=6, and both MAAD cards refetch with ipVersion=6.
  5. Click Addresses. The spectrum chart and its source and side selectors come back.
  6. With measure=bytes, ipVersion=6 and groupBy=5min, click a point on any chart. The file page opens with Bytes and IPv6 selected. Click Next File: both stay selected.
  7. Open the dashboard with ?measure=flows: it falls back to Addresses. /netflow/files/<slug>?measure=flows returns 400.
  8. Open a dataset built with --no-maad. Controls has no MAAD group, and both MAAD cards show the short "MAAD was not computed" placeholder.
  9. Repeat step 2 in dark mode and at about 1024px wide. Sources wrap onto their own line and nothing overflows.

Decisions and edge cases

  • The measure and family live in the global Controls, not in the card headers. Both are URL state that every chart's drilldown carries, and both MAAD cards share them. The cards can also be reordered and end up far apart, so controls in the card headers would either appear twice or be stranded in one card. The card subtitles show the current selection instead.
  • The weighted view charts the stored scalar dimensions (D0/D1/D2), not tau(q) curves. They are the only MAAD result that is defined for every measure and can be summarised over time. Structure curves stay on the file page.
  • Sources past the eighth use a neutral grey instead of cycling hues. A source's colour depends on its position in the full source list, so turning a source off doesn't recolour the others.
  • The x-axis ticks on the IP, protocol, dimensions and spectrum cards now sit on bucket starts, so day boundaries get labels. When fewer than two buckets with data are in view (for example a single 5-minute bucket), they fall back to the default ticks.
  • "MAAD computed" is detected from maad_q_grid, which the pipeline writes only when MAAD runs. In a D1 deployment where one database is shared across datasets, this flag is database-wide, not per dataset.
  • An unknown measure in the dashboard URL falls back to Addresses, which matches how the other search params behave. The APIs and the file page reject it with 400.
  • Buckets whose dimensions are NULL (too few addresses) show as gaps, not zeros.
  • The e2e fixture's MAAD rows store full 33-value tau/tau_sd blobs, so they pass the blob validation added in feat: store MAAD results as compact f32 rows #119.

Verification

  • Automated: bun run format, bun run lint, bun run typecheck, bun run test:web (181 tests), bun run test:db and bun run test:e2e (14 tests) all pass.
    • The web tests add tick-placement and dimension-key tests, plus the api-maad-dimensions, dataset page load, schema, file navigation and file page tests.
    • dashboard-maad-measure.spec.ts covers switching the measure, a single shared family control that refetches both cards, choosing a side and dimension, rejecting an invalid param, and drilling down to a file.
    • maad-ip-version.spec.ts now drives the shared family control, and checks that the file page's measure and family both survive Next File.
  • Manual: once the full reprocess lands, check the dimensions chart against a real reprocessed dataset. The screenshots come from a synthetic two-source database with hourly buckets.

UI Changes

Screenshots are pending upload. They'll be added here: a before image of the first iteration (before the redesign), and after images for Addresses, Bytes, a dataset without MAAD, dark mode and a 1024px width.


Made by Claude Opus 5.5 in Claude Code (T3 Code).

/api/netflow/dimension-stats returns the stored d0, d1 and d2 per source,
bucket and address side for one measure, IP version and direction over a
half-open window, with the usual coverage timelines. /api/netflow/maad-status
reports whether the product ran MAAD, detected from its maad_q_grid rows.
… bytes

The dashboard gets a MAAD measure control whose value lives in the measure
search param. A new MAAD Dimensions card charts D0/D1/D2 for the selected
measure. The spectrum card explains that weighted measures have no spectrum,
and both MAAD cards say when a dataset was built without MAAD. Drilldowns carry
the measure to the file page, whose measure control now reads and writes it.
The Controls card now reads as one toolbar. Reset View sits in the header,
granularity keeps its five options on one line, and a labelled MAAD group holds
the measure and the address family. Both MAAD cards share that family, so it
appears once instead of in each card. Segmented controls lay out horizontally,
including direction and the file page's MAAD controls.

MAAD Dimensions charts one side and one dimension at a time, with one line per
source. Sources take fixed slots from a validated categorical palette with
light and dark steps, the y-axis fits the data, and the tooltip shows three
decimals. The spectrum card picks its source and side with segmented controls.
Card headers name the active measure and family. Unavailable MAAD cards show a
compact placeholder, and their reserved height shrinks to match.

Linear time axes in these cards put ticks on bucket starts, so day boundaries
get labels as they do in Traffic Overview. The default ticks stay when fewer
than two buckets fall inside the plotted range.
@flamboh
flamboh added this pull request to stack #113 September 28, 2026 19:38
@flamboh
flamboh merged commit e0d1be9 into main Sep 28, 2026
3 checks passed
@flamboh
flamboh deleted the maad/08-weighted-dashboard branch September 28, 2026 19:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant