Skip to content

XMAT-021: Headless XMAT - add xmat, a command line that analyzes captures without the GUI - #158

Draft
Gerardo Malpartida (GerardoMal-ATG) wants to merge 5 commits into
gerardomal/capture-file-readingfrom
gerardomal/headless-cli
Draft

Gerardo Malpartida (GerardoMal-ATG) wants to merge 5 commits into
gerardomal/capture-file-readingfrom
gerardomal/headless-cli

Conversation

@GerardoMal-ATG

Copy link
Copy Markdown
Contributor

Adds xmat.exe, a command line that analyzes existing captures without the GUI (#58), and parity checks that classify every discovered GUI entry point as supported, planned or GUI-only, so command-line support is tracked as features are added. It also hardens and speeds up the analysis code that both front ends now share.

Stacked on #156. This diff shows only the headless changes; it retargets to main when #156 merges. xmat opens captures through #156's checks, so it refuses the same files as the GUI.

Try it (from the build output, next to the GUI)

.\xmat.exe analyze capture.saz --out saz-results
.\xmat.exe analyze capture.xmatcap --out xmatcap-results --fail-on findings
.\xmat.exe analyze big.saz --out big-results --max-records 250000
.\xmat.exe doctor --for analyze
.\xmat.exe help

Commits (each builds and passes tests)

  1. UI tests clean up their storage.
  2. Shared analysis: the GUI analyzer becomes an adapter over AnalyzeCaptureOperation, with bounded SAZ reading, faster rules under a shared work budget, and repeated and polling calls grouped by their exact request body.
  3. Save refuses to overwrite an open capture and replaces files atomically.
  4. The xmat command line (analyze, doctor, JSON output, exit codes, --max-* limits), the parity checks, docs/cli.md and CONTRIBUTING.
  5. CI ships and smoke-runs xmat.exe.

Please review first

  • docs/cli.md: exit codes, the xmat-cli/1 and xmat-analysis/1 JSON, limits and budgets.
  • CONTRIBUTING: the parity gate and the shared operations layer.
  • Edits to existing code (50 files, +6,020/−739): the GUI analyzer adapter, rules engine and rules, data layer and Save.
  • The safety and contract tests. Only the bulk fixture data can be skimmed.

GUI behavior changes

  • Small-Batch Detection keeps its statistics per run instead of accumulating them across consoles.
  • Two different request bodies sent to the same URI are no longer counted together as repeated or polling calls when their old 32-bit hashes collide.
  • A capture that exceeds the analysis budget fails with an error instead of continuing unbounded rule evaluation. In testing, a regular 1 Hz polling fixture (same URI and body) completed at 75 minutes and exceeded the budget at 90; that is an observation, not a duration cutoff.
  • Saving over a capture that is open in a tab is refused with an error message, and saves replace files atomically.
  • Otherwise, on four capture fixtures, including a 486-call capture, GUI analysis produces the same reports as Harden capture file open and import #156. Report entry order already varies between runs on main.

Limits: 512 MiB input, 256 MiB data, 32 MiB per record and 100,000 records by default, raisable to 2,048, 512, 128 and 250,000. On the input-limit test corpus, runs at the defaults stayed under 1.4 GiB and 15 seconds, and at the maximums under 2.4 GiB and 25 seconds. These are observations, not bounds: run xmat with time and memory limits. --out must be on storage with Windows permissions, such as NTFS.

Not in this PR: command-line versions of live capture, sanitized conversion, HTML reports and network-trace analysis; the GUI already has these. docs/cli.md lists them as planned, and xmat rejects them as unknown commands until they ship.

Once this merges, a PR that adds GUI commands or windows (for example #145) also classifies each new entry point in CapabilityRegistry.cs; the parity test names any it misses.

The UI test fixture deleted its per-run app storage only at process exit, which
the test host usually kills first, so runs leaked XMAT-UI-Tests-* folders. A
custom assembly runner now stops the app, closes its log and deletes the run's
own folder before the assembly reports completion; a folder that can't be
deleted fails the run instead of leaking.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 270d0c0c-8578-4bd6-aa1d-c32a56075710
AnalyzeCaptureOperation runs the existing rules on .saz files, .xmatcap files
and the GUI's records, and the GUI analyzer becomes an adapter with unchanged
report formats. It returns a summary of findings per device and console without
request data. Fiddler archives pass the same archive check as GUI imports and
are then read within source, entry, total and record limits; HTTP bodies stay
opaque, and malformed parts are skipped with data-free warnings. Capture
databases are opened through the same guarded, read-only path as the GUI, and
the input's SHA-256 is computed from that same open file. The polling and
repeated-call rules pair and link calls without rescanning them and group calls
by their exact request body instead of a 32-bit hash, so findings change only
where two different bodies collided. The throttled-call, batch-frequency and
call-frequency rules no longer rescan their calls or build regular expressions
per call, and Small-Batch Detection keeps its statistics per run. All rule work
in one analysis shares a work and memory budget; a capture that exceeds it
fails with analysis.rules.work-limit, or an error in the GUI, instead of
continuing unbounded rule evaluation. Report JSON is streamed to its files.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 270d0c0c-8578-4bd6-aa1d-c32a56075710
Saving to a .xmatcap file that XMAT has open in a tab, including through a
different letter case or a hard link, is now refused up front with an error
message instead of failing with an unhandled file-in-use error. A save now
writes the capture to a short temporary file in the destination folder and then
moves it over the destination, so a failed or interrupted save leaves the
original file untouched. Save failures show a message that contains no capture
data.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 270d0c0c-8578-4bd6-aa1d-c32a56075710
xmat.exe analyzes .saz and .xmatcap files without the GUI: analyze, doctor,
help and --version, one JSON document on stdout, stable exit codes, staged
atomic output and Ctrl+C handling. xmat analyze takes --max-input-mib,
--max-data-mib, --max-record-mib and --max-records within fixed ranges;
anything else is a usage error that names the option but never its value.
Captures the GUI refuses are refused here too, with exit code 3: SQLite journal
or write-ahead log files next to the capture (-journal, -wal or -shm), or a
database in write-ahead-log mode, report analysis.input.live-database; a
location without a normalized path reports analysis.input.location-unsupported;
and a file another program has open for writing reports
analysis.input.unreadable. xmat asks the .NET garbage collector to conserve
memory; the GUI's setting is unchanged. Every GUI entry point is classified as
supported, planned or GUI-only, non-UI code may not add dialog or dispatcher
calls, and the headless code may not use GUI globals. docs/cli.md covers usage,
output, exit codes, compatibility and the capability table, and CONTRIBUTING
describes the parity rules and the operation layer.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 270d0c0c-8578-4bd6-aa1d-c32a56075710
CI copies xmat.exe next to the GUI and smoke-runs its --version and doctor
commands from the packaged folder, and the README points to the command line.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 270d0c0c-8578-4bd6-aa1d-c32a56075710
@GerardoMal-ATG Gerardo Malpartida (GerardoMal-ATG) changed the title Add xmat, a command line that analyzes captures without the GUI XMAT-021: Headless XMAT - add xmat, a command line that analyzes captures without the GUI Oct 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant