Fix two ALGO_BIOMD defects, and run the MD algorithms on MD data

[ayzk]

GROMACS is about to depend on ALGO_BIOMD and ALGO_BIOMDXTC through the HDF5 filter. Nothing ran
them on a molecular dynamics trajectory in the layout they were written for, and two defects in
ALGO_BIOMD were live.

Two defects in SZBioMDDecomposition

The same input does not compress to the same file. save() writes firstFillFrame_ and
fillValue_ unconditionally; only compress_2d and compress_3d assign them. A 1D compression
puts 12 bytes of whatever the object's memory last held into the stream. Master, one input, five
runs:

308baf4d6e253fd3563f36df289ef503  36dac7922e24e69ca5883391f696a28a
6a9c39852ba515b86d80edc2f3473d4d  1d9db36f3c1a361927c6580651ff4b06
ef8e43ac160ebcaadc4722f1face67dc

A trajectory cannot be checksummed and a pipeline cannot be re-run reproducibly. PR #145 gave the
same two members an initialiser in SZBioMDXtcDecomposition, whose first line is
Based on SZBioMDDecomposition.hpp. The copy was fixed and the original was not.

load() gives back the bytes it charged.

const uchar *c_pos = c;          // the cursor BEFORE the reads
...
remaining_length -= c_pos - c;   // negative, so this adds back

Measured: the cursor advances 37 bytes and remaining_length is charged 0. Everything parsed
afterwards runs on a budget 37 bytes larger than the buffer holds — the accounting #145 existed to
fix, left in place in the algorithm GROMACS uses.

Neither is present anywhere else. Four compressions of one input, each of the seven algorithms
at one, two and three dimensions: ALGO_BIOMD at 1D gives four distinct outputs, the other twenty
combinations are deterministic. Of the seventeen places in include/SZ3 that adjust remaining_length by hand,
this is the only one that subtracts a difference taken in the wrong order.

What was not covered

SZBioMDXtcDecomposition reads its dimensions as {frames, atoms, xyz} and takes a different path
for 3D. Every MD field the suite had — the three EXAALT sets — is 1D or 2D, and every 3D field is a
climate or turbulence field. Nothing had three as its last dimension, so this never ran:

/* Start from the last frame and look for frames filled with the same value.
 * Those frames do not need to be compressed - they can all just be filled. */

Master also had no decomposition unit tests at all.

Seven trajectories

fetch_md_trajectory.py downloads the MDAnalysisData benchmark set from the figshare files that
package points at, verifies each against a recorded sha256, and writes xyz.f32 as
{frames, atoms, 3} plus a 2D file per coordinate. Using the URLs rather than the package keeps
the data to immutable files; MDAnalysis is still needed to read DCD, XTC and NetCDF.

dataset	frames x atoms	format
md-peg-1chain	50 x 13406	NetCDF
md-ifabp-water	500 x 12445	DCD
md-adk-equilibrium	4187 x 3341	DCD
md-cg-fiber	2221 x 8364	DCD
md-membrane-peptide	1001 x 18727	XTC
md-nhaa-equilibrium	5001 x 60702, leading 200	XTC
md-yiip-equilibrium	901 x 111815, leading 100	XTC

Three are GROMACS XTC. nhaa and yiip are 911M and 302M elements in full, past what a job can do
in half an hour, so they are cut to a leading slice. The published frame counts for those two are
each one short of what the files hold; the script validates the shape it is told against the shape
it reads, which is how that surfaced.

Tests

test_decomposition_save_load.cpp covers both halves of the save/load contract for both MD
decompositions at one, two and three dimensions. Determinism is checked by constructing the
decomposition over storage filled with two different patterns and comparing what save() writes,
which does not depend on the allocator returning dirty memory — compressing twice on the heap fires
at 4096 elements and not at 12288 on the same build. Reverting either fix fails exactly the cases
that fix addresses and nothing else: the 1D determinism case, and the three accounting cases with
advanced 37 bytes but charged 0.

test_biomd.cpp covers the two algorithms through the public API: the trajectory layout at four
error bounds, trailing fill frames, every frame after the first being fill, one frame per chunk, the
smallest multi-frame shape, 1D and 2D input, input shorter than one triplet, and determinism at all
three dimensionalities.

An adversarial read of the first version of that file found four cases that could not report the
defect they name, all fixed here: the determinism case ran only at 3D, where the header is a
function of the input whatever the memory held; both fill-frame cases used values sitting on the
quantisation grid, so deleting findFillValueAndFirstFilledFrame left the suite green; {1,1024,3}
is a 2D case because setDims drops a dimension of 1; and kBoundSlack was tighter than the
eb * 1.1 LinearQuantizer accepts.

Compression ratio and timing

Nothing in the suite looked at either, so a change that halved the ratio passed as long as the error
bound held. Every case now reports its compressed size, ratio and both timings; the driver collects
them into integration_metrics.csv and the workflow keeps it as an artifact.

Sizes are checked against compression_baseline.json with a 5% band in either direction —
compression is deterministic, so a size that moved means the output moved, and an improvement is
worth recording too. A key with no baseline is reported, not failed, so a new dataset does not break
its first run. update_compression_baseline.py merges the artifacts of a run into the file. The
baseline is keyed by harness, so the HDF5 filter path is watched separately from the CLI.

Timings are recorded and not gated; runner-to-runner variance needs a few runs of data before a
threshold means anything.

Verification

Linux, gcc 13.3, Release with BUILD_TESTING=ON: 0 warnings, all tests pass. The integration driver
on md-peg-1chain: 64/64, both algorithms and both harnesses at four error bounds over the 3D and
the 2D field. The HDF5 filter passes on the 3D layout in all three chunk modes. All seven
trajectories download, verify and convert. The baseline gate was exercised in all three states:
absent, matching, and drifted (which fails the case and the run).

The compressed format is unchanged for 2D and 3D; 1D ALGO_BIOMD streams differ from master only in
the 12 bytes that were indeterminate there, and master's own 1D streams differ from each other.

🤖 Generated with Claude Code

[Copilot]

🟢 Approval recommended

The changes are test/CI-focused, internally consistent (API + integration harness), and the added metrics/coverage align with the PR goals without introducing correctness regressions in the reviewed diffs.

Pull request overview

This PR expands SZ3’s test coverage to exercise ALGO_BIOMD and ALGO_BIOMDXTC on real molecular-dynamics trajectory layouts ({frames, atoms, 3}), adds an MD trajectory fetch/convert path to the integration suite, and records compression ratio + timing metrics as CI artifacts to help detect regressions beyond error-bound correctness.

Changes:

• Added a new GTest module (test_biomd.cpp) that round-trips synthetic trajectories through the public SZ3 API across both BioMD algorithms, including fill-frame edge cases and determinism.
• Extended the integration driver to support mdtraj:<dataset> datasets via a new fetch/convert script, and added seven MD datasets to datasets.json and the integration workflow matrix.
• Recorded per-case compression ratio and compress/decompress timing via a METRICS ... line, aggregated into integration_metrics.csv, and uploaded as a workflow artifact.

File summaries

File	Description
tools/test/modules/test_biomd.cpp	New unit tests covering BioMD/BioMDXTC on trajectory-shaped data and important edge cases (fill frames, low element counts, determinism).
tools/test/integration/test_sz3_executable.py	Measures compress/decompress time, enforces ratio ≥ 1, and emits a parseable METRICS line for aggregation.
tools/test/integration/integration_test_driver.py	Adds mdtraj: dataset preparation, captures test stdout for metrics parsing, and writes integration_metrics.csv.
tools/test/integration/fetch_md_trajectory.py	New utility to download/checksum/convert MDAnalysisData trajectories into raw float32 field files used by the integration suite.
tools/test/integration/datasets.json	Registers seven MD trajectory datasets (including limited-frame variants for very large trajectories).
.github/workflows/integration_test.yml	Adds workflow concurrency, runs MD trajectory datasets in the matrix, installs MDAnalysis, and uploads metrics artifacts.

Review details

• Files reviewed: 6/6 changed files
• Comments generated: 0
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.