Conversation
SZBioMDDecomposition::save() writes firstFillFrame_ and fillValue_ unconditionally, but only compress_2d and compress_3d assign them. A 1D compression therefore put 12 bytes of whatever the object's memory last held into the stream: the same input compressed five times produced five different files. PR #145 gave the same two members an initialiser in SZBioMDXtcDecomposition, which was written from a copy of this file, and left the original. load() then took back everything it had charged. c_pos is the cursor before the reads, so c_pos - c is negative and the last line added the 37 bytes the bounded read() calls had just subtracted. Every later parse -- the encoder's load, the bin count, the decode -- ran on a budget that large again. The Xtc class has no such line. A sweep over the eleven algorithms at one, two and three dimensions puts the determinism defect in ALGO_BIOMD's 1D path alone; of the seventeen places that adjust remaining_length by hand, this is the only one that subtracts a difference taken in the wrong order. test_decomposition_save_load.cpp covers both halves of the contract for both decompositions at all three dimensionalities. Determinism is checked by constructing the decomposition over storage filled with two different patterns and comparing what save() writes, which does not depend on the allocator handing back dirty memory. Reverting either fix fails exactly the cases that fix addresses: the 1D determinism case, and the three accounting cases with "advanced 37 bytes but charged 0". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
There was a problem hiding this comment.
🟢 Approval recommended
The fixes are covered by regression tests; only a minor include nit remains.
Pull request overview
Fixes nondeterministic 1D ALGO_BIOMD serialization and incorrect load() byte accounting.
Changes:
- Initialize fill-state members deterministically.
- Remove the incorrect
remaining_lengthadjustment. - Add save/load regression tests across dimensions.
File summaries
| File | Summary |
|---|---|
tools/test/modules/test_decomposition_save_load.cpp |
Adds deterministic serialization and byte-accounting coverage; minor nit: include <cstdint> directly (1 vote). |
include/SZ3/decomposition/SZBioMDDecomposition.hpp |
Initializes serialized state and corrects load() accounting. |
Review details
Suppressed comments (1)
tools/test/modules/test_decomposition_save_load.cpp:14
uint32_tis used bynoise()but this new test does not include<cstdint>. It currently compiles only because another header happens to provide the declaration transitively; add the standard header directly so the test is portable across supported standard libraries.
#include <random>
- Files reviewed: 2/2 changed files
- Comments generated: 0
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Collaborator
Author
|
Folded into #147 — the fix and the tests that cover it belong in one change. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two defects in
SZBioMDDecomposition, the decomposition behindALGO_BIOMD. Both matter forGROMACS, which is about to depend on this algorithm through the HDF5 filter.
The same input does not compress to the same file
save()writesfirstFillFrame_andfillValue_unconditionally, and onlycompress_2dandcompress_3dassign them. A 1D compression puts 12 bytes of whatever the object's memory lastheld into the stream. Current master, one input, five runs:
A trajectory cannot be checksummed, a pipeline cannot be re-run reproducibly, and content-addressed
storage of the resulting files does not deduplicate.
PR #145 gave the same two members an initialiser in
SZBioMDXtcDecomposition, whose first line isBased on SZBioMDDecomposition.hpp. The copy was fixed and the original was not.load()gives back the bytes it chargedc_pos - cis negative, so the last line adds back exactly what the boundedread()callssubtracted. Measured: the cursor advances 37 bytes and
remaining_lengthis charged 0. Everythingparsed afterwards — the encoder's
load, the bin count,decode— runs on a budget 37 byteslarger than the buffer holds. This is the accounting #145 existed to fix, left in place in the
algorithm GROMACS uses.
SZBioMDXtcDecomposition::load()has no such line.Neither defect is present anywhere else
Determinism, four compressions of one input, every algorithm at each dimensionality:
ALGO_BIOMDOf the seventeen places in
include/SZ3that adjustremaining_lengthby hand, this is the onlyone that subtracts a difference taken in the wrong order. The rest are
-= consumedor-= <size>, several guarded by a precedingif (consumed > remaining_length) throw.The fix
void load(const uchar *&c, size_t &remaining_length) override { - // clear(); - const uchar *c_pos = c; read(site, c, remaining_length); read(firstFillFrame_, c, remaining_length); read(fillValue_, c, remaining_length); quantizer.load(c, remaining_length); - remaining_length -= c_pos - c; } @@ int site = 0; - size_t firstFillFrame_; - T fillValue_; + size_t firstFillFrame_ = 0; + T fillValue_ = 0;Tests
tools/test/modules/test_decomposition_save_load.cpp: both halves of the save/load contract, forboth MD decompositions, at one, two and three dimensions.
Determinism is checked by constructing the decomposition over storage filled with two different
patterns and comparing what
save()writes. Compressing twice on the heap instead would only catchthis when the allocator happens to return dirty memory — it fires at 4096 elements and not at
12288 on the same build. The dimensionality matrix also records that
SZBioMDXtcDecompositionwasbroken at 1D and 2D before #145 and never at 3D, so a future determinism test written only against
3D data is visibly not covering anything.
Reverting either fix fails exactly the cases that fix addresses, and nothing else:
Verification
Linux, gcc 13.3, Release with
BUILD_TESTING=ON: 0 warnings, 15/15 ctest. The compressed format isunchanged for 2D and 3D; 1D streams differ from master only in the 12 bytes that were indeterminate
there, and master's own 1D streams differ from each other.
🤖 Generated with Claude Code