[mattfil]

Summary

This is an attempt to introduce filters in vtkhdf backend, generated with AI assistance with GPT-sol-6.1. If ok, a next issue could introduce the same filters for h5py backend. The benchmark folder is just to reproduce the timings reported in this pr and can be removed. Closes #96.

Adds lossless HDF5 compression through the existing backend_args argument of write_mesh, write_point_data, and write_cell_data.

storage_args = {
    "compression": "gzip",
    "compression_opts": 4,
    "shuffle": True,
}

io4dolfinx.write_mesh(
    "results.vtkhdf",
    mesh,
    backend="vtkhdf",
    backend_args=storage_args,
)

• Compression remains disabled by default.
• Bulk mesh and field datasets use automatic chunking; small bookkeeping datasets remain uncompressed.
• Filtered MPI writes are collective, including participation from ranks with no local data.
• Invalid options and inconsistent settings across ranks are rejected before writing.
• Existing datasets retain their filters on append; explicitly conflicting settings raise an error before resizing.
• Adds documentation, round-trip and compatibility tests, and reproducible benchmark/report scripts.

Evidence

Before: VTKHDF writers provided no compression configuration.

After: Mesh and scalar/vector point/cell data can be compressed and read transparently through io4dolfinx and a VTK reader.

Validation:

• 55 passed in serial across the compression, VTKHDF, and mesh-writer tests.
• 48 passed per rank with four MPI ranks.
• Tests cover appended timesteps, empty ranks, filter conflicts, unavailable filters, and byte-for-byte preservation of special floating-point values.
• Repository-wide Ruff lint and formatting checks pass.
• The documented example executes successfully; the standalone documentation page builds.
• Full serial testing reached 1,439 passes before stopping because ADIOS2 is unavailable locally.
• Full-project mypy reports the same _serendipity_coordinate_element error reproduced on the unchanged upstream baseline.

Benchmarks

Comparison: uncompressed → gzip level 4 + shuffle.

Each configuration uses one warm-up and five measured repetitions; tables report medians. The workload contains a triangular mesh, a smooth point-vector field, and a smooth cell-scalar field.

Measurements were taken on one Intel Core Ultra 7 255HX workstation using ext4, HDF5 1.14.6, h5py 3.16.0, and DOLFINx 0.11.0. MPI processes were bound to cores with one thread per process.

Write times include mesh and field writes plus a final fsync. Read times include mesh reconstruction and field reads using warm filesystem caches. Sizes are decimal MB.

512 cells

MPI ranks	File size (MB)	Size reduction	Write (ms)	Read (ms)
1	0.089 → 0.064	28.5%	9.8 → 10.3	8.0 → 8.3
2	0.089 → 0.064	28.4%	11.0 → 11.7	8.6 → 9.5
4	0.089 → 0.064	28.6%	13.6 → 13.5	10.7 → 11.3
8	0.089 → 0.064	28.4%	20.0 → 19.1	15.7 → 18.4

131,072 cells

MPI ranks	File size (MB)	Size reduction	Write (ms)	Read (ms)
1	8.618 → 0.812	90.6%	336.1 → 58.8	304.5 → 325.4
2	8.618 → 0.803	90.7%	202.4 → 35.0	185.4 → 190.1
4	8.618 → 0.787	90.9%	217.8 → 33.1	126.7 → 131.6
8	8.618 → 0.814	90.6%	259.1 → 32.5	77.4 → 80.9

1,179,648 cells

MPI ranks	File size (MB)	Size reduction	Write (ms)	Read (ms)
1	76.893 → 5.272	93.1%	3,378.2 → 324.9	3,258.9 → 3,421.8
2	76.893 → 5.247	93.2%	2,356.5 → 187.1	2,236.0 → 2,155.5
4	76.893 → 5.266	93.2%	1,976.6 → 123.7	1,367.4 → 1,553.0
8	76.893 → 6.089	92.1%	2,635.4 → 97.3	897.4 → 1,104.9

For this workload, compression substantially reduces medium/large file sizes and write times. Reads are generally slightly slower, while tiny files show little timing benefit.

These measurements compare the backend’s current I/O paths: compressed writes enable collective I/O and different HDF5 allocation behavior. They do not isolate gzip overhead, and compression ratios will differ for noisy fields.

Merge Danger

Door: Two-way
Blast Radius: VTKHDF

Compression is opt-in and preserves existing public signatures. The main behavior changes requiring care are collective filtered writes and disjoint topology-offset writes; both are covered by MPI round-trip and empty-rank tests.