You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Native Gzip reads currently offer auto and cuda, so callers cannot require the hardware decompression engine. Add decompression_backend="hardware" and --xdr-decompression-backend hardware for pooled native Gzip decoding. Unsupported devices, chunk sizes and allocations fail before submission; existing auto and cuda behavior is preserved.
Incompatible hardware/decoder options fail before input access, and older native extensions produce a rebuild diagnostic. The implementation uses documented nvCOMP/CUDA interfaces, with no dependency changes. Tests use synthetic data.
Validation
GB300, CUDA 13, nvCOMP 5.2: all 11 focused hardware cases passed after rebuilding, with vendor logs confirming hardware launches, byte-identical GZIP_1/GZIP_2 results against Astropy, and successful CUDA decoding after precise allocation/chunk rejection checks.
On a GPU without the decompression engine, the instrumented native coverage script passed all 45 tests with no skips and recorded execution in the hardware device and route checks.
CPU environment: 722 core/xDR tests passed, 200 GPU/optional-dependency skips. The synchronous hardware pool regression fails when the old forwarding expression is restored.
ARM64 and x86-64 native builds, all public pre-commit hooks and lock validation passed. After rebasing onto current main, all 217 focused CPU tests and hooks passed again with the feature file contents unchanged.
Repository hygiene
No credentials, private paths, local datasets, notebook outputs, or generated run artifacts were added.
Reviewing files that changed from the base of the PR and between b300d5b and 18b6ce2.
You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.
Use the checkbox below for a quick retry:
Trigger review
No actionable comments were generated in the recent review.
Walkthrough
The xDR decompression path now accepts hardware for native Gzip decoding. It validates hardware, buffer, and tile-size requirements and does not fall back to CUDA for explicit hardware requests. Reader stream and keepalive requirements, tests, CLI help, and user documentation are updated.
The backend options include hardware. The batch path restricts hardware requests to pooled, gzip-wrapped native Gzip calls and forwards the pooled-allocation flag. Tests cover option validation, backend forwarding, and benchmark CLI forwarding.
Native hardware eligibility and decoding src/cuphoton/xdr/src/nvcomp_batch_ext.cpp, tests/xdr/test_gzip_gpu.py, tests/xdr/test_io_gpu.py
The native path checks device support, maximum chunk size, and input and output buffer eligibility before hardware submission. Tests cover successful decoding and failures for unsupported devices or ineligible inputs.
GpuCompImageReader.read rejects hardware requests without an explicit stream. Device-heap decoding also requires an explicit keepalive owner list. FITS options and user documentation describe hardware requirements, fallback behavior, and failure cases; tests cover reader rejection and preservation of requested settings in metadata.
Possibly related PRs
NVIDIA/cuPhoton#77: Adds automatic hardware selection and explicit CUDA selection for native Gzip, which this change extends with strict hardware-only selection.
NVIDIA/cuPhoton#76: Adds native Gzip decoding and the gzip_decoder controls used by this change.
NVIDIA/cuPhoton#107: Adds deferred native Gzip decoding and completion owners, which intersect with this change’s backend handling and buffer ownership requirements.
Merge Risk: Minimal · previous review b300d
This adds an explicit hardware Gzip decompression option that fails clearly instead of falling back when the device, buffers, or tile sizes are incompatible. Existing auto and cuda behavior is preserved. No actionable merge-blocking risk is evident.
Security architecture risk: Low · up to b300d
The hardware-only option rejects incompatible requests rather than silently switching engines. No introduced privilege expansion or weakened security boundary was identified, but incomplete coverage and device execution that was not independently reproduced leave residual uncertainty.
Retained concerns
No architecture-level concerns identified.
Security review details
Security Blast Radius
inferred — The inspected change operates within the invoking process's existing CUDA context and device-buffer interface. It changes engine selection rather than introducing a remote service or new identity transition. Exposure beyond that process and GPU context, including deployment-specific tenant isolation, is not established by the available evidence.
Trust Boundaries and Controls
observed — The Python-to-native boundary retains caller-side metadata validation. The native raw-pointer interface lacks allocation-extent parameters and therefore cannot independently prove every tile's bounds; that interface and its reliance on callers predate this PR. The new allocation-capability queries establish hardware eligibility, not bounds or payload authenticity.
Resilience and Maintainability Implications
observed — Deferred native completion retains metadata and pooled scratch through event or stream completion, quarantining its resources when synchronization fails fatally. The reader retains input and output owners and uses the existing submission guard and quarantine flow. The PR adds hardware owner requirements without replacing these lifecycle controls.
Pre-merge checks: 2
Check name
Status
Explanation
Linked Issues check
Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check
Check skipped because no linked issues were found for this pull request.
Reviewing files that changed from the base of the PR and between 8307e62 and b300d5b.
Files selected for processing: 9
scripts/coverage/xdr.sh
src/cuphoton/core/fits_options.py
src/cuphoton/xdr/nvcomp_batch.py
src/cuphoton/xdr/reader.py
src/cuphoton/xdr/src/nvcomp_batch_ext.cpp
tests/core/test_fits_io.py
tests/xdr/test_gzip_gpu.py
tests/xdr/test_nvcomp_batch.py
tests/xdr/test_reader.py
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 9 remain after this review.
The reason will be displayed to describe this comment to others. Learn more.
Thanks. I read the native checks against the CUDA 13 and nvCOMP 5.2 headers and the decompression-engine FAQ. Rejections run before submission and before device scratch allocation. Zero-tile batches return before the device check. auto and cuda take the same path as before, and the driver entry-point versions match their typedefs. I could not run the engine paths.
Four inline comments below, plus two smaller points:
An extension built from current main also sets supports_decompression_backend. A hardware request then passes the Python checks and fails in native code with decompression_backend must be 'auto' or 'cuda'. A separate capability attribute would route this case to the existing "requires a rebuilt native extension" message.
GpuCompImageReader.inflate_device_heap and decode_from_device_heap reject hardware without keepalive with "requires use_native_pool=True". Neither method has that parameter. Naming keepalive, as read() names stream, would point users at the fix.
The reason will be displayed to describe this comment to others. Learn more.
Thanks. I checked the fixes at b300d5b4c.
normalize_xdr_options and merge_xdr_options now reject deflate with hardware, including a decoder inherited from a manifest. The direct helper error names gzip_decoder.
With the old pooled-flag expression restored, only test_native_backend_forwarding[hardware-gzip_pooled-False] fails. The focused xDR and FITS CPU tests pass at this head (250 passed, 59 skipped).
The rejection tests match their own messages, and the coverage script runs both engine-free tests.
An extension without supports_hardware_decompression now gets the rebuilt-extension error, and the heap helpers name keepalive.
I did not run the GPU tests. The native coverage job was still running when I checked.
Expose a hardware backend for xDR diagnostics and qualification.
Validate device limits and allocation eligibility before submission,
and preserve automatic and CUDA decoding behavior.
Signed-off-by: Trent Nelson <trentn@nvidia.com>
Reject incompatible decoder choices before input access and identify
older extensions explicitly. Exercise native hardware rejection in GPU
coverage and verify synchronous pool selection and precise error paths.
Signed-off-by: Trent Nelson <trentn@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Native Gzip reads currently offer
autoandcuda, so callers cannot require the hardware decompression engine. Adddecompression_backend="hardware"and--xdr-decompression-backend hardwarefor pooled native Gzip decoding. Unsupported devices, chunk sizes and allocations fail before submission; existingautoandcudabehavior is preserved.Incompatible hardware/decoder options fail before input access, and older native extensions produce a rebuild diagnostic. The implementation uses documented nvCOMP/CUDA interfaces, with no dependency changes. Tests use synthetic data.
Validation
Repository hygiene