Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 25 additions & 9 deletions docs/components/xdr.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,16 @@ floating-point bit patterns; choose explicit modes for comparisons or debugging.
| --- | --- | --- | --- |
| `postprocess` | `--xdr-postprocess` | `auto`, `fused`, `separate`, `native` | Restore byte order and scatter decoded tiles. |
| `gzip_decoder` | `--xdr-gzip-decoder` | `auto`, `gzip`, `deflate` | Use native Gzip when available; otherwise decode aligned raw DEFLATE. |
| `decompression_backend` | `--xdr-decompression-backend` | `auto`, `cuda` | Let nvCOMP choose a compatible decompression engine for native Gzip, with CUDA fallback. |
| `decompression_backend` | `--xdr-decompression-backend` | `auto`, `cuda`, `hardware` | Let nvCOMP choose a compatible decompression engine for native Gzip, with CUDA fallback. |

`postprocess="fused"` selects the same kernel as `auto`; `"separate"` runs
individual restoration kernels. `gzip_decoder="gzip"` requires native Gzip
support. `"deflate"` strips the Gzip framing and aligns payloads for raw
DEFLATE. `decompression_backend="cuda"` requires a native extension with
backend selection support and selects CUDA kernels explicitly. The native
raw DEFLATE decoder uses CUDA in either backend mode.
backend selection support and selects CUDA kernels explicitly.
`decompression_backend="hardware"` requires native Gzip hardware decoding,
with no CUDA fallback. Use it with `gzip_decoder="auto"` or `"gzip"`; the
native raw DEFLATE decoder supports only the `auto` and `cuda` backends.

`postprocess="native"` requires the [native imaging extension](../packaging.md).
For `GZIP_1` and `GZIP_2` image tiles it runs byte-order restoration, unshuffle,
Expand Down Expand Up @@ -109,9 +111,9 @@ defaults. Full batch FITS loading
still requires the native FITS planner; codec fallback does not remove that
requirement.

When xDR decodes compressed HDUs, explicit `gzip` or `cuda` requests fail clearly
if the installed extension cannot honor them. Rebuild an older source extension
using the [native build instructions](#native-extension-availability). The `auto`
When xDR decodes compressed HDUs, explicit `gzip`, `cuda` or `hardware`
requests fail clearly if the installed extension cannot honor them. Rebuild
an older source extension using the [native build instructions](#native-extension-availability). The `auto`
choices select available capabilities before decoding. I/O or decoder exceptions
propagate; they do not trigger another reader, codec or postprocessing attempt.
The supported FITS compression formats remain `GZIP_1` and `GZIP_2`.
Expand All @@ -131,13 +133,27 @@ tiles beyond the hardware limit.
Batch readers retain native pooled scratch buffers. The low-level non-pooled
path, including `GpuCompImageReader.read()` without an explicit stream, uses
`cudaMallocAsync` scratch, which is not hardware-decompression capable. That
path selects CUDA directly, including with `decompression_backend="auto"`,
to avoid a failed hardware attempt on each call. Custom CuPy allocators can
also affect eligibility in pooled calls.
path selects CUDA directly with `auto` or `cuda` and rejects `hardware`.
Pass an explicit stream to `GpuCompImageReader.read` or use a batch reader
for pooled scratch. Custom CuPy allocators can also affect eligibility in
pooled calls.
An `auto` receipt and a compatible GPU therefore do not establish hardware use.
See [nvCOMP logging](../troubleshooting.md#confirm-the-decompression-engine)
to inspect the actual decoder calls.

For a hardware-only diagnostic run on supported compressed FITS inputs:

```bash
cuphoton xdr benchmark-fits --hdu-indices 1 \
--xdr-gzip-decoder gzip --xdr-decompression-backend hardware exposure.fits
```

The decode fails if the device, buffers or tile sizes are incompatible; it
does not retry with CUDA. In workflows, combine the hardware option with
`reader="xdr"` (CLI `--fits-reader xdr`) to require xDR as well. Uncompressed
HDUs perform no decompression, and Astropy-selected reads do not apply this
backend option. A receipt records the request, including these cases.

The benchmark's `--output-json` report includes
`capabilities.hardware_decompression`: the current device's algorithm mask,
`supports_deflate`, and `max_chunk_bytes`. Unavailable queries leave those
Expand Down
8 changes: 5 additions & 3 deletions docs/guide/read-fits.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -205,7 +205,7 @@ With `reader="astropy"`, `device=True` decodes on the CPU and copies the arrays
|---|---|---|---|
| `postprocess` | `--xdr-postprocess` | `auto`, `fused`, `separate`, `native` | restores the pixels with one fused kernel |
| `gzip_decoder` | `--xdr-gzip-decoder` | `auto`, `gzip`, `deflate` | uses native Gzip when available, otherwise raw DEFLATE |
| `decompression_backend` | `--xdr-decompression-backend` | `auto`, `cuda` | lets nvCOMP pick its decompression engine, with CUDA as the fallback |
| `decompression_backend` | `--xdr-decompression-backend` | `auto`, `cuda`, `hardware` | lets nvCOMP pick its decompression engine, with CUDA as the fallback |

`postprocess="native"` restores compressed tiles with the precompiled kernels of cuPhoton's native extension (@sec-install). Every key defaults to `auto`. The keys choose how xDR decodes, not whether it runs, and none changes a pixel value. The call checks them on any machine and records them in the receipt:

Expand All @@ -215,6 +215,8 @@ checked = read_fits_images("sample/target.fits", [0], reader="auto",
print(checked.metadata()["xdr_options"])
```

Use `decompression_backend="hardware"` with `reader="xdr"` to require native Gzip hardware decoding for compressed HDUs. It fails when the hardware engine, buffers or tile sizes are incompatible. Uncompressed HDUs need no decompression. See [hardware eligibility](https://github.com/NVIDIA/cuPhoton/blob/main/docs/components/xdr.md#hardware-eligibility-and-diagnostics) for supported devices and allocation requirements.

The receipt records the requested choices, not the engine that ran. To load the same HDUs from many files of one shape, `batch_to_device` returns one CuPy array per HDU, shaped (file, y, x):

```{python}
Expand Down Expand Up @@ -268,8 +270,8 @@ CPU runs of xFit, xPois and xRep turn `auto` into `astropy` before they read. Th
`No module named 'cupy'`
: `device=True` needs CuPy, even when Astropy decodes. Install the `gpu` profile.

An explicit `gzip_decoder="gzip"` or `decompression_backend="cuda"` fails on a GPU host
: The native extension is older than these choices. Rebuild it, as the [troubleshooting guide](https://github.com/NVIDIA/cuPhoton/blob/main/docs/troubleshooting.md#an-explicit-compression-choice-is-unavailable) describes.
An explicit decoder or backend choice fails on a GPU host
: The native extension may predate the requested choice. Rebuild it, as the [troubleshooting guide](https://github.com/NVIDIA/cuPhoton/blob/main/docs/troubleshooting.md#an-explicit-compression-choice-is-unavailable) describes. A `hardware` request also requires a supported device, compatible buffers and tile sizes.

## Next steps

Expand Down
7 changes: 6 additions & 1 deletion docs/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,9 @@ error. GPUDirect Storage also requires host and storage configuration; use

`--xdr-gzip-decoder gzip` requires native Gzip support;
`--xdr-decompression-backend cuda` requires an extension with backend selection.
`--xdr-decompression-backend hardware` additionally requires native Gzip
hardware selection, a supported GPU, compatible pooled buffers and tile sizes.
Use `auto` or `gzip` as the Gzip decoder for this mode.
An importable extension may predate those entry points. After updating a source
checkout, rebuild it with the [native build instructions](components/xdr.md#native-extension-availability)
in the environment running your command. Use `auto` when capability fallback
Expand All @@ -81,7 +84,9 @@ reader, decoder and backend policies.

### Confirm the decompression engine

Enable [nvCOMP logging](https://docs.nvidia.com/cuda/nvcomp/getting_started.html#logging)
Use `--xdr-decompression-backend hardware` to require hardware execution
for compressed xDR reads. It fails if the hardware engine cannot be used.
To inspect automatic selection, enable [nvCOMP logging](https://docs.nvidia.com/cuda/nvcomp/getting_started.html#logging)
for a diagnostic run. Level 5 includes API calls as well as informational
messages; use a separate log file to keep them out of the command output:

Expand Down
4 changes: 3 additions & 1 deletion scripts/coverage/xdr.sh
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,9 @@ PY

python -m pytest -q -rs -o "pythonpath=$PYTHONPATH" \
--junitxml="$coverage_build/junit.xml" \
tests/xdr/test_native.py tests/xdr/test_deflate_gpu.py
tests/xdr/test_native.py tests/xdr/test_deflate_gpu.py \
tests/xdr/test_gzip_gpu.py::test_forced_hardware_roundtrip_or_unsupported_device \
tests/xdr/test_gzip_gpu.py::test_native_hardware_rejects_incompatible_route

python - <<'PY'
from xml.etree import ElementTree as ET
Expand Down
6 changes: 4 additions & 2 deletions src/cuphoton/core/cli/fits.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,10 @@ class XdrDecompressionBackendArg(SetInvariant):
_help = (
"xDR decompression backend: auto permits compatible Gzip "
"hardware with CUDA fallback; cuda requires explicit CUDA "
"selection support. Omitted preserves input choices or uses "
"xDR defaults; applies when the FITS reader uses xDR."
"selection support; hardware requires native Gzip on the "
"hardware engine, with no CUDA fallback. Omitted preserves "
"input choices or uses xDR defaults; applies when the FITS "
"reader uses xDR."
)
_set = XDR_OPTION_CHOICES["decompression_backend"]
_default = None
Expand Down
9 changes: 6 additions & 3 deletions src/cuphoton/core/fits_io.py
Original file line number Diff line number Diff line change
Expand Up @@ -253,11 +253,14 @@ def read_fits_images(

``xdr_options`` accepts ``postprocess`` (auto/fused/separate/native),
``gzip_decoder`` (auto/gzip/deflate), and ``decompression_backend``
(auto/cuda). Omitted choices use xDR's ``auto`` defaults. For example,
(auto/cuda/hardware). Omitted choices use xDR's ``auto`` defaults.
For example,
``xdr_options={"decompression_backend": "cuda"}`` requests CUDA kernels
when xDR decodes a compressed HDU; it requires a native helper with
backend selection. These choices do not change the reader or its Astropy
fallback policy. Metadata records explicit choices as requested settings,
backend selection. ``hardware`` requires native Gzip hardware decoding
without CUDA fallback. These choices do not change the reader or its
Astropy fallback policy; use ``reader="xdr"`` to require GPU reading.
Metadata records explicit choices as requested settings,
including when Astropy is selected; it does not identify an nvCOMP engine.
"""
validate_fits_reader(reader)
Expand Down
14 changes: 12 additions & 2 deletions src/cuphoton/core/fits_options.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
XDR_OPTION_CHOICES = {
"postprocess": frozenset({"auto", "fused", "separate", "native"}),
"gzip_decoder": frozenset({"auto", "gzip", "deflate"}),
"decompression_backend": frozenset({"auto", "cuda"}),
"decompression_backend": frozenset({"auto", "cuda", "hardware"}),
}


Expand All @@ -32,6 +32,14 @@ def normalize_xdr_options(
choices = ", ".join(sorted(XDR_OPTION_CHOICES[name]))
raise ValueError(f"xDR {name} must be one of: {choices}")
result[name] = value
if (
result.get("decompression_backend") == "hardware"
and result.get("gzip_decoder") == "deflate"
):
raise ValueError(
"xDR decompression_backend='hardware' requires "
"gzip_decoder='auto' or 'gzip'"
)
return result


Expand All @@ -40,4 +48,6 @@ def merge_xdr_options(
overrides: Mapping[str, str] | None,
) -> dict[str, str]:
"""Override only supplied keys and preserve other manifest choices."""
return {**normalize_xdr_options(base), **normalize_xdr_options(overrides)}
return normalize_xdr_options(
{**normalize_xdr_options(base), **normalize_xdr_options(overrides)}
)
7 changes: 4 additions & 3 deletions src/cuphoton/xdr/convenience.py
Original file line number Diff line number Diff line change
Expand Up @@ -78,11 +78,12 @@ def batch_to_device(
gzip_decoder : {"auto", "gzip", "deflate"}
``auto`` prefers native Gzip, with aligned raw DEFLATE fallback.
``gzip`` requires native Gzip support; ``deflate`` selects raw decode.
decompression_backend : {"auto", "cuda"}
decompression_backend : {"auto", "cuda", "hardware"}
``auto`` lets native Gzip select compatible hardware with CUDA
fallback. ``cuda`` requires a native helper with backend selection.
Native raw DEFLATE uses CUDA in either mode. All three runtime
controls apply with either value of ``parallel``.
``hardware`` requires native Gzip on the hardware engine without CUDA
fallback. Native raw DEFLATE supports only ``auto`` and ``cuda``.
All three runtime controls apply with either value of ``parallel``.

Returns
-------
Expand Down
63 changes: 51 additions & 12 deletions src/cuphoton/xdr/nvcomp_batch.py
Original file line number Diff line number Diff line change
Expand Up @@ -894,10 +894,12 @@ def gpu_gzip_decompress_batch(
"auto" uses native Gzip when available, otherwise raw DEFLATE.
"gzip" requires the native Gzip helper and gzip-wrapped inputs.
"deflate" strips gzip framing and aligns payloads before decoding.
decompression_backend : "auto" | "cuda"
decompression_backend : "auto" | "cuda" | "hardware"
"auto" lets native Gzip use compatible hardware with CUDA fallback.
Native raw DEFLATE uses CUDA in either mode.
Native raw DEFLATE supports only "auto" and "cuda" and uses CUDA.
"cuda" requires a rebuilt native helper and uses CUDA kernels.
"hardware" requires native Gzip and pooled allocations, and fails
when the device, buffers, or tile sizes cannot use hardware decoding.
use_cpp_helper : "auto" | True | False
"auto" (default) → use the C++ pybind11 helper when importable, else
fall back to the Python loop. True forces the C++ path (raises if
Expand Down Expand Up @@ -939,12 +941,35 @@ def gpu_gzip_decompress_batch(
"""
if _decode_checks is not None and keepalive is None:
raise ValueError("deferred decode requires an explicit owner list")
if decompression_backend not in ("auto", "cuda"):
raise ValueError("decompression_backend must be 'auto' or 'cuda'")
if decompression_backend not in ("auto", "cuda", "hardware"):
raise ValueError(
"decompression_backend must be 'auto', 'cuda', or 'hardware'"
)
if gzip_decoder not in ("auto", "gzip", "deflate"):
raise ValueError("gzip_decoder must be 'auto', 'gzip', or 'deflate'")
if gzip_decoder == "gzip" and not gzip_wrapped:
raise ValueError("gzip_decoder='gzip' requires gzip_wrapped=True")
if decompression_backend == "hardware":
if gzip_decoder == "deflate":
raise ValueError(
"decompression_backend='hardware' requires "
"gzip_decoder='auto' or 'gzip'"
)
if not gzip_wrapped:
raise ValueError(
"decompression_backend='hardware' requires native Gzip "
"decoding of gzip-wrapped inputs"
)
if not use_native_pool:
raise ValueError(
"decompression_backend='hardware' requires "
"use_native_pool=True"
)
if use_cpp_helper is False:
raise ValueError(
"decompression_backend='hardware' requires "
"use_cpp_helper enabled"
)

import cupy as cp

Expand Down Expand Up @@ -1017,25 +1042,35 @@ def gpu_gzip_decompress_batch(
ext = None
else:
ext = _try_get_cpp_ext()
if ext is None and gzip_decoder != "gzip":
if (
ext is None
and gzip_decoder != "gzip"
and decompression_backend != "hardware"
):
_warn_python_fallback_once()

backend_selection = getattr(ext, "supports_decompression_backend", False)
if decompression_backend != "auto" and not backend_selection:
if (decompression_backend != "auto" and not backend_selection) or (
decompression_backend == "hardware"
and not getattr(ext, "supports_hardware_decompression", False)
):
raise RuntimeError(
"decompression_backend='cuda' requires a rebuilt native "
"extension with backend selection and use_cpp_helper enabled"
f"decompression_backend={decompression_backend!r} requires a "
"rebuilt native extension with backend selection and "
"use_cpp_helper enabled"
)
backend_kwargs = (
{"decompression_backend": decompression_backend}
if backend_selection
else {}
)
gzip_fn = getattr(ext, "batch_gzip_decompress", None)
if gzip_decoder == "gzip" and gzip_fn is None:
if (
gzip_decoder == "gzip" or decompression_backend == "hardware"
) and gzip_fn is None:
raise RuntimeError(
"gzip_decoder='gzip' requires a native extension with Gzip "
"support and use_cpp_helper enabled"
"the requested decoder/backend requires a native extension with "
"Gzip support and use_cpp_helper enabled"
)
use_gzip = (
gzip_wrapped and gzip_decoder != "deflate" and gzip_fn is not None
Expand Down Expand Up @@ -1113,7 +1148,11 @@ def gpu_gzip_decompress_batch(
out_offsets,
size_array,
stream_ptr,
use_native_pool and keepalive is not None,
use_native_pool
and (
keepalive is not None
or decompression_backend == "hardware"
Comment thread
tpn marked this conversation as resolved.
Resolved
),
**backend_kwargs,
)
if keepalive is not None and scratch_owner is not None:
Expand Down
5 changes: 3 additions & 2 deletions src/cuphoton/xdr/prefetch.py
Original file line number Diff line number Diff line change
Expand Up @@ -1771,8 +1771,9 @@ def batch_to_device_stream(
"gzip" requires native Gzip support; "deflate" selects raw DEFLATE.
decompression_backend
"auto" lets Gzip select compatible hardware with CUDA fallback. "cuda"
requires a rebuilt native helper and uses CUDA kernels. Native raw
DEFLATE uses CUDA in either mode.
requires a rebuilt native helper and uses CUDA kernels. "hardware"
requires native Gzip on the hardware engine without CUDA fallback.
Native raw DEFLATE supports only "auto" and "cuda".
section
Optional 2D ROI applied uniformly to CompImageHDUs.
stream
Expand Down
Loading
Loading