RuntimeWarning: invalid value / divide by zero / overflow encountered in divide for sparse/numba_backend

[bpuchala]

Warnings like the following may appear when using casm-bset with numpy>=2.4:

tests/clexwriter/test_write_v1_basic.py: 46 warnings
tests/docs/test_quick_start_examples.py: 20 warnings
tests/test_Hstrain_disp_dof.py: 31 warnings
tests/test_Hstrain_dof.py: 34 warnings
tests/test_disp_dof.py: 22 warnings
tests/test_occ_dof.py: 23 warnings
tests/test_occ_site_functions_chebychev.py: 38 warnings
tests/test_occ_site_functions_occupation.py: 56 warnings
  /opt/hostedtoolcache/Python/3.13.12/x64/lib/python3.13/site-packages/sparse/numba_backend/_sparse_array.py:338: RuntimeWarning: invalid value encountered in divide
    getattr(ufunc, method)(*test_args, out=test_out, **test_kwargs)

tests/test_Hstrain_dof.py::test_Hstrain_lowsym_1
  /opt/hostedtoolcache/Python/3.13.12/x64/lib/python3.13/site-packages/sparse/numba_backend/_sparse_array.py:338: RuntimeWarning: divide by zero encountered in divide
    getattr(ufunc, method)(*test_args, out=test_out, **test_kwargs)

tests/test_Hstrain_dof.py::test_Hstrain_lowsym_1
tests/test_Hstrain_dof.py::test_Hstrain_lowsym_1
  /opt/hostedtoolcache/Python/3.13.12/x64/lib/python3.13/site-packages/sparse/numba_backend/_sparse_array.py:338: RuntimeWarning: overflow encountered in divide
    getattr(ufunc, method)(*test_args, out=test_out, **test_kwargs)

My best understanding of the issue currently is that it is due to behavior of sparse / numba / numpy:

The sparse.numba_backend attempts to pre-verify ufunc operations (like division) by executing them on small test samples to determine output types and behavior. In NumPy 2.4.x, the internal handling of divide, invalid, and overflow warnings during these test calls has become more sensitive or has bypassed previous suppression mechanisms used by sparse.

After reverting to numpy==2.3.5 the warnings go away. I anticipate this will be handled by an update to sparse / numba eventually. I will look into suppressing the warning in casm/bset/polynomial_functions/_polynomial_functions.py.

[bpuchala]

More notes:

This does not work to filter the warnings:

import warnings

# Suppress NumPy 2.4+ warnings triggered by sparse's Numba type-inference
warnings.filterwarnings(
    "ignore",
    message=".*(overflow|invalid value|divide by zero) encountered in divide.*",
    category=RuntimeWarning,
    module="sparse.numba_backend.*"
)

Nor does this:

import warnings

# Use a broader match for the message and omit the module restriction
# if the module-specific filter is being bypassed by Numba/NumPy internals.
warnings.filterwarnings(
    "ignore",
    message=".*(overflow|invalid value|divide by zero) encountered in divide.*",
    category=RuntimeWarning
)

Setting this environment variable does suppress the warnings, though it will also suppress other warnings that may be wanted:

export PYTHONWARNINGS="ignore:overflow encountered in divide:RuntimeWarning,ignore:invalid value encountered in divide:RuntimeWarning"

For now I will leave casm-bset as is rather than pin numpy / numba / sparse. Currently we are at sparse 0.18.0 and numba 0.64.0. If the issue persists after updates to sparse and numba I will revisit.