Update NSVHMEM block for new releases on Github

[mhrywniak]

Releases >= 3.4.5 are published on Github, ensure they get picked up. Fix mpi arg to allow using either True or an explicit MPI_HOME.

I could not run the pydocmd generate command, do I need a different package? uvx -p 3.11 -w pydoc-markdown pydocmd generate seems to expect a .py file instead, and with newer Python versions I get errors about the deprecated/removed imp package.

Pull Request Description

Author Checklist

• Updated documentation (pydocmd generate) if any docstrings have been modified
• Passes all unit tests

[samcmill]

Would you please add a test case for version 3.4.5 to exercise the new behavior?

[samcmill]

@jilliebean @sopcao for viz

[samcmill]

Also, since you updated the docstring, the docs will need to be regenerated. I can do that post-merge since it requires having pydocmd installed.

[mhrywniak]

Thanks for the review! I will likely get around to implementing this on Monday.

Also, since you updated the docstring, the docs will need to be regenerated. I can do that post-merge since it requires having pydocmd installed.

I can also regenerate this, but uvx -p 3.11 -w pydoc-markdown pydocmd generate gave me error messages. Do you need a specific pydocmd version installed?

[mhrywniak]

Sorry for the delay - I ended up doing more a rewrite than an update and so this likely requires a new review.

Re: the else branch. I checked the upstream defaults in cmake_config/NVSHMEMEnv.cmake at tag v3.4.5-0 (and on main): NVSHMEM_MPI_SUPPORT is registered with nvshmem_add_default_on_option, which defaults the option to ON when neither the CMake nor the matching env var is set.
Essentially theelse branch is still needed, and I've restored it as -DNVSHMEM_MPI_SUPPORT=OFF.

To be consistent here I also flipped the mpi parameter's default from None to True. Users who don't want MPI can pass mpi=False.
Please let me know if you'd like me to keep previous logic; my reasoning was it makes sense to track NVSHMEM defaults.

Re: cuda LD_LIBRARY_PATH for the build environment. Generalized this to fire whenever a CUDA path is given, not only when MPI is enabled. I'm still not quite sure why but it looks like NVSHMEM build need to be able to dlopen libs from <cuda>/lib64 at configure time regardless of MPI, and it doesn't seem to hurt

Re: a 3.4.5 test case.

• Added test_github_release_345_mpi exercising the new GitHub-release tarball path (https://github.com/NVIDIA/nvshmem/archive/refs/tags/v3.4.5-0.tar.gz), the corrected source directory (nvshmem-3.4.5-0), the .tar.gz extraction, and mpi='/usr/local/openmpi' (string form instead of just mpi=True, so it results in -DNVSHMEM_MPI_SUPPORT=ON -DMPI_HOME=/usr/local/openmpi).
• repurposed the old test_package_ubuntu to pass mpi=False so the off-case is covered explicitly.

Finally, I sent out Claude to track down the pydocmd setup; it looks like the pydocmd package just had some breaking changes and when I tried reinstalling and running I ran into several version clashes.
I've only included the targeted nvshmem mpi docstring change in docs/building_blocks.md; happy to send a separate PR that pins/documents the toolchain and a script to run. Draft pushed at https://github.com/mhrywniak/hpc-container-maker/tree/docs_updater_script, just let me know if that seems useful.

[samcmill]

Other than the 2 flagged items, LGTM.

[samcmill]

Finally, I sent out Claude to track down the pydocmd setup; it looks like the pydocmd package just had some breaking changes and when I tried reinstalling and running I ran into several version clashes. I've only included the targeted nvshmem mpi docstring change in docs/building_blocks.md; happy to send a separate PR that pins/documents the toolchain and a script to run. Draft pushed at https://github.com/mhrywniak/hpc-container-maker/tree/docs_updater_script, just let me know if that seems useful.

Ideally I'd like to have GitHub automatically regenerate the documentation whenever there is a commit. (I'm not asking you to do it, just noting the direction I'd like to take the documentation.)

[mhrywniak]

Should be done now, please have a look.

Ideally I'd like to have GitHub automatically regenerate the documentation whenever there is a commit. (I'm not asking you to do it, just noting the direction I'd like to take the documentation.)

Makes sense. I've not worked with GH actions (mostly gitlab), but happy to help with this.

[samcmill]

LGTM. Thanks Markus!