Support multiple EESSI versions by using ReFrame environments by casparvl · Pull Request #326 · EESSI/test-suite

casparvl · 2026-03-12T15:47:07Z

Add initial functionality to schedule test for all EESSI environments in a single run, by configuring ReFrame environments and using ReFrame's own find_modules function.

For now, these work:

reframe -c test-suite/eessi/testsuite/tests/apps/gromacs.py --list -t CI -t 1_node
...
[List of matched checks]
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=1_node %module_info=('snellius:rome', 'EESSI-2023.06', 'GROMACS/2024.1-foss-2023b') /ec5fea98
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=1_node %module_info=('snellius:rome', 'EESSI-2023.06', 'GROMACS/2024.3-foss-2023b-PLUMED-2.9.2') /0149252b
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=1_node %module_info=('snellius:rome', 'EESSI-2023.06', 'GROMACS/2024.3-foss-2023b') /7f86cc73
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=1_node %module_info=('snellius:rome', 'EESSI-2023.06', 'GROMACS/2024.4-foss-2023b') /b44c4f26
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=1_node %module_info=('snellius:rome', 'EESSI-2025.06', 'GROMACS/2025.2-foss-2025a') /10fe4b84
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=1_node %module_info=('snellius:genoa', 'EESSI-2023.06', 'GROMACS/2024.1-foss-2023b') /8575f1af
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=1_node %module_info=('snellius:genoa', 'EESSI-2023.06', 'GROMACS/2024.3-foss-2023b-PLUMED-2.9.2') /fa197803
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=1_node %module_info=('snellius:genoa', 'EESSI-2023.06', 'GROMACS/2024.3-foss-2023b') /848b3575
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=1_node %module_info=('snellius:genoa', 'EESSI-2023.06', 'GROMACS/2024.4-foss-2023b') /b4a91b63
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=1_node %module_info=('snellius:genoa', 'EESSI-2025.06', 'GROMACS/2025.2-foss-2025a') /42464906
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=gpu %scale=1_node %module_info=('snellius:gpu_A100', 'EESSI-2023.06', 'GROMACS/2024.4-foss-2023b-CUDA-12.4.0') /91ff3a13
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=gpu %scale=1_node %module_info=('snellius:gpu_H100', 'EESSI-2023.06', 'GROMACS/2024.4-foss-2023b-CUDA-12.4.0') /0aadbdf1
Found 12 check(s)

reframe -c test-suite/eessi/testsuite/tests/apps/numpy/numpy.py --list -t 1_8_node
...
[List of matched checks]
- EESSI_NumPy %scale=1_8_node %module_info=('snellius:rome', 'EESSI-2023.06', 'SciPy-bundle/2023.02-gfbf-2022b') /d008c385
- EESSI_NumPy %scale=1_8_node %module_info=('snellius:rome', 'EESSI-2023.06', 'SciPy-bundle/2023.07-gfbf-2023a') /0b69130e
- EESSI_NumPy %scale=1_8_node %module_info=('snellius:rome', 'EESSI-2023.06', 'SciPy-bundle/2023.11-gfbf-2023b') /794e1d9e
- EESSI_NumPy %scale=1_8_node %module_info=('snellius:rome', 'EESSI-2025.06', 'SciPy-bundle/2024.05-gfbf-2024a') /a29dae40
- EESSI_NumPy %scale=1_8_node %module_info=('snellius:rome', 'EESSI-2025.06', 'SciPy-bundle/2025.06-gfbf-2025a') /e2755723
- EESSI_NumPy %scale=1_8_node %module_info=('snellius:genoa', 'EESSI-2023.06', 'SciPy-bundle/2023.02-gfbf-2022b') /8d434ed0
- EESSI_NumPy %scale=1_8_node %module_info=('snellius:genoa', 'EESSI-2023.06', 'SciPy-bundle/2023.07-gfbf-2023a') /ec49d0f6
- EESSI_NumPy %scale=1_8_node %module_info=('snellius:genoa', 'EESSI-2023.06', 'SciPy-bundle/2023.11-gfbf-2023b') /e2523a64
- EESSI_NumPy %scale=1_8_node %module_info=('snellius:genoa', 'EESSI-2025.06', 'SciPy-bundle/2024.05-gfbf-2024a') /1b4c0e79
- EESSI_NumPy %scale=1_8_node %module_info=('snellius:genoa', 'EESSI-2025.06', 'SciPy-bundle/2025.06-gfbf-2025a') /5dccf227
Found 10 check(s)

Still need to work on the other tests.

… in a single run, by configuring ReFrame environments and using ReFrame's own find_modules function

casparvl · 2026-03-12T15:56:12Z

Ok, I now simply wrap the find_modules function from ReFrame in our own utils. The only change now required in the tests is that the return should define module_info instead of module_name.

…ct instead

casparvl · 2026-03-12T16:03:35Z

Oh, this broke name_only of course:

WARNING: skipping test file 'test-suite/eessi/testsuite/tests/apps/openfoam/openfoam.py': type error: find_modules() got an unexpected keyword argument 'name_only':
| /gpfs/home4/casparl/EESSI/test-suite/eessi/testsuite/tests/apps/openfoam/openfoam.py:90
|     module_info = parameter(find_modules('OpenFOAM/v', name_only=False))

that's something we might want to fix if we do the wrapped thing (if it's easy...)

casparvl · 2026-03-12T16:04:30Z

Metallwalls should also be adapted, since I think it doesn't use EESSI_mixin (?):

WARNING: skipping test 'EESSI_MetalWalls_MW': attribute error: 'EESSI_MetalWalls_MW' object has no attribute 'module_name':
| /gpfs/home4/casparl/EESSI/test-suite/eessi/testsuite/hooks.py:651
|     if not test.module_name:
 (rerun with '-v' for a backtrace)

casparvl · 2026-03-12T16:07:21Z

For me:

reframe -c test-suite/eessi/testsuite/tests/apps/ -t CI -t 1_node --list
...
- EESSI_QuantumESPRESSO_PW %ecut=50 %nbnd=10 %scale=1_node %module_info=('snellius:rome', 'EESSI-2023.06', 'QuantumESPRESSO/7.2-foss-2022b') %device_type=cpu /1a10c479
- EESSI_QuantumESPRESSO_PW %ecut=50 %nbnd=10 %scale=1_node %module_info=('snellius:rome', 'EESSI-2023.06', 'QuantumESPRESSO/7.3.1-foss-2023a') %device_type=cpu /41f20edb
- EESSI_QuantumESPRESSO_PW %ecut=50 %nbnd=10 %scale=1_node %module_info=('snellius:genoa', 'EESSI-2023.06', 'QuantumESPRESSO/7.2-foss-2022b') %device_type=cpu /6943f12d
- EESSI_QuantumESPRESSO_PW %ecut=50 %nbnd=10 %scale=1_node %module_info=('snellius:genoa', 'EESSI-2023.06', 'QuantumESPRESSO/7.3.1-foss-2023a') %device_type=cpu /751de4da
- EESSI_CP2K %scale=1_node %benchmark_info=QS/H2O-32 %module_info=('snellius:rome', 'EESSI-2023.06', 'CP2K/2023.1-foss-2023a') /cb3afdc7
- EESSI_CP2K %scale=1_node %benchmark_info=QS/H2O-32 %module_info=('snellius:rome', 'EESSI-2023.06', 'Libint/2.7.2-GCC-12.3.0-lmax-6-cp2k') /eb966eda
- EESSI_CP2K %scale=1_node %benchmark_info=QS/H2O-32 %module_info=('snellius:genoa', 'EESSI-2023.06', 'CP2K/2023.1-foss-2023a') /db55acb8
- EESSI_CP2K %scale=1_node %benchmark_info=QS/H2O-32 %module_info=('snellius:genoa', 'EESSI-2023.06', 'Libint/2.7.2-GCC-12.3.0-lmax-6-cp2k') /fd4dc36f
- EESSI_LPC3D %scale=1_node %module_info=('snellius:rome', 'EESSI-2023.06', 'LPC3D/0.1.2-foss-2023a') /cca8ba7c
- EESSI_LPC3D %scale=1_node %module_info=('snellius:genoa', 'EESSI-2023.06', 'LPC3D/0.1.2-foss-2023a') /1f8c34a8
- EESSI_NumPy %scale=1_node %module_info=('snellius:rome', 'EESSI-2023.06', 'SciPy-bundle/2023.02-gfbf-2022b') /5f66d2a4
- EESSI_NumPy %scale=1_node %module_info=('snellius:rome', 'EESSI-2023.06', 'SciPy-bundle/2023.07-gfbf-2023a') /74180bd1
- EESSI_NumPy %scale=1_node %module_info=('snellius:rome', 'EESSI-2023.06', 'SciPy-bundle/2023.11-gfbf-2023b') /b13eb6e7
- EESSI_NumPy %scale=1_node %module_info=('snellius:rome', 'EESSI-2025.06', 'SciPy-bundle/2024.05-gfbf-2024a') /4198937c
- EESSI_NumPy %scale=1_node %module_info=('snellius:rome', 'EESSI-2025.06', 'SciPy-bundle/2025.06-gfbf-2025a') /001c65e5
- EESSI_NumPy %scale=1_node %module_info=('snellius:genoa', 'EESSI-2023.06', 'SciPy-bundle/2023.02-gfbf-2022b') /2406ceda
- EESSI_NumPy %scale=1_node %module_info=('snellius:genoa', 'EESSI-2023.06', 'SciPy-bundle/2023.07-gfbf-2023a') /848c2beb
- EESSI_NumPy %scale=1_node %module_info=('snellius:genoa', 'EESSI-2023.06', 'SciPy-bundle/2023.11-gfbf-2023b') /87bf79d8
- EESSI_NumPy %scale=1_node %module_info=('snellius:genoa', 'EESSI-2025.06', 'SciPy-bundle/2024.05-gfbf-2024a') /4d49b0c2
- EESSI_NumPy %scale=1_node %module_info=('snellius:genoa', 'EESSI-2025.06', 'SciPy-bundle/2025.06-gfbf-2025a') /6c960caf
- EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_bw %module_info=('snellius:rome', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.1-1-gompi-2023a') %scale=1_node /28114225
- EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_bw %module_info=('snellius:rome', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.2-gompi-2023b') %scale=1_node /2985e32f
- EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_bw %module_info=('snellius:rome', 'EESSI-2025.06', 'OSU-Micro-Benchmarks/7.5-gompi-2025a') %scale=1_node /66e93c7a
- EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_bw %module_info=('snellius:genoa', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.1-1-gompi-2023a') %scale=1_node /d3089aa5
- EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_bw %module_info=('snellius:genoa', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.2-gompi-2023b') %scale=1_node /84bb5554
- EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_bw %module_info=('snellius:genoa', 'EESSI-2025.06', 'OSU-Micro-Benchmarks/7.5-gompi-2025a') %scale=1_node /087941ec
- EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_latency %module_info=('snellius:rome', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.1-1-gompi-2023a') %scale=1_node /f273f6d3
- EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_latency %module_info=('snellius:rome', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.2-gompi-2023b') %scale=1_node /aacbc5b0
- EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_latency %module_info=('snellius:rome', 'EESSI-2025.06', 'OSU-Micro-Benchmarks/7.5-gompi-2025a') %scale=1_node /ad4c59aa
- EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_latency %module_info=('snellius:genoa', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.1-1-gompi-2023a') %scale=1_node /6d239a22
- EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_latency %module_info=('snellius:genoa', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.2-gompi-2023b') %scale=1_node /943cd266
- EESSI_OSU_pt2pt_CPU %benchmark_info=mpi.pt2pt.osu_latency %module_info=('snellius:genoa', 'EESSI-2025.06', 'OSU-Micro-Benchmarks/7.5-gompi-2025a') %scale=1_node /20052531
- EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_bw %module_info=('snellius:gpu_A100', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.2-gompi-2023a-CUDA-12.1.1') %scale=1_node /072871b6
- EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_bw %module_info=('snellius:gpu_A100', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.5-gompi-2023b-CUDA-12.4.0') %scale=1_node /afc1e049
- EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_bw %module_info=('snellius:gpu_A100', 'EESSI-2025.06', 'OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0') %scale=1_node /1d758e9d
- EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_bw %module_info=('snellius:gpu_H100', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.2-gompi-2023a-CUDA-12.1.1') %scale=1_node /8544f946
- EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_bw %module_info=('snellius:gpu_H100', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.5-gompi-2023b-CUDA-12.4.0') %scale=1_node /bcde6240
- EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_bw %module_info=('snellius:gpu_H100', 'EESSI-2025.06', 'OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0') %scale=1_node /35f1f357
- EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_latency %module_info=('snellius:gpu_A100', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.2-gompi-2023a-CUDA-12.1.1') %scale=1_node /4c2620bf
- EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_latency %module_info=('snellius:gpu_A100', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.5-gompi-2023b-CUDA-12.4.0') %scale=1_node /d12354ac
- EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_latency %module_info=('snellius:gpu_A100', 'EESSI-2025.06', 'OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0') %scale=1_node /f54a219f
- EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_latency %module_info=('snellius:gpu_H100', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.2-gompi-2023a-CUDA-12.1.1') %scale=1_node /ebd15104
- EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_latency %module_info=('snellius:gpu_H100', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.5-gompi-2023b-CUDA-12.4.0') %scale=1_node /e11d33eb
- EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_latency %module_info=('snellius:gpu_H100', 'EESSI-2025.06', 'OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0') %scale=1_node /d6c59637
- EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_info=('snellius:rome', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.1-1-gompi-2023a') %scale=1_node %device_type=cpu /0302bc4a
- EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_info=('snellius:rome', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.2-gompi-2023b') %scale=1_node %device_type=cpu /b5209e9a
- EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_info=('snellius:rome', 'EESSI-2025.06', 'OSU-Micro-Benchmarks/7.5-gompi-2025a') %scale=1_node %device_type=cpu /63912bd7
- EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_info=('snellius:genoa', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.1-1-gompi-2023a') %scale=1_node %device_type=cpu /8b6b78cf
- EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_info=('snellius:genoa', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.2-gompi-2023b') %scale=1_node %device_type=cpu /a8297e2c
- EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_info=('snellius:genoa', 'EESSI-2025.06', 'OSU-Micro-Benchmarks/7.5-gompi-2025a') %scale=1_node %device_type=cpu /28747c55
- EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_info=('snellius:gpu_A100', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.2-gompi-2023a-CUDA-12.1.1') %scale=1_node %device_type=gpu /f27f47b5
- EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_info=('snellius:gpu_A100', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.5-gompi-2023b-CUDA-12.4.0') %scale=1_node %device_type=gpu /dc6c60d9
- EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_info=('snellius:gpu_A100', 'EESSI-2025.06', 'OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0') %scale=1_node %device_type=gpu /ca3b2c97
- EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_info=('snellius:gpu_H100', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.2-gompi-2023a-CUDA-12.1.1') %scale=1_node %device_type=gpu /0dc81150
- EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_info=('snellius:gpu_H100', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.5-gompi-2023b-CUDA-12.4.0') %scale=1_node %device_type=gpu /07afffdd
- EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_info=('snellius:gpu_H100', 'EESSI-2025.06', 'OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0') %scale=1_node %device_type=gpu /641920a7
- EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_info=('snellius:rome', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.1-1-gompi-2023a') %scale=1_node %device_type=cpu /7725a99a
- EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_info=('snellius:rome', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.2-gompi-2023b') %scale=1_node %device_type=cpu /806d6d5f
- EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_info=('snellius:rome', 'EESSI-2025.06', 'OSU-Micro-Benchmarks/7.5-gompi-2025a') %scale=1_node %device_type=cpu /ea33fdf4
- EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_info=('snellius:genoa', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.1-1-gompi-2023a') %scale=1_node %device_type=cpu /e84aa818
- EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_info=('snellius:genoa', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.2-gompi-2023b') %scale=1_node %device_type=cpu /77b25adf
- EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_info=('snellius:genoa', 'EESSI-2025.06', 'OSU-Micro-Benchmarks/7.5-gompi-2025a') %scale=1_node %device_type=cpu /baa4f778
- EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_info=('snellius:gpu_A100', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.2-gompi-2023a-CUDA-12.1.1') %scale=1_node %device_type=gpu /7bd9e7db
- EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_info=('snellius:gpu_A100', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.5-gompi-2023b-CUDA-12.4.0') %scale=1_node %device_type=gpu /10a3d22c
- EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_info=('snellius:gpu_A100', 'EESSI-2025.06', 'OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0') %scale=1_node %device_type=gpu /30435cb0
- EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_info=('snellius:gpu_H100', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.2-gompi-2023a-CUDA-12.1.1') %scale=1_node %device_type=gpu /01d518df
- EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_info=('snellius:gpu_H100', 'EESSI-2023.06', 'OSU-Micro-Benchmarks/7.5-gompi-2023b-CUDA-12.4.0') %scale=1_node %device_type=gpu /4184c316
- EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_info=('snellius:gpu_H100', 'EESSI-2025.06', 'OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0') %scale=1_node %device_type=gpu /253bc1ba
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=1_node %module_info=('snellius:rome', 'EESSI-2023.06', 'GROMACS/2024.1-foss-2023b') /ec5fea98
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=1_node %module_info=('snellius:rome', 'EESSI-2023.06', 'GROMACS/2024.3-foss-2023b-PLUMED-2.9.2') /0149252b
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=1_node %module_info=('snellius:rome', 'EESSI-2023.06', 'GROMACS/2024.3-foss-2023b') /7f86cc73
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=1_node %module_info=('snellius:rome', 'EESSI-2023.06', 'GROMACS/2024.4-foss-2023b') /b44c4f26
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=1_node %module_info=('snellius:rome', 'EESSI-2025.06', 'GROMACS/2025.2-foss-2025a') /10fe4b84
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=1_node %module_info=('snellius:genoa', 'EESSI-2023.06', 'GROMACS/2024.1-foss-2023b') /8575f1af
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=1_node %module_info=('snellius:genoa', 'EESSI-2023.06', 'GROMACS/2024.3-foss-2023b-PLUMED-2.9.2') /fa197803
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=1_node %module_info=('snellius:genoa', 'EESSI-2023.06', 'GROMACS/2024.3-foss-2023b') /848b3575
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=1_node %module_info=('snellius:genoa', 'EESSI-2023.06', 'GROMACS/2024.4-foss-2023b') /b4a91b63
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=1_node %module_info=('snellius:genoa', 'EESSI-2025.06', 'GROMACS/2025.2-foss-2025a') /42464906
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=gpu %scale=1_node %module_info=('snellius:gpu_A100', 'EESSI-2023.06', 'GROMACS/2024.4-foss-2023b-CUDA-12.4.0') /91ff3a13
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=gpu %scale=1_node %module_info=('snellius:gpu_H100', 'EESSI-2023.06', 'GROMACS/2024.4-foss-2023b-CUDA-12.4.0') /0aadbdf1
- EESSI_TensorFlow %scale=1_node %module_info=('snellius:rome', 'EESSI-2023.06', 'tensorflow-probability/0.20.0-foss-2023a') %device_type=cpu /dc8fa7bb
- EESSI_TensorFlow %scale=1_node %module_info=('snellius:rome', 'EESSI-2023.06', 'TensorFlow/2.13.0-foss-2023a') %device_type=cpu /272d9e0b
- EESSI_TensorFlow %scale=1_node %module_info=('snellius:genoa', 'EESSI-2023.06', 'tensorflow-probability/0.20.0-foss-2023a') %device_type=cpu /54df3bda
- EESSI_TensorFlow %scale=1_node %module_info=('snellius:genoa', 'EESSI-2023.06', 'TensorFlow/2.13.0-foss-2023a') %device_type=cpu /214a955a

At first look, that looks quite good: all CUDA-enabled modules are only scheduled on our GPU partitions, all non-CUDA modules only on the CPU partitions. Modules are discovered accross both EESSI 2023.06 and EESSI 2025.06. The execution takes quite long - possibly also because we don't cache (yet). Anyway, I'm not disappointed with the current state: it needs some work to fix the last tests, but not a bad starting point.

…ssi_environments' into support_multiple_eessi_environments

…n of sys:part +feat notation directly

smoors · 2026-04-04T12:10:38Z

i must be doing something wrong here (with my suggested fix above).
if i don't load an EESSI module before running reframe, no GROMACS module is found.
if i load EESSI/2023.06, i only get the GROMACS modules of that EESSI version, even for the EESSI-2025.06 environment:

reframe -c test-suite/eessi/testsuite/tests/apps/gromacs.py -t 4_cores -t CI --system hydra:zen4 -v --list
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=4_cores %module_info=('hydra:zen4', 'EESSI-2023.06', 'GROMACS/2024.1-foss-2023b') /571a2288
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=4_cores %module_info=('hydra:zen4', 'EESSI-2023.06', 'GROMACS/2024.3-foss-2023b-PLUMED-2.9.2') /2c579518
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=4_cores %module_info=('hydra:zen4', 'EESSI-2023.06', 'GROMACS/2024.3-foss-2023b') /a9773f33
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=4_cores %module_info=('hydra:zen4', 'EESSI-2023.06', 'GROMACS/2024.4-foss-2023b') /a50f456d
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=4_cores %module_info=('hydra:zen4', 'EESSI-2025.06', 'GROMACS/2024.1-foss-2023b') /95a816c8
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=4_cores %module_info=('hydra:zen4', 'EESSI-2025.06', 'GROMACS/2024.3-foss-2023b-PLUMED-2.9.2') /65ca6b4f
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=4_cores %module_info=('hydra:zen4', 'EESSI-2025.06', 'GROMACS/2024.3-foss-2023b') /6f8d873c
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=4_cores %module_info=('hydra:zen4', 'EESSI-2025.06', 'GROMACS/2024.4-foss-2023b') /aed0121e

smoors · 2026-04-04T14:03:43Z

ok, solved it by removing 'resolve_module_conflicts': False in my config file (don't remember exactly why that was needed).

Co-authored-by: Sam Moors <smoors@users.noreply.github.com>

casparvl · 2026-04-07T15:16:12Z

ok, solved it by removing 'resolve_module_conflicts': False in my config file (don't remember exactly why that was needed).

I have no clue why resolve_module_conflicts (https://reframe-hpc.readthedocs.io/en/stable/config_reference.html#config.general.resolve_module_conflicts) should interfere with any of this. But, do I understand that you now have the expected behavior? I.e.

Your current MODULEPATH contains the EESSI modules (but no application modules such as GROMACS etc directly)
When you list, reframe correctly finds the modules that become available by loading the respective EESSI versions?

smoors · 2026-04-07T18:03:43Z

yes, i get the expected behavior now:

- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=4_cores %module_info=('hydra:ampere', 'EESSI-2023.06', 'GROMACS/2024.1-foss-2023b') /cc033d46                                
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=4_cores %module_info=('hydra:ampere', 'EESSI-2023.06', 'GROMACS/2024.3-foss-2023b-PLUMED-2.9.2') /e5b52802
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=4_cores %module_info=('hydra:ampere', 'EESSI-2023.06', 'GROMACS/2024.3-foss-2023b') /42c23ed7
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=4_cores %module_info=('hydra:ampere', 'EESSI-2023.06', 'GROMACS/2024.4-foss-2023b') /cb7345ad
- EESSI_GROMACS %benchmark_info=HECBioSim/Crambin %nb_impl=cpu %scale=4_cores %module_info=('hydra:ampere', 'EESSI-2025.06', 'GROMACS/2025.2-foss-2025a') /4a414393
Found 5 check(s)

checking the ReFrame source code, the way it works as i understand: when it tries to load a module (here EESSI/2025.06) from an environment and it fails, if resolve_module_conflicts is True, then it will unload the conflicting module(s) (EESSI/2023.06) and try again.

smoors · 2026-04-15T20:23:56Z

i looked a bit into implementing a cache for the modules, but it's not as trivial as i thought: we need to create a separate cache for each environment.

also, the find_modules function in reframe looks quite inefficient: it runs ms.available_modules(substr) for each substr, environment and system-partition, while it should ideally only run for each environment and return a list all modules, which can be cached, and which we can do multiple substring searches on (like we did before).

so, i'm now thinking to keep using our own find_modules function, updated to work with multiple environments. what do you think?

…an still load environments for testing local modules

casparvl · 2026-04-29T14:33:31Z

Discussed in the test suite sync, we can actually cache the triplets that find_modules(substr) return, but the problem is: there is no point. We will never call find_modules with the same substr again, so we wouldn't benefit from caching that way.

That means @smoors is right and the only way to properly cache is to implement the caching functionality inside the find_modules function. Somewhere around here

Something like

    ms = rt.runtime().modules_system
    current_system = rt.runtime().system
    snap0 = rt.snapshot()
    global cache
    cache = {}
    for p in current_system.partitions:
        if p not in cache:
            cache[p] = {}
        for e in p.environs:
            rt.loadenv(p.local_env, e)
            if e is not in cache[p]:
                 cache[p][e] = ms.available_modules()
            # Then something like a substr match on cache[p][e] 
            # modules = OrderedSet(ms.available_modules(substr))
            snap0.restore()
            for m in cache[p][e]:
                if _is_valid_for_env(m, e.name):
                    yield (p.fullname, e.name, m)

Ok, we need to figure out the last part (what does _is_valid_for_env() actually does).

casparvl · 2026-04-29T14:39:47Z

CI is failing because the eessi mixin class and the metallwalls test set the default as valid programming environment. We should just replace that by *.

…ult', as we now use ReFrame environments to test over multiple versions of EESSI

casparvl · 2026-04-29T15:23:09Z

Ah, the issue with the CI is a bit more subtle: the GH actions site_configuration does define a site_configuration['systems']['partitions']['environs'] = 'default', but it does not set a site_configurations['environments'], because that is not needed for the default. As a result, our common_config now site_configurations['environments'] with a list that does NOT contain default, and this then causes ReFrame to crash since the default environment isn't specfied there.

One thing that's a bit weird is that ReFrame's standard environment is called builtin, not default. So I'm still not entirely sure how this ever worked...

Anyway, I'll just remove it. That way, the common_config should add the EESSI environments as environs.

casparvl · 2026-05-06T10:57:43Z

MetalWalls works now here as well:

[ RUN      ] EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=1_core %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /25c602eb @snellius:ro
me+EESSI-2023.06
[ RUN      ] EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=2_cores %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /e513c7a1 @snellius:r
ome+EESSI-2023.06
[ RUN      ] EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=4_cores %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /e9d3c3d0 @snellius:r
ome+EESSI-2023.06
[ RUN      ] EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=8_cores %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /bc798ef0 @snellius:r
ome+EESSI-2023.06
[ RUN      ] EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=1cpn_2nodes %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /f10015c8 @snelli
us:rome+EESSI-2023.06
[ RUN      ] EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=1cpn_4nodes %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /77e758f2 @snelli
us:rome+EESSI-2023.06
[ RUN      ] EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=1_8_node %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /a555f53d @snellius:rome+EESSI-2023.06
[ RUN      ] EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=1_4_node %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /02dec5af @snellius:rome+EESSI-2023.06
[ RUN      ] EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=1_2_node %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /b70e2271 @snellius:rome+EESSI-2023.06
[ RUN      ] EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=1_node %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /f1f510f9 @snellius:rome+EESSI-2023.06
[ RUN      ] EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=2_nodes %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /73a40f3b @snellius:rome+EESSI-2023.06
[ RUN      ] EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=4_nodes %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /4b6d96d2 @snellius:rome+EESSI-2023.06
[     SKIP ] ( 1/14) EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=4_nodes %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /4b6d96d2 @snellius:rome+EESSI-2023.06 [Number of tasks 512 exceeds maximum task count 256 for hackathonGPU/benchmark]
[ RUN      ] EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=8_nodes %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /1fdf3b8c @snellius:rome+EESSI-2023.06
[     SKIP ] ( 2/14) EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=8_nodes %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /1fdf3b8c @snellius:rome+EESSI-2023.06 [Number of tasks 1024 exceeds maximum task count 256 for hackathonGPU/benchmark]
[ RUN      ] EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=16_nodes %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /bb7e5da2 @snellius:rome+EESSI-2023.06
[     SKIP ] ( 3/14) EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=16_nodes %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /bb7e5da2 @snellius:rome+EESSI-2023.06 [Number of tasks 2048 exceeds maximum task count 256 for hackathonGPU/benchmark]
[       OK ] ( 4/14) EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=1_2_node %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /b70e2271 @snellius:rome+EESSI-2023.06
P: total_elapsed_time: 14.0009 s
[       OK ] ( 5/14) EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=1_node %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /f1f510f9 @snellius:rome+EESSI-2023.06
P: total_elapsed_time: 8.95646 s
WARNING: PROCESS BINDING ERROR: wrong number of nodes: expected 2, found 1
[       OK ] ( 6/14) EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=1_4_node %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /02dec5af @snellius:rome+EESSI-2023.06
P: total_elapsed_time: 24.8333 s
[       OK ] ( 7/14) EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=2_nodes %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /73a40f3b @snellius:rome+EESSI-2023.06
P: total_elapsed_time: 6.13102 s
[       OK ] ( 8/14) EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=1_8_node %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /a555f53d @snellius:rome+EESSI-2023.06
P: total_elapsed_time: 46.6689 s
[       OK ] ( 9/14) EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=4_cores %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /e9d3c3d0 @snellius:rome+EESSI-2023.06
P: total_elapsed_time: 177.33 s
[       OK ] (10/14) EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=8_cores %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /bc798ef0 @snellius:rome+EESSI-2023.06
P: total_elapsed_time: 156.227 s
WARNING: PROCESS BINDING ERROR: wrong number of nodes: expected 4, found 1
[       OK ] (11/14) EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=1cpn_4nodes %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /77e758f2 @snellius:rome+EESSI-2023.06
P: total_elapsed_time: 181.136 s
[       OK ] (12/14) EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=2_cores %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /e513c7a1 @snellius:rome+EESSI-2023.06
P: total_elapsed_time: 352.746 s
WARNING: PROCESS BINDING ERROR: wrong number of nodes: expected 2, found 1
[       OK ] (13/14) EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=1cpn_2nodes %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /f10015c8 @snellius:rome+EESSI-2023.06
P: total_elapsed_time: 358.229 s
[       OK ] (14/14) EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=1_core %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /25c602eb @snellius:rome+EESSI-2023.06
P: total_elapsed_time: 699.566 s
[----------] all spawned checks have finished

satishskamath · 2026-05-07T10:12:44Z

MetalWalls works now here as well:

[ RUN      ] EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=1_core %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /25c602eb @snellius:ro
me+EESSI-2023.06
[ RUN      ] EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=2_cores %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /e513c7a1 @snellius:r
ome+EESSI-2023.06
[ RUN      ] EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=4_cores %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /e9d3c3d0 @snellius:r
ome+EESSI-2023.06
[ RUN      ] EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=8_cores %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /bc798ef0 @snellius:r
ome+EESSI-2023.06
[ RUN      ] EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=1cpn_2nodes %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /f10015c8 @snelli
us:rome+EESSI-2023.06
[ RUN      ] EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=1cpn_4nodes %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /77e758f2 @snelli
us:rome+EESSI-2023.06
[ RUN      ] EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=1_8_node %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /a555f53d @snellius:rome+EESSI-2023.06
[ RUN      ] EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=1_4_node %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /02dec5af @snellius:rome+EESSI-2023.06
[ RUN      ] EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=1_2_node %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /b70e2271 @snellius:rome+EESSI-2023.06
[ RUN      ] EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=1_node %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /f1f510f9 @snellius:rome+EESSI-2023.06
[ RUN      ] EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=2_nodes %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /73a40f3b @snellius:rome+EESSI-2023.06
[ RUN      ] EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=4_nodes %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /4b6d96d2 @snellius:rome+EESSI-2023.06
[     SKIP ] ( 1/14) EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=4_nodes %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /4b6d96d2 @snellius:rome+EESSI-2023.06 [Number of tasks 512 exceeds maximum task count 256 for hackathonGPU/benchmark]
[ RUN      ] EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=8_nodes %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /1fdf3b8c @snellius:rome+EESSI-2023.06
[     SKIP ] ( 2/14) EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=8_nodes %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /1fdf3b8c @snellius:rome+EESSI-2023.06 [Number of tasks 1024 exceeds maximum task count 256 for hackathonGPU/benchmark]
[ RUN      ] EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=16_nodes %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /bb7e5da2 @snellius:rome+EESSI-2023.06
[     SKIP ] ( 3/14) EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=16_nodes %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /bb7e5da2 @snellius:rome+EESSI-2023.06 [Number of tasks 2048 exceeds maximum task count 256 for hackathonGPU/benchmark]
[       OK ] ( 4/14) EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=1_2_node %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /b70e2271 @snellius:rome+EESSI-2023.06
P: total_elapsed_time: 14.0009 s
[       OK ] ( 5/14) EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=1_node %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /f1f510f9 @snellius:rome+EESSI-2023.06
P: total_elapsed_time: 8.95646 s
WARNING: PROCESS BINDING ERROR: wrong number of nodes: expected 2, found 1
[       OK ] ( 6/14) EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=1_4_node %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /02dec5af @snellius:rome+EESSI-2023.06
P: total_elapsed_time: 24.8333 s
[       OK ] ( 7/14) EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=2_nodes %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /73a40f3b @snellius:rome+EESSI-2023.06
P: total_elapsed_time: 6.13102 s
[       OK ] ( 8/14) EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=1_8_node %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /a555f53d @snellius:rome+EESSI-2023.06
P: total_elapsed_time: 46.6689 s
[       OK ] ( 9/14) EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=4_cores %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /e9d3c3d0 @snellius:rome+EESSI-2023.06
P: total_elapsed_time: 177.33 s
[       OK ] (10/14) EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=8_cores %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /bc798ef0 @snellius:rome+EESSI-2023.06
P: total_elapsed_time: 156.227 s
WARNING: PROCESS BINDING ERROR: wrong number of nodes: expected 4, found 1
[       OK ] (11/14) EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=1cpn_4nodes %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /77e758f2 @snellius:rome+EESSI-2023.06
P: total_elapsed_time: 181.136 s
[       OK ] (12/14) EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=2_cores %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /e513c7a1 @snellius:rome+EESSI-2023.06
P: total_elapsed_time: 352.746 s
WARNING: PROCESS BINDING ERROR: wrong number of nodes: expected 2, found 1
[       OK ] (13/14) EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=1cpn_2nodes %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /f10015c8 @snellius:rome+EESSI-2023.06
P: total_elapsed_time: 358.229 s
[       OK ] (14/14) EESSI_MetalWalls_MW %benchmark_info=hackathonGPU/benchmark %scale=1_core %module_info=('snellius:rome', 'EESSI-2023.06', 'MetalWalls/21.06.1-foss-2023a') %device_type=cpu /25c602eb @snellius:rome+EESSI-2023.06
P: total_elapsed_time: 699.566 s
[----------] all spawned checks have finished

@casparvl I am not sure if process binding errors belong to the test right on top or bottom of them. But they are referring to nodes so most likely not or something is wrong with the terminology within the warning. Any comments there?

smoors · 2026-05-10T15:16:21Z

@casparvl I am not sure if process binding errors belong to the test right on top or bottom of them. But they are referring to nodes so most likely not or something is wrong with the terminology within the warning. Any comments there?

this was fixed in #338

Add initial functionality to schedule test for all EESSI environments…

30d4393

… in a single run, by configuring ReFrame environments and using ReFrame's own find_modules function

Make all tests aware that find_modules now returns a module_info obje…

ab0e7ce

…ct instead

satishskamath reviewed Mar 26, 2026

View reviewed changes

Comment thread eessi/testsuite/utils.py Outdated

casparvl commented Mar 26, 2026

View reviewed changes

Comment thread eessi/testsuite/utils.py Outdated

casparvl commented Mar 26, 2026

View reviewed changes

Comment thread eessi/testsuite/utils.py Outdated

casparvl and others added 6 commits March 26, 2026 14:38

Apply suggestion from @casparvl

8f7dad6

Apply suggestion from @casparvl

d6d4c5f

Fix linter errors

0e3a48a

Merge remote-tracking branch 'refs/remotes/origin/support_multiple_ee…

c63fbe0

…ssi_environments' into support_multiple_eessi_environments

Make sure that if reframe is new enough, we simply use the combinatio…

faf21ae

…n of sys:part +feat notation directly

Fix linting issues

8303df6

smoors reviewed Apr 4, 2026

View reviewed changes

Comment thread eessi/testsuite/eessi_mixin.py Outdated

smoors reviewed Apr 5, 2026

View reviewed changes

Comment thread eessi/testsuite/common_config.py

smoors reviewed Apr 5, 2026

View reviewed changes

Comment thread eessi/testsuite/eessi_mixin.py Outdated

casparvl and others added 2 commits April 7, 2026 17:08

add missing 'if'

5d58833

Co-authored-by: Sam Moors <smoors@users.noreply.github.com>

Use at least three letters for variable assignments

63c0eab

Make appending to the existing environment the default, so that one c…

0112ebe

…an still load environments for testing local modules

No longer require that the reframe environment is always called 'defa…

3f4edfa

…ult', as we now use ReFrame environments to test over multiple versions of EESSI

Rely on the common config to set the environs

2a6ddbb

casparvl commented Apr 30, 2026

View reviewed changes

Comment thread eessi/testsuite/tests/apps/MetalWalls.py

Caspar van Leeuwen added 2 commits May 5, 2026 16:51

Resolved merge conflict when mergin EESSI#332 into this branch

67d9506

Merge branch 'main' into support_multiple_eessi_environments

8e0dd27

Conversation

casparvl commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

casparvl commented Mar 12, 2026

Uh oh!

casparvl commented Mar 12, 2026

Uh oh!

casparvl commented Mar 12, 2026

Uh oh!

casparvl commented Mar 12, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

smoors commented Apr 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

smoors commented Apr 4, 2026

Uh oh!

Uh oh!

Uh oh!

casparvl commented Apr 7, 2026

Uh oh!

smoors commented Apr 7, 2026

Uh oh!

smoors commented Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

casparvl commented Apr 29, 2026

Uh oh!

casparvl commented Apr 29, 2026

Uh oh!

casparvl commented Apr 29, 2026

Uh oh!

Uh oh!

casparvl commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

satishskamath commented May 7, 2026

Uh oh!

smoors commented May 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

casparvl commented Mar 12, 2026 •

edited

Loading

smoors commented Apr 4, 2026 •

edited

Loading

smoors commented Apr 15, 2026 •

edited

Loading

casparvl commented May 6, 2026 •

edited

Loading