Allegro LAMMPS on AMD GPUs

[johkl]

Dear developers,

Thanks for the excellent work and set of codes!

Do you have, or know of anyone else who have, any success in using/porting the pair_allegro LAMMPS patch on AMD GPUs? and if so what have been their experience?

Best,
Johan

[Linux-cpp-lisp]

Hi Johan,

Thanks for your your interest in our code!

Yes, we know of at least one or two groups that have done this with some success. My general understanding is that it didn't require too much fiddling, since ROCm devices are exposed at a user level as CUDA devices in PyTorch. I'll ping @svandenhaute in case he's willing to share his experience.

[johkl]

I did (very non-systematically) try adding appropriate paths using combinations of CMAKE_LIBRARY_PATH / CMAKE_INCLUDE_PATH / CMAKE_PREFIX_PATH .

I suppose it should definitely be possible to solve the problem in this way, I just never managed to fully do it this way.

[hmcezar]

Sorry to come back to this after so long, but I'm trying to create a container with NequIP/Allegro/LAMMPS on LUMI and I thought maybe @svandenhaute or @Linux-cpp-lisp could help.

Using LUMI/23.03 I managed to create a container with conda-containerize that allowed me to install NequIP, Allegro and Torch 1.11.0 (from source).
But I'm having problems with pair_allegro now.

After getting LAMMPS stable_2Aug2023_update2 and applying the patch from the multicut branch I had to manually add

find_package(hip REQUIRED)
find_package(rocblas REQUIRED)
find_package(hipfft REQUIRED)
find_package(hipsparse REQUIRED)
find_package(hiprand REQUIRED)

to lammps/cmake/CMakeLists.txt to pass cmake.
However, when I run make I'm getting:

/project/project_465000259/musselic/software/pytorch/torch/include/c10/macros/Macros.h:368:5: error: __host__ __device__ function '__assert_fail' cannot overload __device__ function '__assert_fail'
    __assert_fail(
    ^
/opt/rocm-5.2.3/include/hip/amd_detail/amd_device_functions.h:832:6: note: previous declaration is here
void __assert_fail(const char *assertion,
     ^

Any ideas on how to solve this? I feel that I'm close, but I'm not sure how to move forward.

[Linux-cpp-lisp]

@anjohan ?

[hmcezar]

So, after digging a bit (and checking a few git blames) I think the issue is I'm using pytorch 1.11.0, which had this issue:
pytorch/pytorch#73040

In this case I think I'll have to patch pytorch or use a version that is not 1.11.0.

Is 1.11.0 still the recommended version for production runs @Linux-cpp-lisp ?

If any of you have any suggestions, I'm open to new ideas.

Thanks!

[Linux-cpp-lisp]

Oh, right---on AMD GPUs, specifically, I'm pretty sure later versions are fine. (See mir-group/nequip#311 (reply in thread) and Ctrl-F AMD)