OpenHPC llvm4 module loading old gnu7.1

[CrashLaker]

Summary:

Using up-to-date 1.3 OpenHPC repo module add command to load llvm4 is loading old gnu7.1 path.

Steps to reproduce:
$ module add llvm4
$ clang

Expected results:
$ clang
all good

Actual results:
$ clang
[...]
clang: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by /opt/ohpc/pub/compiler/llvm/4.0.1/bin/../lib/../lib/../lib/../lib/libclangASTMatchers.so.4)
Because it's using base's libstdc++
$ echo $LD_LIBRARY_PATH
/opt/ohpc/pub/compiler/gcc/7.1.0/lib64:/opt/ohpc/pub/compiler/llvm/4.0.1/lib

Solution:

1. export LD_LIBRARY_PATH=/opt/ohpc/pub/compiler/gcc/7.2.0/lib64:/opt/ohpc/pub/compiler/llvm/4.0.1/lib
   or
2. Edit module script /opt/ohpc/pub/modulefiles/llvm4/4.0.1
   Change gcc/7.1.0 to gcc/7.2.0

[adrianreber]

OpenHPC 1.3.3 has llvm-5.0 and gcc 7.2.0? At least I think that is the situation? Which version of llvm and gcc do you have installed from OpenHPC?

[CrashLaker]

I have both llvm4/4.0.1 and llvm5/5.0.0 with openhpc's gnu 7.2.0

Some debugs:
$ module purge
$ module add llvm4/4.0.1
$ echo $LD_LIBRARY_PATH
/opt/ohpc/pub/compiler/gcc/7.1.0/lib64:/opt/ohpc/pub/compiler/llvm/4.0.1/lib
$ clang
[...]clang: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.21' not found[...]

$ module purge
$ module add llvm5/5.0.0
$ echo $LD_LIBRARY_PATH
/opt/ohpc/pub/compiler/gcc/7.2.0/lib64:/opt/ohpc/pub/compiler/llvm/5.0.0/lib
$ clang
clang-5.0: error: no input files

[crbaird]

This is a bug. The llvm module file contains the gcc library path it was built with, so a quick workaround is to edit /opt/ohpc/pub/modulefiles/llvm4/4.0.1 and point LD_LIBRARY_PATH to the gcc 7.2.0 lib directory instead of 7.1.0. We'll implement a permanent fix in ohpc 1.3.4.

[rengolin]

Changing the GCC package should not change LLVM's behaviour.

The most problematic cases here are glibc, libstdc++ and libgcc. While Glibc should NOT be replaced with musl / newlib (not in HPC at least), both libstdc++ and libgcc can be replaced with LLVM's own libcxx and compiler-rt.

So, work around options, change library path, fine. Long term options, there are some, each with its own problems...

1. Use a full LLVM stack, libcxx, compiler-rt, lld (including a stable glibc built by GCC in a separate folder). This won't give errors when using LLVM, but may mismatch symbols if you cross GCC and LLVM objects and try to link them together. By using GCC's own libraries, linking different objects is usually a trouble-free experience.

2. Have a dynamic library path, depending on which GCC is loaded. The problem with this is that now a lot of things depend on environment variables, and unix shells are known for trouble keeping variables intact.

3. Hard-code to the system's GCC. This is a problem if compiling libraries with module GCC and programs with LLVM, but it works out-of-the-box if only using LLVM.

4. Hard-code to module's GCC. This will work with Spack and manual compilation, but will require GCC module to be installed (but not loaded) and may conflict is cross-linking with system built objects.

Some of those problems are not specific to the situations (and is common in compilation environments), but we need to take decisions based on what users are more comfortable with.

Of course, there may be different solutions (I'm no expert in LMod, Spack), probably easier ones, but there will be some form of discussion as to what's the least damaging solution.

[crbaird]

fixed with release of llvm 5.0.1 in OpenHPC 1.3.4