[SOLVED] gromacs doesn't build because of no AVX2 support

E3LDDfrK · 2019-10-25 07:08:56

Hi,

I recently tried to install gromacs package from AUR using pikaur (an AUR helper).
By default, it also tries to install its own fftw library. As far as I understand, because of performance and compatibility reason.

It fails at this step when compiling its own fftw:

libtool: compile:  gcc-8 -DHAVE_CONFIG_H -I. -I/home/user/.cache/pikaur/build/gromacs/src/double/src/external/build-fftw/fftwBuild-prefix/src/fftwBuild/dft/simd/avx2 -I../../.. -I /home/user/.cache/pikaur/build/gromacs/src/double/src/external/build-fftw/fftwBuild-prefix/src/fftwBuild -D_FORTIFY_SOURCE=2 -march=core-avx2 -mfma -march=native -O2 -pipe -fno-plt -MT n1fv_2.lo -MD -MP -MF .deps/n1fv_2.Tpo -c /home/user/.cache/pikaur/build/gromacs/src/double/src/external/build-fftw/fftwBuild-prefix/src/fftwBuild/dft/simd/avx2/n1fv_2.c  -fPIC -DPIC -o n1fv_2.o  
In file included from /home/user/.cache/pikaur/build/gromacs/src/double/src/external/build-fftw/fftwBuild-prefix/src/fftwBuild/dft/simd/n1f.h:21,
                 from /home/user/.cache/pikaur/build/gromacs/src/double/src/external/build-fftw/fftwBuild-prefix/src/fftwBuild/dft/simd/avx2/../common/n1fv_2.c:71,
                 from /home/user/.cache/pikaur/build/gromacs/src/double/src/external/build-fftw/fftwBuild-prefix/src/fftwBuild/dft/simd/avx2/n1fv_2.c:3:
/home/user/.cache/pikaur/build/gromacs/src/double/src/external/build-fftw/fftwBuild-prefix/src/fftwBuild/simd-support/simd-avx2.h:43:2: error: #error "compiling simd-avx2.h without avx2 support"
 #error "compiling simd-avx2.h without avx2 support"

If I build it manually from the website, it doesn't fail. But the line is slightly different:

libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I/home/user/download/gromacs-2019.4/build/src/external/build-fftw/fftwBuild-prefix/src/fftwBuild/dft/simd/avx2 -I../../.. -I /home/user/download/gromacs-2019.4/build/src/external/build-fftw/fftwBuild-prefix/src/fftwBuild -march=core-avx2 -mfma -O3 -fomit-frame-pointer -mtune=native -malign-double -fstrict-aliasing -fno-schedule-insns -MT n1fv_2.lo -MD -MP -MF .deps/n1fv_2.Tpo -c /home/user/download/gromacs-2019.4/build/src/external/build-fftw/fftwBuild-prefix/src/fftwBuild/dft/simd/avx2/n1fv_2.c  -fPIC -DPIC -o n1fv_2.o

From what I can tell, the problem is the failed build has both "-march=core-avx2 -march=native" when compiling fftw. It fails because my "-march=native"/Sandy Bridge CPU doesn't actually support AVX2. The tagged on "-march=native -O2 -pipe -fno-plt -MT" comes from CFLAGS set in /etc/makepkg.conf.

The gcc-8 is just because CUDA needs it, can be changed in the PKGBUILD.

Any suggestion how to solve this?

edit: Just tried with the default CFLAGS="-march=x86-64 -mtune=generic -O2 -pipe -fno-plt" in /etc/makepkg.conf. Doesn't work. The avx2 step of fftw compilation uses -march=core-avx2 -march=x86-64 -mtune=generic this time. It gives the same error of no avx2 support.

edit2: A solution is here https://bbs.archlinux.org/viewtopic.php … 1#p1870411

Last edited by E3LDDfrK (2019-10-25 23:34:14)

Lone_Wolf · 2019-10-25 10:40:14

-DGMX_SIMD={AVX_128_FMA, AVX_256, AVX2_256, AVX2_128, AVX_512}

Have you tried adding that to the cmake command in the PKGBUILD as mentioned in the sticky comments ?

E3LDDfrK · 2019-10-25 18:20:59

Lone_Wolf wrote:

-DGMX_SIMD={AVX_128_FMA, AVX_256, AVX2_256, AVX2_128, AVX_512}
Have you tried adding that to the cmake command in the PKGBUILD as mentioned in the sticky comments ?

Yes, I chose the one supported by my CPU, "-DGMX_SIMD=AVX_256". Both for installing from AUR and from the website manually. Basically, that option works fine for the gromacs part of the build. But the gromacs package also build its own fftw library. And the fftw library is built with sse, avx, avx2, etc. support. At the avx2 step of fftw compilation, it has a default "-march=core-avx2". The problem is, I guess, there's a tagged on CFLAGS="-march=native -O2 -pipe -fno-plt". So the end result has both "-march=core-avx2 -march=native". That part breaks the build, as the last few comments on the AUR page indicate. I think the other 2 commenters have the same problem.

edit: Just to be clear at other stages of the fftw compilation, it has "-msse2" or "-mavx" options for the gcc. The "-march=native" from CFLAGS is also tagged on in those cases. But it's fine because my CPU supports them. But at the avx2 stage, it has a "-march=core-avx2" by default. The tagged on "-march=native" breaks the build, because my CPU doesn't support AVX2.

Last edited by E3LDDfrK (2019-10-25 18:37:11)

loqs · 2019-10-25 20:03:23

Can you test the following PKGBUILD changes

diff --git a/PKGBUILD b/PKGBUILD
index 17e1900..4a1bc14 100644
--- a/PKGBUILD
+++ b/PKGBUILD
@@ -29,6 +29,7 @@ export CC=gcc-8
 export CXX=g++-8 
 
 build() {
+  sed -i 's/set(_fftw_simd_support_level --enable-sse2;--enable-avx;--enable-avx2;--enable-avx512)/set(_fftw_simd_support_level --enable-sse2;--enable-avx)/' gromacs-2019.4/src/external/build-fftw/CMakeLists.txt
   mkdir -p ${srcdir}/{single,double}
 
  
@@ -39,8 +40,8 @@ build() {
         -DCMAKE_INSTALL_LIBDIR=lib \
         -DGMX_DOUBLE=ON \
         -DGMX_BUILD_OWN_FFTW=ON \
-        -DREGRESSIONTEST_DOWNLOAD=ON
-        #-DGMX_SIMD=AVX2_256 \
+        -DREGRESSIONTEST_DOWNLOAD=ON \
+        -DGMX_SIMD=AVX_256
   make
 
   msg2 "Building the single precision files"
@@ -49,8 +50,8 @@ build() {
         -DCMAKE_INSTALL_PREFIX=/usr/ \
         -DCMAKE_INSTALL_LIBDIR=lib\
         -DGMX_BUILD_OWN_FFTW=ON \
-        -DREGRESSIONTEST_DOWNLOAD=ON
-        #-DGMX_SIMD=AVX2_256 \
+        -DREGRESSIONTEST_DOWNLOAD=ON \
+        -DGMX_SIMD=AVX_256
   make
 }

The sed line is just a test to see if it builds correctly on your system.
If the approach is correct then a more complicated patch will be needed for the gromacs-2019.4/src/external/build-fftw/CMakeLists.txt
Edit:
Also is the performance of the package fftw package much worse than the static lib gromacs builds and uses?

Last edited by loqs (2019-10-25 21:35:27)

E3LDDfrK · 2019-10-25 23:24:07

loqs wrote:

Can you test the following PKGBUILD changes

Yes, that works. The build passed all of the regression tests for correctness too. Thank you for that.

On their official installation guide, it says:

From FFTW-3.3.5, you should also add --enable-avx2 also.

There's probably a good reason why they by default uses the option "--enable-avx2" even when they detects no avx2 support. Who knows.

loqs wrote:

Also is the performance of the package fftw package much worse than the static lib gromacs builds and uses?

Probably not. That's what I did before posting the question here. I'm just trying to follow their recommendation from their installation guide:

FFTW is likely to be available for your platform via its package management system, but there can be compatibility and significant performance issues associated with these packages. In particular, GROMACS simulations are normally run in “mixed” floating-point precision, which is suited for the use of single precision in FFTW. The default FFTW package is normally in double precision, and good compiler options to use for FFTW when linked to GROMACS may not have been used.

As far as I understand, the fftw package from Extra has both single and double precision.

Building it manually works. Yes, it still compiles the avx2 part of the fftw. But since my CFLAGS in /etc/makepkg.conf is not appended to it, it compiles just fine. Perhaps the PKGBUILD can be edited to ignore the CFLAGS in /etc/makepkg.conf in my case. I don't know how though.

I consider this is solved. Thanks again.

loqs · 2019-10-25 23:42:28

The reason I was suggesting a more complex patch would be needed for the PKGBUILD is when the system has AVX2 support but not AVX512 the sed fix will cause it to build without AVX2.

E3LDDfrK · 2019-10-26 00:11:03

I got that. I was just saying that, in my case, if the CFLAGS in /etc/makepkg.conf is not involved in the build, it will compile successfully. But I don't know how to edit the PKGBUILD to do that. If I download the package, and build it manually with cmake, it works.

With the AUR package, the problematic compiler options when compiling fftw used are:

-march=core-avx2 -mfma -march=native -O2 -pipe -fno-plt

The "-march=native -O2 -pipe -fno-plt" comes from my CFLAGS in /etc/makepkg.conf.

When I build gromacs manually, the compiler options used by default are:

-march=core-avx2 -mfma -O3 -fomit-frame-pointer -mtune=native -malign-double -fstrict-aliasing -fno-schedule-insns

This one builds successfully. So, for me, if I can somehow tell the AUR one to ignore the CFLAGS in /etc/makepkg.conf, it should work fine.

Last edited by E3LDDfrK (2019-10-26 00:15:37)

loqs · 2019-10-26 00:20:55

options=(!buildflags)

Disables CPPFLAGS, CFLAGS,CXXFLAGS, LDFLAGS from makepkg.conf see man PKGBUILD for more details.
You could also use string substitution to manipulate the value of CFLAGS or just overwrite the value of CFLAGS.
You could argue anyone not using the default.

-march=x86-64 -mtune=generic -O2 -pipe -fno-plt

can expect the PKGBUILD to be untested with such flags and have unexpected results.

E3LDDfrK · 2019-10-26 00:27:39

loqs wrote:

You could argue anyone not using the default.
-march=x86-64 -mtune=generic -O2 -pipe -fno-plt
can expect the PKGBUILD to be untested with such flags and have unexpected results.

But I also tested the default? As I wrote on OP. It doesn't work. I got the same error. I'd guess again because of the double -march thing "-march=core-avx2 -march=x86-64".

loqs · 2019-10-26 00:46:05

Ah I missed the point. If I now understand you want it to build with the specified optimizations irrespective of build system hardware support.

Last edited by loqs (2019-10-26 00:46:22)

E3LDDfrK · 2019-10-26 01:04:01

loqs wrote:

If I now understand you want it to build with the specified optimizations irrespective of build system hardware support.

You mean the "specified optimizations" from gromacs itself? Not exactly.

I just want to build the AUR package of gromacs instead of downloading the package from the website and build it manually. Because it's more convenient in terms of update, uninstall, etc. with the AUR helper. But the straightforward way of doing that doesn't work because my CPU doesn't support avx2.

I honestly have no idea why this line shows up even though it detects that my CPU doesn't support avx2:

-- The GROMACS-managed build of FFTW 3 will configure with the following optimizations: --enable-sse2;--enable-avx;--enable-avx2;--enable-avx512

Is there a good reason why they do that? Who knows. But your solution of manually removing "--enable-avx2;--enable-avx512" solves my problem. Using the fftw library from [Extra] also works. Both pass all of the regression tests for correctness.

Disabling the buildflags may work too. I will not try it, the compilation of this package takes too much time on my laptop.

Last edited by E3LDDfrK (2019-10-26 01:12:52)

loqs · 2019-10-26 01:20:44

It does that because of

# Set library optimizations
set(_fftw_simd_support_level "")
if(${GMX_SIMD_ACTIVE} MATCHES "^(SSE|AVX)" AND APPLE)
    # OS X and --enable-avx causes compilation issues (fftw always picks gcc by default). It's
    # not an important enough performance loss to bother warning the
    # user about.
    set(_fftw_simd_support_level "--enable-sse2")
elseif(${GMX_SIMD_ACTIVE} MATCHES "^(SSE)")
    set(_fftw_simd_support_level "--enable-sse2")
elseif(${GMX_SIMD_ACTIVE} MATCHES "^(AVX)")
    # Testing shows FFTW configured with --enable-sse2 --enable-avx is
    # slightly faster on most architectures than --enable-sse2 alone.
    # Support for --enable-avx2 was only added in 3.3.5, but
    # configuring with it is at worst a warning, even on an earlier
    # version.
    # MSVC, GCC < 4.9, Clang < 3.9 do not support AVX-512, so
    # we should not enable it there. FFTW does not support clang with
    # AVX-512, so we should not enable that either.
if(MSVC OR (CMAKE_COMPILER_IS_GNUCC AND CMAKE_C_COMPILER_VERSION VERSION_LESS 4.9.0) OR
   (CMAKE_C_COMPILER_ID MATCHES "Clang" AND CMAKE_C_COMPILER_VERSION VERSION_LESS 3.9.0) OR
   (CMAKE_C_COMPILER_ID MATCHES "Clang" AND ${GMX_SIMD_ACTIVE} MATCHES "^(AVX_512)"))
    set(_fftw_simd_support_level --enable-sse2;--enable-avx;--enable-avx2)
else()
    set(_fftw_simd_support_level --enable-sse2;--enable-avx;--enable-avx2;--enable-avx512)
endif()
elseif(${GMX_SIMD_ACTIVE} MATCHES "^(VSX)")
    set(_fftw_simd_support_level --enable-vsx)
endif()

So I think something like the following which if DGMX_SIMD=AVX_256 should drop avx2 and avx512

diff --git a/gromacs-2019.4/src/external/build-fftw/CMakeLists.txt b/gromacs-2019.4/src/external/build-fftw/CMakeLists.txt
index e8084f2..c82d5f4 100644
--- a/gromacs-2019.4/src/external/build-fftw/CMakeLists.txt
+++ b/gromacs-2019.4/src/external/build-fftw/CMakeLists.txt
@@ -80,9 +80,13 @@ if(MSVC OR (CMAKE_COMPILER_IS_GNUCC AND CMAKE_C_COMPILER_VERSION VERSION_LESS 4.
    (CMAKE_C_COMPILER_ID MATCHES "Clang" AND CMAKE_C_COMPILER_VERSION VERSION_LESS 3.9.0) OR
    (CMAKE_C_COMPILER_ID MATCHES "Clang" AND ${GMX_SIMD_ACTIVE} MATCHES "^(AVX_512)"))
     set(_fftw_simd_support_level --enable-sse2;--enable-avx;--enable-avx2)
+else()
+if(${GMX_SIMD_ACTIVE} MATCHES "^(AVX_256)")
+    set(_fftw_simd_support_level --enable-sse2;--enable-avx)
 else()
     set(_fftw_simd_support_level --enable-sse2;--enable-avx;--enable-avx2;--enable-avx512)
 endif()
+endif()
 elseif(${GMX_SIMD_ACTIVE} MATCHES "^(VSX)")
     set(_fftw_simd_support_level --enable-vsx)
 endif()

E3LDDfrK · 2019-10-26 01:27:33

Yes, perhaps they have a good reason why they add both --enable-avx and --enable-avx2 together. I don't know.

E3LDDfrK · 2019-10-26 21:02:56

edit: Never mind, probably the list on the official GCC manual is just incomplete.

Just want to share more information for people who experience a similar problem.

It seems gcc doesn't officially support "-march=core-avx2" flag anymore. According to the GCC manual, it should have used "-mavx2". From what I read, the "-march=core-avx2" flag still can be used with the intel compiler.

The "-march=core-avx2" flag still obviously works with gcc though, even though it's not officially supported. When I downloaded it and built it according to the instruction in the manual, it compiled just fine with "-march=core-avx2". The fftw devs probably has a reason why they still use that flag.

Last edited by E3LDDfrK (2019-10-26 21:14:48)

Arch Linux

#1 2019-10-25 07:08:56

[SOLVED] gromacs doesn't build because of no AVX2 support

#2 2019-10-25 10:40:14

Re: [SOLVED] gromacs doesn't build because of no AVX2 support

#3 2019-10-25 18:20:59

Re: [SOLVED] gromacs doesn't build because of no AVX2 support

#4 2019-10-25 20:03:23

Re: [SOLVED] gromacs doesn't build because of no AVX2 support

#5 2019-10-25 23:24:07

Re: [SOLVED] gromacs doesn't build because of no AVX2 support

#6 2019-10-25 23:42:28

Re: [SOLVED] gromacs doesn't build because of no AVX2 support

#7 2019-10-26 00:11:03

Re: [SOLVED] gromacs doesn't build because of no AVX2 support

#8 2019-10-26 00:20:55

Re: [SOLVED] gromacs doesn't build because of no AVX2 support

#9 2019-10-26 00:27:39

Re: [SOLVED] gromacs doesn't build because of no AVX2 support

#10 2019-10-26 00:46:05

Re: [SOLVED] gromacs doesn't build because of no AVX2 support

#11 2019-10-26 01:04:01

Re: [SOLVED] gromacs doesn't build because of no AVX2 support

#12 2019-10-26 01:20:44

Re: [SOLVED] gromacs doesn't build because of no AVX2 support

#13 2019-10-26 01:27:33

Re: [SOLVED] gromacs doesn't build because of no AVX2 support

#14 2019-10-26 21:02:56

Re: [SOLVED] gromacs doesn't build because of no AVX2 support

Board footer