[SOLVED] Compiling with OpenMPI fails

fazky · 2015-04-07 13:39:04

Hello,

I am trying to compile a program (the name is iqtree) with openMPI. Compilation without openMPI works perfectly. However, if I try to use openMPI, I get the following error when running make:

/usr/bin/ld: cannot find -lgomp

I have extra/openmpi 1.8.4-1 installed and if I try to run ld separately, I get:

$ ld -lgomp --verbose
GNU ld (GNU Binutils) 2.25.0
  Supported emulations:
   elf_x86_64
   elf32_x86_64
   elf_i386
   i386linux
   elf_l1om
   elf_k1om
using internal linker script:
==================================================
/* Script for -z combreloc: combine and sort reloc sections */
/* Copyright (C) 2014 Free Software Foundation, Inc.
   Copying and distribution of this script, with or without modification,
   are permitted in any medium without royalty provided the copyright
   notice and this notice are preserved.  */
OUTPUT_FORMAT("elf64-x86-64", "elf64-x86-64",
	      "elf64-x86-64")
OUTPUT_ARCH(i386:x86-64)
ENTRY(_start)
SEARCH_DIR("/usr/x86_64-unknown-linux-gnu/lib64"); SEARCH_DIR("/usr/lib"); SEARCH_DIR("/usr/local/lib"); SEARCH_DIR("/usr/x86_64-unknown-linux-gnu/lib");
SECTIONS
{
  /* Read-only sections, merged into text segment: */
  PROVIDE (__executable_start = SEGMENT_START("text-segment", 0x400000)); . = SEGMENT_START("text-segment", 0x400000) + SIZEOF_HEADERS;
  .interp         : { *(.interp) }
  .note.gnu.build-id : { *(.note.gnu.build-id) }
  .hash           : { *(.hash) }
  .gnu.hash       : { *(.gnu.hash) }
  .dynsym         : { *(.dynsym) }
  .dynstr         : { *(.dynstr) }
  .gnu.version    : { *(.gnu.version) }
  .gnu.version_d  : { *(.gnu.version_d) }
  .gnu.version_r  : { *(.gnu.version_r) }
  .rela.dyn       :
    {
      *(.rela.init)
      *(.rela.text .rela.text.* .rela.gnu.linkonce.t.*)
      *(.rela.fini)
      *(.rela.rodata .rela.rodata.* .rela.gnu.linkonce.r.*)
      *(.rela.data .rela.data.* .rela.gnu.linkonce.d.*)
      *(.rela.tdata .rela.tdata.* .rela.gnu.linkonce.td.*)
      *(.rela.tbss .rela.tbss.* .rela.gnu.linkonce.tb.*)
      *(.rela.ctors)
      *(.rela.dtors)
      *(.rela.got)
      *(.rela.bss .rela.bss.* .rela.gnu.linkonce.b.*)
      *(.rela.ldata .rela.ldata.* .rela.gnu.linkonce.l.*)
      *(.rela.lbss .rela.lbss.* .rela.gnu.linkonce.lb.*)
      *(.rela.lrodata .rela.lrodata.* .rela.gnu.linkonce.lr.*)
      *(.rela.ifunc)
    }
  .rela.plt       :
    {
      *(.rela.plt)
      PROVIDE_HIDDEN (__rela_iplt_start = .);
      *(.rela.iplt)
      PROVIDE_HIDDEN (__rela_iplt_end = .);
    }
  .init           :
  {
    KEEP (*(SORT_NONE(.init)))
  }
  .plt            : { *(.plt) *(.iplt) }
  .plt.bnd        : { *(.plt.bnd) }
  .text           :
  {
    *(.text.unlikely .text.*_unlikely .text.unlikely.*)
    *(.text.exit .text.exit.*)
    *(.text.startup .text.startup.*)
    *(.text.hot .text.hot.*)
    *(.text .stub .text.* .gnu.linkonce.t.*)
    /* .gnu.warning sections are handled specially by elf32.em.  */
    *(.gnu.warning)
  }
  .fini           :
  {
    KEEP (*(SORT_NONE(.fini)))
  }
  PROVIDE (__etext = .);
  PROVIDE (_etext = .);
  PROVIDE (etext = .);
  .rodata         : { *(.rodata .rodata.* .gnu.linkonce.r.*) }
  .rodata1        : { *(.rodata1) }
  .eh_frame_hdr : { *(.eh_frame_hdr) }
  .eh_frame       : ONLY_IF_RO { KEEP (*(.eh_frame)) }
  .gcc_except_table   : ONLY_IF_RO { *(.gcc_except_table
  .gcc_except_table.*) }
  /* These sections are generated by the Sun/Oracle C++ compiler.  */
  .exception_ranges   : ONLY_IF_RO { *(.exception_ranges
  .exception_ranges*) }
  /* Adjust the address for the data segment.  We want to adjust up to
     the same address within the page on the next page up.  */
  . = ALIGN (CONSTANT (MAXPAGESIZE)) - ((CONSTANT (MAXPAGESIZE) - .) & (CONSTANT (MAXPAGESIZE) - 1)); . = DATA_SEGMENT_ALIGN (CONSTANT (MAXPAGESIZE), CONSTANT (COMMONPAGESIZE));
  /* Exception handling  */
  .eh_frame       : ONLY_IF_RW { KEEP (*(.eh_frame)) }
  .gcc_except_table   : ONLY_IF_RW { *(.gcc_except_table .gcc_except_table.*) }
  .exception_ranges   : ONLY_IF_RW { *(.exception_ranges .exception_ranges*) }
  /* Thread Local Storage sections  */
  .tdata	  : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
  .tbss		  : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
  .preinit_array     :
  {
    PROVIDE_HIDDEN (__preinit_array_start = .);
    KEEP (*(.preinit_array))
    PROVIDE_HIDDEN (__preinit_array_end = .);
  }
  .init_array     :
  {
    PROVIDE_HIDDEN (__init_array_start = .);
    KEEP (*(SORT_BY_INIT_PRIORITY(.init_array.*) SORT_BY_INIT_PRIORITY(.ctors.*)))
    KEEP (*(.init_array EXCLUDE_FILE (*crtbegin.o *crtbegin?.o *crtend.o *crtend?.o ) .ctors))
    PROVIDE_HIDDEN (__init_array_end = .);
  }
  .fini_array     :
  {
    PROVIDE_HIDDEN (__fini_array_start = .);
    KEEP (*(SORT_BY_INIT_PRIORITY(.fini_array.*) SORT_BY_INIT_PRIORITY(.dtors.*)))
    KEEP (*(.fini_array EXCLUDE_FILE (*crtbegin.o *crtbegin?.o *crtend.o *crtend?.o ) .dtors))
    PROVIDE_HIDDEN (__fini_array_end = .);
  }
  .ctors          :
  {
    /* gcc uses crtbegin.o to find the start of
       the constructors, so we make sure it is
       first.  Because this is a wildcard, it
       doesn't matter if the user does not
       actually link against crtbegin.o; the
       linker won't look for a file to match a
       wildcard.  The wildcard also means that it
       doesn't matter which directory crtbegin.o
       is in.  */
    KEEP (*crtbegin.o(.ctors))
    KEEP (*crtbegin?.o(.ctors))
    /* We don't want to include the .ctor section from
       the crtend.o file until after the sorted ctors.
       The .ctor section from the crtend file contains the
       end of ctors marker and it must be last */
    KEEP (*(EXCLUDE_FILE (*crtend.o *crtend?.o ) .ctors))
    KEEP (*(SORT(.ctors.*)))
    KEEP (*(.ctors))
  }
  .dtors          :
  {
    KEEP (*crtbegin.o(.dtors))
    KEEP (*crtbegin?.o(.dtors))
    KEEP (*(EXCLUDE_FILE (*crtend.o *crtend?.o ) .dtors))
    KEEP (*(SORT(.dtors.*)))
    KEEP (*(.dtors))
  }
  .jcr            : { KEEP (*(.jcr)) }
  .data.rel.ro : { *(.data.rel.ro.local* .gnu.linkonce.d.rel.ro.local.*) *(.data.rel.ro .data.rel.ro.* .gnu.linkonce.d.rel.ro.*) }
  .dynamic        : { *(.dynamic) }
  .got            : { *(.got) *(.igot) }
  . = DATA_SEGMENT_RELRO_END (SIZEOF (.got.plt) >= 24 ? 24 : 0, .);
  .got.plt        : { *(.got.plt)  *(.igot.plt) }
  .data           :
  {
    *(.data .data.* .gnu.linkonce.d.*)
    SORT(CONSTRUCTORS)
  }
  .data1          : { *(.data1) }
  _edata = .; PROVIDE (edata = .);
  . = .;
  __bss_start = .;
  .bss            :
  {
   *(.dynbss)
   *(.bss .bss.* .gnu.linkonce.b.*)
   *(COMMON)
   /* Align here to ensure that the .bss section occupies space up to
      _end.  Align after .bss to ensure correct alignment even if the
      .bss section disappears because there are no input sections.
      FIXME: Why do we need it? When there is no .bss section, we don't
      pad the .data section.  */
   . = ALIGN(. != 0 ? 64 / 8 : 1);
  }
  .lbss   :
  {
    *(.dynlbss)
    *(.lbss .lbss.* .gnu.linkonce.lb.*)
    *(LARGE_COMMON)
  }
  . = ALIGN(64 / 8);
  . = SEGMENT_START("ldata-segment", .);
  .lrodata   ALIGN(CONSTANT (MAXPAGESIZE)) + (. & (CONSTANT (MAXPAGESIZE) - 1)) :
  {
    *(.lrodata .lrodata.* .gnu.linkonce.lr.*)
  }
  .ldata   ALIGN(CONSTANT (MAXPAGESIZE)) + (. & (CONSTANT (MAXPAGESIZE) - 1)) :
  {
    *(.ldata .ldata.* .gnu.linkonce.l.*)
    . = ALIGN(. != 0 ? 64 / 8 : 1);
  }
  . = ALIGN(64 / 8);
  _end = .; PROVIDE (end = .);
  . = DATA_SEGMENT_END (.);
  /* Stabs debugging sections.  */
  .stab          0 : { *(.stab) }
  .stabstr       0 : { *(.stabstr) }
  .stab.excl     0 : { *(.stab.excl) }
  .stab.exclstr  0 : { *(.stab.exclstr) }
  .stab.index    0 : { *(.stab.index) }
  .stab.indexstr 0 : { *(.stab.indexstr) }
  .comment       0 : { *(.comment) }
  /* DWARF debug sections.
     Symbols in the DWARF debugging sections are relative to the beginning
     of the section so we begin them at 0.  */
  /* DWARF 1 */
  .debug          0 : { *(.debug) }
  .line           0 : { *(.line) }
  /* GNU DWARF 1 extensions */
  .debug_srcinfo  0 : { *(.debug_srcinfo) }
  .debug_sfnames  0 : { *(.debug_sfnames) }
  /* DWARF 1.1 and DWARF 2 */
  .debug_aranges  0 : { *(.debug_aranges) }
  .debug_pubnames 0 : { *(.debug_pubnames) }
  /* DWARF 2 */
  .debug_info     0 : { *(.debug_info .gnu.linkonce.wi.*) }
  .debug_abbrev   0 : { *(.debug_abbrev) }
  .debug_line     0 : { *(.debug_line .debug_line.* .debug_line_end ) }
  .debug_frame    0 : { *(.debug_frame) }
  .debug_str      0 : { *(.debug_str) }
  .debug_loc      0 : { *(.debug_loc) }
  .debug_macinfo  0 : { *(.debug_macinfo) }
  /* SGI/MIPS DWARF 2 extensions */
  .debug_weaknames 0 : { *(.debug_weaknames) }
  .debug_funcnames 0 : { *(.debug_funcnames) }
  .debug_typenames 0 : { *(.debug_typenames) }
  .debug_varnames  0 : { *(.debug_varnames) }
  /* DWARF 3 */
  .debug_pubtypes 0 : { *(.debug_pubtypes) }
  .debug_ranges   0 : { *(.debug_ranges) }
  /* DWARF Extension.  */
  .debug_macro    0 : { *(.debug_macro) }
  .gnu.attributes 0 : { KEEP (*(.gnu.attributes)) }
  /DISCARD/ : { *(.note.GNU-stack) *(.gnu_debuglink) *(.gnu.lto_*) }
}


==================================================
attempt to open /usr/x86_64-unknown-linux-gnu/lib64/libgomp.so failed
attempt to open /usr/x86_64-unknown-linux-gnu/lib64/libgomp.a failed
attempt to open /usr/lib/libgomp.so succeeded
-lgomp (/usr/lib/libgomp.so)
libpthread.so.0 needed by /usr/lib/libgomp.so
found libpthread.so.0 at /usr/lib/libpthread.so.0
libc.so.6 needed by /usr/lib/libgomp.so
found libc.so.6 at /usr/lib/libc.so.6
ld-linux-x86-64.so.2 needed by /usr/lib/libpthread.so.0
found ld-linux-x86-64.so.2 at /usr/lib/ld-linux-x86-64.so.2
ld: warning: cannot find entry symbol _start; not setting start address

I am not sure if anything of this is relevant. I tried to nail it down to the actual command of make that fails:

$ /usr/bin/c++    -fopenmp  -O3 -g0   -static CMakeFiles/iqtree.dir/alignment.cpp.o CMakeFiles/iqtree.dir/alignmentpairwise.cpp.o CMakeFiles/iqtree.dir/circularnetwork.cpp.o CMakeFiles/iqtree.dir/eigendecomposition.cpp.o CMakeFiles/iqtree.dir/greedy.cpp.o CMakeFiles/iqtree.dir/gss.cpp.o CMakeFiles/iqtree.dir/guidedbootstrap.cpp.o CMakeFiles/iqtree.dir/gurobiwrapper.cpp.o CMakeFiles/iqtree.dir/gzstream.cpp.o CMakeFiles/iqtree.dir/hashsplitset.cpp.o CMakeFiles/iqtree.dir/iqtree.cpp.o CMakeFiles/iqtree.dir/maalignment.cpp.o CMakeFiles/iqtree.dir/matree.cpp.o CMakeFiles/iqtree.dir/mexttree.cpp.o CMakeFiles/iqtree.dir/mpdablock.cpp.o CMakeFiles/iqtree.dir/msetsblock.cpp.o CMakeFiles/iqtree.dir/msplitsblock.cpp.o CMakeFiles/iqtree.dir/modelsblock.cpp.o CMakeFiles/iqtree.dir/mtree.cpp.o CMakeFiles/iqtree.dir/mtreeset.cpp.o CMakeFiles/iqtree.dir/ncbitree.cpp.o CMakeFiles/iqtree.dir/ngs.cpp.o CMakeFiles/iqtree.dir/node.cpp.o CMakeFiles/iqtree.dir/optimization.cpp.o CMakeFiles/iqtree.dir/parsmultistate.cpp.o CMakeFiles/iqtree.dir/pattern.cpp.o CMakeFiles/iqtree.dir/pda.cpp.o CMakeFiles/iqtree.dir/pdnetwork.cpp.o CMakeFiles/iqtree.dir/pdtree.cpp.o CMakeFiles/iqtree.dir/pdtreeset.cpp.o CMakeFiles/iqtree.dir/phyloanalysis.cpp.o CMakeFiles/iqtree.dir/phylonode.cpp.o CMakeFiles/iqtree.dir/phylosupertree.cpp.o CMakeFiles/iqtree.dir/phylotree.cpp.o CMakeFiles/iqtree.dir/phylotreesse.cpp.o CMakeFiles/iqtree.dir/pruning.cpp.o CMakeFiles/iqtree.dir/split.cpp.o CMakeFiles/iqtree.dir/splitgraph.cpp.o CMakeFiles/iqtree.dir/splitset.cpp.o CMakeFiles/iqtree.dir/stoprule.cpp.o CMakeFiles/iqtree.dir/superalignment.cpp.o CMakeFiles/iqtree.dir/superalignmentpairwise.cpp.o CMakeFiles/iqtree.dir/supernode.cpp.o CMakeFiles/iqtree.dir/tinatree.cpp.o CMakeFiles/iqtree.dir/tools.cpp.o CMakeFiles/iqtree.dir/whtest_wrapper.cpp.o CMakeFiles/iqtree.dir/lpwrapper.c.o CMakeFiles/iqtree.dir/pllnni.cpp.o CMakeFiles/iqtree.dir/phylosupertreeplen.cpp.o CMakeFiles/iqtree.dir/phylotesting.cpp.o CMakeFiles/iqtree.dir/ecopd.cpp.o CMakeFiles/iqtree.dir/ecopdmtreeset.cpp.o CMakeFiles/iqtree.dir/graph.cpp.o CMakeFiles/iqtree.dir/candidateset.cpp.o CMakeFiles/iqtree.dir/checkpoint.cpp.o CMakeFiles/iqtree.dir/upperbounds.cpp.o  -o iqtree-omp -rdynamic pllrepo/src/libpll.a pllrepo/src/libpllavx.a ncl/libncl.a whtest/libwhtest.a zlib-1./usr/bin/ld: cannot find -lgomp

collect2: error: ld returned 1 exit status
2.7/libz.a sprng/libsprng.a vectorclass/libvectorclass.a model/libmodel.a libavxkernel.a -lm

Do you think that my PC has some issues or could the command itself be wrong? I wondered, why /usr/bin/c++ is used and not gcc? Or is this part of the gcc compiler collection?

Thanks for you help.

Cheers,
Dominik

Last edited by fazky (2015-04-13 08:56:08)

mauritiusdadd · 2015-04-08 08:09:49

Hi fazky,

$ /usr/bin/c++    -fopenmp  -O3 -g0   -static CMakeFiles/iqtree.dir/alignment.cpp.o ...

First of all, It seems you are not using the openmpi compiler (mpic++) but c++ instead. In order to set the compiler, you have to run cmake with the following options:

cmake -DCMAKE_CXX_COMPILER:FILEPATH="/usr/bin/mpic++" -DCMAKE_C_COMPILER:FILEPATH="/usr/bin/mpicc" $sourcedir

where $sourcedir is the directory containing the file CMakeList.txt.
Secondly, your problem is caused by the use of the option '-static' because in Archlinux the use of static libraries is discouraged and almost all the official packages do not provide them (and for this reason ld is not able to find a static version of libgomp). You have to patch the file CMakeList.txt:

--- CMakeLists.txt	2015-04-08 09:46:08.789454325 +0200
+++ CMakeLists.txt	2015-04-08 09:47:37.910946974 +0200
@@ -59,7 +59,7 @@
 if (WIN32)
 	message("Target OS     : Windows")
 	# build as static binary to run on most machines
-	set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -static")
+	set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS}")
     SET(CMAKE_FIND_LIBRARY_SUFFIXES .lib .a ${CMAKE_FIND_LIBRARY_SUFFIXES})
     add_definitions(-DWIN32)
 elseif (APPLE) 
@@ -77,7 +77,7 @@
 	message("Target OS     : Unix")
 	# build as static binary to run on most machines
 #  if (CMAKE_BUILD_TYPE STREQUAL "Release") 	
-    set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -static")
+    set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS}")
 #  endif() 
 else()
 	# Note that IQ-TREE has NOT been tested on other platforms

edit: typo

Last edited by mauritiusdadd (2015-04-09 06:04:21)

arch9699 · 2015-04-11 12:55:20

In addition to the above suggestions, you can try the following

#include <stdio.h>
#include "mpi.h"
#include <omp.h>

/* mpiCC -fopenmp hello.c -o hello */


int main(int argc, char *argv[]) {
  int numprocs, rank, namelen;
  char processor_name[MPI_MAX_PROCESSOR_NAME];
  int iam = 0, np = 1;

  MPI_Init(&argc, &argv);
  MPI_Comm_size(MPI_COMM_WORLD, &numprocs);
  MPI_Comm_rank(MPI_COMM_WORLD, &rank);
  MPI_Get_processor_name(processor_name, &namelen);

  #pragma omp parallel default(shared) private(iam, np)
  {
    np = omp_get_num_threads();
    iam = omp_get_thread_num();
    printf("Hello from thread %d out of %d from process %d out of %d on %s\n",
           iam, np, rank, numprocs, processor_name);
  }

  MPI_Finalize();
}

Make sure that mpic++ is in the path

mpic++ -o hello hello.c -lgomp

or

mpic++ -fopenmp -o hello hello.c

Both of the above commands work.

mpirun -np 4 hello

Last edited by arch9699 (2015-04-11 13:00:00)

fazky · 2015-04-13 08:54:37

Hi mauritiusdadd and arch9699,

thanks for your suggestions. Both of them worked. I will forward this to the maintainer of IQ-Tree, maybe he wants to change the CMakeLists.txt file!

Dominik

edit:
I was also confused about OpenMPI and OpenMP. I did not know that these were different things. I contacted the maintainer of IQ-Tree and he is using OpenMP, not OpenMPI (cf. http://pawangh.blogspot.co.at/2014/05/m … enmp.html). He uses static libraries on purpose. However, applying the patch from above works flawlessly without any need for compiling static libraries.

Thanks!

Last edited by fazky (2015-04-13 11:05:02)

Arch Linux

#1 2015-04-07 13:39:04

[SOLVED] Compiling with OpenMPI fails

#2 2015-04-08 08:09:49

Re: [SOLVED] Compiling with OpenMPI fails

#3 2015-04-11 12:55:20

Re: [SOLVED] Compiling with OpenMPI fails

#4 2015-04-13 08:54:37

Re: [SOLVED] Compiling with OpenMPI fails

Board footer