You are not logged in.

#1 2015-11-03 10:34:56

ADI64
Member
Registered: 2008-12-08
Posts: 13

ccache does not work with nvcc (CUDA)

Hi there,

I enabled ccache in /etc/makepkg.conf as described in the Ccache wiki entry and now I want to build a slightly modified version of the pcl AUR package.

However, compilation fails when compiling NVCC device objects:

[  5%] Building NVCC (Device) object gpu/utils/CMakeFiles/pcl_gpu_utils.dir/src/pcl_gpu_utils_generated_repacks.cu.o
/usr/bin/ccache: invalid option -- 'E'

"full" output here: http://pastebin.com/4w8dVKMU

It seems like nvcc calls gcc (which actually is ccache which in turn should call gcc) in a way that ccache does not know which compiler to invoke. I got it to work by creating simple gcc / g++ wrappers like so:

#!/bin/bash
ccache gcc "$@"

then turning of ccache in /etc/makepkg.conf and editing the package's CMake-Config vars to use these wrappers as c / cxx compiler. This works.

// Edit: Another option is just setting CUDA_NVCC_HOST_COMPILER=/usr/bin/g++ via CMake to bypass ccache.

However, I feel like this is not a proper solution. Whose fault is this? Are the provided ccache gcc / g++ wrappers faulty? Is nvcc calling gcc in a weird way?

Last edited by ADI64 (2015-11-03 10:43:21)

Offline

#2 2015-11-03 16:45:48

olive
Member
From: Belgium
Registered: 2008-06-22
Posts: 1,490

Re: ccache does not work with nvcc (CUDA)

What have you done exactly? If you are using makepkg; the only think to do is to enable ccache in the BUILDENV entry of /etc/makepkg.conf and it takes care of everything. It seems that somehow the compilation process call /usr/bin/ccache, which is bogus; it should cal gcc, which should be a symlink to ccache. ccache check how it has been called to find the real compiler. But again, normally makepkg takes care of all of this if you enable ccache in the BUILDENV entry of /etc/makepkg.conf

Offline

#3 2015-11-03 18:05:17

ADI64
Member
Registered: 2008-12-08
Posts: 13

Re: ccache does not work with nvcc (CUDA)

Thanks for your reply! Enabling ccache in /etc/makepkg.conf is exactly what I did and yes, it seems like nvcc is calling /usr/bin/ccache directly, but I don't know why. Apart from nvcc, the compilation process works like expected.
Are you sure that /usr/bin/gcc should be a symlink to ccache? What I understood was that it basically sets $PATH so that /usr/lib/ccache/bin gets checked first (which includes symlinks called gcc etc.), but you can still explicitly call /usr/bin/gcc without having ccache invoked.
I'm still wondering why nvcc seems to call ccache directly and not via the provided symlinks.

Offline

#4 2015-11-03 18:35:53

olive
Member
From: Belgium
Registered: 2008-06-22
Posts: 1,490

Re: ccache does not work with nvcc (CUDA)

I have not said that /usr/bin/gcc should be a symlink; the gcc that come first in your path should be. If you call /usr/bin/gcc directly, you indeed bypass ccache. Could you test with another package? Maybe nvcc follows the symlink and call ccache directly?

Last edited by olive (2015-11-03 18:36:07)

Offline

#5 2015-11-04 16:12:03

ADI64
Member
Registered: 2008-12-08
Posts: 13

Re: ccache does not work with nvcc (CUDA)

Oh, please excuse me, I misunderstood.
Other packages are building just fine with ccache, it's just that every nvcc-invocation through cmake seems to go wrong.
I modified the PKGBUILD to set CMAKE_VERBOSE_MAKEFILE=ON and saw this call to nvcc:

/opt/cuda/bin/nvcc -M -D__CUDACC__ [...]/gpu/utils/src/repacks.cu -o [...]/gpu/utils/CMakeFiles/pcl_gpu_utils.dir /src/pcl_gpu_utils_generated_repacks.cu.o.NVCC-depend -ccbin /usr/bin/ccache [...]

Especially the --ccbin /usr/bin/ccache is interesting as this should not happen. It seems like cmake indeed followed the symlink /usr/lib/ccache/bin/g++ to /usr/bin/ccache and decided to use that. So I guess this is a CMake problem?

Offline

Board footer

Powered by FluxBB