distccd barely does anything when properly configured

synthead · 2010-11-22 13:38:21

I have a workstation with two cores and a server with four cores. The server is much better at crunching numbers, so I established a small distcc cluster between the two. However, when testing it out by compiling 'komposer' from the AUR on the workstation machine, the server barely ever peaks from distccd. Once in a blue moon I'll see one of the cores peg at 100%, but it's only for a split second and promptly goes back to idling. What am I doing wrong?

Here's the server's /etc/conf.d/distccd:

#
# Parameters to be passed to distccd
#
# You must explicitly add IPs (or subnets) that are allowed to connect,
# using the --allow switch.  See the distccd manpage for more info.
#
DISTCC_ARGS="--user nobody --allow 127.0.0.1 --allow 10.8.0.2 --allow 10.8.0.3"

And my workstation's /etc/makepkg.conf:

#
# /etc/makepkg.conf
#

#########################################################################
# SOURCE ACQUISITION
#########################################################################
#
#-- The download utilities that makepkg should use to acquire sources
#  Format: 'protocol::agent'
DLAGENTS=('ftp::/usr/bin/wget -c --passive-ftp -t 3 --waitretry=3 -O %o %u'
          'http::/usr/bin/wget -c -t 3 --waitretry=3 -O %o %u'
          'https::/usr/bin/wget -c -t 3 --waitretry=3 --no-check-certificate -O %o %u'
          'rsync::/usr/bin/rsync -z %u %o'
          'scp::/usr/bin/scp -C %u %o')

# Other common tools:
# /usr/bin/snarf
# /usr/bin/lftpget -c
# /usr/bin/curl

#########################################################################
# ARCHITECTURE, COMPILE FLAGS
#########################################################################
#
CARCH="x86_64"
CHOST="x86_64-unknown-linux-gnu"

#-- Exclusive: will only run on x86_64
# -march (or -mcpu) builds exclusively for an architecture
# -mtune optimizes for an architecture, but builds for whole processor family
CFLAGS="-march=x86-64 -mtune=generic -O2 -pipe"
CXXFLAGS="-march=x86-64 -mtune=generic -O2 -pipe"
LDFLAGS="-Wl,--hash-style=gnu -Wl,--as-needed"
#-- Make Flags: change this for DistCC/SMP systems
MAKEFLAGS="-j7"

#########################################################################
# BUILD ENVIRONMENT
#########################################################################
#
# Defaults: BUILDENV=(fakeroot !distcc color !ccache)
#  A negated environment option will do the opposite of the comments below.
#
#-- fakeroot: Allow building packages as a non-root user
#-- distcc:   Use the Distributed C/C++/ObjC compiler
#-- color:    Colorize output messages
#-- ccache:   Use ccache to cache compilation
#
BUILDENV=(fakeroot distcc color ccache)
#
#-- If using DistCC, your MAKEFLAGS will also need modification. In addition,
#-- specify a space-delimited list of hosts running in the DistCC cluster.
DISTCC_HOSTS="10.8.0.1"

#########################################################################
# GLOBAL PACKAGE OPTIONS
#   These are default values for the options=() settings
#########################################################################
#
# Default: OPTIONS=(strip docs libtool emptydirs zipman purge)
#  A negated option will do the opposite of the comments below.
#
#-- strip:     Strip symbols from binaries/libraries in STRIP_DIRS
#-- docs:      Save doc directories specified by DOC_DIRS
#-- libtool:   Leave libtool (.la) files in packages
#-- emptydirs: Leave empty directories in packages
#-- zipman:    Compress manual (man and info) pages in MAN_DIRS with gzip
#-- purge:     Remove files specified by PURGE_TARGETS
#
OPTIONS=(strip docs libtool emptydirs zipman purge)

#-- File integrity checks to use. Valid: md5, sha1, sha256, sha384, sha512
INTEGRITY_CHECK=(md5)
#-- Options to be used when stripping binaries. See `man strip' for details.
STRIP_BINARIES="--strip-all"
#-- Options to be used when stripping shared libraries. See `man strip' for details.
STRIP_SHARED="--strip-unneeded"
#-- Options to be used when stripping static libraries. See `man strip' for details.
STRIP_STATIC="--strip-debug"
#-- Manual (man and info) directories to compress (if zipman is specified)
MAN_DIRS=({usr{,/local}{,/share},opt/*}/{man,info})
#-- Doc directories to remove (if !docs is specified)
DOC_DIRS=(usr/{,local/}{,share/}{doc,gtk-doc} opt/*/{doc,gtk-doc})
#-- Directories to be searched for the strip option (if strip is specified)
STRIP_DIRS=(bin lib sbin usr/{bin,lib,sbin,local/{bin,lib,sbin}} opt/*/{bin,lib,sbin})
#-- Files to be removed from all packages (if purge is specified)
PURGE_TARGETS=(usr/{,share}/info/dir .packlist *.pod)

#########################################################################
# PACKAGE OUTPUT
#########################################################################
#
# Default: put built package and cached source in build directory
#
#-- Destination: specify a fixed directory where all packages will be placed
#PKGDEST=/home/packages
#-- Source cache: specify a fixed directory where source files will be cached
#SRCDEST=/home/sources
#-- Source packages: specify a fixed directory where all src packages will be placed
#SRCPKGDEST=/home/srcpackages
#-- Packager: name/email of the person or organization building packages
#PACKAGER="John Doe <john@doe.com>"

#########################################################################
# EXTENSION DEFAULTS
#########################################################################
#
# WARNING: Do NOT modify these variables unless you know what you are
#          doing.
#
#PKGEXT='.pkg.tar.xz'
PKGEXT='.pkg.tar.gz'
SRCEXT='.src.tar.gz'

# vim: set ft=sh ts=2 sw=2 et:

And yes, my /etc/hosts.allow permits distccd activity on the server side with "distccd: LOCAL" I'm doing this over a VPN with OpenVPN if it matters. I'm using UDP for OpenVPN and everything appears to be fine.

synthead · 2010-11-27 20:27:16

Any thoughts?

skunktrader · 2010-11-27 20:52:00

Assuming its this package http://aur.archlinux.org/packages.php?ID=13667 you are testing with, there is a post in the comments that the build does not seem to honor makeflags for some reason...

falconindy · 2010-11-27 21:00:58

Using testing? make 5.82 has some ugly regressions. better off using core/make instead.

synthead · 2011-03-18 17:47:12

Nah, just the regular packages from core and extra. I'm compiling firefox4 with the same results, too. I have 8 threads going where my machine is a dual core and the distcc server is a quad core.

nobody    2882     1  0  2683   272   3 Mar17 ?        00:00:00 /usr/bin/distccd --daemon --user nobody --allow 10.8.0.3
nobody    2883  2882  0  2683   696   3 Mar17 ?        00:00:00 /usr/bin/distccd --daemon --user nobody --allow 10.8.0.3
nobody    2904  2882  0  2683   696   3 Mar17 ?        00:00:00 /usr/bin/distccd --daemon --user nobody --allow 10.8.0.3
nobody    2913  2882  0  2683   692   3 Mar17 ?        00:00:00 /usr/bin/distccd --daemon --user nobody --allow 10.8.0.3
nobody    2915  2882  0  2683   696   3 Mar17 ?        00:00:00 /usr/bin/distccd --daemon --user nobody --allow 10.8.0.3
nobody    2931  2882  0  2683   692   1 Mar17 ?        00:00:00 /usr/bin/distccd --daemon --user nobody --allow 10.8.0.3
nobody    2942  2882  0  2683   700   3 Mar17 ?        00:00:00 /usr/bin/distccd --daemon --user nobody --allow 10.8.0.3

It's like it's not there. I can use nmap to verify that it's open and even telnet into the port, but it's not getting any jobs.

Arch Linux

#1 2010-11-22 13:38:21

distccd barely does anything when properly configured

#2 2010-11-27 20:27:16

Re: distccd barely does anything when properly configured

#3 2010-11-27 20:52:00

Re: distccd barely does anything when properly configured

#4 2010-11-27 21:00:58

Re: distccd barely does anything when properly configured

#5 2011-03-18 17:47:12

Re: distccd barely does anything when properly configured

Board footer