You are not logged in.

Pages: 1

Topic closed

#1 2011-06-23 00:39:17

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,697
Website

is anyone using distcc and does it work [solved]

I setup and configured distcc as per the wiki page but when compiling through makepkg, I am not getting any activity on the my "daemon" machine.  As well, I get "Blocked" in the output of distccmon-text when compiling.

...does this work for anyone?

My setup:
PC1 - the deamon (quad core)
PC2 - the server (dual core)

*Both machines have "distccd: 192.168.0." in their /etc/hosts.allow
*Both machines are running distccd

On PC1:
/etc/makepkg.conf

DLAGENTS=('ftp::/usr/bin/wget -c --passive-ftp -t 3 --waitretry=3 -O %o %u'
          'http::/usr/bin/wget -c -t 3 --waitretry=3 -O %o %u'
          'https::/usr/bin/wget -c -t 3 --waitretry=3 --no-check-certificate -O %o %u'
          'rsync::/usr/bin/rsync -z %u %o'
          'scp::/usr/bin/scp -C %u %o')

# Other common tools:
# /usr/bin/snarf
# /usr/bin/lftpget -c
# /usr/bin/curl

#########################################################################
# ARCHITECTURE, COMPILE FLAGS
#########################################################################
#
CARCH="x86_64"
CHOST="x86_64-unknown-linux-gnu"

#-- Exclusive: will only run on x86_64
# -march (or -mcpu) builds exclusively for an architecture
# -mtune optimizes for an architecture, but builds for whole processor family
CFLAGS="-march=native -O2 -pipe"
CXXFLAGS="${CFLAGS}"
LDFLAGS="-Wl,--hash-style=gnu -Wl,--as-needed"
#-- Make Flags: change this for DistCC/SMP systems
MAKEFLAGS="-j7"

#########################################################################
# BUILD ENVIRONMENT
#########################################################################
#
# Defaults: BUILDENV=(fakeroot !distcc color !ccache check)
#  A negated environment option will do the opposite of the comments below.
#
#-- fakeroot: Allow building packages as a non-root user
#-- distcc:   Use the Distributed C/C++/ObjC compiler
#-- color:    Colorize output messages
#-- ccache:   Use ccache to cache compilation
#-- check:    Run the check() function if present in the PKGBUILD
#
BUILDENV=(fakeroot distcc color !ccache !check)
#
#-- If using DistCC, your MAKEFLAGS will also need modification. In addition,
#-- specify a space-delimited list of hosts running in the DistCC cluster.
DISTCC_HOSTS="localhost 192.168.0.3"

#########################################################################
# GLOBAL PACKAGE OPTIONS
#   These are default values for the options=() settings
#########################################################################
#
# Default: OPTIONS=(strip docs libtool emptydirs zipman purge)
#  A negated option will do the opposite of the comments below.
#
#-- strip:     Strip symbols from binaries/libraries
#-- docs:      Save doc directories specified by DOC_DIRS
#-- libtool:   Leave libtool (.la) files in packages
#-- emptydirs: Leave empty directories in packages
#-- zipman:    Compress manual (man and info) pages in MAN_DIRS with gzip
#-- purge:     Remove files specified by PURGE_TARGETS
#
OPTIONS=(strip docs libtool emptydirs zipman purge)

#-- File integrity checks to use. Valid: md5, sha1, sha256, sha384, sha512
INTEGRITY_CHECK=(sha256)
#-- Options to be used when stripping binaries. See `man strip' for details.
STRIP_BINARIES="--strip-all"
#-- Options to be used when stripping shared libraries. See `man strip' for details.
STRIP_SHARED="--strip-unneeded"
#-- Options to be used when stripping static libraries. See `man strip' for details.
STRIP_STATIC="--strip-debug"
#-- Manual (man and info) directories to compress (if zipman is specified)
MAN_DIRS=({usr{,/local}{,/share},opt/*}/{man,info})
#-- Doc directories to remove (if !docs is specified)
DOC_DIRS=(usr/{,local/}{,share/}{doc,gtk-doc} opt/*/{doc,gtk-doc})
#-- Directories to be searched for the strip option (if strip is specified)
STRIP_DIRS=(bin lib sbin usr/{bin,lib,lib32,sbin,local/{bin,lib,sbin}} opt/*/{bin,lib,sbin})
#-- Files to be removed from all packages (if purge is specified)
PURGE_TARGETS=(usr/{,share}/info/dir .packlist *.pod)

#########################################################################
# PACKAGE OUTPUT
#########################################################################
#
# Default: put built package and cached source in build directory
#
#-- Destination: specify a fixed directory where all packages will be placed
#PKGDEST=/home/packages
#-- Source cache: specify a fixed directory where source files will be cached
#SRCDEST=/home/sources
#-- Source packages: specify a fixed directory where all src packages will be placed
#SRCPKGDEST=/home/srcpackages
#-- Packager: name/email of the person or organization building packages
#PACKAGER=""

#########################################################################
# EXTENSION DEFAULTS
#########################################################################
#
# WARNING: Do NOT modify these variables unless you know what you are
#          doing.
#
PKGEXT='.pkg.tar.xz'
SRCEXT='.src.tar.gz'

# vim: set ft=sh ts=2 sw=2 et:

On PC2:
/etc/conf.d/distccd

DISTCC_ARGS="--user nobody --allow 192.168.0.2"

Here is the output of distccmon-text at more or less any given point in time:

 31069  Blocked                                                            [0]
 30655  Blocked                                                            [0]
 30121  Blocked                                                            [0]
 31464  Compile                                                   localhost[0]
 31472  Compile                                                   localhost[1]
 31488  Compile                                                   localhost[2]
 31480  Compile                                                   localhost[3]

Last edited by graysky (2011-06-24 20:43:40)

CPU-optimized Linux-ck packages @ Repo-ck  • AUR packages • Zsh and other configs

Offline

#2 2011-06-23 20:05:00

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,697
Website

Re: is anyone using distcc and does it work [solved]

EDIT: Finally got it working and updated the wiki page.

Last edited by graysky (2011-06-24 20:43:26)

CPU-optimized Linux-ck packages @ Repo-ck  • AUR packages • Zsh and other configs

Offline

#3 2012-04-03 14:26:21

travnick
Member
From: Poland
Registered: 2011-04-24
Posts: 75

Re: is anyone using distcc and does it work [solved]

I'm interested in distcc and installed it on two machines:
1: core 2 duo
2: 64-bit pentium4 with HT

It look like there is some communication between this Two machines (distccmon-gnome) but I see no speed up. The second machine (p4) is doing nothing (htop).
My configuration:

/etc/makepkg.conf:

MAKEFLAGS="-j4"
BUILDENV=(fakeroot distcc color !cache check !sign)
DISTCC_HOSTS="10.42.0.44/2 localhost/2" #with or without localhost, I'm trying all combinations

/etc/conf.d/distccd:
DISTCC_ARGS=" --user nobody --allow 10.42.0.0/24" #--listen 10.42.0.1"

/etc/hosts.allow:
distccd: 10.42.0.0/24 : ALLOW

I tested with makepkg and "pump make -j4 CC=distcc"
I turned off iptables on both machines, but still the second is idle ... from time to time something appears in distccmon-gnome.
Any ideas?

Offline

#4 2012-04-03 20:04:18

vwyodajl
Member
Registered: 2012-01-21
Posts: 183

Re: is anyone using distcc and does it work [solved]

Please post back if you figure it out, I was trying a couple weeks ago to get a cross-compile distcc setup going and never had any luck no matter what I tried. Sometimes it would communicate but not compile the program properly then other times they would refuse each others connection's even with no iptables. I finally gave up since there was no big need, will probably try again in next few weeks, just so I can know how to do it.

Last edited by vwyodajl (2012-04-03 20:04:52)

Offline

#5 2015-01-26 18:55:56

SeBO
Member
From: Warsaw, Poland
Registered: 2008-03-06
Posts: 7

Re: is anyone using distcc and does it work [solved]

I have the same problem here - distcc works only for localhost ... i'm trying to compile on arm7h as master(with slave capabilities, distccd is running on this machine too) and amd64 as slave, but compilation is distributed only to 127.0.0.1 - i don't have this host in my makepkg nor in DISTCC_HOSTS env variable.

there is one more bug/problem: setting workers (slaves) throught /etc/makepkg.conf dosen't work (see below)

root@markone ~]# cat /etc/makepkg.conf
#
# /etc/makepkg.conf
#

#########################################################################
# SOURCE ACQUISITION
#########################################################################
#
#-- The download utilities that makepkg should use to acquire sources
#  Format: 'protocol::agent'
DLAGENTS=('ftp::/usr/bin/curl -fC - --ftp-pasv --retry 3 --retry-delay 3 -o %o %u'
          'http::/usr/bin/curl -fLC - --retry 3 --retry-delay 3 -o %o %u'
          'https::/usr/bin/curl -fLC - --retry 3 --retry-delay 3 -o %o %u'
          'rsync::/usr/bin/rsync --no-motd -z %u %o'
          'scp::/usr/bin/scp -C %u %o')

# Other common tools:
# /usr/bin/snarf
# /usr/bin/lftpget -c
# /usr/bin/wget

#-- The the package required by makepkg to download VCS sources
#  Format: 'protocol::package'
VCSCLIENTS=('bzr::bzr'
            'git::git'
            'hg::mercurial'
            'svn::subversion')

#########################################################################
# ARCHITECTURE, COMPILE FLAGS
#########################################################################
#
CARCH="armv7h"
CHOST="armv7l-unknown-linux-gnueabihf"

#-- Compiler and Linker Flags
# -march (or -mcpu) builds exclusively for an architecture
# -mtune optimizes for an architecture, but builds for whole processor family
CPPFLAGS="-D_FORTIFY_SOURCE=2"
CFLAGS="-march=armv7-a -mfloat-abi=hard -mfpu=vfpv3-d16 -O2 -pipe -fstack-protector --param=ssp-buffer-size=4"
CXXFLAGS="-march=armv7-a -mfloat-abi=hard -mfpu=vfpv3-d16 -O2 -pipe -fstack-protector --param=ssp-buffer-size=4"
LDFLAGS="-Wl,-O1,--sort-common,--as-needed,-z,relro"
#-- Make Flags: change this for DistCC/SMP systems
MAKEFLAGS="-j4"
#-- Debugging flags
DEBUG_CFLAGS="-g -fvar-tracking-assignments"
DEBUG_CXXFLAGS="-g -fvar-tracking-assignments"

#########################################################################
# BUILD ENVIRONMENT
#########################################################################
#
# Defaults: BUILDENV=(!distcc color !ccache check !sign)
#  A negated environment option will do the opposite of the comments below.
#
#-- fakeroot: Allow building packages as a non-root user
#-- distcc:   Use the Distributed C/C++/ObjC compiler
#-- color:    Colorize output messages
#-- ccache:   Use ccache to cache compilation
#-- check:    Run the check() function if present in the PKGBUILD
#-- sign:     Generate PGP signature file
#
BUILDENV=(distcc color fakeroot !ccache !check !sign)
#
#-- If using DistCC, your MAKEFLAGS will also need modification. In addition,
#-- specify a space-delimited list of hosts running in the DistCC cluster.
DISTCC_HOSTS="192.168.2.8/2 192.168.2.16/2"
#
#-- Specify a directory for package building.
#BUILDDIR=/tmp/makepkg

#########################################################################
# GLOBAL PACKAGE OPTIONS
#   These are default values for the options=() settings
#########################################################################
#
# Default: OPTIONS=(strip docs !libtool !staticlibs emptydirs zipman purge !upx !debug)
#  A negated option will do the opposite of the comments below.
#
#-- strip:      Strip symbols from binaries/libraries
#-- docs:       Save doc directories specified by DOC_DIRS
#-- libtool:    Leave libtool (.la) files in packages
#-- staticlibs: Leave static library (.a) files in packages
#-- emptydirs:  Leave empty directories in packages
#-- zipman:     Compress manual (man and info) pages in MAN_DIRS with gzip
#-- purge:      Remove files specified by PURGE_TARGETS
#-- upx:        Compress binary executable files using UPX
#-- debug:      Add debugging flags as specified in DEBUG_* variables
#
OPTIONS=(strip docs !libtool !staticlibs emptydirs zipman purge !upx !debug)

#-- File integrity checks to use. Valid: md5, sha1, sha256, sha384, sha512
INTEGRITY_CHECK=(md5)
#-- Options to be used when stripping binaries. See `man strip' for details.
STRIP_BINARIES="--strip-all"
#-- Options to be used when stripping shared libraries. See `man strip' for details.
STRIP_SHARED="--strip-unneeded"
#-- Options to be used when stripping static libraries. See `man strip' for details.
STRIP_STATIC="--strip-debug"
#-- Manual (man and info) directories to compress (if zipman is specified)
MAN_DIRS=({usr{,/local}{,/share},opt/*}/{man,info})
#-- Doc directories to remove (if !docs is specified)
DOC_DIRS=(usr/{,local/}{,share/}{doc,gtk-doc} opt/*/{doc,gtk-doc})
#-- Files to be removed from all packages (if purge is specified)
PURGE_TARGETS=(usr/{,share}/info/dir .packlist *.pod)

#########################################################################
# PACKAGE OUTPUT
#########################################################################
#
# Default: put built package and cached source in build directory
#
#-- Destination: specify a fixed directory where all packages will be placed
#PKGDEST=/home/packages
#-- Source cache: specify a fixed directory where source files will be cached
#SRCDEST=/home/sources
#-- Source packages: specify a fixed directory where all src packages will be placed
#SRCPKGDEST=/home/srcpackages
#-- Log files: specify a fixed directory where all log files will be placed
#LOGDEST=/home/makepkglogs
#-- Packager: name/email of the person or organization building packages
#PACKAGER="John Doe <john@doe.com>"
#-- Specify a key to use for package signing
#GPGKEY=""

#########################################################################
# COMPRESSION DEFAULTS
#########################################################################
#
COMPRESSGZ=(gzip -c -f -n)
COMPRESSBZ2=(bzip2 -c -f)
COMPRESSXZ=(xz -c -z -)
COMPRESSLRZ=(lrzip -q)
COMPRESSLZO=(lzop -q)
COMPRESSZ=(compress -c -f)


#########################################################################
# EXTENSION DEFAULTS
#########################################################################
#
# WARNING: Do NOT modify these variables unless you know what you are
#          doing.
#
PKGEXT='.pkg.tar.xz'
SRCEXT='.src.tar.gz'

# vim: set ft=sh ts=2 sw=2 et:

only explcitice setinng env varaible export DISTCC_HOSTS="192.168.2.16 192.168.2.8"
I've also checked ports that are open for distccd and they are open as ... smile but I can not connect to them sad

Offline

#6 2015-01-26 20:30:21

jasonwryan
Anarchist
From: .nz
Registered: 2009-05-09
Posts: 30,426
Website

Re: is anyone using distcc and does it work [solved]

Please don't necrobump: https://wiki.archlinux.org/index.php/Fo … Bumping.27

..and Arch ARM is a separate distro: https://bbs.archlinux.org/viewtopic.php?id=153431



Closing

Arch + dwm   •   Mercurial repos  •   Surfraw

Registered Linux User #482438

Offline

Pages: 1

Topic closed

Board footer

Powered by FluxBB