Compiling freezes PC

hugsie · 2018-10-30 16:48:28

Compiling packages freezes my pc to a point where i have to press the reset button.

Is there a way set the amount of cores for that purpose?

I edited the file /etc/makepkg.conf and set MAKEFLAGS="-j8" but this does not help.

I also edit the PKGBUILD files frequently but this is really getting on my nerves. I want a method to set it once for good.

V1del · 2018-10-30 17:16:31

Setting MAKEFFLAGS to that is the purpose of it, and by setting -j8 you made it use 8 cores/separate jobs. However a more relevant cause is that it leads to a high I/O contention and CFQ seems to be having some troubles with that, FWIW you could build in tmpfs if that is applicable/usable and/or try to set a different IO scheduler, I've made good experiences with BFQ: https://wiki.archlinux.org/index.php/Im … _scheduler

Also FWIW, what hardware are you trying this on? CPU/RAM/hard disk? maybe you are simply expecting too much.

Slithery · 2018-10-30 17:17:38

Where are you building the packages?

hugsie · 2018-11-06 13:13:02

Slithery wrote:

Where are you building the packages?

on a local m2 ssd

hugsie · 2018-11-06 13:16:28

V1del wrote:

Setting MAKEFFLAGS to that is the purpose of it, and by setting -j8 you made it use 8 cores/separate jobs. However a more relevant cause is that it leads to a high I/O contention and CFQ seems to be having some troubles with that, FWIW you could build in tmpfs if that is applicable/usable and/or try to set a different IO scheduler, I've made good experiences with BFQ: https://wiki.archlinux.org/index.php/Im … _scheduler
Also FWIW, what hardware are you trying this on? CPU/RAM/hard disk? maybe you are simply expecting too much.

thanks for your help and the link. but i dont quiet understand what i am supposed to do and why i have to change the IO scheduler? what user should pofit from that change?

my hardware is a threadripper 1950x with 32GB RAM and a M.2 SSD

Lone_Wolf · 2018-11-06 16:20:16

-j8 seems rather low for a 16 core / 32 threads processor, personally i'd set it much higher.

Is the M2 SSD the only drive in this system ?
Which software are you trying to build and what exact command ?

post your makepkg.conf please.

loqs · 2018-11-06 17:05:46

When you write "freezes my pc to a point where i have to press the reset button" this happened before you edited MAKEFLAGS which would mean it happened when using only one thread?
Does freeze mean you just can get any response from the system while it is building and when the job finishes the system responds as expected or does freeze mean the build never ends, display is frozen etc.

hugsie · 2018-11-08 06:52:22

The M2 SSD is not the only drive - i have another 3 SSDs in my system.
I was trying to build ungoogled-chromium and firefox and i had to edit both PKGBUILD files because i wasnt able to compile both of them.

i changed MAKEFLAGS to -j32 because my cpu should be made better use of but changed it back to -j8 when building chromium and firefox started to freeze my system.

In that state i am not even able to connect to ssh from another pc, I have to press the reset button.

here is my makepkg.conf

#!/hint/bash
#
# /etc/makepkg.conf
#

#########################################################################
# SOURCE ACQUISITION
#########################################################################
#
#-- The download utilities that makepkg should use to acquire sources
#  Format: 'protocol::agent'
DLAGENTS=('file::/usr/bin/curl -gqC - -o %o %u'
          'ftp::/usr/bin/curl -gqfC - --ftp-pasv --retry 3 --retry-delay 3 -o %o %u'
          'http::/usr/bin/curl -gqb "" -fLC - --retry 3 --retry-delay 3 -o %o %u'
          'https::/usr/bin/curl -gqb "" -fLC - --retry 3 --retry-delay 3 -o %o %u'
          'rsync::/usr/bin/rsync --no-motd -z %u %o'
          'scp::/usr/bin/scp -C %u %o')

# Other common tools:
# /usr/bin/snarf
# /usr/bin/lftpget -c
# /usr/bin/wget

#-- The package required by makepkg to download VCS sources
#  Format: 'protocol::package'
VCSCLIENTS=('bzr::bzr'
            'git::git'
            'hg::mercurial'
            'svn::subversion')

#########################################################################
# ARCHITECTURE, COMPILE FLAGS
#########################################################################
#
CARCH="x86_64"
CHOST="x86_64-pc-linux-gnu"

#-- Compiler and Linker Flags
CPPFLAGS="-D_FORTIFY_SOURCE=2"
CFLAGS="-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -fno-plt"
CXXFLAGS="-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -fno-plt"
LDFLAGS="-Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now"
#-- Make Flags: change this for DistCC/SMP systems
MAKEFLAGS="-j8"
#-- Debugging flags
DEBUG_CFLAGS="-g -fvar-tracking-assignments"
DEBUG_CXXFLAGS="-g -fvar-tracking-assignments"

#########################################################################
# BUILD ENVIRONMENT
#########################################################################
#
# Defaults: BUILDENV=(!distcc !color !ccache check !sign)
#  A negated environment option will do the opposite of the comments below.
#
#-- distcc:   Use the Distributed C/C++/ObjC compiler
#-- color:    Colorize output messages
#-- ccache:   Use ccache to cache compilation
#-- check:    Run the check() function if present in the PKGBUILD
#-- sign:     Generate PGP signature file
#
BUILDENV=(!distcc color !ccache check !sign)
#
#-- If using DistCC, your MAKEFLAGS will also need modification. In addition,
#-- specify a space-delimited list of hosts running in the DistCC cluster.
#DISTCC_HOSTS=""
#
#-- Specify a directory for package building.
#BUILDDIR=/tmp/makepkg

#########################################################################
# GLOBAL PACKAGE OPTIONS
#   These are default values for the options=() settings
#########################################################################
#
# Default: OPTIONS=(!strip docs libtool staticlibs emptydirs !zipman !purge !debug)
#  A negated option will do the opposite of the comments below.
#
#-- strip:      Strip symbols from binaries/libraries
#-- docs:       Save doc directories specified by DOC_DIRS
#-- libtool:    Leave libtool (.la) files in packages
#-- staticlibs: Leave static library (.a) files in packages
#-- emptydirs:  Leave empty directories in packages
#-- zipman:     Compress manual (man and info) pages in MAN_DIRS with gzip
#-- purge:      Remove files specified by PURGE_TARGETS
#-- debug:      Add debugging flags as specified in DEBUG_* variables
#
OPTIONS=(strip docs !libtool !staticlibs emptydirs zipman purge !debug)

#-- File integrity checks to use. Valid: md5, sha1, sha256, sha384, sha512
INTEGRITY_CHECK=(md5)
#-- Options to be used when stripping binaries. See `man strip' for details.
STRIP_BINARIES="--strip-all"
#-- Options to be used when stripping shared libraries. See `man strip' for details.
STRIP_SHARED="--strip-unneeded"
#-- Options to be used when stripping static libraries. See `man strip' for details.
STRIP_STATIC="--strip-debug"
#-- Manual (man and info) directories to compress (if zipman is specified)
MAN_DIRS=({usr{,/local}{,/share},opt/*}/{man,info})
#-- Doc directories to remove (if !docs is specified)
DOC_DIRS=(usr/{,local/}{,share/}{doc,gtk-doc} opt/*/{doc,gtk-doc})
#-- Files to be removed from all packages (if purge is specified)
PURGE_TARGETS=(usr/{,share}/info/dir .packlist *.pod)
#-- Directory to store source code in for debug packages
DBGSRCDIR="/usr/src/debug"

#########################################################################
# PACKAGE OUTPUT
#########################################################################
#
# Default: put built package and cached source in build directory
#
#-- Destination: specify a fixed directory where all packages will be placed
#PKGDEST=/home/packages
#-- Source cache: specify a fixed directory where source files will be cached
#SRCDEST=/home/sources
#-- Source packages: specify a fixed directory where all src packages will be placed
#SRCPKGDEST=/home/srcpackages
#-- Log files: specify a fixed directory where all log files will be placed
#LOGDEST=/home/makepkglogs
#-- Packager: name/email of the person or organization building packages
#PACKAGER="John Doe <john@doe.com>"
#-- Specify a key to use for package signing
#GPGKEY=""

#########################################################################
# COMPRESSION DEFAULTS
#########################################################################
#
COMPRESSGZ=(gzip -c -f -n)
COMPRESSBZ2=(bzip2 -c -f)
COMPRESSXZ=(xz -c -z -)
COMPRESSLRZ=(lrzip -q)
COMPRESSLZO=(lzop -q)
COMPRESSZ=(compress -c -f)

#########################################################################
# EXTENSION DEFAULTS
#########################################################################
#
# WARNING: Do NOT modify these variables unless you know what you are
#          doing.
#
PKGEXT='.pkg.tar.xz'
SRCEXT='.src.tar.gz'

mpan · 2018-11-08 07:17:35

Try ionice from the util-linux package on the whole makepkg process:

ionice -c 3 makepkg …other …options

That should give priority to all other stuff over makepkg and its children.

Also, if you’re using GNU version of make, consider using -l instead of simple -j. For a 32-core system: -l 32 -j 50. Note that if you use -l, -j still should be used — it then defines the jobs “reservoir” size. Failing to set -j for -l will usually cause the build to fail immedietely with resources exhaustion.

Last edited by mpan (2018-11-08 07:18:51)

Arch Linux

#1 2018-10-30 16:48:28

Compiling freezes PC

#2 2018-10-30 17:16:31

Re: Compiling freezes PC

#3 2018-10-30 17:17:38

Re: Compiling freezes PC

#4 2018-11-06 13:13:02

Re: Compiling freezes PC

#5 2018-11-06 13:16:28

Re: Compiling freezes PC

#6 2018-11-06 16:20:16

Re: Compiling freezes PC

#7 2018-11-06 17:05:46

Re: Compiling freezes PC

#8 2018-11-08 06:52:22

Re: Compiling freezes PC

#9 2018-11-08 07:17:35

Re: Compiling freezes PC

Board footer