You are not logged in.
I like to build my aur packages semi-regularly in a clean chroot to verify dependencies , build issues and such .
Lately this gives me problems due to lack of space . My / partition uses ext4, has 30+ Gib space free before a chroot is created, but that amount decreases fast while building with pkgctl / extra-x86_64-build / multilib-build & friends.
atm I'm trying to build my llvm-minimal-git and free space on root is now less than 1 Gib according to df -h .
My ~/.bashrc has export TMPDIR=/data/panoramix/tmp in it, but the drive that is on sees no change in free space during build .
Ah, the build just failed with 'no space left on device' .
df -h (as user) indicates there is 2.7M available.
removing /var/lib/archbuild/extra-x86_64 shows 34 Gib available .
Have others encountered such issues and could this be a bug in devtools ?
how can I redirect /var/lib/archbuild to another location ?
Last edited by Lone_Wolf (2024-03-20 14:12:28)
Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.
clean chroot building not flexible enough ?
Try clean chroot manager by graysky
Online
Try a bind-moun onto an existing filesystem, or just a direct mount of another filesystem?
Preferably a btrfs filesystem, which allows devtools to use copy-on-write snapshots which consums a lot less disk space.
Last edited by swsnr (2024-02-26 16:41:59)
Offline
I didn't need a bind mount, the -c <dir> option of extra-x86_64-build (& makechrootpkg) allowed to put the chroot on a different place with more free space .
Unfortunately this didn't solve the problem as the oom killer aborted the build.
Meanwhile running makepkg on the live system with the same PKGBUILD doesn't show this huge decrease in free space or the abortion .
Since my /etc/makepkg.conf limits the number of jobs to 75% of available cores (MAKEFLAGS="-j 18") I'm now testing extra-x86_64-build with lower job numbers hardcoded in the PKGBUILD. (currently testing with MAKEFLAGS="-j 12" )
the free space decrease is still present, and the build aborts around the same point where it did with no limit to jobs.
I'm certain I didn't have this issue early in january, so I'm going to downgrade devtools to the previous version , 1.1.04-1 from sept 2023 to test.
ADDED
oom killer interferes also with devtools 1.1.04-1 , so the cause must lie somewhere else.
Last edited by Lone_Wolf (2024-02-27 16:39:19)
Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.
clean chroot building not flexible enough ?
Try clean chroot manager by graysky
Online
I have made progress.
Using my own (customized) /etc/pacman.conf & /etc/makepkg.conf with a classic chroot everything works fine just like makepkg -Crs as user.
Combining the same files with aur clrean-chroot-manager from graysky also works fine .
pkgctl build aborted when /var/lib/archbuild needed diskspace increased to above 25 Gib and doesn't appear to have an option to build in a different location (with more space) .
Going to test extra-x86_64-build (which does allow that) next.
Last edited by Lone_Wolf (2024-03-09 14:55:06)
Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.
clean chroot building not flexible enough ?
Try clean chroot manager by graysky
Online
extra-x86_64-build -c -r /data/panoramix/chroot/ just finished .
time needed was 110 minutes, typical time for this package (with ccache disabled) is around 60 minutes.
some numbers :
# du -s /data/panoramix/chroot/root
964548 /data/panoramix/chroot/root
# du -s /data/panoramix/chroot/panoramix
9659688 /data/panoramix/chroot/panoramix
# du -s /data/panoramix/chroot/extra-x86_64/
88931916 /data/panoramix/chroot/extra-x86_64/
#
The first 2 are the sizes of the folders used by my classic chroot, the almost 10 times as big one is for the extra-x86_64 build !
During investigating I increased my swapspace from 8 to 24 GiB .
Some fun results of free -m during the extra-x86_64-build :
$ free -m
total used free shared buff/cache available
Mem: 15845 13832 1108 12 1339 2013
Swap: 24575 917 23658
$ free -m
total used free shared buff/cache available
Mem: 15845 11551 3266 12 1461 4293
Swap: 24575 917 23658
$ free -m
total used free shared buff/cache available
Mem: 15845 15595 209 5 423 250
Swap: 24575 12375 12200
$ free -m
total used free shared buff/cache available
Mem: 15845 15389 409 10 439 455
Swap: 24575 24266 309
$ free -m
total used free shared buff/cache available
Mem: 15845 3321 8211 16 4662 12523
Swap: 24575 981 23594
$ free -m
total used free shared buff/cache available
Mem: 15845 15570 173 4 544 275
Swap: 24575 6021 18554
$ free -m
total used free shared buff/cache available
Mem: 15845 2020 5641 16 8533 13824
Swap: 24575 527 24048
$
Time to do other things, will try using my custom /etc/makepkg.conf with extra-x86_64-build in the coming days.
Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.
clean chroot building not flexible enough ?
Try clean chroot manager by graysky
Online
I found the cause of the memory & space issues during build and changed the title of the thread.
The custom makepg.conf-file I use has !lto in OPTIONS.
The PKGBUILD has options=('!lto!) which according to man PKGBUILD is correct to disable lto for this package.
Unfortunately this doesn't work and the pacakge is build with lto which drastically increases free space & memory used .
Using OPTIONS=('!lto') in the PKGBUILD does prevent lto building as desired.
Man makepkg.conf uses all caps for env vars and man PKGBUILD lists lowercase env vars only, this causes confusion about the correct usage.
Should I file a bug report ?
And if so, should it be created against the archlinux pacman package or pacman upstream ?
Please advise.
Last edited by Lone_Wolf (2024-03-19 15:03:04)
Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.
clean chroot building not flexible enough ?
Try clean chroot manager by graysky
Online
Mod note: moving to Pacman issues.
Sakura:-
Mobo: MSI MAG X570S TORPEDO MAX // Processor: AMD Ryzen 9 5950X @4.9GHz // GFX: AMD Radeon RX 5700 XT // RAM: 32GB (4x 8GB) Corsair DDR4 (@ 3000MHz) // Storage: 1x 3TB HDD, 6x 1TB SSD, 2x 120GB SSD, 1x 275GB M2 SSD
Making lemonade from lemons since 2015.
Online
I found the cause of the memory & space issues during build and changed the title of the thread.
The custom makepg.conf-file I use has !lto in OPTIONS.
The PKGBUILD has options=('!lto!) which according to man PKGBUILD is correct to disable lto for this package.Unfortunately this doesn't work and the pacakge is build with lto which drastically increases free space & memory used .
Using OPTIONS=('!lto') in the PKGBUILD does prevent lto building as desired.
Man makepkg.conf uses all caps for env vars and man PKGBUILD lists lowercase env vars only, this causes confusion about the correct usage.
Should I file a bug report ?
And if so, should it be created against the archlinux pacman package or pacman upstream ?Please advise.
This is the wrong assessment. There are many packages using options=('!lto') in the repos, and that disables LTO.
What you are doing with OPTIONS=('!lto') in the PKGBUILD is overriding the OPTIONS= line in makepkg.conf. My guess is that stops a debug package being built - for some reason Arch enabled that by default. Try adding "!debug" to the options in the PKGBUILD.
Offline
Further testing confirmed it was not lto but debug that caused the issue.
Marking as Solved.
My custom makepkg.conf uses
BUILDENV=(!distcc color ccache check !sign)
OPTIONS=(strip docs !libtool !staticlibs emptydirs zipman purge !debug !lto)
I'm going to check all my aur packages to ensure they explicitly enable/disable lto & debug to avoid this issue in the future.
Thanks for the input, Allan .
Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.
clean chroot building not flexible enough ?
Try clean chroot manager by graysky
Online