[SOLVED] Simultaneously building multiple packages in a clean chroot

colinkeenan · 2016-07-27 17:32:14

In my previous question (https://bbs.archlinux.org/viewtopic.php?id=215170), I learned about building packages in a clean chroot (https://wiki.archlinux.org/index.php/De … ean_Chroot). It solved the problem I had, and now I build everything that way. However, I have noticed that if I open multiple terminal tabs and start a build in each one, instead of making multiple copies of a clean chroot and building everything simultaneously, it just locks the chroot and waits for one build to finish before starting another build.

Is it possible to configure it so that all the packages are built at the same time, each in a copy of the clean chroot?

Last edited by colinkeenan (2016-07-27 18:28:54)

Alad · 2016-07-27 17:54:53

makechrootpkg -h will tell you the option you need. That said, you could instead increase the -j setting in your MAKEFLAGS.

Last edited by Alad (2016-07-27 17:56:48)

alphaniner · 2016-07-27 18:01:33

$ makechrootpkg -h
...
-l <copy>  The directory to use as the working copy of the chroot
           Useful for maintaining multiple copies
           Default: alphaniner

I think specifying a different copy name for each package should do it. But keep in mind resource usage: if you've allowed make to use all your cores, building multiple things at a time won't necessarily be practical.

colinkeenan · 2016-07-27 18:03:50

Ok, thank you. I will try that option and see if it works before marking this solved.

Last edited by colinkeenan (2016-07-27 18:04:34)

colinkeenan · 2016-07-27 18:28:13

It works. I can open one tab and do

makechrootpkg -c -r $CHROOT -l colini

and another tab (for a different package) and do

makechrootpkg -c -r $CHROOT -l colinii

without making the colini and colinii sub-directories myself, and it builds each package simultaneously.

graysky · 2016-07-27 21:04:33

@op - glad you figured out a solution but I'm curious what use-case would demand a simultaneous build.

colinkeenan · 2016-07-27 21:11:16

I maintain a lot of packages on archlinuxcn, and some git versions need to be rebuilt every couple of days. Some of those take a really long time to build. So, I start those longer-build time ones first, then start all the others. I then go back through all the open tabs signing and uploading completed packages, marking those as done, then closing each tab as done. While doing all this, some longer packages may finish until there's only one I have to continue waiting for (usually inkscape-bzr).

graysky · 2016-07-27 21:19:30

Interesting. I would think that eventually, you'll have all cores on the machine tapped by build A and build B. Not sure how efficient it is to have it running like that but if it works for you and you don't get any MCEs while building (you do have mcelog installed on the host machine I assume) I guess both will eventually finish. Rock on

colinkeenan · 2016-07-27 21:29:38

Just installed mcelog.

Never heard of MCE before, but I see mcelog is a Community package, and the Arch Wiki is informative: https://wiki.archlinux.org/index.php/Ma … _exception

Last edited by colinkeenan (2016-07-27 21:30:09)

graysky · 2016-07-27 21:43:40

@colin - You're probably fine if not overlclocked and adequately cooled. I will say that GCC can be pretty good means of stress testing stability: https://wiki.archlinux.org/index.php/St … g_Programs

severach · 2016-07-31 06:38:37

Sounds to me like you're underpowered. E3-1245 are near give-away prices so I build them into everything now, expressly for the purpose of fast compiles. If that's not enough, you can build 16c/32t systems for cheap! I hope you're at least using ECC memory.

colinkeenan · 2016-07-31 06:51:37

GIGABYTE GA-Z77N-WIFI LGA 1155 Intel Z77 Motherboard
G.SKILL Ares Series 8GB (2 x 4GB) DDR3 1866
Samsung 840 Pro Series 128 gb SSD
Intel Core i5-3570K Quad-Core Processor 3.4 GHz
Nvidia GTX 750 Ti 2GB

Getting a little dated, but it's good enough for me.

Arch Linux

#1 2016-07-27 17:32:14

[SOLVED] Simultaneously building multiple packages in a clean chroot

#2 2016-07-27 17:54:53

Re: [SOLVED] Simultaneously building multiple packages in a clean chroot

#3 2016-07-27 18:01:33

Re: [SOLVED] Simultaneously building multiple packages in a clean chroot

#4 2016-07-27 18:03:50

Re: [SOLVED] Simultaneously building multiple packages in a clean chroot

#5 2016-07-27 18:28:13

Re: [SOLVED] Simultaneously building multiple packages in a clean chroot

#6 2016-07-27 21:04:33

Re: [SOLVED] Simultaneously building multiple packages in a clean chroot

#7 2016-07-27 21:11:16

Re: [SOLVED] Simultaneously building multiple packages in a clean chroot

#8 2016-07-27 21:19:30

Re: [SOLVED] Simultaneously building multiple packages in a clean chroot

#9 2016-07-27 21:29:38

Re: [SOLVED] Simultaneously building multiple packages in a clean chroot

#10 2016-07-27 21:43:40

Re: [SOLVED] Simultaneously building multiple packages in a clean chroot

#11 2016-07-31 06:38:37

Re: [SOLVED] Simultaneously building multiple packages in a clean chroot

#12 2016-07-31 06:51:37

Re: [SOLVED] Simultaneously building multiple packages in a clean chroot

Board footer