[Solved] Compiling the kernel is rather slow and looks unoptimized

germoney · 2020-02-10 15:32:33

Solved: https://bbs.archlinux.org/viewtopic.php … 7#p1887357

----

This is probably not the right place to post this, but let me ask anyway. Also, since I'm not a C-developer, my knowledge regarding build tools is rather limited, so please excuse my lack of experience.

I've been building the Linux kernel on my own for a while now and especially after finally having upgraded to a Ryzen 3950X (16C/32T) two months ago, I have noticed how unoptimized the kernel's build procedure is. What I mean by that is that most of the time it is compiling its list/tree of modules with only one thread while the remaining cores of my CPU are just idling. In some cases the system's load average drops down to a 1.5 during the kernel compilation, which is a bit surprising. I'm not sure what the reason for this is, as I've set the `MAKEFLAGS` env var to `-j32` and even tried setting it explicitly in the `make` call. I would expect that if there are modules which are for whatever reason being built sequentially with just one job that other modules which are independent of those modules could be built in parallel, but this is not the case. Instead, the modules which get built slowly block the entire thing until they have finished compiling. Then it builds a few more modules in parallel, or at least tries to, as it is not utilizing every CPU core either, until the next blocking module gets built with just one single thread. This leads to an unnecessary long build time considering the available CPU cores.

When I'm compiling other projects/applications like for example ungoogled-chromium or clementine, the entire CPU gets utilized and is maxed out at 100% almost the entire time. Since these projects are not using Make as build tool, I suppose that Make is the issue here, or at least the kernel's makefiles.

The kernel I'm building is the mainline kernel with a few additional patches and I'm using the kernel config and a slightly customized PKGBUILD of Arch's default kernel for building/packaging it because I don't have any experience with kernel configs and don't want to go down the endless rabbithole of learning the various flags. This means that I'm building lots of modules which I don't actually need, but I don't really mind other than the modules' unnecessary "long" build times when compiling the kernel. I know that it's possible to automate filtering out unneeded modules, but I don't want to do this as it has its downsides, too. Also, I'm not asking for improving recurring builds with ccache, etc, as this is not the issue here.

So what is the reason that building the kernel is unnecessary slow on my system? Did I miss something? Or is this expected behavior?
I'd appreciate any kind of help or further directions. Thank you!

Last edited by germoney (2020-02-12 15:47:13)

sabroad · 2020-02-10 15:56:01

germoney wrote:

compiling its list/tree of modules with only one thread while the remaining cores of my CPU are just idling.
[...] I'm using the kernel config and a slightly customized PKGBUILD

For makepkg parallel compilation have you set MAKEFLAGS in makepkg.conf ?

MAKEFLAGS="-j$(nproc)"

Last edited by sabroad (2020-02-10 16:00:06)

germoney · 2020-02-10 16:12:06

> as I've set the `MAKEFLAGS` env var to `-j32` and even tried setting it explicitly in the `make` call

yes, I've set it basically everywhere (with a static "-j32" value), just to be extra sure: in the makepkg.conf, while executing makepkg and also in the PKGBUILD file itself where it executes make, even though it's unnecessary

Slithery · 2020-02-12 15:11:15

https://www.phoronix.com/scan.php?page= … el-Job-Opt

germoney · 2020-02-12 15:45:24

I read this phoronix article, which was part of the motivation for making this forum post, but it looks like I misunderstood the changes. I didn't realize that I had to be running 5.6 in order to get the improvements when building something with make and I thought that it would also apply to kernel 5.6 when compiling on an older kernel.

So I just did a recompile of 5.5-rc1 while using the 5.6-rc1 kernel and it properly used all of my CPU's cores while system load average was at around 32.5, and the compile time dropped from 35 mins to ~5 mins. Looks like the problem is solved.

Arch Linux

#1 2020-02-10 15:32:33

[Solved] Compiling the kernel is rather slow and looks unoptimized

#2 2020-02-10 15:56:01

Re: [Solved] Compiling the kernel is rather slow and looks unoptimized

#3 2020-02-10 16:12:06

Re: [Solved] Compiling the kernel is rather slow and looks unoptimized

#4 2020-02-12 15:11:15

Re: [Solved] Compiling the kernel is rather slow and looks unoptimized

#5 2020-02-12 15:45:24

Re: [Solved] Compiling the kernel is rather slow and looks unoptimized

Board footer