cmake ignores max job count, makes system unresponsive

germoney · Today 05:19:53

Trying to look for answers here on the Arch forums, since this affects me when building packages for projects which use cmake... I also want to note that I don't have much experience (or any at all TBH) with anything C/C++ related, so please bear with me.

I've been maintaining my own kwin and plasma-workspace PKGBUILDs for a long time now, since I'm applying a couple of patches for specific stuff which I don't agree with upstream. This is irrelevant to the problem though.

The problem is that when building kwin or plasma-workspace, my system always becomes super unresponsive and basically unusable. Since I don't have any problems when for example building the Linux kernel (also using my own PKGBUILD), I guess that the problem comes from cmake-based projects, like kwin and plasma-workspace.

My system's CPU is a Ryzen 3950X (16C32T), hence why I have set `MAKEFLAGS=-j32` in my `makepkg.conf`. As described, this works as expected when building the kernel for example, without any responsiveness issues while building. Process scheduling works perfectly fine and the system stays usable despite 100% CPU utilization.

However, building those cmake-based projects seems to ignore the number of parallel build-jobs. Setting the `-j32` parameter on the `cmake --build` command itself (or setting the `CMAKE_BUILD_PARALLEL_LEVEL=32` env var) also doesn't help. And surprisingly, reducing it to something lower like for example 8 also doesn't help (including `MAKEFLAGS=-j8` set at the same time). It always results in 100% CPU utilization, so it looks like (way) more than 32 jobs are run in parallel, choking the CPU scheduler and making the system totally unresponsive.

Setting `-j1` on `cmake --build` does not do that, but that's obviously not the solution. According to the docs/manual: "Some native build tools always build in parallel. The use of <jobs> value of 1 can be used to limit to a single job."
https://cmake.org/cmake/help/latest/man … ke-build-j

So what's the solution here? I want to build those packages as fast as possible with my system staying responsive while it's building, which means I want to limit the max number of parallel build jobs to the number of my CPU's max thread count, which works fine when building make-based projects and other non-cmake-based stuff.

I'm aware that cmake is a wrapper/generator for make (and other build tools), but why does it ignore the `MAKEFLAGS=-j32` then? Is it because it invokes multiple make commands under the hood in parallel, causing this wild "process expansion" without each individual make process being aware of this? Or is this dependent on the projects' CMakeLists.txt files themselves? As said in the beginning, I don't have much experience here.

Can the number of spawned build processes be limited in a different way without causing any failures? I've tried `systemd-run --user --pty --same-dir --wait -p TasksAccounting=1 -p TasksMax=32 makepkg ...`, but this results in "sh: fork: retry: Resource temporarily unavailable".

Thanks!

seth · Today 08:43:17

So what's the solution here? I want to build those packages as fast as possible with my system staying responsive while it's building

nice/ionice (latter depends on disk scheduler support)

Just to be clear, you're suggesting MAKEFLAGS, CMAKE_BUILD_PARALLEL_LEVEL and -j1 are *all* ignored?
Or does

Setting `-j1` on `cmake --build` does not do that, but that's obviously not the solution.

mean that constraining the build to a single job (*only*) prevents the overload? Did you try -j8 *there* as well?
Does that only affect

The problem is that when building kwin or plasma-workspace, my system always becomes super unresponsive and basically unusable. Since I don't have any problems when for example building the Linux kernel (also using my own PKGBUILD), I guess that the problem comes from cmake-based projects, like kwin and plasma-workspace.

?Did you try any other cmake based AUR build?

Process scheduling works perfectly fine and the system stays usable despite 100% CPU utilization.
…
It always results in 100% CPU utilization, so it looks like (way) more than 32 jobs are run in parallel, choking the CPU

Sanity check

It always results in 100% CPU utilization

for one core or all combined?

Lone_Wolf · Today 10:31:06

Does the cmake invocation include -G Ninja ?

germoney · Today 10:55:09

seth wrote:

Just to be clear, you're suggesting MAKEFLAGS, CMAKE_BUILD_PARALLEL_LEVEL and -j1 are *all* ignored?

I'm saying that despite `MAKEFLAGS=-j8` and `-j8` being set on the `cmake --build` command, more than 8 build jobs (way more than 32 build jobs actually) are run in parallel, taking 100% on all of my CPU cores and choking the system.
If `-j1` is set on the `cmake --build` command, only one build job is run. Hence why I think that cmake is running multiple make commands in parallel, which themselves are running multiple build jobs.

seth wrote:

Did you try any other cmake based AUR build?

When building `strawberry` (or `clementine-git`) for example, the problem does not occur. Full utilization of all CPU cores, but the system stays responsive. Smaller projects also don't cause any issues.

So this looks like it's related to `kwin` and `plasma-workspace`. Btw, same problem when using the PKGBUILDs from the extra repo, so this is not caused by my modified PKGBUILDs.

Lone_Wolf wrote:

Does the cmake invocation include -G Ninja ?

No, it does not on the `kwin` and `plasma-workspace` PKGBUILDs. The ninja build tool does not cause these issues. I've built `qt6-base` in order to check, which uses Ninja via cmake. Full CPU usage on all cores, no system responsiveness issues.

Arch Linux

#1 Today 05:19:53

cmake ignores max job count, makes system unresponsive

#2 Today 08:43:17

Re: cmake ignores max job count, makes system unresponsive

#3 Today 10:31:06

Re: cmake ignores max job count, makes system unresponsive

#4 Today 10:55:09

Re: cmake ignores max job count, makes system unresponsive

Board footer