Correct usage of 'make localmodconfig'

IrvineHimself · 2016-10-26 03:52:28

Since starting with Arch a few months ago, I have been compiling custom kernels based on official builds using 'localmodconfig'. Apart from the learning opportunities, my main goal in doing this was to automate the enabling of Apparmor. Currently, I keep a clone of the zen-kernel git repository and ensure compatibility with Arch by compiling it from the official Arch-zen build. Things progress however and I am now ready to move onto the next step in the learning process, which is applying patches to compile other kernel flavours. While researching this topic, I realised that my basic understanding of 'localmodconfig' is incomplete.

My basic problem is that some of these patches, for example Liquorix, have an associated '.configure' file, and I strongly suspect that this default configuration is less than optimum. I know that last night when, (as a first step,) I installed the AUR 'linux-lqs' package, I was very surprised at how long it took to compile.

This brings into focus some basic questions I have about the usage of 'localmodconfig'.

These are:

1) Would it make any difference to compile a custom Zen, (or Liquorix,) kernel from the official Arch kernel using 'localmodconfig'?
2) In the long term, how serious would the divergence be if, starting with an official Zen kernel, I successively configure each new release from the preceding custom version. (At the moment, this is something I take pains to avoid.)

Your thoughts are greatly appreciated

Irvine

basica · 2016-10-26 04:02:33

localmodconfig basically checks to see what modules you have running and creates a config that only builds those modules. It doesn't change the module settings to be inbuilt or not, it just turns off ones you don't have running at the time (even if you may otherwise use them but don't have the device inserted at the time for example). localyesconfig does the same, but makes them all inbuilt if possible which can get rid of the need for an initramfs.

I don't know what the Liquorix kernel does, but part of it is choosing settings aside from kernel module selections. Since the config is catered to a wide audience it has to enable a lot of modules, just like the Arch kernel and as such will take a lot longer than a localmodconfig config file. If you wanted to improve the compile time, you would have to run a localmod config on the Liquorix config file (I don't recommend this since I am unfamiliar with the Liquorix kernel and don't know if there are potential issues in doing this if they've patched the kernel to do certain things etc).

IrvineHimself · 2016-10-26 05:38:37

Thanks for your reply.

basica wrote:

....you would have to run a localmod config on the Liquorix config file...

By that you mean using 'localmodconfig' while running in the Liquorix kernel?

My main concern, is the following warning on the Kernels/Arch_Build_System

Warning: systemd has a number of kernel configuration requirements for general use, for specific usecases (e.g., UEFI) and for specific systemd functionality (e.g., bootchart). Failure to meet these requirements can result in your system being degraded or unusable. The list of required and recommended kernel CONFIGs can be found in /usr/share/doc/systemd/README. Check them before you compile.These requirements also change over time. Because Arch assumes you are using the official kernel, there will be no announcement of these changes. Before you install a new version of systemd, check the version release notes to make sure your current custom kernel meets any new systemd requirements.

The Wiki/Kernel/Traditional_Build certainly gives the impression that 'localmodconfig' is a sparse equivalent to importing the official Arch configuration. Having said that, I am by no means an expert and fully expect to be corrected on this.

Compiling with 'localmodconfig' from an official build appears to meet the all the requirements. For example in my custom Zen kernel, the only points of note in the dmesg output are

iwlwifi .... Unsupported splx structure
platform MSFT0101:00: failed to claim resource 1
acpi MSFT0101:00: platform device creation failed: -16

I also have these errors with the official builds of Arch and Zen, and have researched them extensively: The iwlwifi thingy is self corrected by loading another module, and the MSFT0101 problem is undergoing active development. ,

On the other hand, the AUR Liquorix kernel has a host of issues, not the least being a loop where it continuously initialises one of the ACPI modules?

Irvine

eschwartz · 2016-10-26 05:51:30

If you are going to use localmodconfig, I would suggest also using modprobed-db to keep track of used kernel modules over time.

IrvineHimself · 2016-10-26 06:06:59

Eschwartz wrote:

If you are going to use localmodconfig, I would suggest also using modprobed-db to keep track of used kernel modules over time.

Thanks, good tip. It should be in the Traditional Build Wiki page which describes the localmodconfig method.

I only recently came across modprobe.db while researching this topic. I like to keep my installs to the bare minimum, but, in this case, I made an exception and immediately installed it.

Irvine

basica · 2016-10-26 06:23:37

IrvineHimself wrote:

Thanks for your reply.
By that you mean using 'localmodconfig' while running in the Liquorix kernel?
My main concern, is the following warning on the Kernels/Arch_Build_System
The Wiki/Kernel/Traditional_Build certainly gives the impression that 'localmodconfig' is a sparse equivalent to importing the official Arch configuration. Having said that, I am by no means an expert and fully expect to be corrected on this.
Compiling with 'localmodconfig' from an official build appears to meet the all the requirements. For example in my custom Zen kernel, the only points of note in the dmesg output are
I also have these errors with the official builds of Arch and Zen, and have researched them extensively: The iwlwifi thingy is self corrected by loading another module, and the MSFT0101 problem is undergoing active development. ,
On the other hand, the AUR Liquorix kernel has a host of issues, not the least being a loop where it continuously initialises one of the ACPI modules?
Irvine

The localmodconfig command strips unloaded modules from the config file you provide it and that's it. If you don't provide it with a config file, it uses the default one. So if you're using the ABS to check out the Arch kernel package, you'll be stripping the Arch config file of these unloaded modules, but keeping enabled whatever options that have been configured (such as the name for example). Thus when you compile it, you're compiling far fewer modules than before and still having everything you need to load the system. What kernel you're currently running isn't too much of an issue so long as any modules you have running are all available in the kernel sources you plan on building (e.g. you're not trying to run localmodconfig while running a zfs patched kernel).

If you're using this Liquorix kernel, like the Arch kernel it will have a lot of modules configured to be built in the config file in order for it to work for a wide variety of people. A lot of these modules you won't need and to strip them you'd need to do it through something like localmodconfig to strip them (or otherwise manually remove them). As pointed out in the wiki, it may not be configured to work with Arch out of the box, but if it's in the AUR my guess that it will be. Since I am not familiar with the project I can't recommend doing this as I don't know what effect (if any) it will have on any patches they may have applied. If you're using their kernel straight from their site, then you will probably need to run make nconfig to enable any options that are needed for it to boot under Arch.

Also, I forgot to mention it myself, but I also recommend the package Eschwartz suggested for the reasons I outlined before.

graysky · 2016-10-26 08:50:43

As I noted in on the wiki and in the modprobed-db man page, you want to really use your system for a few days while populating the database of modules before you build a kernel. The risk is that you don't build in everything you need and try to troubleshoot why a new piece of hardware you bought is just not working... well, it could be because you never built in support!

IrvineHimself · 2016-10-26 09:40:30

basica wrote:

.....strips unloaded modules from the config file you provide it and that's it. If you don't provide it with a config file, it uses the default one ......

I think this more or less answers my basic question, Rather than running, (or after running,) 'make clean && make mrproper', I should just paste into the build directory the default configuration before running 'make localmodconfig'?

basica wrote:

..... What kernel you're currently running isn't too much of an issue so long as any modules you have running are all available in the kernel sources you plan on building ......

This should be the case, but is worth taking note of.

basica wrote:

..... If you're using their kernel straight from their site, then you will probably need to run make nconfig to enable any options that are needed for it to boot under Arch......

This is exactly what I intend. The whole point of the exercise is to learn about patching kernels. I have absolutely no interest in Liquorix other than as a useful test-bed for developing my non-existent kernel-hacking skills.

graysky wrote:

.... you want to really use your system for a few days while populating the database of modules before you build a kernel....

I did take note of this and even used your suggestion to set up modprobed-db as a service.

Thanks for all your advice

Irvine

edited for bad grammar

Last edited by IrvineHimself (2016-10-26 09:42:21)

basica · 2016-10-26 10:03:37

IrvineHimself wrote:

I think this more or less answers my basic question, Rather than running, (or after running,) 'make clean && make mrproper', I should just paste into the build directory the default configuration before running 'make localmodconfig'?

If you plan on using a patched kernel like Liquorix, then copying and pasting another config file into the sources probably isn't a good idea as they might have their config file set up in a certain way to take advantage of their patches and using another config file potentially negates using the patched kernel. This is the same reason I'd be hesitant to run a localmodconfig on their config file until I have a good idea about what they're doing. Chances are it probably wouldn't even be an issue, but I am an overly paranoid type..

IrvineHimself wrote:

This is exactly what I intend. The whole point of the exercise is to learn about patching kernels. I have absolutely no interest in Liquorix other than as a useful test-bed for developing my non-existent kernel-hacking skills.

The Linux Kernel in a Nutshell, while old, is still a pretty decent book to read by one of the kernel developers. You can grab the full PDF here (Note: It's released under the creative commons, so it's a legal copy). It covers the kernel build process with some interesting insights into things, you may find it useful as you're exploring this subject.

Depending on exactly where you're looking to go with this, I'd suggest really just start off with the vanilla kernel, doing a make defconfig, building it, installing it and then troubleshooting it till you can get it to boot under Arch. You don't really need to get involved with the Liquorix kernel since in of itself it doesn't require anything of you, it's prepatched, has a config file and it's ready to compile. It's no different than using a vanilla kernel for your purposes really.

IrvineHimself · 2016-10-26 14:03:11

Thanks for the advice, I just downloaded the book and will definitely be studying it.

I did start my experiments with the vanilla kernel. Then, since I am mostly interested in Blender modelling and animations, I moved to the Zen kernel, (with Apparmor enabled,) for everyday use.

Perhaps I misstated myself, I didn't mean just copy any configure file. Rather, I meant, copy/paste the default Liquorix configuration file. Or, more generally, the default configuration file, (if any,) for whatever patch I may wish to apply in the future.

Irvine

Arch Linux

#1 2016-10-26 03:52:28

Correct usage of 'make localmodconfig'

#2 2016-10-26 04:02:33

Re: Correct usage of 'make localmodconfig'

#3 2016-10-26 05:38:37

Re: Correct usage of 'make localmodconfig'

#4 2016-10-26 05:51:30

Re: Correct usage of 'make localmodconfig'

#5 2016-10-26 06:06:59

Re: Correct usage of 'make localmodconfig'

#6 2016-10-26 06:23:37

Re: Correct usage of 'make localmodconfig'

#7 2016-10-26 08:50:43

Re: Correct usage of 'make localmodconfig'

#8 2016-10-26 09:40:30

Re: Correct usage of 'make localmodconfig'

#9 2016-10-26 10:03:37

Re: Correct usage of 'make localmodconfig'

#10 2016-10-26 14:03:11

Re: Correct usage of 'make localmodconfig'

Board footer