Kernel module management

B-Con · 2008-02-20 22:43:39

I'm having some trouble visualizing/understanding how modules interface with the kernel.

Along the lines of my previous thread, *when* does a kernel recompile require that a module be recompiled? The repository-provided kernel and nvidia drivers, I presume, are compiled on the same(ish) machine with the same settings, thus they work together. But if I manually compile the kernel, must the modules from 3rd-parties also be recompiled against it? Do all modules have to be? Ie, if I have modules for stuff like madwifi, nvidia, truecrypt, etc, how do I know what needs to be recompiled?

And say I'm upgrading a kernel, the normal way through pacman: I assume all extra modules are easily locatable and these modules are then recopied to the new kernel's storage location? Ie, is everything under /lib/modules/X.Y.Z-ARCH/extra/ automatically copied from the old kernel to the new? Would everything in there need to be recompiled?

Sorry for my lack of kernel knowledge, just trying to get everything straight. I was about to install a custom kernel (for the first time, obviously ) and wanted to make sure I'm not cluelessly messing up other stuff.

brebs · 2008-02-21 00:19:54

The modules should *always* be recompiled whenever the kernel is compiled.

Just because we can sometimes get away with not doing so (for the same kernel version), is not an excuse - modules can still work, but they might also be unstable.

Don't use pacman for the kernel - the kernel is practically the one exception where pacman is just not well suited. But use pacman for the kernel modules.

No, modules are not copied over, because they are expected to be recompiled.

Last edited by brebs (2008-02-21 00:21:09)

tomk · 2008-02-21 00:27:10

brebs wrote:

Don't use pacman for the kernel - the kernel is practically the one exception where pacman is just not well suited.

You forgot to add "IMO".

Many of us use pacman for the kernel, and for everything else - personal preference is the deciding factor.

brebs · 2008-02-21 00:54:13

For recompiling kernel modules, see nvidia thread.

Last edited by brebs (2008-02-21 00:54:48)

B-Con · 2008-02-21 01:13:57

brebs wrote:

No, modules are not copied over, because they are expected to be recompiled.

How is it then that Truecrypt always works after a kernel upgrade, as it requires a kernel module?

brebs · 2008-02-21 01:23:58

Is this a trick question

Whatever kernel module it requires (probably crypto-related) is probably *built-in* to the kernel, therefore is recompiled along with the rest of the kernel.

B-Con · 2008-02-21 02:42:09

Heh, not intended to be a trick question:

/lib/modules/2.6.24-ARCH/extra/truecrypt.ko

It has its own module. I assume the module is only responsible for creating the virtual drive and managing the encryption layer.

Last edited by B-Con (2008-02-21 02:42:58)

brebs · 2008-02-21 03:02:47

This doesn't happen by magic. /lib/modules/2.6.24-ARCH/extra/truecrypt.ko is provided by /var/abs/extra/system/truecrypt/PKGBUILD

The Arch kernel calls itself 2.6.24-ARCH, rather than 2.6.24.1-blah like it should, which is why the module is still "present" when the Arch kernel is upgraded - the modules are still in /lib/modules/2.6.24-ARCH/

So, as I've already explained, this is naughty, because the truecrypt module doesn't necessarily get recompiled for the new kernel.

This could explain the recent nvidia instability, and why I was blissfully unaffected by it - I use a custom "compilekernel" script which always recompiles the nvidia package at the end, along with any other packages I have installed which contain kernel modules.

aport · 2008-02-21 05:15:55

brebs wrote:

This could explain the recent nvidia instability, and why I was blissfully unaffected by it - I use a custom "compilekernel" script which always recompiles the nvidia package at the end, along with any other packages I have installed which contain kernel modules.

Slightly offtopic: Could you post your "compilekernel" script? I am looking to do something similar to this as I don't particularly enjoy tracking down and compiling the packages I have which provide modules. I don't have *that* many installed, but it seems that whenever I upgrade my kernel (I compile it myself with some extra patches) I always seem to miss a few and get greeted with some "module does not exist" messages.
I would appreciate it.

B-Con · 2008-02-21 06:36:01

brebs wrote:

This doesn't happen by magic. /lib/modules/2.6.24-ARCH/extra/truecrypt.ko is provided by /var/abs/extra/system/truecrypt/PKGBUILD
The Arch kernel calls itself 2.6.24-ARCH, rather than 2.6.24.1-blah like it should, which is why the module is still "present" when the Arch kernel is upgraded - the modules are still in /lib/modules/2.6.24-ARCH/
So, as I've already explained, this is naughty, because the truecrypt module doesn't necessarily get recompiled for the new kernel.
This could explain the recent nvidia instability, and why I was blissfully unaffected by it - I use a custom "compilekernel" script which always recompiles the nvidia package at the end, along with any other packages I have installed which contain kernel modules.

So upgrading the kernel from 2.6.24 to 2.6.24.1 I could understand the Truecrypt module slipping between the cracks staying put, since the directory never gets cleared out. But I've had TC since 2.6.22, and it's never stopped working.

I'm wondering, it possible they've been releasing a new version of TC with every kernel? I haven't noticed this explicitly, but I do remember, over the past couple months, installing (I think two) updated TC versions even though I knew there was no updated version upstream, and I do remember one of those TC updates being at the same time as a kernel upgrade. That would mean they were compiling the TC module against the stock kernel and pushing it out into the repositories with the new kernel; that would explain it.

Oh, and also, how do I compile against a certain kernel? Does the new kernel have to first be installed, then I can compile and install the package with the kernel modules?

Last edited by B-Con (2008-02-21 06:41:49)

brebs · 2008-02-21 13:21:54

For my module compilation script, see thread.

brebs · 2008-02-21 15:12:57

B-Con, look through /var/log/pacman.log and look at the datestamps of the truecrypt.ko files in the /lib/modules/ tree.

B-Con · 2008-02-21 22:57:10

b-con@beacon:~$ cat /var/log/pacman.log | grep kernel26
[2008-01-09 05:27] ==> Running command: /sbin/mkinitcpio -k 2.6.23-ARCH -c /etc/mkinitcpio.conf -g /boot/kernel26.img
[2008-01-09 05:28] :: Generating image '/boot/kernel26.img'...SUCCESS
[2008-01-09 05:28] ==> Running command: /sbin/mkinitcpio -k 2.6.23-ARCH -c /etc/mkinitcpio.d/kernel26-fallback.conf -g /boot/kernel26-fallback.img
[2008-01-09 05:28] :: Generating image '/boot/kernel26-fallback.img'...SUCCESS
[2008-01-09 05:28] upgraded kernel26 (2.6.22.9-1 -> 2.6.23.12-3)
[2008-01-19 21:52] ==> Running command: /sbin/mkinitcpio -k 2.6.23-ARCH -c /etc/mkinitcpio.conf -g /boot/kernel26.img
[2008-01-19 21:52] :: Generating image '/boot/kernel26.img'...SUCCESS
[2008-01-19 21:52] ==> Running command: /sbin/mkinitcpio -k 2.6.23-ARCH -c /etc/mkinitcpio.d/kernel26-fallback.conf -g /boot/kernel26-fallback.img
[2008-01-19 21:53] :: Generating image '/boot/kernel26-fallback.img'...SUCCESS
[2008-01-19 21:53] upgraded kernel26 (2.6.23.12-3 -> 2.6.23.14-1)
[2008-02-16 03:00] ==> Running command: /sbin/mkinitcpio -k 2.6.24-ARCH -c /etc/mkinitcpio.conf -g /boot/kernel26.img
[2008-02-16 03:00] :: Generating image '/boot/kernel26.img'...SUCCESS
[2008-02-16 03:00] ==> Running command: /sbin/mkinitcpio -k 2.6.24-ARCH -c /etc/mkinitcpio.d/kernel26-fallback.conf -g /boot/kernel26-fallback.img
[2008-02-16 03:01] :: Generating image '/boot/kernel26-fallback.img'...SUCCESS
[2008-02-16 03:01] upgraded kernel26 (2.6.23.14-1 -> 2.6.24.1-2)
[2008-02-20 14:23] ==> Running command: /sbin/mkinitcpio -k 2.6.24-ARCH -c /etc/mkinitcpio.conf -g /boot/kernel26.img
[2008-02-20 14:24] :: Generating image '/boot/kernel26.img'...SUCCESS
[2008-02-20 14:24] ==> Running command: /sbin/mkinitcpio -k 2.6.24-ARCH -c /etc/mkinitcpio.d/kernel26-fallback.conf -g /boot/kernel26-fallback.img
[2008-02-20 14:24] :: Generating image '/boot/kernel26-fallback.img'...SUCCESS
[2008-02-20 14:24] upgraded kernel26 (2.6.24.1-2 -> 2.6.24.1-2)
[2008-02-20 16:16] ==> Running command: /sbin/mkinitcpio -k 2.6.24-ARCH -c /etc/mkinitcpio.conf -g /boot/kernel26.img
[2008-02-20 16:16] :: Generating image '/boot/kernel26.img'...SUCCESS
[2008-02-20 16:16] ==> Running command: /sbin/mkinitcpio -k 2.6.24-ARCH -c /etc/mkinitcpio.d/kernel26-fallback.conf -g /boot/kernel26-fallback.img
[2008-02-20 16:17] :: Generating image '/boot/kernel26-fallback.img'...SUCCESS
[2008-02-20 16:17] upgraded kernel26 (2.6.24.1-2 -> 2.6.24.1-2)

b-con@beacon:~$ cat /var/log/pacman.log | grep truecrypt
[2008-01-09 05:27] installed truecrypt-utils (4.3a-2)
[2008-01-09 05:28] installed truecrypt (4.3a-11)
[2008-01-16 19:33] ==> since by default all sudo and truecrypt activities
[2008-01-16 19:33] ==> This entry disables the logging for the truecrypt group.
[2008-02-16 03:02] upgraded truecrypt (4.3a-11 -> 4.3a-12)

How about it, every time the minor release of the kernel changes (22 to 23, and 23 to 24) Truecrypt gets updated.

B-Con · 2008-02-26 07:32:29

brebs wrote:

B-Con, look through /var/log/pacman.log and look at the datestamps of the truecrypt.ko files in the /lib/modules/ tree.

I looked through the script and I'm not quite clear on something. What exactly is it that you're doing that compiles the module against the new, but yet uninstalled, kernel?

redguardtoo · 2008-02-26 07:49:56

The Linux Kernel Driver Interface
This is being written to try to explain why Linux does not have a binary kernel interface, nor does it have a stable kernel interface. ,,,

http://www.kroah.com/log/linux/stable_api_nonsense.html

brebs · 2008-02-26 12:11:36

B-Con wrote:

but yet uninstalled

It's impossible to compile against an *uninstalled* kernel.

You're installing pre-compiled binaries. Obviously, they were compiled on a different PC, which *did* have the relevant kernel installed.

I, on the other hand, install the kernel and then compile the modules (e.g. nvidia) against it.

B-Con · 2008-02-26 21:54:28

brebs wrote:

B-Con wrote:
but yet uninstalled
It's impossible to compile against an *uninstalled* kernel.
You're installing pre-compiled binaries. Obviously, they were compiled on a different PC, which *did* have the relevant kernel installed.
I, on the other hand, install the kernel and then compile the modules (e.g. nvidia) against it.

OK. That's what was confusing me in your script, I wasn't sure if the new kernel was installed or not. It looked like you were, but I wanted to be sure.

Arch Linux

#1 2008-02-20 22:43:39

Kernel module management

#2 2008-02-21 00:19:54

Re: Kernel module management

#3 2008-02-21 00:27:10

Re: Kernel module management

#4 2008-02-21 00:54:13

Re: Kernel module management

#5 2008-02-21 01:13:57

Re: Kernel module management

#6 2008-02-21 01:23:58

Re: Kernel module management

#7 2008-02-21 02:42:09

Re: Kernel module management

#8 2008-02-21 03:02:47

Re: Kernel module management

#9 2008-02-21 05:15:55

Re: Kernel module management

#10 2008-02-21 06:36:01

Re: Kernel module management

#11 2008-02-21 13:21:54

Re: Kernel module management

#12 2008-02-21 15:12:57

Re: Kernel module management

#13 2008-02-21 22:57:10

Re: Kernel module management

#14 2008-02-26 07:32:29

Re: Kernel module management

#15 2008-02-26 07:49:56

Re: Kernel module management

#16 2008-02-26 12:11:36

Re: Kernel module management

#17 2008-02-26 21:54:28

Re: Kernel module management

Board footer