[SOLVED] How and when are kernel module updates applied?

Archemore · 2020-12-08 05:32:17

Hi,

I'm trying to better understand how upgrading kernel modules takes place in Linux. This is less an urgent tech support request, and more a curiosity. Here's an example I'd like to understand.

Suppose I upgrade mesa which seems to provide the amdgpu kernel module based on this Wiki page. (Or maybe amdgpu already comes preinstalled right in the kernel, because it's too fundamental? In any case, it seems mesa at least provides some kernel driver, maybe just some "DRI driver for 3D acceleration.") When exactly will the new driver be loaded into the kernel for use? Next time I reboot? Next time a process attempts to interact with a device that requires the module? In the case of something involving graphics, like mesa, would it just automatically be applied since graphics are being used constantly?

Based on this warning here, it seems that not rebooting after a kernel upgrade certainly means one can no longer mess around with kernel modules using modprobe, which seems reasonable. But given the kernel can exist without any number of modules, to what extent is a reboot required after upgrading modules? And what sort of problems could one run into without rebooting after upgrading some modules?

If my simplistic understanding of kernel modules is so far off, some further reading for a beginner would be appreciated!

P.S. Regarding this particular example, I noticed that even though I have the packages listed under the AMDGPU page installed, running e.g. lsmod | grep -i mesa or lsmod | grep -i vulkan and so forth produce nothing. Running lsmod | grep -i amdgpu however produces a few kernel modules I don't recognize from the Wiki page. What's up with that discrepancy?

Last edited by Archemore (2020-12-09 21:41:23)

loqs · 2020-12-08 05:52:45

The amdgpu kernel module is supplied by kernel packages not mesa.
If the kernel is updated the modules it supplies are also updated but can not be loaded as they will not match the running kernel hence the need for a reboot.
If a kernel module supplied by another package also known as an out of tree module / driver is updated provided it matches the running kernel it can be loaded without a reboot, assuming the module is not already loaded or that the loaded version can be unloaded.

Lone_Wolf · 2020-12-08 14:53:29

Archemore, check https://www.reddit.com/r/archlinux/comm … m/djsc8at/ for an overview of linux graphics stack.

Mesa is typically loaded on gui start.
If you update it while in X / wayland , log out and re-login to ensure you're using the new version.

Archemore · 2020-12-08 17:51:57

loqs wrote:

The amdgpu kernel module is supplied by kernel packages not mesa.
If the kernel is updated the modules it supplies are also updated but can not be loaded as they will not match the running kernel hence the need for a reboot.
If a kernel module supplied by another package also known as an out of tree module / driver is updated provided it matches the running kernel it can be loaded without a reboot, assuming the module is not already loaded or that the loaded version can be unloaded.

So am I correct in assuming that when a kernel module is updated, the package stored on the disk is updated, but the module running in memory will remain unchanged if in current use? Then the new module won't be used until the module actually gets reloaded by some other means (e.g. with a reboot, or other "resetting" type option like relogging in)?

Lone_Wolf wrote:

Archemore, check https://www.reddit.com/r/archlinux/comm … m/djsc8at/ for an overview of linux graphics stack.
Mesa is typically loaded on gui start.
If you update it while in X / wayland , log out and re-login to ensure you're using the new version.

Thanks! This clears up a lot of my misunderstanding about graphics drivers on Linux. But about the relogging in: if I'm using a graphical DM, then wouldn't mesa need to be loaded by the time that appears? If I logged out then, I'd still need some graphics to show it, but would mesa still get reloaded?

Last edited by Archemore (2020-12-08 17:52:39)

loqs · 2020-12-08 18:10:52

Archemore wrote:

So am I correct in assuming that when a kernel module is updated, the package stored on the disk is updated, but the module running in memory will remain unchanged if in current use? Then the new module won't be used until the module actually gets reloaded by some other means (e.g. with a reboot, or other "resetting" type option like relogging in)?

Correct, updating the package does not change the running kernel or its loaded modules. Apart from a reboot I think you would need to manually unload and reload a module to update it, it would also require the running kernel version matched the version both the old and new modules were built for.

V1del · 2020-12-08 19:03:07

Afaik mesa gets reloaded every time a new application requests a graphic context. So just restarting "affected" programs should suffice.

Archemore · 2020-12-08 19:26:42

V1del wrote:

Afaik mesa gets reloaded every time a new application requests a graphic context. So just restarting "affected" programs should suffice.

Hm, that sounds a bit strange to me. Is there a reason to reload mesa every time a new application with graphics is launched? In the context of this thread, it ensures a mesa update will take effect "immediately" so to speak, but seems horribly inefficient as the ratio of mesa upgrades to new programs with graphics being launched must be incredibly small for the average user.

Lone_Wolf · 2020-12-08 19:30:30

Archemore wrote:

But about the relogging in: if I'm using a graphical DM, then wouldn't mesa need to be loaded by the time that appears? If I logged out then, I'd still need some graphics to show it, but would mesa still get reloaded?

V1del wrote:

Afaik mesa gets reloaded every time a new application requests a graphic context. So just restarting "affected" programs should suffice.

V1del is correct, but some affected programs are harder to restart then others (like Desktop Environment / Window Manager / Display Manager ).

You are right, Archemore . logging out/in is not enough to reload mesa for everything when using a graphical display manager.
You'd have to stop the DM and restart it.

Last edited by Lone_Wolf (2020-12-08 20:22:02)

loqs · 2020-12-08 19:40:42

Archemore wrote:

Hm, that sounds a bit strange to me. Is there a reason to reload mesa every time a new application with graphics is launched? In the context of this thread, it ensures a mesa update will take effect "immediately" so to speak, but seems horribly inefficient as the ratio of mesa upgrades to new programs with graphics being launched must be incredibly small for the average user.

When a process is started all the libraries it uses are loaded from disk and mapped into that processes memory space, the kernel may have the library cached already but that is transparent to the process.

Archemore · 2020-12-08 20:41:14

Lone_Wolf wrote:

V1del is correct, but some affected programs are harder to restart then others (like Desktop Environment / Window Manager / Display Manager ).
You are right, Archemore . logging out/in is not enough to reload mesa for everything when using a graphical display manager.
You'd have to stop the DM and restart it.

That makes sense. So logging out and back in would mean all DE/WM/etc applications will use the new mesa driver, but the DM will continue using the old driver until it's restarted (realistically upon rebooting)? Are there any risks at having multiple versions of the same driver running at once?

loqs wrote:

When a process is started all the libraries it uses are loaded from disk and mapped into that processes memory space, the kernel may have the library cached already but that is transparent to the process.

I see, so in this case the libraries can include relevant kernel modules? If multiple programs want to use the same graphics libraries (e.g. OpenGL?), wouldn't it make sense to have the kernel load that once into memory and allow multiple programs to use it as needed? But then I suppose when a new application is launched, the kernel will check what libraries it requires, and if any aren't loaded, they would be now. That's how a newly launched program that needs mesa can get the newer version, while the old apps will keep using the old one. Does that sound accurate?

loqs · 2020-12-08 22:32:14

Archemore wrote:

loqs wrote:
When a process is started all the libraries it uses are loaded from disk and mapped into that processes memory space, the kernel may have the library cached already but that is transparent to the process.
I see, so in this case the libraries can include relevant kernel modules? If multiple programs want to use the same graphics libraries (e.g. OpenGL?), wouldn't it make sense to have the kernel load that once into memory and allow multiple programs to use it as needed? But then I suppose when a new application is launched, the kernel will check what libraries it requires, and if any aren't loaded, they would be now. That's how a newly launched program that needs mesa can get the newer version, while the old apps will keep using the old one. Does that sound accurate?

Kernel modules such as amdgpu, i915, nvidia are only loaded once as there is only one running kernel so there can only be one version of them in use.
Userspace libraries such as libGLX_mesa.so are loaded once per process into that processes address space. See https://unix.stackexchange.com/question … ed-library
Edit:
Once a process is started all the processes libraries are loaded so having the library on disk changed or a different process load a different process use a different version of the library is not usually an issue.
There are corner cases such as if process itself calls dlopen at some point after it is started and the library it was going to call has been updated that can fail.

Last edited by loqs (2020-12-08 22:38:11)

Archemore · 2020-12-09 06:49:37

loqs wrote:

Archemore wrote:
loqs wrote:
When a process is started all the libraries it uses are loaded from disk and mapped into that processes memory space, the kernel may have the library cached already but that is transparent to the process.
I see, so in this case the libraries can include relevant kernel modules? If multiple programs want to use the same graphics libraries (e.g. OpenGL?), wouldn't it make sense to have the kernel load that once into memory and allow multiple programs to use it as needed? But then I suppose when a new application is launched, the kernel will check what libraries it requires, and if any aren't loaded, they would be now. That's how a newly launched program that needs mesa can get the newer version, while the old apps will keep using the old one. Does that sound accurate?
Kernel modules such as amdgpu, i915, nvidia are only loaded once as there is only one running kernel so there can only be one version of them in use.
Userspace libraries such as libGLX_mesa.so are loaded once per process into that processes address space. See https://unix.stackexchange.com/question … ed-library
Edit:
Once a process is started all the processes libraries are loaded so having the library on disk changed or a different process load a different process use a different version of the library is not usually an issue.
There are corner cases such as if process itself calls dlopen at some point after it is started and the library it was going to call has been updated that can fail.

I see, so one module can provide libraries to various processes. But if the module gets updated, but not reloaded, how is it ok for the module to be old but the libraries provided be the new ones? Maybe I'm misunderstanding since I have a lot to learn about the kernel, but thank you for the link the led to some interesting reading!

V1del · 2020-12-09 08:51:20

You are misunderstanding something. Mesa and the kernel module (respectively userspace libs in general) are irrelevant to each other. A new mesa's library release will use the "old" kernel module just the same. But if you have multiple versions of mesa then old processes still have the "old" version in RAM and new processes the "new" version. This is userspace. Mesa (whether new or old) will call down to the kernel for certain functionalities but these will work as long as the relevant kernel modules are loaded (... which they are, you are currently actively using them)

Archemore · 2020-12-09 21:41:13

Thanks everyone for bearing with my questions! This has been very informative.

Arch Linux

#1 2020-12-08 05:32:17

[SOLVED] How and when are kernel module updates applied?

#2 2020-12-08 05:52:45

Re: [SOLVED] How and when are kernel module updates applied?

#3 2020-12-08 14:53:29

Re: [SOLVED] How and when are kernel module updates applied?

#4 2020-12-08 17:51:57

Re: [SOLVED] How and when are kernel module updates applied?

#5 2020-12-08 18:10:52

Re: [SOLVED] How and when are kernel module updates applied?

#6 2020-12-08 19:03:07

Re: [SOLVED] How and when are kernel module updates applied?

#7 2020-12-08 19:26:42

Re: [SOLVED] How and when are kernel module updates applied?

#8 2020-12-08 19:30:30

Re: [SOLVED] How and when are kernel module updates applied?

#9 2020-12-08 19:40:42

Re: [SOLVED] How and when are kernel module updates applied?

#10 2020-12-08 20:41:14

Re: [SOLVED] How and when are kernel module updates applied?

#11 2020-12-08 22:32:14

Re: [SOLVED] How and when are kernel module updates applied?

#12 2020-12-09 06:49:37

Re: [SOLVED] How and when are kernel module updates applied?

#13 2020-12-09 08:51:20

Re: [SOLVED] How and when are kernel module updates applied?

#14 2020-12-09 21:41:13

Re: [SOLVED] How and when are kernel module updates applied?

Board footer