You are not logged in.

#1 2019-03-09 14:02:52

Jason P.
Member
Registered: 2015-02-26
Posts: 171

[solved] Missing modules after compiling kernel with localmodconfig

I've compiled a linux kernel inside chroot using aurutils. Before doing the build i called modprobed-db to recall modules from its database. I've also edited the PKGBUILD to uncoditionally call 'make localmodconfig' and it worked. I was prompted about two different modules before compilation but nothing else. Except that when installing the package I got this errors during the creation of initramfs:

[lvm2]
==> ERROR: module not found: `dm_snapshot'
==> ERROR: module not found: `dm_mirror'
==> ERROR: module not found: `dm_cache'
==> ERROR: module not found: `dm_cache_smq'
==> ERROR: module not found: `dm_thin_pool'

Those modules were not added to modprobed-db probably because anything on my system uses them, although I do have LVM and dm_crypt. Can those errors be treated as warnings? Can I just simply add them to modprobed-db (is a plain text file)?

I don't have a clue about the two modules I was asked before compilation but they were not related to LVM neither crypt. Is there a way to remember my preferences for next time if I'm doing chroot builds?

Last edited by Jason P. (2019-03-12 15:41:33)

Offline

#2 2019-03-09 15:18:22

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,595
Website

Re: [solved] Missing modules after compiling kernel with localmodconfig

If you build a from a custom kernel and you ask it to load modules that aren't part of that build, modprobe will obviously not be able to find them whether you manually add them to the database or not.  Install and boot into the standard Arch kernel, run the system for a while and run `modprobed-db s` ... do they get added to the database?

You may find that on major kernel bumps, (ie 4.20 --> 5.0) new modules will be available but if you don't first do as I outlined, you will never get a chance to load them and thus build them in subsequent builds.


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#3 2019-03-09 16:47:24

loqs
Member
Registered: 2014-03-06
Posts: 17,196

Re: [solved] Missing modules after compiling kernel with localmodconfig

Did the resulting initrd boot correctly without those modules?
Edit:
You could cp /usr/lib/initcpio/install/lvm2 to /etc/initcpio/install/lvm2 then remove the missing modules from line 8 and rebuild the initrd.

Last edited by loqs (2019-03-09 16:53:33)

Offline

#4 2019-03-09 18:49:06

Jason P.
Member
Registered: 2015-02-26
Posts: 171

Re: [solved] Missing modules after compiling kernel with localmodconfig

graysky wrote:

If you build a from a custom kernel and you ask it to load modules that aren't part of that build, modprobe will obviously not be able to find them whether you manually add them to the database or not.

I've built from a standard Arch kernel that I use on a daily basis.

graysky wrote:

run the system for a while and run `modprobed-db s` ... do they get added to the database?

That was exactly what I did. Install modprobed-db, enable user units, let it run for a couple of days, booting also into other kernel and plugging some devices (USB, MIDI keyboard...). Modules database counts 115 entries and modules loaded by the system are also 115.

As I've said my setup uses LVM and crypt, otherwise the system wouldn't boot, but for some reason several specific modules were not listed.

graysky wrote:

You may find that on major kernel bumps, (ie 4.20 --> 5.0) new modules will be available but if you don't first do as I outlined, you will never get a chance to load them and thus build them in subsequent builds.

So you mean that I should try to do the same as before (plugging / unplugging stuff and so on)? How to activate 'dm_snapshot' or 'dm_thin_pool' then? That's why I've suggested to just enter those specific modules directly into database and rebuild again.

Offline

#5 2019-03-09 18:58:59

Jason P.
Member
Registered: 2015-02-26
Posts: 171

Re: [solved] Missing modules after compiling kernel with localmodconfig

loqs wrote:

Did the resulting initrd boot correctly without those modules?

Yep. Booting was fine. At the end I guess those modules are not probably used by my standard installation, that could be the reason they are not present in database.

Offline

#6 2019-03-09 22:36:38

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,595
Website

Re: [solved] Missing modules after compiling kernel with localmodconfig

Jason P. wrote:

As I've said my setup uses LVM and crypt, otherwise the system wouldn't boot, but for some reason several specific modules were not listed.

Ah... likely, the first time you built the custom kernel, these dm modules were compiled into the kernel (ie always on) which is why your image booted.  I don't understand why you couldn't load them on a subsequent build though.  Did you define these in the MODULES array of /etc/mkinitcpio.conf by chance?

Jason P. wrote:

So you mean that I should try to do the same as before (plugging / unplugging stuff and so on)? How to activate 'dm_snapshot' or 'dm_thin_pool' then? That's why I've suggested to just enter those specific modules directly into database and rebuild again.

I'm just saying make sure your database is complete.  When you do experience oddness, it is likely because you're running new software or hardware and your minimized image is missing key modules.  Always think about that when you get weirdness and boot into the standard kernel to see if the weirdness reproduces.  If it doesn't reproduce, run `modprobed_db s` and see if new modules are detected then rebuild the minimized kernel.


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#7 2019-03-10 13:49:48

Jason P.
Member
Registered: 2015-02-26
Posts: 171

Re: [solved] Missing modules after compiling kernel with localmodconfig

Just to clarify a little bit in case I didn't explain myself well enoguh. I've tried to compile inside a chroot and recalling modules from modprobed-db but until this build I was not able. So this is my first and only build that used 'make localmodconfig'. I didn't try yet to rebuild.

I always do these builds from a standard Arch kernel, the one I boot into for using my computer everyday. Kernel version recently changed to 5.0.0 and that build with 'make localmodconfig' was in fact using that kernel version. modprobed-db is running as a user service before the upgrade but I didn't notice any new moduled added with the new version except for one. I know it because of the total count.

All custom kernel builds were functional to at least the point of booting into them, even this one with much less modules compiled (around 134 IIRC). Although the difference happened when installing the kernel package itself and initramfs was built (not sure if normal or fallback). I've only got errors with the smaller build, this last one.

- What you mean by "compiled into the kernel"? If that's the situation on my Arch standard kernel using LVM, should have modprobed-db detected those modules that initramfs complains about?
- I don't have access to my computer today so I have to confirm how it looks like my MODULES array in /etc/mkinitcpio.conf
- Does 'modprobed-db store' do something different from the system user unit that it's already on?

Sorry for the long explanation but I was having the feeling of not being clear enough.

Last edited by Jason P. (2019-03-10 13:55:21)

Offline

#8 2019-03-11 13:42:43

Jason P.
Member
Registered: 2015-02-26
Posts: 171

Re: [solved] Missing modules after compiling kernel with localmodconfig

Result of calling as my regular user 'modprobed-db store' => 'No new modules detected. Taking no action.' Seems logic because the systemd user unit is running for days.

➜ % ~  grep MODULES /etc/mkinitcpio.conf 
MODULES=(nouveau)

➜ % ~  grep HOOKS /etc/mkinitcpio.conf 
HOOKS=(consolefont base udev autodetect modconf block keymap encrypt lvm2 resume filesystems keyboard fsck)

Offline

#9 2019-03-11 17:17:06

Jason P.
Member
Registered: 2015-02-26
Posts: 171

Re: [solved] Missing modules after compiling kernel with localmodconfig

I've rebuilt the package but result is the same. Now I can paste full output of the error:

➜ # ~  pacman -S linux-rt-bfq
warning: linux-rt-bfq-4.19.25.rt16-8 is up to date -- reinstalling
resolving dependencies...
looking for conflicting packages...

Packages (1) linux-rt-bfq-4.19.25.rt16-8

Total Installed Size:  15.56 MiB
Net Upgrade Size:       0.01 MiB

:: Proceed with installation? [Y/n] Y
(1/1) checking keys in keyring                                  [###################################] 100%
(1/1) checking package integrity                                [###################################] 100%
(1/1) loading package files                                     [###################################] 100%
(1/1) checking for file conflicts                               [###################################] 100%
(1/1) checking available disk space                             [###################################] 100%
:: Processing package changes...
(1/1) reinstalling linux-rt-bfq                                 [###################################] 100%
:: Running post-transaction hooks...
(1/3) Updating linux-rt-bfq module dependencies...
(2/3) Updating linux-rt-bfq initcpios...
==> Building image from preset: /etc/mkinitcpio.d/linux-rt-bfq.preset: 'default'
  -> -k /boot/vmlinuz-linux-rt-bfq -c /etc/mkinitcpio.conf -g /boot/initramfs-linux-rt-bfq.img
==> Starting build: 4.19.25-rt16-8-rt-bfq
  -> Running build hook: [consolefont]
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [autodetect]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
  -> Running build hook: [keymap]
  -> Running build hook: [encrypt]
  -> Running build hook: [lvm2]
==> ERROR: module not found: `dm_snapshot'
==> ERROR: module not found: `dm_mirror'
==> ERROR: module not found: `dm_cache'
==> ERROR: module not found: `dm_cache_smq'
==> ERROR: module not found: `dm_thin_pool'
  -> Running build hook: [resume]
  -> Running build hook: [filesystems]
  -> Running build hook: [keyboard]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/initramfs-linux-rt-bfq.img
==> WARNING: errors were encountered during the build. The image may not be complete.
==> Building image from preset: /etc/mkinitcpio.d/linux-rt-bfq.preset: 'fallback'
  -> -k /boot/vmlinuz-linux-rt-bfq -c /etc/mkinitcpio.conf -g /boot/initramfs-linux-rt-bfq-fallback.img -S autodetect
==> Starting build: 4.19.25-rt16-8-rt-bfq
  -> Running build hook: [consolefont]
  -> Running build hook: [base]
  -> Running build hook: [udev]
  -> Running build hook: [modconf]
  -> Running build hook: [block]
  -> Running build hook: [keymap]
  -> Running build hook: [encrypt]
  -> Running build hook: [lvm2]
==> ERROR: module not found: `dm_snapshot'
==> ERROR: module not found: `dm_mirror'
==> ERROR: module not found: `dm_cache'
==> ERROR: module not found: `dm_cache_smq'
==> ERROR: module not found: `dm_thin_pool'
  -> Running build hook: [resume]
  -> Running build hook: [filesystems]
  -> Running build hook: [keyboard]
  -> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/initramfs-linux-rt-bfq-fallback.img
==> WARNING: errors were encountered during the build. The image may not be complete.
error: command failed to execute correctly
(3/3) Arming ConditionNeedsUpdate...

Modules in modprobed-db were recalled correctly:

➜ % ~  lsmod | tail -n +2 | wc -l
115

➜ % ~  modprobed-db list
Modprobed-db v2.40

115 modules currently loaded per /proc/modules
115 modules are in /path-to-database

➜ % ~  diff <(lsmod | tail -n +2 | cut -d ' ' -f1 | sort) <(sort modprobed.db)
➜ % ~  

But dm_snapshot, dm_mirror and friends are not in database. As I've said before, I compile every time from my standard Arch kernel.

➜ % ~  modprobed-db list | grep dm_
dm_crypt
dm_mod

Last edited by Jason P. (2019-03-11 17:20:43)

Offline

#10 2019-03-12 12:07:57

Jason P.
Member
Registered: 2015-02-26
Posts: 171

Re: [solved] Missing modules after compiling kernel with localmodconfig

I've been looking closer at this. The issue probably comes from differences between running kernel (5.0.0) and compiled kernel (4.19.25). Although their config files are the same for the modules in question.

Running kernel: 5.0.0

➜ % ~  zcat /proc/config.gz | grep -i config_dm_
CONFIG_DM_DEBUG=y
CONFIG_DM_BUFIO=m
# CONFIG_DM_DEBUG_BLOCK_MANAGER_LOCKING is not set
CONFIG_DM_BIO_PRISON=m
CONFIG_DM_PERSISTENT_DATA=m
CONFIG_DM_UNSTRIPED=m
CONFIG_DM_CRYPT=m
CONFIG_DM_SNAPSHOT=m
CONFIG_DM_THIN_PROVISIONING=m
CONFIG_DM_CACHE=m
CONFIG_DM_CACHE_SMQ=m
CONFIG_DM_WRITECACHE=m
CONFIG_DM_ERA=m
CONFIG_DM_MIRROR=m
CONFIG_DM_LOG_USERSPACE=m
CONFIG_DM_RAID=m
CONFIG_DM_ZERO=m
CONFIG_DM_MULTIPATH=m
CONFIG_DM_MULTIPATH_QL=m
CONFIG_DM_MULTIPATH_ST=m
CONFIG_DM_DELAY=m
CONFIG_DM_UEVENT=y
CONFIG_DM_FLAKEY=m
CONFIG_DM_VERITY=m
CONFIG_DM_VERITY_FEC=y
CONFIG_DM_SWITCH=m
CONFIG_DM_LOG_WRITES=m
CONFIG_DM_INTEGRITY=m
CONFIG_DM_ZONED=m

Compiled kernel: 4.19.25

CONFIG_DM_MQ_DEFAULT=y
CONFIG_DM_DEBUG=y
CONFIG_DM_BUFIO=m
# CONFIG_DM_DEBUG_BLOCK_MANAGER_LOCKING is not set
CONFIG_DM_BIO_PRISON=m
CONFIG_DM_PERSISTENT_DATA=m
CONFIG_DM_UNSTRIPED=m
CONFIG_DM_CRYPT=m
CONFIG_DM_SNAPSHOT=m
CONFIG_DM_THIN_PROVISIONING=m
CONFIG_DM_CACHE=m
CONFIG_DM_CACHE_SMQ=m
CONFIG_DM_WRITECACHE=m
CONFIG_DM_ERA=m
CONFIG_DM_MIRROR=m
CONFIG_DM_LOG_USERSPACE=m
CONFIG_DM_RAID=m
CONFIG_DM_ZERO=m
CONFIG_DM_MULTIPATH=m
CONFIG_DM_MULTIPATH_QL=m
CONFIG_DM_MULTIPATH_ST=m
CONFIG_DM_DELAY=m
CONFIG_DM_UEVENT=y
CONFIG_DM_FLAKEY=m
CONFIG_DM_VERITY=m
CONFIG_DM_VERITY_FEC=y
CONFIG_DM_SWITCH=m
CONFIG_DM_LOG_WRITES=m
CONFIG_DM_INTEGRITY=m
CONFIG_DM_ZONED=m

CONFIG_DM_MQ_DEFAULT=y is the only different setting. Config file of the compiled kernel can be found here.

Offline

#11 2019-03-12 12:43:07

progandy
Member
Registered: 2012-05-17
Posts: 5,184

Re: [solved] Missing modules after compiling kernel with localmodconfig

The lvm2 hook in mkinitcpio tries to add all dm- modules to the initramfs. That doesn't mean you need them or that the kernel will load them. Since your configuration doesn't need them they aren't loaded and not part of your modprobed-db. You should be able to ignore the errors when mkinitcpio tries to add these modules.

If you want to silence these warnings, just add all the dm- modules manually to your modprobed-db file before recalling and compiling the kernel.


| alias CUTF='LANG=en_XX.UTF-8@POSIX ' |

Offline

#12 2019-03-12 15:41:18

Jason P.
Member
Registered: 2015-02-26
Posts: 171

Re: [solved] Missing modules after compiling kernel with localmodconfig

@progandy Thanks for the clear explanation. Your suggestion worked perfectly. Closing.

Offline

Board footer

Powered by FluxBB