Arch Linux

Eispala

Member

Registered: 2018-05-13

Posts: 6

[SOLVED] Can't boot after upgrade -> /lib/[]/modules.devname not found

Hello all,

I am using linux (arch linux to be precise) for about 3-4 weeks now, and set me up a gaming VM with gpu passthrough.
Everything went pretty well, i wrote some scripts etc. and everything is running fine.

Today i wanted to upgrade my system.

This is my first post in this forum, as I am kind of depressed and out of solutions after trying to fix this problem. (Solutions means i am out of things i can try to google)

I can't boot into my system anymore (Restored a snapshot to post this), after i used the following command to upgrade my system:

$ sudo pacman -Syu

While upgrading, pacman does not give any special outputs.

After a reboot, i get the following messages:

:: running early hook [udev]
warning: /lib/modules/4.16.8-1-ARCH/modules.devname not found - ignring
starting version 238
:: running hook [udev]
:: Triggering uevents...
Waiting 10 seconds for device /dev/disk/by-partuuid/[...]
ERROR: device 'PARTUUID=[...]' not found. Skipping fsck.
mount: /new_root: can't find PARTUUID=[...].
You are now being dropped into an emergency shell.
sh: can't access tty; job control turned off
[rootfs ]#

Of course i did some research, and the overall answer was:
Just boot from live, mount your partitions, arch-chroot into it, mkinitcpio -p linux

That just does not help.

Output of

$ mkinitcpio -p linux

==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'default'
-> -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux.img
==> Starting build: 4.16.8-1-ARCH
-> Running build hook: [base]
-> Running build hook: [udev]
-> Running build hook: [autodetect]
-> Running build hook: [modconf]
-> Running build hook: [block]
-> Running build hook: [filesystems]
-> Running build hook: [keyboard]
-> Running build hook: [resume]
-> Running build hook: [fsck]
==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/initramfs-linux.img
==> Image generation successful
==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'fallback'
-> -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux-fallback.img 
==> Starting build: 4.16.8-1-ARCH
-> Running build hook: [base]
-> Running build hook: [udev]
-> Running build hook: [modconf]
-> Running build hook: [block]
==> Warning: Possibly missing firmware for module: wd719x
==> Warning: Possibly missing firmware for module: aic94xx
-> Running build hook: [filesystems]
-> Running build hook: [keyboard]
-> Running build hook: [resume]
-> Running build hook: [fsck]

==> Generating module dependencies
==> Creating gzip-compressed initcpio image: /boot/initamfs-linux-fallback.img
==> Image generation successful

Strange thing:
When i boot from a live ISO, mount my broken installation, arch-chroot into it and issue

$ uname -r

the output is my old kernel. (4.16.5)

I tried to use pacman -Syu when chrooted into my system, but it just says "all packages up to date, there is nothing to do".

My fstab:

UUID=3ace7ab6-baac-4752-a80e-881091359045	/         	ext4      	rw,defaults,noatime,discard	0 1

UUID=4790-7878      	/boot     	vfat      	rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro	0 2

UUID=3f4edb19-8077-4c50-8229-74bf98dfe90e	none      	swap      	defaults,noatime,discard0 0

hugetlbfs  /dev/hugepages/  hugetlbfs  defaults 0 0

Output of

$ lsblk -f /dev/sdd1 /dev/sdd2 /dev/sd3

NAME FSTYPE LABEL UUID                                 MOUNTPOINT
sdd1 vfat         4790-7878                            /boot
sdd2 swap         3f4edb19-8077-4c50-8229-74bf98dfe90e [SWAP]
sdd3 ext4         3ace7ab6-baac-4752-a80e-881091359045 /

my /etc/mkinitcpio.conf:

# vim:set ft=sh
# MODULES
# The following modules are loaded before any boot hooks are
# run.  Advanced users may wish to specify all system modules
# in this array.  For instance:
#     MODULES=(piix ide_disk reiserfs)
MODULES=(vfio_pci vfio vfio_iommu_type1 vfio_virqfd)

# BINARIES
# This setting includes any additional binaries a given user may
# wish into the CPIO image.  This is run last, so it may be used to
# override the actual binaries included by a given hook
# BINARIES are dependency parsed, so you may safely ignore libraries
BINARIES=()

# FILES
# This setting is similar to BINARIES above, however, files are added
# as-is and are not parsed in any way.  This is useful for config files.
FILES=()

# HOOKS
# This is the most important setting in this file.  The HOOKS control the
# modules and scripts added to the image, and what happens at boot time.
# Order is important, and it is recommended that you do not change the
# order in which HOOKS are added.  Run 'mkinitcpio -H <hook name>' for
# help on a given hook.
# 'base' is _required_ unless you know precisely what you are doing.
# 'udev' is _required_ in order to automatically load modules
# 'filesystems' is _required_ unless you specify your fs modules in MODULES
# Examples:
##   This setup specifies all modules in the MODULES setting above.
##   No raid, lvm2, or encrypted root is needed.
#    HOOKS=(base)
#
##   This setup will autodetect all modules for your system and should
##   work as a sane default
#    HOOKS=(base udev autodetect block filesystems)
#
##   This setup will generate a 'full' image which supports most systems.
##   No autodetection is done.
#    HOOKS=(base udev block filesystems)
#
##   This setup assembles a pata mdadm array with an encrypted root FS.
##   Note: See 'mkinitcpio -H mdadm' for more information on raid devices.
#    HOOKS=(base udev block mdadm encrypt filesystems)
#
##   This setup loads an lvm2 volume group on a usb device.
#    HOOKS=(base udev block lvm2 filesystems)
#
##   NOTE: If you have /usr on a separate partition, you MUST include the
#    usr, fsck and shutdown hooks.
HOOKS=(base udev autodetect modconf block filesystems keyboard resume fsck)

# COMPRESSION
# Use this to compress the initramfs image. By default, gzip compression
# is used. Use 'cat' to create an uncompressed image.
#COMPRESSION="gzip"
#COMPRESSION="bzip2"
#COMPRESSION="lzma"
#COMPRESSION="xz"
#COMPRESSION="lzop"
#COMPRESSION="lz4"

# COMPRESSION_OPTIONS
# Additional options for the compressor
#COMPRESSION_OPTIONS=()

My /boot/loader/loader.conf:

timeout 12
default arch-uefi-pass

/boot/loader/entries/arch-uefi-pass.conf:

title    Arch Linux Pass
linux    /vmlinuz-linux
initrd  /intel-ucode.img
initrd    /initramfs-linux-pass.img
options root=PARTUUID=7a63b918-a2de-432d-892c-397b61336fe4 rw intel_iommu=on iommu=pt hugepages=4608

So yeah...

I also tried to reinstall the kernel after i used

$ arch-chroot

with

$ pacman -S linux

This did not help either.

Does anyone have any ideas?

Last edited by Eispala (2018-05-14 18:58:06)

jasonwryan

Anarchist

From: .nz

Registered: 2009-05-09

Posts: 30,426

Website

Re: [SOLVED] Can't boot after upgrade -> /lib/[]/modules.devname not found

Your /boot was likely unmounted during the upgrade.

---

Arch + dwm   •   Mercurial repos  •   Surfraw

Registered Linux User #482438

Eispala

Member

Registered: 2018-05-13

Posts: 6

Re: [SOLVED] Can't boot after upgrade -> /lib/[]/modules.devname not found

When i issue

$ lsblk

i get:

[...]
sdd      8:48   0 111,8G  0 disk 
├─sdd1   8:49   0     1G  0 part /boot
├─sdd2   8:50   0     8G  0 part [SWAP]
└─sdd3   8:51   0 102,8G  0 part /

Doesn't that mean my boot partition is mounted in /boot?

$ ls -lsh /boot

Gives me:

4,0K drwxr-xr-x 4 root root 4,0K 14. Apr 18:26 EFI
 27M -rwxr-xr-x 1 root root  27M 21. Apr 14:35 initramfs-linux-fallback.img
7,0M -rwxr-xr-x 1 root root 7,0M 21. Apr 14:35 initramfs-linux.img
7,0M -rwxr-xr-x 1 root root 7,0M 14. Apr 18:32 initramfs-linux-pass.img
1,6M -rwxr-xr-x 1 root root 1,6M 14. Mär 21:17 intel-ucode.img
4,0K drwxr-xr-x 3 root root 4,0K 14. Apr 18:26 loader
5,0M -rwxr-xr-x 1 root root 5,0M  1. Apr 03:01 vmlinuz-linux

With the live iso i did:

mount /dev/sdd3 /mnt
mount /dev/sdd1 /mnt/boot

arch-chroot /mnt

mkinitcpio -p linux

Last edited by Eispala (2018-05-13 19:05:18)

jasonwryan

Anarchist

From: .nz

Registered: 2009-05-09

Posts: 30,426

Website

Re: [SOLVED] Can't boot after upgrade -> /lib/[]/modules.devname not found

From the chroot, make sure /boot is mounted and reinstall linux (not just rebuild the initrd). Your `pacman -Q linux` and `uname -r` should then report the same kernel.

---

Arch + dwm   •   Mercurial repos  •   Surfraw

Registered Linux User #482438

Eispala

Member

Registered: 2018-05-13

Posts: 6

Re: [SOLVED] Can't boot after upgrade -> /lib/[]/modules.devname not found

Yeah i did that (I just editet the entry post)

I did the following (When i reinstalled the kernel):

mount /dev/sdd3 /mnt
mount /dev/sdd1 /mnt/boot

arch-chroot /mnt

pacman -S linux

mkinitcpio -p linux

I made sure, that there are my bootloader config files in /boot/loader/entries when i chrooted in the live iso.

Thats why i didn't know what to do anymore.

EDIT:
I found a post (https://bbs.archlinux.org/viewtopic.php … 7#p1415627) which sounds like it could have to do something with my problem, as the OP has the same "different kernel versions" issue as i have.

$ pacman -Qi linux

did actually return the new kernel.
But

$ uname -r

did not. (Even after i rebooted)

But i didn't really get what to do

Last edited by Eispala (2018-05-13 19:14:48)

mrunion

Member

From: Jonesborough, TN

Registered: 2007-01-26

Posts: 1,938

Website

Re: [SOLVED] Can't boot after upgrade -> /lib/[]/modules.devname not found

Check the UUID's of the devices. It sounds like you do not have the "proper" /boot mounted when doing a Kernel upgrade. If you get a different running Kernel (shown with uname -a) than what you just installed from pacman (and after a reboot), you either do not have /boot mounted, or you are not booting from where you think you are.

---

Matt

"It is very difficult to educate the educated."

Eispala

Member

Registered: 2018-05-13

Posts: 6

Re: [SOLVED] Can't boot after upgrade -> /lib/[]/modules.devname not found

This is my /etc/fstab:

# Static information about the filesystems.
# See fstab(5) for details.

# <file system> <dir> <type> <options> <dump> <pass>
# /dev/sdc3
UUID=3ace7ab6-baac-4752-a80e-881091359045	/         	ext4      	rw,defaults,noatime,discard	0 1

# /dev/sdc1
UUID=4790-7878      	/boot     	vfat      	rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro	0 2

# /dev/sdc2
UUID=3f4edb19-8077-4c50-8229-74bf98dfe90e	none      	swap      	defaults,noatime,discard0 0

So the UUIDs I'm looking for are:
3ace7ab6-baac-4752-a80e-881091359045    (Root)
4790-7878 (Boot)
3f4edb19-8077-4c50-8229-74bf98dfe90e     (Swap)

The output of

$ lsblk -f

is:

NAME   FSTYPE LABEL             UUID                                 MOUNTPOINT
sda                                                                  
├─sda1 ntfs                     96BCC587BCC5627B                     
├─sda2 vfat                     D4C7-1DD5                            
├─sda3                                                               
├─sda4 ntfs   Image             B60C05670C0523C9                     
├─sda5 ntfs   Volume            C68E845F8E844A3F                     
├─sda6 swap   p_swap            345567fa-5d97-42e1-8727-bf034ef12edc 
└─sda7 vfat   EFIBOOT           2525-4949                            
sdb                                                                  
├─sdb1                                                               
└─sdb2 ntfs   EVO2              9A60E74260E723AB                     
sdc                                                                  
├─sdc1                                                               
└─sdc2 ntfs   EVO1              F8E4DFD6E4DF956A                     
sdd                                                                  
├─sdd1 vfat                     4790-7878                            /boot
├─sdd2 swap                     3f4edb19-8077-4c50-8229-74bf98dfe90e [SWAP]
└─sdd3 ext4                     3ace7ab6-baac-4752-a80e-881091359045 /
sde                                                                  
├─sde1 ntfs   Wiederherstellung 7E8E321C8E31CD81                     
├─sde2 vfat                     4632-2F12                            
├─sde3                                                               
└─sde4 ntfs                     F6BC33F6BC33AFCB 

Which checks out with the /etc/fstab config.

I also checked the other efi partitions, but there are no installed bootloaders / bootloader config files in them.

you either do not have /boot mounted, or you are not booting from where you think you are

I have this passthrough configuration, with some special kernel parameters.

When i boot the machine, these are active:

$ cat /proc/cmdline

Result:

initrd=\intel-ucode.img initrd=\initramfs-linux-pass.img root=PARTUUID=7a63b918-a2de-432d-892c-397b61336fe4 rw intel_iommu=on iommu=pt hugepages=4608

The PARTUUID of my /dev/sdd3 is:

$ blkid /dev/sdd3

Result:

/dev/sdd3: UUID="3ace7ab6-baac-4752-a80e-881091359045" TYPE="ext4" PARTLABEL="root" PARTUUID="7a63b918-a2de-432d-892c-397b61336fe4"

The fact that these are loaded, and the other efi partitions are not configured, means i am booted from the right partition, doesn't it?

And as you can see, when i issue

$ lsblk

it shows

[...]
sdd1 vfat                     4790-7878                            /boot

And 4790-7878 is the UUID of /boot in my /etc/fstab

This is the way i tried to "make sure i am booting from where i should".

Is this the wrong way? Did i do some wrong conclusions?

Scimmia

Fellow

Registered: 2012-09-01

Posts: 13,119

Re: [SOLVED] Can't boot after upgrade -> /lib/[]/modules.devname not found

That error is coming from the initramfs. It doesn't match the kernel you're loading.

Eispala

Member

Registered: 2018-05-13

Posts: 6

Re: [SOLVED] Can't boot after upgrade -> /lib/[]/modules.devname not found

How can i solve this Problem?

Reinstalling The Kernel in arch-chrooted system, and doing mkinitcpio -p linux + bootctl -update does not help.

When running Pacman - Syu it says there is nothing to do, and The installed Kernel Version is 4.16.8-1-ARCH.

But uname -r Reports Kernel 4.16.5-1-ARCH.

$ file /boot/vmlinuz-linux 

Says the Version is 4.16.8-1-ARCH from may 9th.

Edith: under /lib/modules/4.16.8-1-ARCH there is a modules.devname file

Edit2: There are no files in /boot If i Only mount /Dev/sdd3(root Partition)

If i mount /Dev/sdd1 to /mnt/boot, my Bootfiles are there

I'm typing on my mobile because The System is broke n again, please disregard Text mistakes

Edit 3:
I fixed it.

Just to make sure other people with the same problem can get some information:

TLDR; In the chrooted system i reinstalled the bootloader.

What i did:

1. Boot live System

$ mount /dev/sdd3 /mnt
$ mount /dev/sdd1 /mnt/boot
$ swapon /dev/sdd2

$ rm -rf /mnt/boot

$ arch-chroot /mnt

$ bootctl install

$ pacman -S linux

$ pacman -S intel-ucode

Then i created a new bootloader entry in /boot/loader/entries/, named arch-uefi-pass.conf

The content:

Titel = Arch Linux Pass
linux vmlinuz-linux
initrd /intel-ucode.img
initrd /initramfs-linux.img

options root=PARTUUID=7a63b918-a2de-432d-892c-397b61336fe4 rw intel_iommu=on iommu=pt hugepages=4608

Then i edited the new loader.conf in /boot/loader/loader.conf

default = arch-*

Then:

$ mkinitcpio -p linux

$ exit

$ umount -R /mnt

$ reboot

I hope i do not have to do this every time i upgrade my kernel

Last edited by Eispala (2018-05-14 19:44:11)

mrunion

Member

From: Jonesborough, TN

Registered: 2007-01-26

Posts: 1,938

Website

Re: [SOLVED] Can't boot after upgrade -> /lib/[]/modules.devname not found

I hope i do not have to do this every time i upgrade my kernel

You won't as long as your ESP is mounted correctly, and available when a Kernel upgrade occurs. Verify you're is correctly mounted by rechecking your setup against the Wiki.

Also, if the problem is "fixed" now, see what is currently in the /boot location on your machine when you are logged in as you normally run the machine.

---

Matt

"It is very difficult to educate the educated."

Eispala

Member

Registered: 2018-05-13

Posts: 6

Re: [SOLVED] Can't boot after upgrade -> /lib/[]/modules.devname not found

When i am in my system, and issue:

$ ls -lsh /boot

The output is:

4,0K drwxr-xr-x 4 root root 4,0K 14. Mai 20:28 EFI
 28M -rwxr-xr-x 1 root root  28M 14. Mai 20:40 initramfs-linux-fallback.img
7,2M -rwxr-xr-x 1 root root 7,2M 14. Mai 20:40 initramfs-linux.img
1,6M -rwxr-xr-x 1 root root 1,6M  7. Mai 23:11 intel-ucode.img
4,0K drwxr-xr-x 3 root root 4,0K 14. Mai 20:28 loader
5,0M -rwxr-xr-x 1 root root 5,0M  9. Mai 14:21 vmlinuz-linux

When i unmount /boot:

$ sudo umount /boot

and do

$ ls -lsh /boot

again, the output is nothing.

If i issue

$ cat /proc/cmdline

The result is:

initrd=\intel-ucode.img initrd=\initramfs-linux.img root=PARTUUID=7a63b918-a2de-432d-892c-397b61336fe4 rw intel_iommu=on iommu=pt hugepages=4608

Which is the right configuration, with the right kernel parameters. And this config exists only on a single partition.

In this answer, i showed how i tried to make sure that i mounted the right ESP: https://bbs.archlinux.org/viewtopic.php … 5#p1785325

If this was the wrong way, could you tell me the right way?

If this was the right way (or one of the right ways), then my problem probably was not the unmounted boot partition, because it's the only configured boot partition.

Maybe i did something wrong when i initially installed the system and the bootloader?