Arch Linux

eichan

Member

Registered: 2018-01-08

Posts: 31

[SOLVED] Boot failure: Failed to start Load Kernel Modules

Hi all,

I can't boot my Arch system anymore:

[FAILED] Failed to start Load Kernel Modules
See 'systemctl status systemd-modules-load.service' for details.

And a bit further:

[FAILED] Failed to mount /boot/efi.
See 'systemctl status boot-efi.mount' for details.
[DEPEND] Dependency failed for Local File Systems.

Then I get a message saying I am in emergency mode and asking for my root password.

Here is the result of the commands suggested:

[tan@bobby ~]$ systemctl status systemd-modules-load.service
● systemd-modules-load.service - Load Kernel Modules
   Loaded: loaded (/usr/lib/systemd/system/systemd-modules-load.service; static; vendor preset: disabled)
1   Active: failed (Result: exit-code) since Sun 2018-01-28 02:10:16 NZDT; 7min ago
     Docs: man:systemd-modules-load.service(8)
           man:modules-load.d(5)
  Process: 224 ExecStart=/usr/lib/systemd/systemd-modules-load (code=exited, status=1/FAILURE)
 Main PID: 224 (code=exited, status=1/FAILURE)

Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.

[tan@bobby ~]$ systemctl status boot-efi.mount
● boot-efi.mount - /boot/efi
   Loaded: loaded (/etc/fstab; generated)
   Active: failed (Result: exit-code) since Sun 2018-01-28 02:10:17 NZDT; 7min ago
   Where: /boot/efi
   What: /dev/disk/by-uuid/F62F-E8CB
     Docs: man:fstab(5)
           man:systemd-fstab-generator(8)
  Process: 227 ExecStart=/usr/bin/mount /dev/disk/by-uuid/F62F-E8CB /boot/efi -t vfat -o rw,realtime,fmask-0022,codepage=437,iocharset=iso-8859-1,shortname=mixed,utf8,errors=remount-ro (code=exited, status=32)

I tried restarting with systemctl status systemd-modules-load.service but it says the control process exited. For boot-efi.mount, it just says mount failed. I notice that many modules must be missing. For example I don't even see my wireless interface.

The only thing I can think of that could have caused this is that I created a package, and then installed it with pacman -U --force. I did this because I had previously installed that software from source without creating a package and wanted a "cleaner" install of it. I guess I was wrong, or I screwed up the package creation part...

Can anyone help me troubleshoot this?

Last edited by eichan (2018-01-28 17:58:38)

loafer

Member

From: the pub

Registered: 2009-04-14

Posts: 1,772

Re: [SOLVED] Boot failure: Failed to start Load Kernel Modules

Which package was it you created and forced the install? Post the package build please.

---

All men have stood for freedom...
For freedom is the man that will turn the world upside down.
Gerrard Winstanley.

Scimmia

Fellow

Registered: 2012-09-01

Posts: 12,093

Re: [SOLVED] Boot failure: Failed to start Load Kernel Modules

What you're seeing usually happens when you're booting the wrong kernel, such as when /boot wasn't mounted when you updated the system. Check uname -r vs pacman -Q <kernelpackage>

eichan

Member

Registered: 2018-01-08

Posts: 31

Re: [SOLVED] Boot failure: Failed to start Load Kernel Modules

The package I was trying to build is libindi_3rdparty, which is a set of drivers for the INDI library (for astronomical devices such as mounts or CCD cameras). This package exists in the AUR but it's an outdated version, and the source downloaded from the author's Github page is missing a file (log4z.h) in the indi-qhy folder. This is the reason my PKGBUILD (see here) is pointing at a local source. That tarball on my local drive is just the exact same source as found on Github, just with the added log4z.h file in the right folder. makepkg and pacman -U pkg.tar.gz seemed happy with it. I uninstalled the package as root under the emergency environment provided by the failed boot, and it didn't change anything.

As for uname -a vs paman -Qi linux, both commands give me the same version number (4.14.15-1). Trying the fallback kernel on the grub menu doesn't help. Exact same problem.

eichan

Member

Registered: 2018-01-08

Posts: 31

Re: [SOLVED] Boot failure: Failed to start Load Kernel Modules

I'll just add to this a photo taken of my screen as it fails its boot session. Sorry it's an image file. The emergency environment doesn't know anything about my wifi adapter (does not appear with 'ip link'), so it's a bit challenging to share some results and logs.

robg

Member

Registered: 2015-03-05

Posts: 219

Re: [SOLVED] Boot failure: Failed to start Load Kernel Modules

1. What is your bootloader? Could you please post the bootloader configuration related to you Arch install?
2. Please provide the outputs of `lsblk` and `cat /etc/fstab`.
3. Does reinstalling the kernel and/or bootloader help with the issue? (of course after uninstalling libindi_3rdparty)

eichan

Member

Registered: 2018-01-08

Posts: 31

Re: [SOLVED] Boot failure: Failed to start Load Kernel Modules

1. I use grub. Here is grub.cfg.

2. lsblk

NAME        MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
nvme0n1     259:0    0   477G  0 disk 
├─nvme0n1p1 259:1    0   500M  0 part 
├─nvme0n1p2 259:2    0   128M  0 part 
├─nvme0n1p3 259:3    0 191.6G  0 part 
└─nvme0n1p4 259:4    0 284.7G  0 part /

EDIT: p1 is the /boot partition, p2 is some Windows obscure partition, p3 is my Windows partition and p4 the Arch one. I don't have swap.

cat /etc/fstab

# Static information about the filesystems.
# See fstab(5) for details.

# <file system> <dir> <type> <options> <dump> <pass>
# /dev/nvme0n1p4
UUID=278106c9-fa11-4d83-b2b6-f849cf8b8a7d	/         	ext4      	rw,relatime,data=ordered	0 1

# /dev/nvme0n1p1 LABEL=ESP
UUID=F62F-E8CB      	/boot/efi 	vfat      	rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro	0 2

3. 'pacman -S linux' exits with an error:

resolving dependencies...
looking for conflicting packages...

Packages (1) linux-4.14.15-1

Total Installed Size:  101.19 MiB
Net Upgrade Size:        0.00 MiB

checking keyring...
checking package integrity...
loading package files...
checking for file conflicts...
checking available disk space...
:: Processing package changes...
reinstalling linux...
:: Running post-transaction hooks...
(1/3) Updating linux module dependencies...
depmod: ERROR: could not open directory /lib/modules/4.14.15-1-ARCH: No such file or directory
depmod: FATAL: could not search modules: No such file or directory
(2/3) Updating linux initcpios...
==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'default'
  -> -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux.img
==> ERROR: '/lib/modules/4.14.15-1-ARCH' is not a valid kernel module directory
==> Building image from preset: /etc/mkinitcpio.d/linux.preset: 'fallback'
  -> -k /boot/vmlinuz-linux -c /etc/mkinitcpio.conf -g /boot/initramfs-linux-fallback.img -S autodetect
==> ERROR: '/lib/modules/4.14.15-1-ARCH' is not a valid kernel module directory
(3/3) Arming ConditionNeedsUpdate...

I thought grub wasn't related because it still works in the sense that it tries to run Arch and it runs Windows without a problem. And its installation is the trickiest... Do you confirm, after seeing the above, that it would be useful to try reinstalling it?

EDIT: Unsure if that's relevant but I should inform that I'm running all those commands as root, from the emergency mode.

Last edited by eichan (2018-01-28 16:53:52)

seth

Member

Registered: 2012-09-03

Posts: 58,791

Re: [SOLVED] Boot failure: Failed to start Load Kernel Modules

Should have been

lsblk -f

to compare the UUIDs

---

How to upload text · How to boot w/o GUI · Disable Windows Fast-Start! · Fix your xinitrc

Scimmia

Fellow

Registered: 2012-09-01

Posts: 12,093

Re: [SOLVED] Boot failure: Failed to start Load Kernel Modules

What does `ls -l /lib` give you?

eichan

Member

Registered: 2018-01-08

Posts: 31

Re: [SOLVED] Boot failure: Failed to start Load Kernel Modules

lsblk -f

NAME        FSTYPE LABEL UUID                                 MOUNTPOINT
nvme0n1                                                       
├─nvme0n1p1 vfat   ESP   F62F-E8CB                            
├─nvme0n1p2                                                   
├─nvme0n1p3 ntfs   OS    CEFC69F9FC69DBE9                     
└─nvme0n1p4 ext4         278106c9-fa11-4d83-b2b6-f849cf8b8a7d /

eichan

Member

Registered: 2018-01-08

Posts: 31

Re: [SOLVED] Boot failure: Failed to start Load Kernel Modules

ls -l /lib

 ls: cannot access '/lib': No such file or directory

There is a /lib64, so 'ls -l /lib64':

lrwxrwxrwx 1 root root 7 Oct 17 20:32 /lib64 -> usr/lib

And /usr/lib is full of library, here is the 'ls -l /usr/lib' in case it's relevant.

loqs

Member

Registered: 2014-03-06

Posts: 18,055

Re: [SOLVED] Boot failure: Failed to start Load Kernel Modules

Just to be sure what is the output of

$ stat /lib /lib64 /usr/lib

eichan

Member

Registered: 2018-01-08

Posts: 31

Re: [SOLVED] Boot failure: Failed to start Load Kernel Modules

stat /lib /lib64 /usr/lib

  File: /lib64 -> usr/lib
  Size: 7         	Blocks: 0          IO Block: 4096   symbolic link
Device: 10304h/66308d	Inode: 14          Links: 1
Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2018-01-29 18:09:55.006666738 +1300
Modify: 2017-10-17 20:32:47.000000000 +1300
Change: 2018-01-05 17:19:30.660587086 +1300
 Birth: -
  File: /usr/lib
  Size: 196608    	Blocks: 392        IO Block: 4096   directory
Device: 10304h/66308d	Inode: 12453953    Links: 206
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2018-01-29 18:17:02.270828858 +1300
Modify: 2018-01-27 21:01:14.050616206 +1300
Change: 2018-01-27 21:01:14.050616206 +1300
 Birth: -

loqs

Member

Registered: 2014-03-06

Posts: 18,055

Re: [SOLVED] Boot failure: Failed to start Load Kernel Modules

Please reinstall the filesystem package that should create the symlink from /lib to /usr/lib and check the output of the command again.
Edit:
Was expecting the following output for stat /lib (assuming it was omitted)

stat: cannot stat '/lib': No such file or directory

Last edited by loqs (2018-01-28 17:26:41)

Scimmia

Fellow

Registered: 2012-09-01

Posts: 12,093

Re: [SOLVED] Boot failure: Failed to start Load Kernel Modules

So somehow, your /usr symlink got deleted. It should point to usr/lib, same as /lib64. This is the cause of all of your problems.

eichan

Member

Registered: 2018-01-08

Posts: 31

Re: [SOLVED] Boot failure: Failed to start Load Kernel Modules

Reinstalling the filesystem package solved the problem. And indeed, the No such file or directory showed was omitted by the '>' pipe.

For information, the output of the command (now run as normal user) is:

tan@bobby ~$ stat /lib /lib64 /usr/lib
  File: /lib -> usr/lib
  Size: 7         	Blocks: 0          IO Block: 4096   symbolic link
Device: 10304h/66308d	Inode: 13          Links: 1
Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2018-01-29 19:53:11.046666744 +1300
Modify: 2017-10-17 20:32:47.000000000 +1300
Change: 2018-01-29 19:52:07.698039494 +1300
 Birth: -
  File: /lib64 -> usr/lib
  Size: 7         	Blocks: 0          IO Block: 4096   symbolic link
Device: 10304h/66308d	Inode: 14          Links: 1
Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2018-01-29 19:52:07.818039500 +1300
Modify: 2017-10-17 20:32:47.000000000 +1300
Change: 2018-01-29 19:52:07.698039494 +1300
 Birth: -
  File: /usr/lib
  Size: 196608    	Blocks: 392        IO Block: 4096   directory
Device: 10304h/66308d	Inode: 12453953    Links: 206
Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2018-01-29 19:52:07.711372828 +1300
Modify: 2018-01-29 19:52:07.698039494 +1300
Change: 2018-01-29 19:52:07.698039494 +1300
 Birth: -

So do you know how this could have happened? Is my PKGBUILD completely wrong? Maybe by pacman -R libindi_3rdparty is somehow deleted the /usr simlink?

Many thanks to all.

Scimmia

Fellow

Registered: 2012-09-01

Posts: 12,093

Re: [SOLVED] Boot failure: Failed to start Load Kernel Modules

If you used --force to install it, it could have. If it contained files in /lib, using --force would have overwritten the symlink with a dir and everything went to hell from there.

eichan

Member

Registered: 2018-01-08

Posts: 31

Re: [SOLVED] Boot failure: Failed to start Load Kernel Modules

Aaaah OK it all makes sense now. Lesson learned. Thanks again.