Unable to boot after kernel update

pataf · 2025-01-23 20:27:22

Hi everyone,
Yesterday I updated my system as I usually do almost daily and today I can't even boot.

In the live arch usb I mounted my root partition with

 mount /dev/nvme0n1p2 /mnt

my boot partition with

 mount /dev/nvme0n1p1 /mnt/efi

my home partition with

 mount /dev/nvme0n1p3 /mnt/home

and then used

arch-chroot /mnt

to chroot into it.
Inside the chroot I tried to reinstall the kernel with

pacman -S linux

The linux version is trying to install is the 6.12.10.
When "running the post-transaction hooks" I get the following error:

(1/4) Arming ConditionNeedsUpdate...
(2/4) Updating module dependencies...
depmod: ERROR: could not open directory /lib/modules/6.2.8-arch1-1: No such file or directory
depmod: FATAL: could not search modules: No such file or directory
error: command failed to execute correctly
(3/4) Updating Linux initcpios...
call fo execv failed (Exec format error)
error: command failed to execute correctly
(4/4) Refresh PackageKit...
Error connecting: Could not connect: No such file or directory

Currently stuck here, looked around but didn't find much related to this.
Thanks for your help!

Last edited by pataf (2025-01-23 20:53:17)

mackin_cheese · 2025-01-23 20:50:36

The latest kernel for Arch is 6.12.10. Why are you on 6.2.8?

pataf · 2025-01-23 20:54:52

mackin_cheese wrote:

The latest kernel for Arch is 6.12.10. Why are you on 6.2.8?

Seems like I got confused with the error message.
It is trying to install 6.12.10 indeed, but the error mentions 6.2.8

mackin_cheese · 2025-01-23 21:02:39

what is your output of

 lsmod

pataf · 2025-01-23 21:21:47

mackin_cheese wrote:

what is your output of
 lsmod 

here's the output: https://katb.in/lijojelepot

seth · 2025-01-24 13:47:08

From the chroot:

uname -a
pacman -Qs kernel
file /boot/vmlinuz-linux*

an please also post the complete "pacman -S linux" output, you can

pacman -S linux |& tee /tmp/pacman_S.txt; cat /tmp/pacman_S.txt curl -F 'file=@-' 0x0.st

musbur · 2025-01-24 16:34:05

I once had a kernel / module mismatch when I updated linux while for some reason the EFI partition wasn't mounted. So the new kernel and initrd went into a directory on the root file system, not the EFI partition. On reboot, the old kernel was loaded from EFI and couldn't find its matching modules in /usr/lib/modules. This may be something similar.

pataf · 2025-01-24 18:22:51

seth wrote:

From the chroot:
uname -a
pacman -Qs kernel
file /boot/vmlinuz-linux*
an please also post the complete "pacman -S linux" output, you can
pacman -S linux |& tee /tmp/pacman_S.txt; cat /tmp/pacman_S.txt curl -F 'file=@-' 0x0.st

from chroot "uname -a" return nothing.

here is the output ot the others commands:
pacman -Qs kernel: https://katb.in/udicateyemo
file /boot/wmlinuz-liux*: https://katb.in/ceniqalekon
pacman -S linux: https://katb.in/ajiwayodiba

seth · 2025-01-24 18:29:14

/boot/vmlinuz-linux: Linux kernel x86 boot executable, bzImage, version 6.4.7-arch1-1

Forgot to mount a boot partition or is the ESP actually your boot partition and belongs into /boot ?

ls -lR /mnt/efi # resp. from the chroot "ls -lR /efi"

But another, major issue is

ldconfig: File /usr/lib32/libfontconfig.so.1 is empty, not checked.
ldconfig: File /usr/lib32/libfontconfig.so.1.15.0 is empty, not checked.
ldconfig: File /usr/lib32/libfontconfig.so is empty, not checked.
ldconfig: File /usr/lib/libbrlapi.so.0.8 is empty, not checked.
ldconfig: File /usr/lib/libbrlapi.so.0.8.6 is empty, not checked.
ldconfig: File /usr/lib/libnm.so.0 is empty, not checked.
ldconfig: File /usr/lib/libbrlapi.so is empty, not checked.
ldconfig: File /usr/lib/libnbd.so.0 is empty, not checked.
ldconfig: File /usr/lib/libnbd.so.0.0.0 is empty, not checked.
ldconfig: File /usr/lib/libfontconfig.so.1 is empty, not checked.
ldconfig: File /usr/lib/libnbd.so is empty, not checked.
ldconfig: File /usr/lib/libnm.so.0.1.0 is empty, not checked.
ldconfig: File /usr/lib/libnm.so is empty, not checked.
ldconfig: File /usr/lib/libfontconfig.so.1.15.0 is empty, not checked.
ldconfig: File /usr/lib/libfontconfig.so is empty, not checked.

sudo LC_ALL=C pacman -Qkk |& grep -v ', 0 altered files' | grep -v backup

pataf · 2025-01-24 18:29:16

musbur wrote:

I once had a kernel / module mismatch when I updated linux while for some reason the EFI partition wasn't mounted. So the new kernel and initrd went into a directory on the root file system, not the EFI partition. On reboot, the old kernel was loaded from EFI and couldn't find its matching modules in /usr/lib/modules. This may be something similar.

Thanks for the advice, do you remember how did you check and fix that?

seth · 2025-01-24 23:19:06

You have broken libraries on your disk, your issue goes far beyond the dated kernel in /boot - since it gets copied there by an alpm hook that might have already failed.
You need to address the wider package corruption.

pataf · 2025-01-25 09:28:21

seth wrote:

/boot/vmlinuz-linux: Linux kernel x86 boot executable, bzImage, version 6.4.7-arch1-1

Forgot to mount a boot partition or is the ESP actually your boot partition and belongs into /boot ?

ls -lR /mnt/efi # resp. from the chroot "ls -lR /efi"

But another, major issue is

ldconfig: File /usr/lib32/libfontconfig.so.1 is empty, not checked.
ldconfig: File /usr/lib32/libfontconfig.so.1.15.0 is empty, not checked.
ldconfig: File /usr/lib32/libfontconfig.so is empty, not checked.
ldconfig: File /usr/lib/libbrlapi.so.0.8 is empty, not checked.
ldconfig: File /usr/lib/libbrlapi.so.0.8.6 is empty, not checked.
ldconfig: File /usr/lib/libnm.so.0 is empty, not checked.
ldconfig: File /usr/lib/libbrlapi.so is empty, not checked.
ldconfig: File /usr/lib/libnbd.so.0 is empty, not checked.
ldconfig: File /usr/lib/libnbd.so.0.0.0 is empty, not checked.
ldconfig: File /usr/lib/libfontconfig.so.1 is empty, not checked.
ldconfig: File /usr/lib/libnbd.so is empty, not checked.
ldconfig: File /usr/lib/libnm.so.0.1.0 is empty, not checked.
ldconfig: File /usr/lib/libnm.so is empty, not checked.
ldconfig: File /usr/lib/libfontconfig.so.1.15.0 is empty, not checked.
ldconfig: File /usr/lib/libfontconfig.so is empty, not checked.

sudo LC_ALL=C pacman -Qkk |& grep -v ', 0 altered files' | grep -v backup

I mounted the boot parrtition in "/mnt/efi" there was not an efi folder in "/mnt/boot".
Checked the 2 fodlers mentioned inside chroot and I got this:

# ls /boot
amd-ucode.img
initramfs-linux.img
initramfs-linux-fallback.img
vmlinuz-linux

# ls /efi
EFI
amd-ucode.img
initramfs-linux.img
initramfs-linux-fallback.img
loader
vmlinuz-linux

from outside chroot:

# stat mnt/boot/initramfs-linux.img
  File: /mnt/boot/initramfs-linux.img
  Size: 9565425   	Blocks: 18688      IO Block: 4096   regular file
Device: 259,2	Inode: 284629      Links: 1
Access: (0600/-rw-------)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2023-07-31 18:29:39.827323129 +0000
Modify: 2023-07-31 18:29:39.873990234 +0000
Change: 2023-07-31 18:29:39.883990328 +0000
 Birth: 2023-07-31 18:29:39.827323129 +0000

# stat /mnt/efi/initramfs-linux.img
  File: /mnt/efi/initramfs-linux.img
  Size: 9197748   	Blocks: 17968      IO Block: 4096   regular file
Device: 259,1	Inode: 11          Links: 1
Access: (0755/-rwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2025-01-21 00:00:00.000000000 +0000
Modify: 2025-01-21 20:10:00.000000000 +0000
Change: 2025-01-21 20:10:00.000000000 +0000
 Birth: 2025-01-21 20:10:00.360000000 +0000

the one in /efi seems to be the one updated recently, verion 6.12.10.
Could it be a mountig issue?

seth · 2025-01-25 09:35:59

Please avoid bloating the thread with pointless full quotes.

Yes, it looks like you're booting from the partition you've now mounted into /mnt/efi and when installing the kernel from the chroot, you've to mount that partition into /mnt/boot
You also want to clean up the boot path to avoid further confusion.

While that explains the depmod behavior, there are still broken libraries
You have to address that.

pataf · 2025-01-25 09:54:32

seth wrote:

Please avoid bloating the thread with pointless full quotes.
Yes, it looks like you're booting from the partition you've now mounted into /mnt/efi and when installing the kernel from the chroot, you've to mount that partition into /mnt/boot
You also want to clean up the boot path to avoid further confusion.

thanks, I'll try to mount to /mnt/boot instead and retry

seth wrote:

While that explains the depmod behavior, there are still broken libraries
You have to address that.

here's the output from the "pacman -Qkk" you shared: https://katb.in/omoteyaboya
what's the best way to fix the broken libraries?

pataf · 2025-01-25 10:16:33

After mounting correctly the boot partition it still gives me the same depmod issue

pataf · 2025-01-25 13:44:07

Managed to fix the broken libraries and now at least it displays the login screen.
In the live chroot I still get the same depmod error tho, even tried updating the live usb to latest install.

After inserting the password in login screen wayland crashes immediately while X11 get stuck in the loading screen after.
Looked at journalctl and found this: https://katb.in/igofiluyowe
Seems like dnsmasq is going into timeout, maybe because of dbus connection pending

Last edited by pataf (2025-01-25 13:44:42)

seth · 2025-01-25 13:58:59

Managed to fix the broken libraries but now it displays the login screen.

How?

brltty: no mtree file
coreutils: no mtree file
discord: no mtree file
firefox: no mtree file
fontconfig: no mtree file
hugo: no mtree file
jre-openjdk: no mtree file
lib32-fontconfig: no mtree file
libnbd: no mtree file
libnm: no mtree file
networkmanager: no mtree file
telegram-desktop: no mtree file
unzip: no mtree file
wine-staging: no mtree file

The packages are massively broken and need to be fixd in two stages

pacman -S --dbonly brltty coreutils discord firefox fontconfig hugo jre-openjdk lib32-fontconfig libnbd libnm networkmanager telegram-desktop unzip wine-staging
pacman -S brltty coreutils discord firefox fontconfig hugo jre-openjdk lib32-fontconfig libnbd libnm networkmanager telegram-desktop unzip wine-staging

pataf · 2025-01-25 14:21:33

seth wrote:

How?

Searched online and came to the solution you suggested, thank you
Here's the new output of "pacman -Qkk |& grep -v ', 0 altered files' | grep -v backup": https://katb.in/uqolificoci

seth · 2025-01-25 14:23:20

Ok, then we'll have to see what's actually broken.
Please post your complete system journal for the boot:

sudo journalctl -b | curl -F 'file=@-' 0x0.st

pataf · 2025-01-25 14:43:30

seth wrote:

Ok, then we'll have to see what's actually broken.
Please post your complete system journal for the boot:
sudo journalctl -b | curl -F 'file=@-' 0x0.st

Here's the output, I tried to log in with both X11 and Wayland: https://katb.in/rubakagebay
btw, now kernel seems fine, in chroot "uname -a" gives me the correct version 6.12.10

pataf · 2025-01-25 15:01:13

Update: seems this is the correct log, the one linked before should be the live boot one.
https://katb.in/eqeqenozuqo

seth · 2025-01-25 15:52:58

Ftr, that service gets you the plain text by appending "/raw" to the url - the html stuff is seriously not helpful.

This looks bad

Jan 25 15:48:37 home bluetoothd[704]: Failed to set mode: Failed (0x03)
Jan 25 15:48:37 home kernel: amdgpu 0000:08:00.0: amdgpu: SMU: response:0xFFFFFFFF for index:40 param:0x00000000 message:AllowGfxOff?
Jan 25 15:48:37 home kernel: amdgpu 0000:08:00.0: amdgpu: Failed to enable gfxoff!
Jan 25 15:48:37 home lact[784]: thread 'main' panicked at lact-daemon/src/server/gpu_controller/amd.rs:247:26:
Jan 25 15:48:37 home lact[784]: Could not get temperature by given key
Jan 25 15:48:37 home lact[784]: note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Jan 25 15:48:37 home kernel: amdgpu 0000:08:00.0: amdgpu: SMU: response:0xFFFFFFFF for index:18 param:0x00000005 message:TransferTableSmu2Dram?
Jan 25 15:48:37 home kernel: amdgpu 0000:08:00.0: amdgpu: Failed to export SMU metrics table!
Jan 25 15:48:38 home kernel: amdgpu 0000:08:00.0: amdgpu: SMU: response:0xFFFFFFFF for index:40 param:0x00000000 message:AllowGfxOff?
Jan 25 15:48:38 home kernel: amdgpu 0000:08:00.0: amdgpu: Failed to enable gfxoff!

Primarily the amdgpu stuff.
Try to disable lact and if that doesn't cut it: the LTS kernel.

pataf · 2025-01-25 17:02:55

seth wrote:

Try to disable lact and if that doesn't cut it: the LTS kernel.

Disabling lact doesn't change the result, still crashes after login
Here's a new log with maybe more info: https://katb.in/enavemuyayi/raw (thanks for the /raw tip)
I'll try with the lts kernel

seth · 2025-01-25 17:34:15

Jan 25 17:05:04 home sddm[771]: Authentication for user  "erikdrozina"  successful
Jan 25 17:05:04 home sddm-helper[926]: pam_unix(sddm:session): session opened for user erikdrozina(uid=1000) by erikdrozina(uid=0)
Jan 25 17:05:04 home systemd-logind[713]: New session 2 of user erikdrozina.
Jan 25 17:05:04 home systemd[1]: Started Session 2 of User erikdrozina.
Jan 25 17:05:07 home ksplashqml[1007]: file:///home/erikdrozina/.local/share/plasma/look-and-feel/com.github.vinceliuice.Layan/contents/splash/Splash.qml:89: ReferenceError: bottomRect is not defined
Jan 25 17:05:07 home ksplashqml[1007]: file:///home/erikdrozina/.local/share/plasma/look-and-feel/com.github.vinceliuice.Layan/contents/splash/Splash.qml:88: ReferenceError: bottomRect is not defined
Jan 25 17:05:07 home ksplashqml[1007]: file:///home/erikdrozina/.local/share/plasma/look-and-feel/com.github.vinceliuice.Layan/contents/splash/Splash.qml:89: ReferenceError: bottomRect is not defined
Jan 25 17:05:13 home kwin_wayland[1004]: kwin_wayland_drm: Pageflip timed out! This is a kernel bug
Jan 25 17:05:18 home kwin_wayland[1004]: kwin_wayland_drm: Pageflip timed out! This is a kernel bug

What happens if you disable the autologin? Does SDDM start?

pataf · 2025-01-25 17:36:15

kernel-lts 6.6.72 boots and I was able to log in just fine in both Wayland and X11.
I checked boot logs and there are no amdgpu errors

Regarding 6.12.10, I'll keep an eye on updates and check if something changes

Thanks for your help

Arch Linux

#1 2025-01-23 20:27:22

Unable to boot after kernel update

#2 2025-01-23 20:50:36

Re: Unable to boot after kernel update

#3 2025-01-23 20:54:52

Re: Unable to boot after kernel update

#4 2025-01-23 21:02:39

Re: Unable to boot after kernel update

#5 2025-01-23 21:21:47

Re: Unable to boot after kernel update

#6 2025-01-24 13:47:08

Re: Unable to boot after kernel update

#7 2025-01-24 16:34:05

Re: Unable to boot after kernel update

#8 2025-01-24 18:22:51

Re: Unable to boot after kernel update

#9 2025-01-24 18:29:14

Re: Unable to boot after kernel update

#10 2025-01-24 18:29:16

Re: Unable to boot after kernel update

#11 2025-01-24 23:19:06

Re: Unable to boot after kernel update

#12 2025-01-25 09:28:21

Re: Unable to boot after kernel update

#13 2025-01-25 09:35:59

Re: Unable to boot after kernel update

#14 2025-01-25 09:54:32

Re: Unable to boot after kernel update

#15 2025-01-25 10:16:33

Re: Unable to boot after kernel update

#16 2025-01-25 13:44:07

Re: Unable to boot after kernel update

#17 2025-01-25 13:58:59

Re: Unable to boot after kernel update

#18 2025-01-25 14:21:33

Re: Unable to boot after kernel update

#19 2025-01-25 14:23:20

Re: Unable to boot after kernel update

#20 2025-01-25 14:43:30

Re: Unable to boot after kernel update

#21 2025-01-25 15:01:13

Re: Unable to boot after kernel update

#22 2025-01-25 15:52:58

Re: Unable to boot after kernel update

#23 2025-01-25 17:02:55

Re: Unable to boot after kernel update

#24 2025-01-25 17:34:15

Re: Unable to boot after kernel update

#25 2025-01-25 17:36:15

Re: Unable to boot after kernel update

Board footer