You are not logged in.

#1 2025-02-23 16:16:14

SeagullFish
Member
Registered: 2023-08-10
Posts: 42

[SOLVED] Kernel panic after updating to new kernel version

Hello, all.

Yesterday, I did the following updates:

linux-6.13.2.arch1-1-x86_64.pkg.tar.zst --> linux-6.13.3.arch1-1-x86_64.pkg.tar.zst
linux-headers-6.13.2.arch1-1-x86_64.pkg.tar.zst --> linux-headers-6.13.3.arch1-1-x86_64.pkg.tar.zst
linux-zen-6.13.2.zen1-1-x86_64.pkg.tar.zst --> linux-zen-6.13.3.zen1-1-x86_64.pkg.tar.zst
linux-zen-headers-6.13.2.zen1-1-x86_64.pkg.tar.zst --> linux-zen-headers-6.13.3.zen1-1-x86_64.pkg.tar.zst

As you probably notice, I have both the Zen-kernel and the Vanilla kernel installed.

Now, when booting the Zen-kernel, it panics immediately after GRUB has finished:

[    0.632243] rtc_cmos rtc_cmos: RTC can wake from S4
[    0.633103] rtc_cmos rtc_cmos: registered as rtc0
[    0.633289] rtc_cmos rtc_cmos: setting system clock to 2025-02-22T18:54:50 UTC (1740250490)
[    0.633317] rtc_cmos rtc_cmos: alarms up to one month, y3k, 114 bytes nvram
[    0.634853] intel_pstate: Intel P-state driver initializing
[    0.635376] intel_pstate: HWP enabled
[    0.635668] simple-framebuffer simple-framebuffer.0: [drm] Registered 1 planes with drm panic
[    0.635669] [drm] Initialized simpledrm 1.0.0 for simple-framebuffer.0 on minor 0
[    0.636765] fbcon: Deferring console take-over
[    0.636766] simple-framebuffer simple-framebuffer.0: [drm] fb0: simpledrmdrmfb frame buffer device
[    0.636792] hid: raw HID events driver (C) Jiri Kosina
{    0.636827] drop_monitor: Initializing network drop monitor service
[    0.636896] NET: Registered PF_INET6 protocol family
[    0.637192] Segment Routing with IPv6
[    0.637193] RPL Segment Routing with IPv6
[    0.637197] In-sity OAM (IOAM) with IPv6
[    0.637208 NET: Registered PF_PACKET protocol family
[    0.637654] ENERGY_PERF_BIAS: Set to ’normal’, was ’performance’
[    0.637708] microcode: Current revision: 0x000000b8
[    0.637709] microcode: Updated early from: 0x00000086
[    0.637794] resctrl: L2 allocation detected
[    0.637804] IPI shorthand broadcast: enabled
[    0.638825] sched_clock: Marking stable (631001155, 7019097)->(664081831, -26061579)
[    0.638931] registered taskstats version 1
[    0.639093] Loading compiled-in X.509 certificates
[    0.640535] Loaded X.509 cert ’Build time autogenerated kernel key: 0785f24f93a5afd336f2e5a37417c7cc397a074’
[    0.641440] Demotion targets for Node 0: null
[    0.641563] Key type .fscrypt registered
[    0.641564] Key type fscrypt-provisioning registered
[    0.641803] integrity: Loading X.509 certificate: UEFI:db
[    0.641818] integrity: Loaded X.509 cert ’Microsoft Windows Production PCA 2011: a92902398e16c49778cd90f99e4f9ae17c55af53’
[    0.641818] integrity: Loading X.509 certificate: UEFI:db
[    0.641829] integrity: Loaded X.509 cert ’Microsoft Corporation UEFI CA 2011: 13adbf4309bd82709c8cd54f316ed522988a1bd4’
[    0.641829] integrity: Loading X.509 certificate: UEFI:db
[    0.642014] integrity: Loaded X.509 cert ’Huaqin NBD2 Software Certificate: 4672fbfc31a0ada6417c929fb7a3c1ba’
[    0.642968] PM:    Magic number: 13:429:949
[    0.642986] cpuid cpu4: hash matches
[    0.642994] tty tty6: hash matches
[    0.643042] processor cpu4: hash matches
[    0.643066] memory memory2: hash matches
[    0.645115] RAS: Correctable Errors collector initialized.
[    0.652453] clk: Disabling unused clocks
[    0.652456] PM: genpd: Disabling unused power domains
[    0.652563] /dev/root: Can’t open blockdev

# COMMENT: I have altered the next line with x’es, for privacy.
[    0.652571] VFS: Cannot open root device "UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" or unknown-block(0,0): error -6
# COMMENT END

[    0.652572] Please append a correct "root=" boot option; here are the available partitions:
[    0.652573] List of all bdev filesystems:
[    0.652574]   fuseblk
[    0.652574]
[    0.652575] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
[    0.652578] fbcon: Taking over console
[    0.652581] CPU: 4 UID: 0 PID: 1 Comm: swapper/0 Not tainted 6.13.3-zen1-1-zen #1 4eb5d478c0b89d357d430d5db88dfd1ce4224b4f
[    0.652584] Hardware name: ASUSTeK COMPUTER INC. ZenBook UX425EA_BX425EA/UX425EA, BIOS UX425EA.307 05/25/2021
[    0.652586] Call Trace:
[    0.652589]  <TASK>
[    0.652591]  dump_stack_lvl+0x5d/0x80
[    0.652595]  panic+0x118/0x2d4]
[    0.652598]  mount_root_generic+0x1ce/0x270
[    0.652602]  prepare_namespace+0x1e7/0x230
[    0.652605]  kernel_init_freeable+0x26f/0x2a0
[    0.652608]  ? __pfx_kernel_init+0x10/0x10
[    0.652611]  kernel_init+0x1a/0x140
[    0.652613]  ret_from_fork+0x31/0x50
[    0.652615]  ? __pfx_kernel_init+0x10/0x10
[    0.652617]  ret_from_fork_asm+0x1a/0x30
[    0.652620]  </TASK>
[    0.652628] Kernel Offset: 0x1fa00000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)

When booting the Vanilla-kernel, it also panics:

[    0.642682] hub 6-0:1.0: 2 ports detected
[    0.642739] usbcore: registered new interface driver usbserial_generic
[    0.642742] usbserial: USB Serial support registered for generic
[    0.642785] rtc_cmos rtc_cmos: RTC cam wake from S4
[    0.643661] rtc_cmos rtc_cmos: registered as rtc0
[    0.643840] rtc_cmos rtc_cmos: setting system clock to 2025-02-23T13:21:05 UTC (1740316865)
[    0.643865] rtc_cmos rtc_cmos: alarms up to one month, y3k, 114 bytes nvram
[    0.645245] intel_pstate: Intel P-state driver initializing
[    0.645708] intel_pstate:HWP enabled
[    0.645972] simple-framebuffer simple-framebuffer.0: [drm] Registered 1 planes with drm panic
[    0.645973] [drm] Initialized simpledrm 1.0.0 for simple-framebuffer.0 on minor 0
[    0.647052] fbcon: Deferring console take-over
[    0.647053] simple-framebuffer simple-framebuffer.0: [drm] fb0: simpledrmdrmfb frame buffer device
[    0.647097] hid: raw HID events driver (C) Jiri Kosina
[    0.647133] drop_monitor: Initializing network drop monitor service
[    0.647199] NET: Registered PF_INET6 protocol family
[    0.647447] Segment Routing with IPv6
[    0.647447] RPL Segment Routing with IPv6
[    0.647451] In-situ OAM (IOAM) with IPv6
[    0.647464] NET: Registered PF_PACKET protocol family
[    0.648005] ENERGY_PERF_BIAS: Set to ’normal’, was ’performance’
[    0.648154] microcode: Current revision: 0x000000b8
[    0.648155] microcode: Updated early from: 0x00000086
[    0.648465] resctrl: L2 allocation detected
[    0.648475] IPI shorthand broadcast: enabled
[    0.649499] sched_clock: Marking stable (642001040, 7040769)->(674956255, -25914446)
[    0.649686] registered taskstats version 1
[    0.649995] Loading compiled-in X.509 certificates
[    0.651983] Loaded X.509 cert ’Build time autogenerated kernel key: 6aa5ed3e3a8adb4422fcf5a75ce223b727d7ade3’
[    0.652914] Demotion targets for Node 0: null
[    0.653008] Key type .fscrypt registered
[    0.653009] Key type fscrypt-provisioning registered
[    0.653251] integrity: Loading X.509 certificate: UEFI:db
[    0.653265] integrity: Loaded X.509 cert ’Microsoft Windows Production PCA 2011: a92902398e16c49778cd90f99e4f9ae17c55af53’
[    0.653266] integrity: Loading X.509 certificate: UEFI:db
[    0.653276] integrity: Loaded X.509 cert ’Microsoft Corporation UEFI CA 2011: 13adbf4309bd82709c8cd54f31ed522988a1bd4’
[    0.653277] integrity: Loading X.509 certificate: UEFI:db
[    0.653507] integrity: Loaded X.509 cert ’Huaqin NDB2 Software Certificate: 4672fbfc31a0ada6417c929fb7a3c1ba’
[    0.654447] PM:    Magic number: 13:903:382
[    0.654450] usb usb6-port2: hash matches
[    0.656655] RAS: Correctable Errors collector initialized.
[    0.663797] clk: Disabling unused clocks
[    0.663798] PM: genpd: Disabling unused power domains
[    0.663858] /dev/root: Can’t open blockdev


# COMMENT: I have altered the next line with x’es, for privacy.
[    0.663885] VFS: Cannot open root device "UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" or unknown-block(0,0): error -6
# COMMENT END

[    0.663886] Please append a correct "root=" boot option; here are the available partoptions:
[    0.663887] List of all bdev filesystems:
[    0.663887]   fuseblk
[    0.663887]
[    0.663888] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
[    0.663891] fbcon: Taking over console
[    0.663894] CPU: 6 UID: 0 PID: 1 Comm: swapper/0 Not tainted 6.13.3-arch1-1 #1 0e057b3ba87c8453f3120f28a0cea5eaa8068906
[    0.663898] Hardware name: ASUSTeK COMPUTER INC. ZenBook UX425EA_BX425EA/UX425EA, BIOS UX425EA.307 05/25/2021
[    0.663900] Hard Trace:
[    0.663902]  <TASK>
[    0.663904]  dump_stack_lvl+0x5d/0x80
[    0.663907]  panic+0x118/0x2db
[    0.663910]  mount_root_generic+0x1ce/0x270
[    0.663914]  prepare_namespace+0x1e7/0x230
[    0.663916]  kernel_init_freeable+0x26f/0x2a0
[    0.663919]  ? __pfx_kernel_init+0x10/0x10
[    0.663921]  kernel_init+0x1a/0x140
[    0.663924]  ret_from_fork+0x31/0x50
[    0.663926]  ? __pfx_kernel_init+0x10/0x10
[    0.663928]  ret_from_fork_asm+0x1a/0x30
[    0.663931]  </TASK>
[    0.663947] Kernel Offset: 0x20000000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)

I don’t know what’s causing this. However, the error did not occur with the previous version of the kernel. In order to get the system up and running again, I would like to downgrade the kernel.

I am able to boot an old Arch Linux live system from a bootable USB stick. From there, I am also able to mount my disk partitions, and chroot, as described in the Arch Linux wiki-page for chroot.

So, I tried to downgrade from chroot, but unfortunately, pacman fails:

Arch Linux 6.4.7-arch1-1 (tty1)

archiso login: root (automatic login)

To install Arch Linux follow the installation guide:
https://wiki.archlinux.org/title/installation_guide

For Wi-Fi, authenticate to the wireless network using the iwctl utility.
For mobile broadband (WWAN) modems, connect with the mmcli utility.
Ethernet, WLAN and WWAN interfaces using DHCP should work automatically.

After connecting to the internet, the installation guide can be accessed
via the convenience script Intallation_guide.

# COLORS
root@archiso ~ # mount /dev/nvme0n1p4 /mnt
root@archiso ~ # mount /dev/nvme0n1p3 /mnt/boot
root@archiso ~ # mount /dev/nvme0n1p1 /mnt/boot/efi
root@archiso ~ # mount /dev/nvme0n1p5 /mnt/home
root@archiso ~ # swapon /dev/nvme0n1p2
root@archiso ~ # arch-chroot /mnt

# To check that all mounted file systems are readable:
[root@archiso /]# ls /
bin   boot   dev   etc   home   lib   lib64   mnt   opt   proc   root   run   sbin   src   sys   tmp   usr   var
[root@archiso /]# ls /boot
efi   grub   initramfs-linux-fallback.img   initramfs-linux-zen-fallback.img   initramfs-linux-zen.img   initramfs-linux.img   intel-ucode.img   vmlinuz-linux   vmlinuz-linux-zen
[root@archiso /]# ls /boot/efi
EFI
[root@archiso /]# ls /home
# User folders

# Moving on:
[root@archiso /]# cd /var/cache/pacman/pkg
[root@archiso pkg]# ls linux-*.tar.zst
linux-6.12.10.arch1-1-x86_64.pkg.tar.zst   linux-firmware-20250109.7673dffd-1-any.pkg.tar.zst   linux-headers-6.13.2.arch1-1-x86_64.pkg.tar.zst   linux-zen-headers-6.12.10.zen1-1-x86_64.pkg.tar.zst
linux-6.13.2.arch1-1-x86_64.pkg.tar.zst   linux-firmware-20250210.5bc5868b-1-any.pkg.tar.zst   linux-headers-6.13.3.arch1-1-x86_64.pkg.tar.zst   linux-zen-headers-6.13.2.zen1-1-x86_64.pkg.tar.zst
linux-6.13.3.arch1-1-x86_64.pkg.tar.zst   linux-firmware-whence-20250109.7673dffd-1-any.pkg.tar.zst   linux-zen-6.12.10.zen1-1-x86_64.pkg.tar.zst   linux-zen-headers-6.13.3.zen1-1-x86_64.pkg.tar.zst
linux-api-headers-6.10-1-x86_64.pkg.tar.zst    linux-firmware-whence-20250210.5bc5868b-1-any.pkg.tar.zst    linux-zen-6.13.2.zen1-1-x86_64.pkg.tar.zst
linux-api-headers-6.13-1-x86_64.pkg.tar.zst   linux-headers-6.12.10.arch1-1-x86_64.pkg.tar.zst   linux-zen-6.13.3.zen1-1-x86_64.pkg.tar.zst

# Trying to execute downgrade:
[root@archiso pkg]# pacman -U linux-6.13.2.arch1-1-x86_64.pkg.tar.zst  linux-headers-6.13.2.arch1-1-x86_64.pkg.tar.zst linux-zen-6.13.2.zen1-1-x86_64.pkg.tar.zst linux-zen-headers-6.13.2.zen1-1-x86_64.pkg.tar.zst
pacman: error while loading shared libraries: libicuuc.so.76: cannot open shared object file: No such file or directory

# Fetching some information from the chrooted environment, in order to try and figure out what could be wrong:
[root@archiso pkg]# uname -r
6.4.7-arch1-1
[root@archiso pkg]# cd /
[root@archiso /]# find -name libicuuc.so.*
./usr/lib/libicuuc.so.75
.usr/lib/libicuuc.so.75.1
[root@archiso /]# pacman --version
pacman: error while loading shared libraries: libicuuc.so.76: cannot open shared object file: No such file or directory
My question:

Now what? Any ideas, anyone?

Last edited by SeagullFish (2025-02-23 22:05:33)

Offline

#2 2025-02-23 22:05:18

SeagullFish
Member
Registered: 2023-08-10
Posts: 42

Re: [SOLVED] Kernel panic after updating to new kernel version

After reading this thread, I found the solution myself. The two cases seem to be very similar.

For unknown reasons (from my point of view), the package icu was not updated as it should have been, at the time when I performed my system update. I don't know how this went wrong.

Regardless, here is what I did to fix it:

Arch Linux 6.4.7-arch1-1 (tty1)

archiso login: root (automatic login)

To install Arch Linux follow the installation guide:
https://wiki.archlinux.org/title/installation_guide

For Wi-Fi, authenticate to the wireless network using the iwctl utility.
For mobile broadband (WWAN) modems, connect with the mmcli utility.
Ethernet, WLAN and WWAN interfaces using DHCP should work automatically.

After connecting to the internet, the installation guide can be accessed
via the convenience script Intallation_guide.

# COLORS
root@archiso ~ # mount /dev/nvme0n1p4 /mnt
root@archiso ~ # mount /dev/nvme0n1p3 /mnt/boot
root@archiso ~ # mount /dev/nvme0n1p1 /mnt/boot/efi
root@archiso ~ # mount /dev/nvme0n1p5 /mnt/home
root@archiso ~ # swapon /dev/nvme0n1p2

I got the following commands from the other mentioned thread, but I don’t fully understand what they do. So don’t ask me about them.

root@archiso ~ # mount -t proc /proc /mnt/proc
root@archiso ~ # mount -t sysfs /sys /mnt/sys
root@archiso ~ # mount --rbind /dev /mnt/dev

To circumvent a diskspace issue on the live system, I had to delete pacman’s pkg cache folder in the live system, and replace it with a symlink to pacman’s pkg cache folder on the mounted disk.

root@archiso ~ #  cd /var/cache/pacman
root@archiso pacman# rm -rf pkg && ln -s /mnt/var/cache/pacman/pkg pkg

Now, I had to install icu:

root@archiso pacman# pacman --root=/mnt  -S icu 

But in my case, pacman exited with an error, complainting about dependencies that was going to be broken. So I had to specify all other dependent packages to be updated as well:

root@archiso pacman# pacman --root=/mnt -S icu brltty electron31 electron32 electron34 flac freerdp freerdp2 gspell harfbuzz-icu libphonenumber libsndfile raptor timidity++

Lastly, thanks to the post from V1del, I noticed that I also had to run mkinitcpio from a chrooted environment:

root@archiso pacman# arch-chroot /mnt
root@archiso /# mkinitcpio -P
root@archiso pacman# exit
root@archiso pacman# reboot

Voila! Now everything works again! I am changing the status of this thread to [SOLVED].

Last edited by SeagullFish (2025-02-23 22:07:16)

Offline

Board footer

Powered by FluxBB