You are not logged in.

#1 2022-10-19 02:48:45

redeadsarescary
Member
Registered: 2022-10-19
Posts: 4

Can't boot system after updating from Kernel 5.19 > 6.02 [SOLVED]

Hi,

I recently did pacman -Syu and it upgraded my linux-zen and main linux kernel to 6.0.2
I have four kernels installed because I do gpu passthrough to a windows vm for gaming and work. They are: Linux-Lts (5.15.74), linux-vfio (compiled manually and on 5.19.12), and the aforementioned linux-zen and linux
I have two graphics cards:
-> GTX 1650 to be used by linux with the nvidia driver
-> GTX970 that is driven by vfio-pci

I have nvidia-dkms installed, and it auto-builds the driver

Since the update to 6.0.2 (and whatever else I upgraded during that pacman update), booting linux or linux-zen will freeze at

 Starting version 251.6-1-arch 

before asking for me to decrypt. There do not appear to be journalctl entries for these boots

linux-vfio on 5.19.12 loads successfully, but the gtx 1650 has no kernel driver in use and so I cannot start gnome. I have two goals here:
1. Determine the reason why my system freezes while starting linux for the 6.02 kernels
-> I don't see a kernel panic message, so this isn't a kernel panic, right?

According to the user seth in https://bbs.archlinux.org/viewtopic.php?id=280528 the problem is with the nvidia drivers and linux  6.02
wait for " nvidia to align the the ACPI API changes in the kernel."

How does seth know that/how can I verify that is the problem or read about it?

2. Determine why the gtx 1650 has no kernel driver in use in linux-vfio when it did before the update.

Here's a relevant section of journalctl from a recent linux-vfio boot

Oct 18 20:46:55 archevan kernel: nvidia: loading out-of-tree module taints kernel.
Oct 18 20:46:55 archevan kernel: nvidia: module license 'NVIDIA' taints kernel.
Oct 18 20:46:55 archevan kernel: Disabling lock debugging due to kernel taint
Oct 18 20:46:55 archevan kernel: mc: Linux media interface: v0.10
Oct 18 20:46:55 archevan kernel: nvidia: module verification failed: signature and/or required key missing - tainting kernel
Oct 18 20:46:55 archevan mtp-probe[552]: checking bus 1, device 3: "/sys/devices/pci0000:00/0000:00:14.0/usb1/1-2"
Oct 18 20:46:55 archevan mtp-probe[552]: bus: 1, device: 3 was not an MTP device
Oct 18 20:46:55 archevan mtp-probe[554]: checking bus 1, device 8: "/sys/devices/pci0000:00/0000:00:14.0/usb1/1-14"
Oct 18 20:46:55 archevan mtp-probe[554]: bus: 1, device: 8 was not an MTP device
Oct 18 20:46:55 archevan mtp-probe[555]: checking bus 1, device 4: "/sys/devices/pci0000:00/0000:00:14.0/usb1/1-3"
Oct 18 20:46:55 archevan mtp-probe[555]: bus: 1, device: 4 was not an MTP device
Oct 18 20:46:55 archevan mtp-probe[556]: checking bus 1, device 6: "/sys/devices/pci0000:00/0000:00:14.0/usb1/1-7"
Oct 18 20:46:55 archevan mtp-probe[556]: bus: 1, device: 6 was not an MTP device
Oct 18 20:46:55 archevan kernel: BPF: [112277] ENUM perf_event_task_context 
Oct 18 20:46:55 archevan kernel: BPF: size=4 vlen=4
Oct 18 20:46:55 archevan kernel: BPF:  
Oct 18 20:46:55 archevan kernel: BPF: Invalid btf_info kind_flag
Oct 18 20:46:55 archevan kernel: BPF: 
Oct 18 20:46:55 archevan kernel: failed to validate module [nvidia] BTF: -22
Oct 18 20:46:55 archevan kernel: videodev: Linux video capture interface: v2.00
Oct 18 20:46:55 archevan kernel: snd_hda_intel 0000:00:1f.3: enabling device (0000 -> 0002)
Oct 18 20:46:55 archevan kernel: snd_hda_intel 0000:01:00.1: Disabling MSI
Oct 18 20:46:55 archevan kernel: snd_hda_intel 0000:01:00.1: Handle vga_switcheroo audio client
Oct 18 20:46:55 archevan systemd-modules-load[475]: Failed to insert module 'nvidia_uvm': Invalid argument
Oct 18 20:46:55 archevan audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-modules-load comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=failed'
Oct 18 20:46:55 archevan systemd[1]: systemd-modules-load.service: Main process exited, code=exited, status=1/FAILURE
Oct 18 20:46:55 archevan systemd[1]: systemd-modules-load.service: Failed with result 'exit-code'.
Oct 18 20:46:55 archevan systemd[1]: Failed to start Load Kernel Modules.
Oct 18 20:46:55 archevan systemd[1]: Starting Apply Kernel Variables...
Oct 18 20:46:55 archevan systemd[1]: Finished Apply Kernel Variables.
Oct 18 20:46:55 archevan audit[1]: SERVICE_START pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-sysctl comm="systemd" exe="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success'
Oct 18 20:46:55 archevan kernel: input: PC Speaker as /devices/platform/pcspkr/input/input33

Here is the result of grep -i upgraded /var/log/pacman.log

[2022-10-17T10:21:09-0500] [PACMAN] starting full system upgrade
[2022-10-17T10:21:56-0500] [ALPM] upgraded tzdata (2022d-1 -> 2022e-1)
[2022-10-17T10:21:56-0500] [ALPM] upgraded zlib (1:1.2.12-2 -> 1:1.2.13-1)
[2022-10-17T10:21:56-0500] [ALPM] upgraded btrfs-progs (5.19.1-1 -> 6.0-1)
[2022-10-17T10:21:56-0500] [ALPM] upgraded systemd-libs (251.5-1 -> 251.6-1)
[2022-10-17T10:21:56-0500] [ALPM] upgraded ca-certificates-mozilla (3.83-1 -> 3.84-1)
[2022-10-17T10:21:56-0500] [ALPM] upgraded dbus (1.14.2-1 -> 1.14.4-1)
[2022-10-17T10:21:56-0500] [ALPM] upgraded dmraid (1.0.0.rc16.3-13 -> 1.0.0.rc16.3-14)
[2022-10-17T10:21:57-0500] [ALPM] upgraded mesa (22.1.7-1 -> 22.2.1-1)
[2022-10-17T10:21:57-0500] [ALPM] upgraded harfbuzz (5.2.0-1 -> 5.3.0-1)
[2022-10-17T10:21:57-0500] [ALPM] upgraded systemd (251.5-1 -> 251.6-1)
[2022-10-17T10:21:57-0500] [ALPM] upgraded libwacom (2.4.0-1 -> 2.5.0-1)
[2022-10-17T10:21:58-0500] [ALPM] upgraded nvidia-utils (515.76-1 -> 520.56.06-1)
[2022-10-17T10:21:58-0500] [ALPM] upgraded freeglut (3.2.2-2 -> 3.4.0-1)
[2022-10-17T10:21:58-0500] [ALPM] upgraded gettext (0.21-2 -> 0.21.1-1)
[2022-10-17T10:21:58-0500] [ALPM] upgraded gnupg (2.2.39-1 -> 2.2.40-1)
[2022-10-17T10:21:58-0500] [ALPM] upgraded graphviz (6.0.1-1 -> 6.0.2-1)
[2022-10-17T10:21:58-0500] [ALPM] upgraded harfbuzz-icu (5.2.0-1 -> 5.3.0-1)
[2022-10-17T10:21:58-0500] [ALPM] upgraded imagemagick (7.1.0.50-1 -> 7.1.0.51-1)
[2022-10-17T10:21:58-0500] [ALPM] upgraded lib32-nvidia-utils (515.76-1 -> 520.56.06-1)
[2022-10-17T10:21:58-0500] [ALPM] upgraded lib32-mesa (22.1.7-1 -> 22.2.1-1)
[2022-10-17T10:21:58-0500] [ALPM] upgraded nss (3.83-1 -> 3.84-1)
[2022-10-17T10:21:58-0500] [ALPM] upgraded lib32-nss (3.83-1 -> 3.84-1)
[2022-10-17T10:21:58-0500] [ALPM] upgraded libbsd (0.11.6-2 -> 0.11.7-1)
[2022-10-17T10:21:58-0500] [ALPM] upgraded libical (3.0.15-1 -> 3.0.15-2)
[2022-10-17T10:21:58-0500] [ALPM] upgraded libpackagekit-glib (1.2.5-1 -> 1.2.5-2)
[2022-10-17T10:21:58-0500] [ALPM] upgraded libxnvctrl (515.76-1 -> 520.56.06-1)
[2022-10-17T10:21:59-0500] [ALPM] upgraded linux (5.19.13.arch1-1 -> 6.0.2.arch1-1)
[2022-10-17T10:21:59-0500] [ALPM] upgraded pahole (1:1.23-1 -> 1:1.24-1)
[2022-10-17T10:22:00-0500] [ALPM] upgraded linux-headers (5.19.13.arch1-1 -> 6.0.2.arch1-1)
[2022-10-17T10:22:01-0500] [ALPM] upgraded linux-zen (5.19.13.zen1-1 -> 6.0.2.zen1-1)
[2022-10-17T10:22:02-0500] [ALPM] upgraded linux-zen-headers (5.19.13.zen1-1 -> 6.0.2.zen1-1)
[2022-10-17T10:22:02-0500] [ALPM] upgraded man-db (2.10.2-1 -> 2.11.0-1)
[2022-10-17T10:22:02-0500] [ALPM] upgraded minizip (1:1.2.12-2 -> 1:1.2.13-1)
[2022-10-17T10:22:02-0500] [ALPM] upgraded nodejs (18.10.0-1 -> 18.11.0-1)
[2022-10-17T10:22:02-0500] [ALPM] upgraded nvidia-dkms (515.76-1 -> 520.56.06-1)
[2022-10-17T10:22:02-0500] [ALPM] upgraded pacman-mirrorlist (20220828-1 -> 20221016-1)
[2022-10-17T10:22:02-0500] [ALPM] upgraded python-pip (22.2.2-2 -> 22.3-1)
[2022-10-17T10:22:02-0500] [ALPM] upgraded python-setuptools (1:63.3.0-1 -> 1:63.4.1-1)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-common (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-audio-alsa (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-audio-dbus (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-audio-jack (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-audio-oss (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-audio-pa (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-audio-sdl (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-ui-opengl (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-ui-spice-core (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-audio-spice (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-block-curl (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-block-dmg (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-block-nfs (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-block-ssh (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-chardev-spice (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-hw-display-qxl (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-hw-display-virtio-gpu (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-hw-display-virtio-gpu-gl (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-hw-display-virtio-gpu-pci (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-hw-display-virtio-gpu-pci-gl (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-hw-display-virtio-vga (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-hw-display-virtio-vga-gl (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-hw-s390x-virtio-gpu-ccw (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-hw-usb-host (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-hw-usb-redirect (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-hw-usb-smartcard (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-img (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-pr-helper (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-system-x86-firmware (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-system-x86 (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-tools (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-ui-curses (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-ui-dbus (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-ui-egl-headless (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-ui-gtk (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-ui-sdl (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-ui-spice-app (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-vhost-user-gpu (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-virtiofsd (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded qemu-desktop (7.1.0-8 -> 7.1.0-9)
[2022-10-17T10:22:03-0500] [ALPM] upgraded systemd-sysvcompat (251.5-1 -> 251.6-1)
[2022-10-17T10:43:56-0500] [PACMAN] starting full system upgrade
[2022-10-17T10:57:46-0500] [PACMAN] starting full system upgrade
[2022-10-17T10:57:53-0500] [ALPM] running '70-dkms-upgrade.hook'...
[2022-10-17T10:57:55-0500] [ALPM] upgraded python-sphinx (5.2.3-1 -> 5.3.0-1)
[2022-10-17T11:00:30-0500] [PACMAN] starting full system upgrade
[2022-10-17T11:00:33-0500] [ALPM] running '70-dkms-upgrade.hook'...
[2022-10-17T11:01:34-0500] [PACMAN] starting full system upgrade
[2022-10-17T11:14:56-0500] [PACMAN] starting full system upgrade
[2022-10-17T11:14:58-0500] [ALPM] running '70-dkms-upgrade.hook'...
[2022-10-18T20:35:50-0500] [PACMAN] starting full system upgrade
[2022-10-18T20:36:10-0500] [ALPM] running '70-dkms-upgrade.hook'...
[2022-10-18T20:36:12-0500] [ALPM] upgraded libpciaccess (0.16-4 -> 0.17-1)
[2022-10-18T20:36:12-0500] [ALPM] upgraded libxshmfence (1.3-3 -> 1.3.1-1)
[2022-10-18T20:36:12-0500] [ALPM] upgraded libxkbfile (1.1.0-3 -> 1.1.1-1)
[2022-10-18T20:36:12-0500] [ALPM] upgraded nvidia-utils (520.56.06-1 -> 520.56.06-2)
[2022-10-18T20:36:13-0500] [ALPM] upgraded firefox (105.0.3-1 -> 106.0-1)
[2022-10-18T20:36:13-0500] [ALPM] upgraded libde265 (1.0.8-2 -> 1.0.9-1)
[2022-10-18T20:36:13-0500] [ALPM] upgraded libical (3.0.15-2 -> 3.0.16-1)
[2022-10-18T20:36:13-0500] [ALPM] upgraded libxmu (1.1.3-3 -> 1.1.4-1)
[2022-10-18T20:36:13-0500] [ALPM] upgraded nodejs (18.11.0-1 -> 18.11.0-2)
[2022-10-18T20:36:13-0500] [ALPM] upgraded nvidia-dkms (520.56.06-1 -> 520.56.06-2)
[2022-10-18T20:36:13-0500] [ALPM] upgraded python-setuptools (1:63.4.1-1 -> 1:63.4.2-1)
[2022-10-18T20:36:13-0500] [ALPM] upgraded xcb-util-cursor (0.1.3-4 -> 0.1.4-1)

I would appreciate any help - I apologize if I've misunderstood something in the wiki, I've been using arch for about 2 months and installing on an setting up my encrypted install took me about 3 hours with the arch wiki. I believe in using  the resources available to me, but I am struggling to determine my problem here

Last edited by redeadsarescary (2022-10-19 03:35:18)

Offline

#2 2022-10-19 03:02:06

loqs
Member
Registered: 2014-03-06
Posts: 18,964

Re: Can't boot system after updating from Kernel 5.19 > 6.02 [SOLVED]

Try downgrading pahole to 1:1.23 https://bbs.archlinux.org/viewtopic.php?id=279132

Offline

#3 2022-10-19 03:08:50

redeadsarescary
Member
Registered: 2022-10-19
Posts: 4

Re: Can't boot system after updating from Kernel 5.19 > 6.02 [SOLVED]

I wanted to post a brief update: This is my /etc/default/grub

GRUB_CMDLINE_LINUX="cryptdevice=UUID=c4072de4-560b-45a9-a9f7-79f279cee7aa:root intel_iommu=on pcie_acs_override=downstream,multifunction vfio-pci.ids=10de:13c2,10de:0fbb"

Based on https://bbs.archlinux.org/viewtopic.php?id=276056 I tried booting without the intel_iommu, pcie_acs_override, and vfio-pci.ids (I pressed e in grub to remove these entirely) and I was able to boot into linux zen 6.0.2


-----------
Thanks loqs, trying to downgrade pahole now and then I will reboot

Offline

#4 2022-10-19 03:17:47

redeadsarescary
Member
Registered: 2022-10-19
Posts: 4

Re: Can't boot system after updating from Kernel 5.19 > 6.02 [SOLVED]

Okay loqs, I did the following:

pacman -U file://var/cache/pacman/pkg/pahole-1:1.23-1-x86_64.pkg.tar.zst

and I can verify

[root@archevan evan]# pacman -Qs pahole
local/pahole 1:1.23-1
    Pahole and other DWARF utils

but I still freeze at  "Starting version 251.6-1-arch" when loading the 6.0.2 linux-zen kernel.

I should also mention I can run my windows vm in linux-vfio via

 virsh start win11 

so I believe that pahole is not causing me problems. I am also using qemu/kvm

thanks for the help - any other ideas?

Offline

#5 2022-10-19 03:27:58

loqs
Member
Registered: 2014-03-06
Posts: 18,964

Re: Can't boot system after updating from Kernel 5.19 > 6.02 [SOLVED]

Did you rebuild the nvidia module after downgrading pahole?

Offline

#6 2022-10-19 03:33:53

redeadsarescary
Member
Registered: 2022-10-19
Posts: 4

Re: Can't boot system after updating from Kernel 5.19 > 6.02 [SOLVED]

Okay update: I read another post on here:

https://bbs.archlinux.org/viewtopic.php?id=280512 and this associated post https://forum.level1techs.com/t/linux-k … h/190039/7

According to the user in the arch forums post,

"Boot is not broken but if you have full disk encryption the password prompt is not visible. If you blindly type the password boot works "

Sounds like the problem is affecting people who do pci passthrough and have full disk encryption.

So, I booted to

 Starting version 251.6-1-arch 

1. Typed in my encryption password. Enter. Nothing happens.
2. Type in my username. Enter. Password. Enter. Nothing happens
3. I do

 systemctl start gdm.service 

Enter.
4. I type in my root password
5. gnome starts! I am able to login

So I just have to deal with my screen being frozen until I start gnome. Not the biggest deal - I'm very happy to be able to use gnome and win11 in the same boot again.

Let's mark this as resolved! Thanks for the attempts at help!

Offline

Board footer

Powered by FluxBB