Updating from Linux 6.5.4 to 6.6 breaks many things on Thinkpad P16s

mFixman · 2023-11-12 18:29:17

Installing any version of the Linux kernel over 6.5. 4, including the current kernel 6.6, breaks the touchpad, trackpad, sound, and Internet of my Thinkpad P16s Gen 2 (Intel) after a reboot. Weirdly, the keyboard works fine.

These issues can be reverted by reinstalling 6.5.4 kernel with `pacman -U` and rebooting again, but installing any new kernel breaks these two things again.

Where can I begin troubleshooting this? The network controller and audio device appear fine in `lspci`.

The result of `sudo dmesg` on the new (broken) kernel is in https://pastebin.com/PK2TGyBs.

This line could be a clue to what is broken (note that this was executed in 6.6 kernel).

systemd[1]: Create List of Static Device Nodes was skipped because of an unmet condition check (ConditionFileNotEmpty=/lib/modules/6.5.4-arch2-1/modules.devname).

Last edited by mFixman (2023-11-12 18:49:01)

seth · 2023-11-13 09:31:03

The result of `sudo dmesg` on the new (broken) kernel is

[ 0.000000] Linux version 6.5.4-arch2-1 (linux@archlinux) (gcc (GCC) 13.2.1 20230801, GNU ld (GNU Binutils) 2.41.0) #1 SMP PREEMPT_DYNAMIC Thu, 21 Sep 2023 11:06:39 +0000

lsblk -f

mFixman · 2023-11-13 14:15:13

$ lsblk -f
NAME        FSTYPE    FSVER LABEL     UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
nvme0n1                                                                                   
├─nvme0n1p1 vfat      FAT32 SYSTEM    401C-8D15                                           
├─nvme0n1p2                                                                               
├─nvme0n1p3 BitLocker 2                                                                   
├─nvme0n1p4 ntfs            WinRE_DRV B86E1F4E6E1F04B2                                    
├─nvme0n1p5 ext4      1.0             8e91f946-bdc8-4c9f-bde3-ef8b74f2ebd8  474.4G    32% /
└─nvme0n1p6 swap      1               82acedd6-7d8c-416c-9a8b-e640b59bae2a                [SWAP]

seth · 2023-11-13 14:16:30

nvme0n1p1 vfat      FAT32 SYSTEM    401C-8D15

I'll bet your right arm that this is the boot partition you didn't mount?

mFixman · 2023-11-13 14:28:26

You might be onto something.

The following lines appears as output in `mount` using the old kernels, but not in the (broken) new one.

systemd-1 on /efi type autofs (rw,relatime,fd=51,pgrp=1,timeout=120,minproto=5,maxproto=5,direct,pipe_ino=32588)
/dev/nvme0n1p1 on /efi type vfat (rw,nosuid,nodev,noexec,relatime,nosymfollow,fmask=0077,dmask=0077,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro)

Trying to mount /dev/nvme0n1p1 into /efi fails with this error.

$ sudo mount /dev/nvme0n1p1 /efi
mount: /efi: unknown filesystem type 'vfat'.
       dmesg(1) may have more information after failed mount system call.

dmesg contains no information about this call.

seth · 2023-11-13 14:38:27

Not "might"
You're not booting from the root partition, but an older kernel from some boot partition (very likely /dev/nvme0n1p1)
You cannot mount that from the system that boots the old kernel against the new package because you're lacking most modules (specifically vfat) but you can fix this from the install iso.

However, whether you can mount that at /efi or it should be /boot higly depends on your bootloader/configuration.

mFixman · 2023-11-13 14:42:34

That makes sense. I think I messed up my GRUB when setting up Arch.

Here is the GRUB menu entry for Arch in `/efi/grub/grub.cfg`.

menuentry 'Arch Linux' --class arch --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-8e91f946-bdc8-4c9f-bde3-ef8b74f2ebd8' {
	load_video
	set gfxpayload=keep
	insmod gzio
	insmod part_gpt
	insmod fat
	search --no-floppy --fs-uuid --set=root 401C-8D15
	echo	'Loading Linux linux ...'
	linux	/vmlinuz-linux root=UUID=8e91f946-bdc8-4c9f-bde3-ef8b74f2ebd8 rw root=/dev/nvme0n1p5 rw quiet loglevel=3 quiet
	echo	'Loading initial ramdisk ...'
	initrd	/initramfs-linux.img
}

The root partition is correct, but I suspect it's loading the `vmzlinuz-linux` file in `/efi` (`/dev/nvme0n1p1`).

Does this look fine to you? What else would you check to fix this?

seth · 2023-11-13 14:50:46

The grub.cfg is ok, "--set=root 401C-8D15" means you're booting from that partition and w/ the default grub behavior, you'd rather mount that into /boot than /efi (because that's where kernel and initramfs will get copied)

You can inspect the present /boot and /efi paths (w/o mounting anything there) to see where you've copied the last kernel, that's where you want to mount 401C-8D15

Arch Linux

#1 2023-11-12 18:29:17

Updating from Linux 6.5.4 to 6.6 breaks many things on Thinkpad P16s

#2 2023-11-13 09:31:03

Re: Updating from Linux 6.5.4 to 6.6 breaks many things on Thinkpad P16s

#3 2023-11-13 14:15:13

Re: Updating from Linux 6.5.4 to 6.6 breaks many things on Thinkpad P16s

#4 2023-11-13 14:16:30

Re: Updating from Linux 6.5.4 to 6.6 breaks many things on Thinkpad P16s

#5 2023-11-13 14:28:26

Re: Updating from Linux 6.5.4 to 6.6 breaks many things on Thinkpad P16s

#6 2023-11-13 14:38:27

Re: Updating from Linux 6.5.4 to 6.6 breaks many things on Thinkpad P16s

#7 2023-11-13 14:42:34

Re: Updating from Linux 6.5.4 to 6.6 breaks many things on Thinkpad P16s

#8 2023-11-13 14:50:46

Re: Updating from Linux 6.5.4 to 6.6 breaks many things on Thinkpad P16s

Board footer