Arch Linux

eblau

Member

Registered: 2016-02-11

Posts: 43

[SOLVED] /dev/nvme* missing in initial ramdisk env after upgrade

Hi folks,

I'm having an issue with my initial ramdisk environment after upgrading from linux-5.17.1.arch1-1 to linux-5.17.3.arch1-1.

I have some mkinitcpio hook scripts that allow me to open my LUKS protected swap file and root partition during boot in the initial ramdisk environment. When booting 5.17.3, these hooks fail because /dev/nvme0n1p3 is inaccessible. When I'm dropped to the emergency root prompt, I can see that none of my /dev/nvme* devices are present. Downgrading to linux-5.17.1.arch1-1 fixes the problem.

Has anyone else noticed this issue? Is there anything else I can check on here? Thanks!

Last edited by eblau (2022-04-22 17:20:04)

loqs

Member

Registered: 2014-03-06

Posts: 17,304

Re: [SOLVED] /dev/nvme* missing in initial ramdisk env after upgrade

Have you seen https://bugs.archlinux.org/task/74412

eblau

Member

Registered: 2016-02-11

Posts: 43

Re: [SOLVED] /dev/nvme* missing in initial ramdisk env after upgrade

Have you seen https://bugs.archlinux.org/task/74412

Thanks. No, I had not seen that. I did look for missing modules when booting 5.17.1.arch1 to see if I was missing them in 5.17.3.arch1-1, but I guess if they were built-in to 5.17.1.arch1, I wouldn't have seen them in any case.

I also would have appreciated a news item or a doc update before a kernel update that would make the system unbootable, as noted in the bug.

eblau

Member

Registered: 2016-02-11

Posts: 43

Re: [SOLVED] /dev/nvme* missing in initial ramdisk env after upgrade

Unfortunately, the advice in the bug report did not help.

V1del

Forum Moderator

Registered: 2012-10-16

Posts: 21,626

Re: [SOLVED] /dev/nvme* missing in initial ramdisk env after upgrade

How do you hope that that would be achieved regarding a news item? There isn't an inherent issue here, I have no trouble booting my nvme drive on the 5.17.3 kernel, hence it stands to reason that none of the package maintainers were able to reproduce this.

How sure are you that you are actually booting the correct kernel? This sounds to me like you might not be booting the correct kernel. Which bootloader are you using? After updating without yet having rebooted what outputs do you get for

file /boot/vmlinuz-linux
sudo lsinitcpio /boot/initramfs-linux.img | grep nvme
lsblk -f

as well as your bootloader config. It's also possible that your script runs before the nvme module is actually loaded, in which case you might have to delay/pick a higher hook entry point e.g. a run_latehook rather than  just  run_hook

Last edited by V1del (2022-04-22 12:35:51)

eblau

Member

Registered: 2016-02-11

Posts: 43

Re: [SOLVED] /dev/nvme* missing in initial ramdisk env after upgrade

@V1del I was able to get past this issue by upgrading my bootable USB drive to 5.17.4, booting 5.17.4, mounting my root filesystem and running "mkinitcpio -P" with nvme and nvme_core both added to the MODULES list in /etc/mkinitcpio.conf inside an arch-chroot environment for my root filesystem.

I was definitely booting the correct kernel as reverting back to 5.17.1 fixed the issue.

The inherent issue here is that NVMe functionality, which was built in to the kernel in 5.17.1, was made modular in 5.17.2. Anyone relying on NVMe being built-in could be affected. In my case, I have a custom initial ramdisk and had to modify it to add the NVMe modules so that I could mount my root filesystem. It would be nice to have a heads up for this kind of change that could make root filesystems inaccessible at boot time.

Last edited by eblau (2022-04-22 17:23:14)

Bubbel

Member

Registered: 2015-04-16

Posts: 7

Re: [SOLVED] /dev/nvme* missing in initial ramdisk env after upgrade

I had similar issues. The root cause was a missing block hook in mkinitcpio.conf.
Adding that (and removing the nvme/nvme_core modules), resolved the issue.