[SOLVED] LUKS Systemd fresh install times out "waiting for device ..."

ggiuffre · 2024-05-25 12:45:19

I want to install Arch Linux on a fully-encrypted drive (with LUKS on a partition) with systemd-boot as a bootloader.

I have already successfully installed Arch Linux before, just never with LUKS.

Once I've installed a bootloader and finished configuring my new system, I reboot. Systemd hangs for 1:30 minutes ("A start job is running for /dev/disk/...") before showing the following message (see attached image for more output):

Timed out waiting for device /dev/disk/by-uuid/9d28bc4c-4035-460c-b923-06e9f97a6e62.

Why do I see this error, and what can I do to successfully reboot into my new system?

---

This is the /etc/fstab that I generated:

# Static information about the filesystems.
# See fstab(5) for details.

# <file system> <dir> <type> <options> <dump> <pass>
# /dev/mapper/root
UUID=3530b9d5-be63-4dde-b6e4-f8ea32811b1e	/         	ext4      	rw,relatime	0 1

# /dev/nvme0n1p1
UUID=7CA6-14A3      	/boot     	vfat      	rw,relatime,fmask=0022,dmask=0022,codepage=437,iocharset=ascii,shortname=mixed,utf8,errors=remount-ro	0 2

And this is the output of blkid:

/dev/nvme0n1p1: UUID="7CA6-14A3" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="731def83-b1a7-4e9b-98e3-ddfe1e299b6e"
/dev/nvme0n1p2: UUID="9d28bc4c-4035-460c-b923-06e9f97a6e62" TYPE="crypto_LUKS" PARTUUID="e0594543-7036-4027-bfc6-f5dd7bdbee60"
/dev/loop0: BLOCK_SIZE="1048576" TYPE="squashfs"
/dev/mapper/root: UUID="3530b9d5-be63-4dde-b6e4-f8ea32811b1e" BLOCK_SIZE="4096" TYPE="ext4"
/dev/sda2: SEC_TYPE="msdos" LABEL_FATBOOT="ARCHISO_EFI" LABEL="ARCHISO_EFI" UUID="6665-2677" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="8da10534-02"
/dev/sda1: BLOCK_SIZE="2048" UUID="2024-05-01-17-04-31-00" LABEL="ARCH_202405" TYPE="iso9660" PARTUUID="8da10534-01"

Under /boot/loader/entries/ I created two files, arch.conf and arch-lts.conf. They're identical except for the title and kernel that they use. Here's the content of the first one:

title Arch Linux
linux /vmlinuz-linux
initrd /intel-ucode.img
initrd /initramfs-linux.img
options rd.luks.name=9d28bc4c-4035-460c-b923-06e9f97a6e62=root root=/dev/mapper/root

No matter which kernel I use (linux or linux-lts) I get the error mentioned above.

Moreover, I see 7 "Linux bootloader" entries as possible boot sources in the UEFI of my machine. I reinstalled the system a few times. Every time, a new one was added. At every new install I format my disk with fdisk, so I'm not sure exactly why every new installation adds an entry. See second image attached. I mention this because perhaps it's related to the main problem discussed in this post. Please be kind if my knowledge of Linux is not as good as yours I'm learning.

My machine is a Lenovo Thinkpad X1 Carbon 12th generation.

I'm glad to provide further information.

output from Systemd

UEFI boot sources

Last edited by ggiuffre (2024-05-28 07:28:07)

impossibleveins23 · 2024-05-26 11:26:13

Did you configure initramfs with the sd-* hooks?

See that they are configured in /etc/mkinitcpio.conf and that you generated new initramfs.

ggiuffre · 2024-05-26 16:20:54

impossibleveins23 wrote:

Did you configure initramfs with the sd-* hooks?
See that they are configured in /etc/mkinitcpio.conf and that you generated new initramfs.

Yes, I set the hooks in my /etc/mkinitcpio.conf to be as follows:

HOOKS=(base systemd autodetect microcode modconf kms keyboard block sd-encrypt filesystems fsck)

Then I regenerated initramfs by running mkinitcpio -P.

(By the way I just noticed now that some people place the "block" hook after the "sd-encrypt" hook. I swapped them, regenerated initramfs, but the problem is still there.)

Is there any other config file or log file that you suggest I should check?

impossibleveins23 · 2024-05-26 16:41:36

Try adding sd-vconsole as well.

kermit63 · 2024-05-26 17:51:52

options rd.luks.name=9d28bc4c-4035-460c-b923-06e9f97a6e62=root root=/dev/mapper/root

Shouldn't this be rd.luks.uuid?

I'm more conversant with grub2 + luks instead of systemd-boot, but that line just doesn't look right. Maybe a systemd-boot user can chime in.

ggiuffre · 2024-05-26 19:59:17

kermit63 wrote:

options rd.luks.name=9d28bc4c-4035-460c-b923-06e9f97a6e62=root root=/dev/mapper/root

Shouldn't this be rd.luks.uuid?

Thanks for the suggestion. That line is a bit counterintuitive, but indeed this is what the documentation suggests to do if you use systemd-boot. See also the system configuration docs for LUKS, which specify that you can omit rd.luks.uuid if you provide rd.luks.name.

ggiuffre · 2024-05-26 20:17:35

impossibleveins23 wrote:

Try adding sd-vconsole as well.

This too unfortunately did not solve the problem. Thanks for the suggestion, though.

impossibleveins23 · 2024-05-27 08:22:59

Try to unlock the root account, drop into emergency shell, mount the encrypted volume the file system.

Regarding multiple EFI bootloaders, you can probably remove them and leave just one.

frostschutz · 2024-05-27 08:57:04

Does /dev/disk/by-uuid/9d28bc4c-4035-460c-b923-06e9f97a6e62 exist on the live cd? This could fail for some strange reasons, custom udev rules, ...

Check with lsinitcpio that the nvme module is included? Otherwise the initramfs won't detect the drive at all. If it's missing, add it to MODULES

It would help to verify directly on the initramfs shell, if you can unlock it (this might still require a custom shadow hook - https://bugs.archlinux.org/task/70408 )

Alternatively build an initcpio without systemd where the rescue shell works.

Last edited by frostschutz (2024-05-27 08:57:35)

ggiuffre · 2024-05-27 11:24:55

frostschutz wrote:

Does /dev/disk/by-uuid/9d28bc4c-4035-460c-b923-06e9f97a6e62 exist on the live cd?

Yes. I hope I got your question right, but from the live USB I see a symbolic link from /dev/disk/by-uuid/9d28bc4c-4035-460c-b923-06e9f97a6e62 to /dev/nvme0n1p2:

root@archiso ~ # ls -l /dev/disk/by-uuid
total 0
lrwxrwxrwx 1 root root 15 May 27 10:30 7CA6-14A3 -> ../../nvme0n1p1
lrwxrwxrwx 1 root root 15 May 27 10:30 9d28bc4c-4035-460c-b923-06e9f97a6e62 -> ../../nvme0n1p2
lrwxrwxrwx 1 root root 10 May 27 10:30 2024-05-01-17-04-31-00 -> ../../sda1
lrwxrwxrwx 1 root root 10 May 27 10:30 6665-2677 -> ../../sda2

frostschutz wrote:

Check with lsinitcpio that the nvme module is included?

Yes. After chrooting into my system if I run lsinitcpio -a /boot/initramfs-linux.img I see "nvme" included in the modules. Same for initramfs-linux-lts.img

(For what concerns the rest of your message I'll get back as soon as I manage to try those suggestions.)

impossibleveins23 wrote:

Try to unlock the root account, drop into emergency shell, mount the encrypted volume the file system.

I'm not sure I got this right: are you talking about the emergency shell that I get dropped in at startup time, or about the shell of the live UBS/CD installer?

impossibleveins23 wrote:

Regarding multiple EFI bootloaders, you can probably remove them and leave just one.

I didn't know you can do this I successfully removed the extra entries with efibootmgr -b <bootnum> -B. Thanks for the tip!

impossibleveins23 · 2024-05-27 12:30:33

I'm not sure I got this right: are you talking about the emergency shell that I get dropped in at startup time, or about the shell of the live UBS/CD installer?

See this.
Never tried it but it might allow you drop in to the emergency shell as root by adding to /etc/mkinitcpio.conf:

FILES=(/etc/shadow.initramfs)

ggiuffre · 2024-05-27 18:40:57

The problem is gone. I'm not 100% sure, but I think it was a combination of two things:

a mistake that I made when declaring my bootloader entries
possibly a bug in the Linux kernel, now apparently fixed

Yesterday I ran pacman -Syu, but then didn't think of giving another try to the "linux" kernel. Today I used the "linux" kernel instead of "linux-lts", and I was able to successfully decrypt the disk and log in to my new system. I'm pretty sure that the non-LTS kernel was not working two days ago, when I opened this topic. I could be wrong, though.

But then I found strange that this same bug made it to the LTS kernel, and therefore I started looking for mistakes from my side in the setup of the LTS bootloader entry. I found that in /boot/loader/entries/arch-lts.conf I had wrongly typed "initramfs-linux.img" instead of "initramfs-linux-lts.img". I don't know exactly what problem this typo created, but fixing that typo made the LTS kernel work again in my setup.

Unless anyone has something against it, I will then mark this topic as solved.

Once again, thank you all very much for the help.

frostschutz · 2024-05-27 20:24:17

Initramfs is interexchangable only for the same kernel version (e.g. initramfs and initramfs-fallback). Loading the initramfs of another kernel will cause incompatibility between kernel and module versions. As a result modules won't be loaded, devices won't be detected, and you end up with UUID missing and other errors. Unfortunately this isn't checked anywhere so there is no clearer error message.

ggiuffre · 2024-05-28 07:26:59

frostschutz wrote:

Initramfs is interexchangable only for the same kernel version (e.g. initramfs and initramfs-fallback). Loading the initramfs of another kernel will cause incompatibility between kernel and module versions. As a result modules won't be loaded, devices won't be detected, and you end up with UUID missing and other errors. Unfortunately this isn't checked anywhere so there is no clearer error message.

Thank you. Now I understand better.

I'm marking this as solved.

Arch Linux

#1 2024-05-25 12:45:19

[SOLVED] LUKS Systemd fresh install times out "waiting for device ..."

#2 2024-05-26 11:26:13

Re: [SOLVED] LUKS Systemd fresh install times out "waiting for device ..."

#3 2024-05-26 16:20:54

Re: [SOLVED] LUKS Systemd fresh install times out "waiting for device ..."

#4 2024-05-26 16:41:36

Re: [SOLVED] LUKS Systemd fresh install times out "waiting for device ..."

#5 2024-05-26 17:51:52

Re: [SOLVED] LUKS Systemd fresh install times out "waiting for device ..."

#6 2024-05-26 19:59:17

Re: [SOLVED] LUKS Systemd fresh install times out "waiting for device ..."

#7 2024-05-26 20:17:35

Re: [SOLVED] LUKS Systemd fresh install times out "waiting for device ..."

#8 2024-05-27 08:22:59

Re: [SOLVED] LUKS Systemd fresh install times out "waiting for device ..."

#9 2024-05-27 08:57:04

Re: [SOLVED] LUKS Systemd fresh install times out "waiting for device ..."

#10 2024-05-27 11:24:55

Re: [SOLVED] LUKS Systemd fresh install times out "waiting for device ..."

#11 2024-05-27 12:30:33

Re: [SOLVED] LUKS Systemd fresh install times out "waiting for device ..."

#12 2024-05-27 18:40:57

Re: [SOLVED] LUKS Systemd fresh install times out "waiting for device ..."

#13 2024-05-27 20:24:17

Re: [SOLVED] LUKS Systemd fresh install times out "waiting for device ..."

#14 2024-05-28 07:26:59

Re: [SOLVED] LUKS Systemd fresh install times out "waiting for device ..."

Board footer