GRUB fails to open encrypted /boot after installation

6i5 · 2024-06-10 20:05:10

I changed the partitioning layout a little bit since my last post to make it usable. Although GRUB installed successfully without errors, I could not access the system after reboot, as GRUB was unable to unlock /boot. Are there any steps that I missed? Assume other things such as installing basic packages and configuring misc. settings was done properly.

Partitions

1. Create Partitions

> fdisk /dev/nvme0n1
> fdisk /dev/nvme1n1
> lsblk
NAME		RM	SIZE	RO	TYPE	MOUNTPOINTS
loop0		0	...		1	loop	/run/archiso/airootfs
nvme1n1		0	3.6T	0	disk
└─nvme1n1p1	0	...		0	part
nvme0n1		0	3.6T	0	disk
└─nvme0n1p1	0	...		0	part
└─nvme0n1p2	0	...		0	part
> mkfs.fat -F 32 /dev/nvme0n1p1
> mount --mkdir /dev/nvme0n1p1 /mnt/efi

> pvcreate /dev/nvme0n1p2
> pvcreate /dev/nvme1n1p1
> vgcreate vg001 /dev/nvme0n1p2
> vgextend vg001 /dev/nvme1n1p1
> pvs
PV				VG		Fmt		Attr	PSize	PFree
/dev/nvme0n1p2	vg001	lvm2	---		<3.64t	<3.64t
/dev/nvme1n1p1	vg001	lvm2	---		<3.64t	<3.64t

> lvcreate -L 1G -n cryptboot vg001
> lvcreate -L 64G -n crypttmp vg001
> lvcreate -L 64G -n cryptvar vg001
> lvcreate -L 128G -n cryptroot vg001
> lvcreate -l 100%FREE -n crypthome vg001
> lvs
LV			VG		Attr		LSize
cryptboot	vg001	-wi-a-----	1.00g
crypthome	vg001	-wi-a-----	...
cryptroot	vg001	-wi-a-----	128.00g
crypttmp	vg001	-wi-a-----	64.00g
cryptvar	vg001	-wi-a-----	64.00g

2. Encrypt Root Partition

> cryptsetup luksFormat --type luks2 --cipher aes-xts-plain64 --hash sha512 --iter-time 5000 --key-size 512 --pbkdf=pbkdf2 --use-urandom --verify-passphrase /dev/vg001/cryptroot
> cryptsetup open /dev/vg001/cryptroot root
> mkfs.ext4 /dev/mapper/root
> mount /dev/mapper/root /mnt

3. Encrypt Remaining Partitions

> cryptsetup luksFormat --type luks2 --cipher aes-xts-plain64 --hash sha512 --iter-time 5000 --key-size 512 --pbkdf=pbkdf2 --use-urandom --verify-passphrase /dev/vg001/cryptboot
> cryptsetup open /dev/vg001/cryptboot boot
> mkfs.ext4 /dev/mapper/boot
> mount --mkdir /dev/mapper/boot /mnt/boot
...
> lsblk -f
NAME				FSTYPE		FSVER	LABEL	UUID	FSAVAIL	FSUSE%	MOUNTPOINTS
loop0				squashfs	4.0							0	100%	/run/archiso/airootfs
nvme1n1
└─nvme1n1p1			LVM2_member	LVM2 001		...
  └─vg001-crypthome	crypto_LUKS	2				...
    └─home			ext4		1.0				...		...		0%		/mnt/home
nvme0n1
├─nvme0n1p1			vfat		FAT32			...						/mnt/efi
└─nvme0n1p2			LVM2_member	LVM2 001		...
  ├─vg001-cryptboot	crypto_LUKS	2				...
  │ └─boot			ext4		1.0				...		907M	0%		/mnt/boot
  ├─vg001-crypttmp	crypto_LUKS	2				...
  │ └─tmp			ext4		1.0				...		59.5G	0%		/mnt/tmp
  ├─vg001-cryptvar	crypto_LUKS	2				...
  │ └─var			ext4		1.0				...		59.5G	0%		/mnt/var
  ├─vg001-cryptroot	crypto_LUKS	2				...
  │ └─root			ext4		1.0				...		119G	0%		/mnt
  └─vg001-crypthome	crypto_LUKS	2				...
    └─home			ext4		1.0				...		...		0%		/mnt/home

4. Generate fstab

> genfstab -U /mnt >> /mnt/etc/fstab
> cat /mnt/etc/fstab
...
# <file system> 	<dir> 	<type> 	<options>
# /dev/nvme0n1p1
UUID=...			/efi	vfat	...
# /dev/mapper/root
UUID=...			/		ext4	rw,realtime
# /dev/mapper/boot
UUID=...			/boot	ext4	rw,realtime
# /dev/mapper/tmp
UUID=...			/tmp	ext4	rw,realtime
# /dev/mapper/var
UUID=...			/var	ext4	rw,realtime
# /dev/mapper/home
UUID=...			/home	ext4	rw,realtime

5. Change Root

root@archiso ~ # arch-chroot /mnt
[root@archiso /] #

6. Edit /etc/mkinitcpio.conf

> nano /etc/mkinitcpio.conf
> cat /etc/mkinitcpio.conf
...
HOOKS=(base udev autodetect microcode modconf kms keyboard keymap consolefont block lvm2 encrypt filesystems fsck)
...

7. Configure GRUB

> nano /etc/default/grub
> cat /etc/default/grub
...
GRUB_CMDLINE_LINUX="... cryptdevice=UUID=(Insert UUID of vg001-cryptboot):cryptlvm ..."
...
GRUB_ENABLE_CRYPTODISK=y
...

8. Install GRUB

> grub-install --target=x86_64-efi --efi-directory=/efi --bootloader-id=GRUB --recheck
> grub-mkconfig -o /boot/grub/grub.cfg
> reboot

9. Unlock Disk

Enter passphrase for lvm/vg001-cryptboot (...):
error: Invalid passphrase.
erorr: disk '...' not found.
Entering rescue mode...
grub rescue>

Last edited by 6i5 (2024-06-10 20:05:31)

frostschutz · 2024-06-10 20:09:36

the linux cmdline will need the root luks uuid

not sure why grub can't open it if you used pbkdf2, show luksdump? (you can still use argon2id for the other partitions that grub won't be touching - also you can use multiple keyslots anyway)

how complicated is your passphrase, can you rule out keyboard layout issues etc.? try a simple one just for testing

Last edited by frostschutz (2024-06-10 20:09:47)

6i5 · 2024-06-10 20:34:22

frostschutz wrote:

the linux cmdline will need the root luks uuid

Not exactly sure what you mean, would you clarify?
Do you mean that more than one UUID needs to be present in GRUB_CMDLINE_LINUX?

/boot /tmp /var / /home were all encrypted separately with LUKS2, using the same password.

I do notice that the UUID presented in the "Enter Passphrase" prompt is different from what I input in GRUB_CMDLINE_LINUX.
The UUID I inserted there was from the /dev/mapper/boot entry of /etc/fstab.
Perhaps this is not the right UUID?
If so can I fix this from grub rescue by finding the correct UUID?

frostschutz · 2024-06-10 21:09:00

Grub will try to open the /boot partition (if you encrypted /boot, /boot/grub, /boot/your kernel and initramfs).

The kernel,initrd however will not care about the /boot partition at all, it will try to open the / root filesystem. So the cmdline linux should have the LUKS UUID which contains the encrypted root filesystem. According to what you posted above, you put the cryptboot there, that would be wrong.

That's not your immediate issue with Grub rescue though. You'll have to find out why Grub fails there. argon2id keyslot would be one reason (was already mentioned in the other thread). Otherwise an issue with the passphrase or keyboard layout.

Last edited by frostschutz (2024-06-10 21:09:42)

6i5 · 2024-06-11 06:38:44

frostschutz wrote:

That's not your immediate issue with Grub rescue though. You'll have to find out why Grub fails there. argon2id keyslot would be one reason (was already mentioned in the other thread). Otherwise an issue with the passphrase or keyboard layout.

Following

grub-mkconfig -o /boot/grub/grub.cfg

I inspected /boot/grub/grub.cfg. I see the line

insmod gcry_sha256

. Since my LUKS2 containers use sha512, I believed that was the reason why GRUB was unable to unlock the volumes. I manually edited the line to gcry_sha512 then booted. Same issue.

I tried

cryptomount -a

in the rescue shell. I was prompted to enter the password for all five modules one by one but none of the five encrypted LVMs could be unlocked, although they shared the same password 'a'. Since I can type the character just fine on the shell it is not a keymap problem. I am out of ideas why I cannot unlock the volumes. GRUB 2.12 clearly supports LUKS2 pbkdf2. Why is it refusing to unlock it?

Or is the gcry module not loaded because /boot is locked? In this case how do I load it in GRUB rescue?

Last edited by 6i5 (2024-06-11 06:39:26)

frostschutz · 2024-06-11 06:47:12

Can you check luksDump if it's pbkdf2 as intended? Otherwise I'm not sure, sorry.

Editing your encrypted grub.cfg will not help. Grub uses a built-in cfg (generated at grub-install time) for the initial setup. If you make any changes in early configuration you have to re-run grub-install (from within arch-chroot).

In Grub rescue you can't load any modules (unless it's in a reachable, unencrypted location). Basically Grub rescue uses built-in modules (included at grub-install time, grub-install has options to include extra modules) and this should include everything necessary to make the jump (to access the real boot filesystem and full grub cfg and modules paths).

Any module you specify at grub-install time does not have to be insmod later so this is one way to simplify grub.cfg if you write it by hand. Not your issue though; it's supposed to work as is (grub-install is supposed to be smart enough to include the crypto stuff. it could be missing LVM I guess but then I'd expect a different error message, but then again Grub's crypto support is rudimentary and rarely gives any good errors).

Last edited by frostschutz (2024-06-11 06:51:34)

6i5 · 2024-06-11 21:05:46

frostschutz wrote:

Can you check luksDump if it's pbkdf2 as intended? Otherwise I'm not sure, sorry.

Yeah it is luks2 sha512 pbkdf2.

frostschutz wrote:

Editing your encrypted grub.cfg will not help. Grub uses a built-in cfg (generated at grub-install time) for the initial setup. If you make any changes in early configuration you have to re-run grub-install (from within arch-chroot).

If so how do I force insmod gcry_sha512 instead of insmod gcry_sha256 during grub-install?

frostschutz wrote:

Any module you specify at grub-install time does not have to be insmod later so this is one way to simplify grub.cfg if you write it by hand. Not your issue though; it's supposed to work as is (grub-install is supposed to be smart enough to include the crypto stuff. it could be missing LVM I guess but then I'd expect a different error message, but then again Grub's crypto support is rudimentary and rarely gives any good errors).

grub.cfg does show insmod lvm2 a few lines above insmod gcry_sha256. If replacing it with gcry_sha512 does not work, I guess I will have to give up trying to encrypt my drives. Too bad.

Last edited by 6i5 (2024-06-11 21:06:12)

kermit63 · 2024-06-11 21:53:43

One option is to leave /boot either unencrypted or encrypted with luks1, then encrypt the remaining partitions with luks2 argon2.

For convenience, you can register a keyfile with the luks2 argon2 encrypted partitions, then use /etc/crypttab to decrypt them automatically during boot.

6i5 · 2024-06-11 22:22:50

kermit63 wrote:

One option is to leave /boot either unencrypted or encrypted with luks1, then encrypt the remaining partitions with luks2 argon2. For convenience, you can register a keyfile with the luks2 argon2 encrypted partitions, then use /etc/crypttab to decrypt them automatically during boot.

Really? Official GRUB documentation explicitly states that Argon2 is not supported yet.

Page 92/160 of PDF wrote:

GRUB suports devices encrypted using LUKS, LUKS2 and geli. Note that necessary modules (luks, luks2 and geli) have to be loaded manually before this command can be used. For LUKS2 only the PBKDF2 key derivation function is supported, as Argon2 is not yet supported.

kermit63 · 2024-06-11 22:52:25

That only applies to /boot. Since /boot as I stated is either unencrypted or luks1, then it can access the kernel and initram. After that, GRUB has already done its job, so the restriction on argon2 no longer applies. This is actually my current set-up.

Please refer to post #4 by frostschutz.

Last edited by kermit63 (2024-06-11 22:57:23)

6i5 · 2024-06-12 02:10:16

kermit63 wrote:

That only applies to /boot. Since /boot as I stated is either unencrypted or luks1, then it can access the kernel and initram. After that, GRUB has already done its job, so the restriction on argon2 no longer applies. This is actually my current set-up.
Please refer to post #4 by frostschutz.

I followed the same steps but this time left cryptboot unencrypted. Successfully booted to GRUB. However following selecting a kernel from GRUB menu,

A password is required to access the cryptlvm volume:
Enter passphrase for /dev/mapper/vg001-cryptroot:
ERROR: device '/dev/mapper/root' not found. Skipping fsck.
mount: /new_root: no valid filesystem type specified.
ERROR: Failed to mount '/dev/mapper/root' on real root
You are now being dropped into an emergency shell.
sh: can't access tty; job control turned off

[rootfs ~]# cryptomount -a
sh: cryptomount: not found
mount --mkdir /dev/mapper/vg001-cryptroot /root
[rootfs ~]# mount: /root: unknown filesystem type 'crypto_LUKS'.

I assume the password did get through, since there is no "wrong password" message. However once I type the password it hangs for ~30 seconds then spits out the errors. cryptomount is not available either this time. Not sure what to do from here. Manually installing Arch is so painstaking.

kermit63 · 2024-06-12 05:21:14

This is the problem with obfuscating the UUIDs in your previous posts. We have no way of checking whether you are specifying the correct UUID in /etc/grub/default. I suggest you take a look at GRUB_CMDLINE_LINUX and check if you're using the correct UUID. This is what frostschutz has been telling you all along.

EDIT: corrected name typo.

Last edited by kermit63 (2024-06-12 09:11:22)

frostschutz · 2024-06-12 07:16:27

Show `cat /proc/cmdline` ... it's looking for /dev/mapper/root but if your cmdline you posted above is still correct, you named it /dev/mapper/cryptlvm instead.

If you don't use root=UUID=your-rootfs-uuid then root= has to match the device name you assigned.

Last edited by frostschutz (2024-06-12 07:16:42)

6i5 · 2024-06-12 16:58:22

kermit63 wrote:

This is the problem with obfuscating the UUIDs in your previous posts. We have no way of checking whether you are specifying the correct UUID in /etc/grub/default. I suggest you take a look at GRUB_CMDLINE_LINUX and check if you're using the correct UUID. This is what frostschutz has been telling you all along.

frostschutz wrote:

Show `cat /proc/cmdline` ... it's looking for /dev/mapper/root but if your cmdline you posted above is still correct, you named it /dev/mapper/cryptlvm instead. If you don't use root=UUID=your-rootfs-uuid then root= has to match the device name you assigned.

Trying one more time. Is everything good now? Will my system be functional following a reboot? I do notice that fstab is missing /efi, does that matter?

> blkid
/dev/mapper/vg001-crypttmp: UUID="aa"
/dev/nvme0n1p1: UUID="bb"
/dev/nvme0n1p2: UUID="cc"
/dev/mapper/home: UUID="dd"
/dev/mapper/tmp: UUID="ee"
/dev/mapper/vg001-crypthome: UUID="ff"
/dev/mapper/vg001-cryptvar: UUID="00"
/dev/loop0: UUID="11"
/dev/mapper/vg001-cryptboot: UUID="22"
/dev/nvme1n1p1: UUID="33"
/dev/mapper/root: UUID="44"
/dev/mapper/vg001-cryptroot: UUID="55"

> cat /etc/default/grub
...
GRUB_CMDLINE_LINUX="... cryptdevice=UUID=55:cryptroot ..."
...

> cat /proc/cmdline
initrd=\arch\boot\x86_64\initramfs-linux.img archisobasedir=arch archisosearchuuid=2024-06-01-09-07-18-00

> cat /etc/fstab
# <file system> 	<dir> 	<type> 	<options>
# /dev/mapper/root
UUID=...			/		ext4	rw,realtime
# /dev/mapper/vg001-cryptboot
UUID=...			/boot	ext4	rw,realtime
# /dev/mapper/tmp
UUID=...			/tmp	ext4	rw,realtime
# /dev/mapper/var
UUID=...			/var	ext4	rw,realtime
# /dev/mapper/home
UUID=...			/home	ext4	rw,realtime

Last edited by 6i5 (2024-06-12 16:58:45)

6i5 · 2024-06-13 08:32:16

frostschutz wrote:

Show `cat /proc/cmdline` ... it's looking for /dev/mapper/root but if your cmdline you posted above is still correct, you named it /dev/mapper/cryptlvm instead. If you don't use root=UUID=your-rootfs-uuid then root= has to match the device name you assigned.

Good news: changing cryptlvm to boot in /etc/default/grub, root is mounted successfully.

GRUB_CMDLINE_LINUX="... cryptdevice=UUID=(UUID of vg001-cryptboot):root ..."

However the system does not mount the other three locked drives automatically.

Enter passphrase for /dev/mapper/vg001-cryptroot:
/dev/mapper/root: clean. .../... files, .../... blocks
[ TIME ] Timed out waiting for device /dev/disk/by-uuid/...
[DEPEND] Dependency failed for /home.
[DEPEND] Dependency failed for Local File Systems.
[DEPEND] Dependency failed for File system Check on /dev/disk/by-uuid/...
...
You are in emergency mode. After logging in, type "journalctl -xb" to view
system logs, "systemctl reboot" to reboot, or "exit"
to continue bootup.
Give root password for maintenance
(or press Control-D to continue):
# cryptomount -a
bash: cryptomount: command not found

And I can't use cryptomount to mount the drives either. I think I am almost done, just need to figure out how to have the system auto-unlock and mount /home /tmp /var once I unlock the root directory. Can I set this up now or do I need to go through the installation yet another time?

Last edited by 6i5 (2024-06-13 08:32:39)

frostschutz · 2024-06-13 08:38:41

do you have them in crypttab?

6i5 · 2024-06-14 02:24:06

frostschutz wrote:

do you have them in crypttab?

I had not setup /etc/crypttab at all. I edited the file as follows in maintenance mode. I also created a file /etc/password.txt which contains the password shared by all three remaining directories in plaintext.

> cat /etc/crypttab
...
# <name>	<device>	<password>			<options>
tmp			UUID=...	/etc/password.txt	luks
var			UUID=...	/etc/password.txt	luks
home		UUID=...	/etc/password.txt	luks

> cat /etc/password.txt
iluvarchlinux

A reboot following the changes, after I unlock cryptroot, I am prompted to enter the password for the three drives. The three disks unlock successfully, and I can finally log in to my desktop.

Please enter passphrase for disk vg001-cryptvar (var):
Please enter passphrase for disk vg001-crypttmp (tmp):
Please enter passphrase for disk vg001-crypthome (home):

Any idea why the three disks do not unlock automatically even though the path to the password is specified in /etc/crypttab?

frostschutz · 2024-06-14 08:22:57

a passphrase as a keyfile will work only if there is no newline at the end of file (check with hexdump -C)

also make sure the file permissions are so that regular users can not access it

SimonJ · 2024-06-14 08:46:05

frostschutz wrote:

a passphrase as a keyfile will work only if there is no newline at the end of file (check with hexdump -C)

This was my main issue when creating a key file.

I used this

echo -n '<your luks password>' /run/media/simon/USBDRIVE/<filename>

6i5 · 2024-06-14 20:09:55

frostschutz wrote:

a passphrase as a keyfile will work only if there is no newline at the end of file (check with hexdump -C) also make sure the file permissions are so that regular users can not access it

SimonJ wrote:

I used this

echo -n '<your luks password>' /run/media/simon/USBDRIVE/<filename>

Removed the newline and now it is working! Is it possible to encrypt the boot partition now with LUKS1 without destroying its contents? If it needs to be reformatted, how do I regenerate the directory after formatting?

> lsblk -f
NAME				FSTYPE		FSVER	MOUNTPOINTS
nvme0n1
└─nvme0n1p1			LVM2_member	LVM2 001
  └─vg001-crypthome	crypto_LUKS	2
    └─home			ext4		1.0		/mnt/home
nvme1n1
├─nvme1n1p1			vfat		FAT32
└─nvme1n1p2			LVM2_member	LVM2 001
  ├─vg001-cryptboot	ext4		1.0		/mnt/boot
  ├─vg001-crypttmp	crypto_LUKS	2
  │ └─tmp			ext4		1.0		/mnt/tmp
  ├─vg001-cryptvar	crypto_LUKS	2
  │ └─var			ext4		1.0		/mnt/var
  ├─vg001-cryptroot	crypto_LUKS	2
  │ └─root			ext4		1.0		/mnt
  └─vg001-crypthome	crypto_LUKS	2
    └─home			ext4		1.0		/mnt/home

Arch Linux

#1 2024-06-10 20:05:10

GRUB fails to open encrypted /boot after installation

#2 2024-06-10 20:09:36

Re: GRUB fails to open encrypted /boot after installation

#3 2024-06-10 20:34:22

Re: GRUB fails to open encrypted /boot after installation

#4 2024-06-10 21:09:00

Re: GRUB fails to open encrypted /boot after installation

#5 2024-06-11 06:38:44

Re: GRUB fails to open encrypted /boot after installation

#6 2024-06-11 06:47:12

Re: GRUB fails to open encrypted /boot after installation

#7 2024-06-11 21:05:46

Re: GRUB fails to open encrypted /boot after installation

#8 2024-06-11 21:53:43

Re: GRUB fails to open encrypted /boot after installation

#9 2024-06-11 22:22:50

Re: GRUB fails to open encrypted /boot after installation

#10 2024-06-11 22:52:25

Re: GRUB fails to open encrypted /boot after installation

#11 2024-06-12 02:10:16

Re: GRUB fails to open encrypted /boot after installation

#12 2024-06-12 05:21:14

Re: GRUB fails to open encrypted /boot after installation

#13 2024-06-12 07:16:27

Re: GRUB fails to open encrypted /boot after installation

#14 2024-06-12 16:58:22

Re: GRUB fails to open encrypted /boot after installation

#15 2024-06-13 08:32:16

Re: GRUB fails to open encrypted /boot after installation

#16 2024-06-13 08:38:41

Re: GRUB fails to open encrypted /boot after installation

#17 2024-06-14 02:24:06

Re: GRUB fails to open encrypted /boot after installation

#18 2024-06-14 08:22:57

Re: GRUB fails to open encrypted /boot after installation

#19 2024-06-14 08:46:05

Re: GRUB fails to open encrypted /boot after installation

#20 2024-06-14 20:09:55

Re: GRUB fails to open encrypted /boot after installation

Board footer