Arch Linux

causasui

Member

Registered: 2013-04-04

Posts: 64

[SOLVED] dm-crypt failing to mount encrypted swap

Problem: I have fully encrypted root device & bootloader with LVM on LUKS, and now I'm trying to get resume from an encrypted swap partition working. Decryption of the root partition with a keyfile is working so that I only have to enter the passphrase once before grub starts.

When I boot up I get to "Reached target Initrd Root Device" and then "A start job is running for /dev/disk/by-uuid/c19468dd-0393-41d1-b7c0-b85f1e23b270" which runs for 1.5min before timing out. Then after "Reached target Local File Systems (pre)" I get the exact same start job timing out again.

When log in, I have not resumed from my previous session -- it's a cold boot -- and the swap partition is not decrypted.

Aside: I don't know if this is related, but if I typo the passphrase when decrypting the bootloader, grub drops me to a recovery console. Kind of weird but only mildly annoying. Resuming from swap is my goal for now -- I only mention it in case it's a clue to the root cause somehow.

I'm working off Dm-crypt/Swap_encryption on the wiki, but I've been over it several times and I can't figure out what I missed.

$ ls -lh /dev/disk/by-uuid/
total 24
lrwxrwxrwx  1 root  wheel    15B Aug  7 21:12 3787ae93-b1b5-4356-b70d-ee764aa52d8c -> ../../nvme0n1p2
lrwxrwxrwx  1 root  wheel    15B Aug  7 21:12 58709add-62a9-4794-8097-c95f9f87ba3b -> ../../nvme0n1p3
lrwxrwxrwx  1 root  wheel    10B Aug  7 21:12 e9f4aa98-e978-4396-88ab-6518d76c6b17 -> ../../dm-0

nvme0n1p2 is the encrypted swap partition. nvme0n1p3 is the encrypted root partition. dm-0 is the decrypted root partition.

I can decrypt and mount the swap partition manually like so:

# cryptsetup open --allow-discards --key-file /crypto_keyfile.bin /dev/disk/by-uuid/3787ae93-b1b5-4356-b70d-ee764aa52d8c swap
# mount -a

So I'm a bit lost as to why the setup I have below isn't doing what I needed.

Here are some relevant configuration files with comments stripped out for brevity.

/etc/initcpio/hooks/openswap
---
run_hook ()
{
    # Optional: To avoid race conditions
    x=0;
    while [ ! -b /dev/mapper/58709add-62a9-4794-8097-c95f9f87ba3b ] && [ $x -le 10 ]; do
       x=$((x+1))
       sleep .2
    done
    # End of optional

    mkdir -p /mnt/crypto_key_device
    mount /dev/mapper/cryptlvm /mnt/crypto_key_device
    cryptsetup open --allow-discards --key-file crypto_key_device/crypto_keyfile.bin /dev/disk/by-uuid/3787ae93-b1b5-4356-b70d-ee764aa52d8c swap
    umount crypto_key_device
}

/etc/initcpio/install/openswap
---
build ()
{
   add_runscript
}

help ()
{
cat<<HELPEOF
  This opens the swap encrypted partition /dev/nvme0n1p2 in /dev/mapper/swap
HELPEOF
}

mkinitcpio.conf
---
MODULES=(nvidia nvidia_modeset nvidia_uvm nvidia_drm)
BINARIES=("/usr/bin/btrfs")
FILES=("/crypto_keyfile.bin")
HOOKS=(base systemd autodetect keyboard sd-vconsole modconf block sd-encrypt openswap resume filesystems fsck)

/etc/default/grub
---
UUID_NVME="58709add-62a9-4794-8097-c95f9f87ba3b"  # encrypted root partition
UUID_ROOT="e9f4aa98-e978-4396-88ab-6518d76c6b17"  # decrypted root volume
UUID_ESWP="3787ae93-b1b5-4356-b70d-ee764aa52d8c"  # encrypted swap partition
UUID_SWAP="c19468dd-0393-41d1-b7c0-b85f1e23b270"  # decrypted swap volume

KEYFILE="/crypto_keyfile.bin"

GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="Arch"
GRUB_CMDLINE_LINUX_DEFAULT="audit=0 loglevel=3 quiet"
GRUB_CMDLINE_LINUX="rd.luks.name=$UUID_NVME=cryptlvm rd.luks.key=$KEYFILE root=UUID=$UUID_ROOT resume=UUID=$UUID_SWAP"

GRUB_PRELOAD_MODULES="part_gpt part_msdos"

GRUB_ENABLE_CRYPTODISK=y

GRUB_TIMEOUT_STYLE=menu
GRUB_TERMINAL_INPUT=console
GRUB_GFXMODE=auto
GRUB_GFXPAYLOAD_LINUX=keep

/etc/fstab
---
# <file system>                             <dir>       <type>      <options>                                                                   <dump> <pass>
# /dev/mapper/cryptlvm LABEL=root
UUID=e9f4aa98-e978-4396-88ab-6518d76c6b17   /           btrfs       rw,relatime,compress=zstd:3,ssd,space_cache,subvolid=256,subvol=/@,subvol=@ 0       0

# /dev/mapper/swap LABEL=swap
UUID=c19468dd-0393-41d1-b7c0-b85f1e23b270   none        swap        defaults,discard                                                            0       0

After I set that up (or change things) I do

# mkinitcpio -p linux
# grub-mkconfig -o /boot/grub/grub.cfg

before rebooting.

I've beet at this long enough I'm sure that some obvious thing must be staring me in the face. Help?

Last edited by causasui (2019-08-13 15:45:40)

nl6720

The Evil Wiki Admin

Registered: 2016-07-02

Posts: 671

Re: [SOLVED] dm-crypt failing to mount encrypted swap

The issue is simple. Your custom "openswap" hooks is a hook for busybox-based initramfs, but looking at mkinitcpio.conf, you're using a systemd-based initramfs. For systemd-based initramfs you don't need custom hooks to unlock multiple encrypted devices, just specify multiple rd.luks.* parameters:

GRUB_CMDLINE_LINUX="rd.luks.name=$UUID_NVME=cryptlvm rd.luks.name=$UUID_ESWP=swap rd.luks.key=$KEYFILE root=UUID=$UUID_ROOT resume=UUID=$UUID_SWAP"

What I don't understand is, if you're using LVM on LUKS, why is swap a separate encrypted partition instead of just a logical volume in LVM? If it was a logical volume in LVM it would not require separate unlocking.

Last edited by nl6720 (2019-08-08 06:58:52)

frostschutz

Member

Registered: 2013-11-15

Posts: 1,503

Re: [SOLVED] dm-crypt failing to mount encrypted swap

I'm trying to get resume from an encrypted swap partition working.

You are trying to corrupt your data.

/etc/initcpio/hooks/openswap
---
run_hook ()
{
    # Optional: To avoid race conditions
    x=0;
    while [ ! -b /dev/mapper/58709add-62a9-4794-8097-c95f9f87ba3b ] && [ $x -le 10 ]; do
       x=$((x+1))
       sleep .2
    done
    # End of optional

    mkdir -p /mnt/crypto_key_device
    mount /dev/mapper/cryptlvm /mnt/crypto_key_device
    cryptsetup open --allow-discards --key-file crypto_key_device/crypto_keyfile.bin /dev/disk/by-uuid/3787ae93-b1b5-4356-b70d-ee764aa52d8c swap
    umount crypto_key_device
}

You are not allowed to mount cryptlvm (root filesystem) before resume from swap. (Technically this is even true for read-only mounts.)

https://www.kernel.org/doc/Documentatio … swsusp.txt

 * BIG FAT WARNING *********************************************************
 *
 * If you touch anything on disk between suspend and resume...
 *				...kiss your data goodbye.
 *
 * If you do resume from initrd after your filesystems are mounted...
 *				...bye bye root partition.
 *			[this is actually same case as above]
 *

You should think of a way to obtain key elsewhere w/o mounting anything or re-use same key already used for cryptlvm.

In fact when resuming there's no point to even open the cryptlvm at all, (unless the swap itself is inside of it), the resume would still have it open after all.

So when resuming, you open the swap first, and only when nothing to resume, open the cryptlvm.

----

Another thing to note, when you change your initramfs (the order it opens devices, changing dm-X names in the process, etc.) it's safer to boot the new initramfs normally first. Not hibernate from old way of booting and try to resume with new way of booting. Things can go wrong. Same if you switch kernels etc., shouldn't try to hibernate old kernel and resume with new kernel.

Last edited by frostschutz (2019-08-08 09:50:31)

causasui

Member

Registered: 2013-04-04

Posts: 64

Re: [SOLVED] dm-crypt failing to mount encrypted swap

What I don't understand is, if you're using LVM on LUKS, why is swap a separate encrypted partition instead of just a logical volume in LVM? If it was a logical volume in LVM it would not require separate unlocking.

That... is a good question. I'm not sure either, now that you mention it. I think that would neatly solve the issues raised by frostschutz also. I'm not too terribly far into this install so I think I'll just nuke it from orbit and start over with that plan.

I guess that's the obvious thing I was missing

You are not allowed to mount cryptlvm (root filesystem) before resume from swap. (Technically this is even true for read-only mounts.)

That is alarming. Thank you for pointing that out since it settles the question of whether I'm redoing my partition scheme.

I see now that there is a notice that the article "needs expansion" to discuss this issue more. I think that should just be a red warning. Maybe I'll post a suggestion on the talk page.

P.S. When I get it working I'll come back and mark the thread solved.

Last edited by causasui (2019-08-08 16:42:21)

frostschutz

Member

Registered: 2013-11-15

Posts: 1,503

Re: [SOLVED] dm-crypt failing to mount encrypted swap

otherwise you could create a tiny LV (1 extent) just for holding the key (no filesystem, avoiding the mount issue) or use `filefrag` to create a blocklist for dd-reading the keyfile directly w/o mounting. (example in a different context https://unix.stackexchange.com/a/85880/30851 ). another idea would be to put it in the last sector of the encrypted PV ( LVM tends to have unused space at the end, verify it says "unused space: non-zero value" in vgdisplay )

the blocklist approach (sometimes also used by grub) only works as long as the file is not touched or moved. if you want to go the extra mile you could generate blocklist dynamically in the install part of the hook

but if the issue can be avoided in the first place, so much the better

Last edited by frostschutz (2019-08-08 17:11:36)

causasui

Member

Registered: 2013-04-04

Posts: 64

Re: [SOLVED] dm-crypt failing to mount encrypted swap

Yeah it really seems like having a separate swap partition is totally unnecessary and the cause of all this grief I've been having. I'm not super far into this setup and so I'd rather just start over and not have to deal with any of these moving parts going forward. I'm using LVM anyway so I can't think of an advantage of what I'm doing over what nl6720 suggested.

causasui

Member

Registered: 2013-04-04

Posts: 64

Re: [SOLVED] dm-crypt failing to mount encrypted swap

Edit: brain fart

Last edited by causasui (2019-08-13 02:51:20)

causasui

Member

Registered: 2013-04-04

Posts: 64

Re: [SOLVED] dm-crypt failing to mount encrypted swap

Alright, swap is much better. I'm still getting this weirdness where if I typo the passphrase I get dropped to grub rescue. I would think I would get to type it again?

Edit: Actually I'll make a new thread and mark this solved. Thanks again!

Last edited by causasui (2019-08-13 15:44:54)