Arch Linux

agrahamlincoln

Member

Registered: 2020-05-02

Posts: 3

[Solved] cryptsetup isLuks hangs in initramfs

Hello!

I have a number of arch systems that have been running for some time, but given that this is very much a startup process, I figured it was most appropriate to put in "installation" section.

TLDR
Problem: My machine doesn't boot anymore

background on this machine:

* LUKS on LVM on mdadm RAID0
* Both /etc/default/grub and /etc/mkinitcpio.conf were configuring and working fine. I was using this machine as a daily driver until this broke on me.

I rebooted my machine yesterday after installing updates to find that it would no longer boot. The boot would hang after reporting systemd was initializing, and there was no feedback from there. I booted to my liveusb, and pieced together my filesystem and couldn't find anything out of the ordinary, so I increased the log verbosity and enabled xtrace for init ('sd.debug=true').

After enabling xtrace, I found that the init scripts were hanging in the [encrypt] hook, when probing the disk for the luks headers. Heres the line it's stuck on, and a few lines before it:

+ echo /dev/mapper/vg0-root0
+ return 0
+ resolved=/dev/mapper/vg0-root0
+ cryptsetup isLuks /dev/mapper/vg0-root0
# it hangs here, every time

I went further and removed all of the GRUB_CMDLINE_LINUX options except for the log level and sd.debug, and enabled sd.shell to get an emergency shell intentionally on next boot. Now I have a shell in the booted initramfs image, and am able to run commands manually. At this point, I can run "cryptsetup isLuks" on files that are obviously not a cryptdevice, and checked they were returning false correctly, but when I use this directly on /dev/vg0/root0 or /dev/mapper/vg0-root0, the command hangs forever.

Things I've tried:

* pacman -Syu from archiso
* rebuilding initramfs manually
* linux-lts
* downgrading linux to the version I last booted on
* downgrading cryptsetup (there was some recent updates related to json-c, but issues reported that were relevant to recent updates were all from creating volumes, not checking them)

I did not try downgrading systemd, only because I did have a clean boot on 245-5.2 prior to this occurring.

I have no idea where to go next with this.. Any clues?

Last edited by agrahamlincoln (2020-05-06 00:22:26)

junaru

Member

Registered: 2007-01-14

Posts: 12

Re: [Solved] cryptsetup isLuks hangs in initramfs

< Eddited because suggestion was a long shot and not related to the issue. >

Last edited by junaru (2020-05-03 09:30:27)

saidie

Member

Registered: 2020-05-03

Posts: 1

Re: [Solved] cryptsetup isLuks hangs in initramfs

I have encountered this issue too after system update.
By downgrading json-c (0.14-1 -> 0.13.1-3) and cryptsetup (2.3.2-1 -> 2.3.1-1), I could workaround this issue; [encrypt] hook shows password prompt now.

But some packages seems to depend on json-c implicitly and showed strange behavior. So I rolled back every packages what I updated recently and it looks everything is fine now.

frostschutz

Member

Registered: 2013-11-15

Posts: 1,505

Re: [Solved] cryptsetup isLuks hangs in initramfs

there was another issue like this just the other day https://gitlab.com/cryptsetup/cryptsetup/-/issues/556

but it was supposed to be fixed in 2.3.1-3 or newer

can't reproduce a hang for isLuks myself, although it's noticably slower on LUKS2 headers than LUKS1, guess it does more validation/parsing for LUKS2

Last edited by frostschutz (2020-05-03 07:51:11)

chris_l

Member

Registered: 2010-12-01

Posts: 390

Re: [Solved] cryptsetup isLuks hangs in initramfs

Well, its not fixed in my case. I tried with the latest version of cryptsetup, which is 2.3.2-1. And it just hangs.

It works fine with 2.3.1-1

---

"open source is about choice"
No.
Open source is about opening the source code complying with this conditions, period. The ability to choose among several packages is just a nice side effect.

yoant

Member

Registered: 2020-05-04

Posts: 3

Re: [Solved] cryptsetup isLuks hangs in initramfs

Well, its not fixed in my case. I tried with the latest version of cryptsetup, which is 2.3.2-1. And it just hangs.

It works fine with 2.3.1-1

Same here.

I've seen [this workaround](https://gitlab.com/cryptsetup/cryptsetu … _335279806), but I haven't tested it yet.
(However

cryptsetup luksDump /dev/sdax

does not show any flag / allow-discard flag, so I'm not sure this is related).

frostschutz

Member

Registered: 2013-11-15

Posts: 1,505

Re: [Solved] cryptsetup isLuks hangs in initramfs

so, any of you should probably file a bug, then

a wild bug report appeared: https://bugs.archlinux.org/task/66531

Last edited by frostschutz (2020-05-04 19:52:19)

yoant

Member

Registered: 2020-05-04

Posts: 3

Re: [Solved] cryptsetup isLuks hangs in initramfs

That has been fixed (for me) by latest json-c patch.

It was apparently an RDRAND related issue (occurring with AMD Ryzen CPUs):

The new json-c is reported to have issues with broken RDRAND during initialization.

Thanks!

agrahamlincoln

Member

Registered: 2020-05-02

Posts: 3

Re: [Solved] cryptsetup isLuks hangs in initramfs

thanks guys! I was able to get my system to boot after downgrading both json-c and cryptsetup as mentioned by others in here.

My machine is on AMD Ryzen and I do encounter the RDRAND issue as well - I started booting with `nordrand` in the kernel parameters while mucking around with the settings there and the warnings went away.

Marking this thread as solved!

yoant

Member

Registered: 2020-05-04

Posts: 3

Re: [Solved] cryptsetup isLuks hangs in initramfs

My machine is on AMD Ryzen and I do encounter the RDRAND issue as well - I started booting with `nordrand` in the kernel parameters while mucking around with the settings there and the warnings went away.

FYI for me a motherboard firmware update solved the RDRAND issue

agrahamlincoln

Member

Registered: 2020-05-02

Posts: 3

Re: [Solved] cryptsetup isLuks hangs in initramfs

My machine is on AMD Ryzen and I do encounter the RDRAND issue as well - I started booting with `nordrand` in the kernel parameters while mucking around with the settings there and the warnings went away.

FYI for me a motherboard firmware update solved the RDRAND issue

Good call - I thought I had updated the bios when I got this machine, but I had not. I updated the firmware and turned off `nordrand` and there are no more RDRAND messages