Arch Linux

dcepelik

Member

From: Prague

Registered: 2013-11-23

Posts: 10

Booting from LUKS partition slow, slow opening of LUKS partition

Hello to everyone,

I have completed pretty straightforward installation of Arch Linux following this wiki and the boot takes too long.

The output of systemd-analyze shows that the loader takes almost 18 seconds, whereas the other times are negligible.

The system is a brand new T460p. I have tried a minimal installation with a single ext3 partition and the boot time was 11 seconds total compared to current 32 seconds total. The difference is in the loader.

The only difference is the use of Btrfs and LUKS. Since I've used Btrfs many times before without problems, I suspect this problem is LUKS-related.

I have also noticed that

cryptsetup open <luks-partition>

takes about 6 secs, while on my friend's T460s takes about a second. I think this is the cause.

I have compared outputs of

cryptsetup luksDump <luks-formatted-device>

on both machines and the only (interesting) difference is NM bits, which is 512 for me compared to his 256. Surprisingly, his Iterations values are about three times greater than mine.

Could anyone provide hints?

Thanks
David

Last edited by dcepelik (2016-09-16 20:51:47)

frostschutz

Member

Registered: 2013-11-15

Posts: 1,472

Re: Booting from LUKS partition slow, slow opening of LUKS partition

Was the LUKS container created on the same machine, using the same cryptsetup binary in the same environment, without crazy --iter-time parameters? How many keyslots are in use and which is the one you're trying to unlock?

The iteration count used by LUKS is entirely dynamic. By default LUKS attempts to invest 1 second of CPU time when adding a key. If you do that while the system is idle, but it's not idle when booting up, it might take longer then. So say it takes 2 seconds to unlock the first key. LUKS supports up to 8 keys, it takes that amount of CPU time for each key it tries to unlock so if you're using the 8th key (without telling LUKS to try the 8th key first) you'll look at 16 seconds opening time.

If you created the LUKS container on a considerably faster machine and then transfer the disk to a slow rust bucket you'll look at very long opening time... 1 second CPU time for a fast CPU will take a considerably longer time on a slow one. Same way the other way around, if you move your stone age LUKS to a new monster machine it will open so much faster.

There is also a huge difference depending which crypto library your cryptsetup binary happens to use... I benchmarked this a while ago with 4 libraries supported by cryptsetup http://www.metamorpher.de/files/cryptse … hmark.html

It's possible to change the key itercounts by luksChangeKey (change the key to the same key). Changing the master key itercount requires the LUKS header to be re-created as a whole, using cryptsetup-reencrypt --keep-key or the manual method http://unix.stackexchange.com/a/178722/30851

You should not do any of this without a backup.

Edit: I guess dynamic CPU clock could also be an issue, if it ran in turbo mode when creating the LUKS header, but won't go into turbo mode [or might even be stuck in power saving] when unlocking it, the process will take so much longer...

Last edited by frostschutz (2016-09-16 21:11:12)

dcepelik

Member

From: Prague

Registered: 2013-11-23

Posts: 10

Re: Booting from LUKS partition slow, slow opening of LUKS partition

Was the LUKS container created on the same machine, using the same cryptsetup binary in the same environment, without crazy --iter-time parameters?

Yes to all.

How many keyslots are in use and which is the one you're trying to unlock?

Only two and I'm unlocking using the first key.

If you created the LUKS container on a considerably faster machine and then transfer the disk to a slow rust bucket you'll look at very long opening time... 1 second CPU time for a fast CPU will take a considerably longer time on a slow one. Same way the other way around, if you move your stone age LUKS to a new monster machine it will open so much faster.

That's the strange thing - my friend's machine is at most the same power as mine, yet the Iteration value is much higher.

There is also a huge difference depending which crypto library your cryptsetup binary happens to use... I benchmarked this a while ago with 4 libraries supported by cryptsetup http://www.metamorpher.de/files/cryptse … hmark.html

Will look into it.

It's possible to change the key itercounts by luksChangeKey (change the key to the same key). Changing the master key itercount requires the LUKS header to be re-created as a whole, using cryptsetup-reencrypt --keep-key or the manual method http://unix.stackexchange.com/a/178722/30851

Will try.

Edit: I guess dynamic CPU clock could also be an issue, if it ran in turbo mode when creating the LUKS header, but won't go into turbo mode [or might even be stuck in power saving] when unlocking it, the process will take so much longer...

Good point. The machine was zeroing the whole SSD before I did it. Maybe given my higher NM bits and lower Iterations count, this could still be it.

dcepelik

Member

From: Prague

Registered: 2013-11-23

Posts: 10

Re: Booting from LUKS partition slow, slow opening of LUKS partition

It's possible to change the key itercounts by luksChangeKey (change the key to the same key). Changing the master key itercount requires the LUKS header to be re-created as a whole, using cryptsetup-reencrypt --keep-key or the manual method http://unix.stackexchange.com/a/178722/30851

Edit: I guess dynamic CPU clock could also be an issue, if it ran in turbo mode when creating the LUKS header, but won't go into turbo mode [or might even be stuck in power saving] when unlocking it, the process will take so much longer...

Hi,

so I tried reencrypting the LUKS partition setting iter-time to 1000 ms. When I unlock the parition from archiso, it's a bit quicker (about ~ 30 %) and the iteration count shown in cryptsetup luksDump is lower, but when booting, the unlock time (or what I think is time spent unlocking the partition) is still bad.

I enter the password on the GRUB prompt, wait for about 6 secs and then "Using keyslot 0" apperars on the screen, the boot continues immediately.

As I followed the tutorial to not have to unlock the parition twice, I have a key embedded in initramfs. I believe I therefore wait twice for the parition to unlock; for the first time on the prompt and for the second time sometime during the boot process. (For the second time even longer, as I unlock using the second key.)

I will be grateful for any further advice on the topic.

Last edited by dcepelik (2016-09-17 11:28:24)

frostschutz

Member

Registered: 2013-11-15

Posts: 1,472

Re: Booting from LUKS partition slow, slow opening of LUKS partition

I enter the password on the GRUB prompt,

Is it the GRUB prompt, i.e. encrypted /boot? I never used that before, my /boot is unencrypted. Or do you mean the Initramfs prompt after all, that happens when GRUB is already no longer involved?

For GRUB it depends which implementation GRUB uses and how fast that is. Since GRUB happens very early in the boot process... it might not be using a potent crypto library, I don't know.

Same setup for the other machine?

As I followed the tutorial to not have to unlock the parition twice, I have a key embedded in initramfs.

Technically you're still unlocking the partition twice. GRUB does it once to get at its encrypted files and load the encrypted kernel/initramfs. Then the kernel itself does it again (thanks to your embedded key, automatically).

And if the passphrase you type, and the embedded key, is not identical, you are using two keyslots after all which means one time you're using the 2nd keyslot to unlock, so for the entire process you are actually doing it three times.

GRUB: unlock using first key (succeds),
Kernel/Initramfs: unlock using first key (fails), unlock using second key (succeeds)

So this will take time.

The machine was zeroing the whole SSD before I did it.

Usually you don't overwrite SSDs...

If you want to zero the whole SSD (or a partition on it), you just use blkdiscard. It takes only a second. (mkfs will do the same, unasked)

For getting rid of free space in a filesystem, there is fstrim.

Last edited by frostschutz (2016-09-17 11:39:26)

dcepelik

Member

From: Prague

Registered: 2013-11-23

Posts: 10

Re: Booting from LUKS partition slow, slow opening of LUKS partition

Is it the GRUB prompt, i.e. encrypted /boot? I never used that before, my /boot is unencrypted. Or do you mean the Initramfs prompt after all, that happens when GRUB is already no longer involved?

Yes, my /boot is encrypted, just like in the tutorial. My EFI system partition is unencrypted. From here the GRUB's EFI boot program starts. It is configured to unencrypt the LUKS volume early and start booting the kernel from the encrypted root partition.

For GRUB it depends which implementation GRUB uses and how fast that is. Since GRUB happens very early in the boot process... it might not be using a potent crypto library, I don't know.

Same setup for the other machine?

That's what confuses me. The other machine's setup is all the same as much as I can tell.

Technically you're still unlocking the partition twice. GRUB does it once to get at its encrypted files and load the encrypted kernel/initramfs. Then the kernel itself does it again (thanks to your embedded key, automatically).

Yes, that's what I meant -- I think this contributes to the overall loader time to be as hight -- it includes twice the penalty for slow unlocking.

And if the passphrase you type, and the embedded key, is not identical, you are using two keyslots after all which means one time you're using the 2nd keyslot to unlock, so for the entire process you are actually doing it three times.

GRUB: unlock using first key (succeds),
Kernel/Initramfs: unlock using first key (fails), unlock using second key (succeeds)

So this will take time.

Oh, I see, so three times it is.

The machine was zeroing the whole SSD before I did it.

Usually you don't overwrite SSDs...

If you want to zero the whole SSD (or a partition on it), you just use blkdiscard. It takes only a second. (mkfs will do the same, unasked)

For getting rid of free space in a filesystem, there is fstrim.

If I understood the tutorial correctly, what I do is create a plain-type container over the parition I want to safely overwrite and dd if=/dev/zero onto it till it's full. As the container is opened using /dev/urandom as a key, the device get's random data as a result.

But thanks for the tip and your help so far, greatly appreciated.

Few ideas of my own; could this be BIOS settings (UEFI settings, system configuration) related? Like not having something vital enabled?

Could it be I've chosen a bad algorithm blindly following the tutorial? I created the container with

# cryptsetup -s 512 luksFormat /dev/sda1

frostschutz

Member

Registered: 2013-11-15

Posts: 1,472

Re: Booting from LUKS partition slow, slow opening of LUKS partition

The algorithm would make the data encryption/decryption faster/slower (run 'cryptsetup benchmark' to get an idea... if your box has AES-NI it shouldn't matter as long as it's AES)

It would not make unlocking faster/slower. That's just the iter count taking however much time it thinks is "safe" to protect against bruteforceing.

Overwriting the entire SSD can make the SSD itself slower. With SSD you just don't do that anymore, encryption or no, random data or no.

Last edited by frostschutz (2016-09-17 13:01:57)

dcepelik

Member

From: Prague

Registered: 2013-11-23

Posts: 10

Re: Booting from LUKS partition slow, slow opening of LUKS partition

The algorithm would make the data encryption/decryption faster/slower (run 'cryptsetup benchmark' to get an idea... if your box has AES-NI it shouldn't matter as long as it's AES)

It would not make unlocking faster/slower. That's just the iter count taking however much time it thinks is "safe" to protect against bruteforceing.

Would you mind elaborating further on how this actually improves data safety (ie., what the iteration count is)? I understand that this mechanism ensures that every password an attacker may try will take some time to probe, no matter if it's correct or not, without providing any information to the attacker about the correctnes of the code before some pre-set time passes. But how's this acutally implemented?

I've been thinking about it and I just don't think it's the unlocking. From the archiso, the container now unlocks in a second just as I want it to. Something else must be taking time when unlocking on the GRUB prompt, or you're right about GRUB using slow decryption library. (But that's not likely, the other machine does it quick with the same setup.)

Overwriting the entire SSD can make the SSD itself slower. With SSD you just don't do that anymore, encryption or no, random data or no.

I see, what was the point of doing it for the hard drives?

Thanks again for sharing insight.

---

Edit: Solved! You were right when you said that GRUB probably does not use a very sophisticated method of decrypting the key. After changing iter-time to 60, the problematic step in GRUB takes about a second .

Thanks a lot!

Now this is solved, I am not sure which one is better -- whether keeping iter-time this low, or not encrypting the /boot partition at all and keeping the iter-time about 1 second for the encrypted root.

What do you think?

PS: Turned out I was wrongly assuming my friend also encrypted his /boot partition -- he did not.

Last edited by dcepelik (2016-09-17 17:09:52)