Arch Linux

wrongturn

Member

Registered: 2021-10-17

Posts: 10

Overwriting a LUKS container with zeros is extremely slow on NVMe

Hi. I just purchased a WD Black SN850 1TB SSD to install Arch on. My plan is to use full system encryption with LUKS. I booted the arch install disk and set up LUKS encryption on a partition using

cryptsetup luksFormat --type luks2 /dev/disk/by-partlabel/cryptsystem
cryptsetup open /dev/disk/by-partlabel/cryptsystem system

I've read that it's recommended to write over the entire drive after using cryptsetup so that the entire partition looks like it's just filled with random data. I used the dd utility for this

dd if=/dev/zero of=/dev/mapper/system status=progress bs=1M

However, my problem is that when trying to write zeros to the container, I only get transfer speeds less than 20 MB/s, and it keeps dropping slowly. I tried writing /dev/zero to the drive directly and it never dropped below 1.8 GB/s, so there's a big difference in performance. I'm guessing this may have something to do with lack of entrophy on my system, but I'm not aware of methods to generate more from the install disk. If anyone has suggestions for how to speed this process up, it would be appreciated.

OS: archlinux-2022.03.01-x86_64.iso
CPU: AMD Ryzen 5 5600X

Last edited by wrongturn (2022-03-23 20:16:47)

frostschutz

Member

Registered: 2013-11-15

Posts: 1,418

Re: Overwriting a LUKS container with zeros is extremely slow on NVMe

I've read that it's recommended to write over the entire drive after using cryptsetup so that the entire partition looks like it's just filled with random data.

Only if you care about making it look filled and also plan to never use TRIM/discard with this drive.

Another purpose is to get rid of old unencrypted data, but this can be achieved by performing blkdiscard or secure erase before installing Linux...

Otherwise not much point to do this for SSDs, it will only waste write cycles and once you TRIM all free space will be visible again anyways.

However, my problem is that when trying to write zeros to the container, I only get transfer speeds less than 20 MB/s, and it keeps dropping slowly. I tried writing /dev/zero to the drive directly and it never dropped below 1.8 GB/s, so there's a big difference in performance.

Your SSD might be overheating, and throttling massively, or it might just have bad write performance. Some SSDs are surprisingly bad at writing data, and sometimes the manufacturer replaces the flash chips w/o changing model numbers, so whatever performance values you saw in early reviews is not what you actually get when you purchase the drive...

Another possibility is that AES acceleration is not active. This can sometimes happen if the LUKS container was opened when the aes module was not loaded (and loading it later does not necessarily help already opened containers). Or maybe you didn't use AES at all...

dd is not a good benchmark, neither is writing zeroes... if the SSD were smart enough to detect zeroes it would not have to write anything at all...

also check dmesg etc. for potential error messages, sometimes it's also a problem on the pci bus etc.

I'm guessing this may have something to do with lack of entrophy on my system

Unless you are using ancient kernel (unlikely in arch linux) entropy essentially no longer matters... algorithms change and the Linux random device no longer accounts for entropy and no longer blocks and it's much faster than before... also outside of the first-time setup (luksformat) entropy is not used in encryption at all, it will always keep using the same old key or you would be unable to read previously written data...

wrongturn

Member

Registered: 2021-10-17

Posts: 10

Re: Overwriting a LUKS container with zeros is extremely slow on NVMe

Thanks frostschutz.

Only if you care about making it look filled and also plan to never use TRIM/discard with this drive.

Another purpose is to get rid of old unencrypted data, but this can be achieved by performing blkdiscard or secure erase before installing Linux...

Otherwise not much point to do this for SSDs, it will only waste write cycles and once you TRIM all free space will be visible again anyways.

I didn't consider TRIM. It's probably best to just install the filesystem directly on the LUKS container then.

Your SSD might be overheating, and throttling massively, or it might just have bad write performance. Some SSDs are surprisingly bad at writing data, and sometimes the manufacturer replaces the flash chips w/o changing model numbers, so whatever performance values you saw in early reviews is not what you actually get when you purchase the drive...

The SSD is rated for 7000 MB/s read and 5300 MB/s write and the SSD is mounted to my motherboard's heat sink, so I don't think overheating is the problem here. Hopefully there are no problems with the chips.

Another possibility is that AES acceleration is not active. This can sometimes happen if the LUKS container was opened when the aes module was not loaded (and loading it later does not necessarily help already opened containers). Or maybe you didn't use AES at all...

I tried loading dm_crypt before opening as https://wiki.archlinux.org/title/Dm-cry … encryption recommends, but it didn't make a difference.

dd is not a good benchmark, neither is writing zeroes... if the SSD were smart enough to detect zeroes it would not have to write anything at all...

Good point, since the drive is new it probably already was filled with zeros.

also check dmesg etc. for potential error messages, sometimes it's also a problem on the pci bus etc.

I checked, but there doesn't seem to be any relevant errors logged there.

Unless you are using ancient kernel (unlikely in arch linux) entropy essentially no longer matters... algorithms change and the Linux random device no longer accounts for entropy and no longer blocks and it's much faster than before... also outside of the first-time setup (luksformat) entropy is not used in encryption at all, it will always keep using the same old key or you would be unable to read previously written data...

This is good to know.

I'll try to install the system and run proper benchmarking tools to see if that fixes my problems.