You are not logged in.

#1 2018-09-02 09:16:57

whitesnow
Member
Registered: 2015-01-04
Posts: 52

real speed of luks encryption hdd, noticable on ssd

Lately I noticed a speed difference in the reading from a luks encrypted ssd.
I changed to a newer ssd, which did read substantially slower than the older one.

In real world probably most of the time not noticeable.

As I did not find much about that in the net, I wanted to post it here.

In summary for the machine I did notice it, the speed of the encrypted ssd (hdd) changed when changing the entry in
/sys/class/scsi_host/host0/link_power_management_policy
whereas the speed of the unencrypted ssd did not change much.

Here some lines from my tests:

machine% dmesg | grep "Core(TM)"
[    0.027561] smpboot: CPU0: Intel(R) Core(TM) i5-2520M CPU @ 2.50GHz (family: 0x6, model: 0x2a, stepping: 0x7)
machine% lsblk /dev/sda4
NAME        MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINT
sda4          8:4    0  128G  0 part  
└─cryptroot 254:0    0  128G  0 crypt 

machine% cat  /sys/class/scsi_host/host*/link_power_management_policy
min_power
min_power
min_power
min_power
max_performance
max_performance
machine% sudo hdparm -t /dev/sda4
[sudo] password for user: 

/dev/sda4:
 Timing buffered disk reads: 1572 MB in  3.00 seconds = 523.96 MB/sec
machine% sudo hdparm -t /dev/mapper/cryptroot 

/dev/mapper/cryptroot:
 Timing buffered disk reads: 774 MB in  3.01 seconds = 257.35 MB/sec
machine% su
Password: 
[root@machine user]# echo medium_power > /sys/class/scsi_host/host0/link_power_management_policy
[root@machine user]# cat  /sys/class/scsi_host/host*/link_power_management_policy
medium_power
min_power
min_power
min_power
max_performance
max_performance
[root@machine user]# hdparm -t /dev/sda4

/dev/sda4:
 Timing buffered disk reads: 1566 MB in  3.00 seconds = 521.96 MB/sec
[root@machine user]# hdparm -t /dev/mapper/cryptroot 

/dev/mapper/cryptroot:
 Timing buffered disk reads: 780 MB in  3.01 seconds = 259.56 MB/sec
[root@machine user]# echo med_power_with_dipm > /sys/class/scsi_host/host0/link_power_management_policy
[root@machine user]# cat  /sys/class/scsi_host/host*/link_power_management_policy
med_power_with_dipm
min_power
min_power
min_power
max_performance
max_performance
[root@machine user]# hdparm -t /dev/sda4

/dev/sda4:
 Timing buffered disk reads: 1568 MB in  3.00 seconds = 522.37 MB/sec
[root@machine user]# hdparm -t /dev/mapper/cryptroot 

/dev/mapper/cryptroot:
 Timing buffered disk reads: 772 MB in  3.01 seconds = 256.78 MB/sec
[root@machine user]# echo max_performance > /sys/class/scsi_host/host0/link_power_management_policy
[root@machine user]# cat  /sys/class/scsi_host/host0/link_power_management_policy
max_performance

[root@machine user]# hdparm -t /dev/sda4

/dev/sda4:
 Timing buffered disk reads: 1572 MB in  3.00 seconds = 523.54 MB/sec
[root@machine user]# hdparm -t /dev/mapper/cryptroot 

/dev/mapper/cryptroot:
 Timing buffered disk reads: 1284 MB in  3.00 seconds = 427.30 MB/sec
[root@machine user]# cryptsetup benchmark
# Tests are approximate using memory only (no storage IO).
PBKDF2-sha1       951520 iterations per second for 256-bit key
PBKDF2-sha256    1216445 iterations per second for 256-bit key
PBKDF2-sha512     801663 iterations per second for 256-bit key
PBKDF2-ripemd160  684449 iterations per second for 256-bit key
PBKDF2-whirlpool  480117 iterations per second for 256-bit key
argon2i       4 iterations, 772988 memory, 4 parallel threads (CPUs) for 256-bit key (requested 2000 ms time)
argon2id      4 iterations, 788007 memory, 4 parallel threads (CPUs) for 256-bit key (requested 2000 ms time)
#     Algorithm |       Key |      Encryption |      Decryption
        aes-cbc        128b       502.6 MiB/s      1662.8 MiB/s
    serpent-cbc        128b        58.9 MiB/s       241.0 MiB/s
    twofish-cbc        128b       132.2 MiB/s       272.7 MiB/s
        aes-cbc        256b       379.5 MiB/s      1279.4 MiB/s
    serpent-cbc        256b        60.9 MiB/s       225.0 MiB/s
    twofish-cbc        256b       138.5 MiB/s       279.0 MiB/s
        aes-xts        256b      1113.3 MiB/s      1076.3 MiB/s
    serpent-xts        256b       243.7 MiB/s       236.2 MiB/s
    twofish-xts        256b       262.4 MiB/s       243.3 MiB/s
        aes-xts        512b       883.0 MiB/s       885.4 MiB/s
    serpent-xts        512b       253.6 MiB/s       236.3 MiB/s
    twofish-xts        512b       264.9 MiB/s       272.8 MiB/s
[root@machine user]# cryptsetup status cryptroot
/dev/mapper/cryptroot is active and is in use.
  type:    LUKS2
  cipher:  aes-xts-plain64
  keysize: 512 bits
  key location: keyring
  device:  /dev/sda4
  sector size:  4096
  offset:  8192 sectors
  size:    268427264 sectors
  mode:    read/write


machine% sudo cryptsetup status virtual  
/dev/mapper/virtual is active.
  type:    LUKS1
  cipher:  aes-xts-plain64
  keysize: 256 bits
  key location: dm-crypt
  device:  /dev/sda7
  sector size:  512
  offset:  4096 sectors
  size:    134213632 sectors
  mode:    read/write
machine% sudo cryptsetup status        
machine% su                                     
Password: 
[root@machine user]# hdparm -t /dev/mapper/virtual 

/dev/mapper/virtual:
 Timing buffered disk reads: 1248 MB in  3.00 seconds = 415.74 MB/sec
[root@machine user]# echo med_power_with_dipm > /sys/class/scsi_host/host0/link_power_management_policy
[root@machine user]# hdparm -t /dev/mapper/virtual 

/dev/mapper/virtual:
 Timing buffered disk reads: 1196 MB in  3.00 seconds = 398.10 MB/sec
[root@machine user]# echo min_power > /sys/class/scsi_host/host0/link_power_management_policy
[root@machine user]# hdparm -t /dev/mapper/virtual 

/dev/mapper/virtual:
 Timing buffered disk reads: 732 MB in  3.00 seconds = 243.77 MB/sec
[root@machine user]# echo med_power_with_dipm > /sys/class/scsi_host/host0/link_power_management_policy
[root@machine user]# hdparm -t /dev/mapper/virtual 

/dev/mapper/virtual:
 Timing buffered disk reads: 738 MB in  3.00 seconds = 245.77 MB/sec
[root@machine user]# echo max_performance > /sys/class/scsi_host/host0/link_power_management_policy
[root@machine user]# hdparm -t /dev/mapper/virtual 

/dev/mapper/virtual:
 Timing buffered disk reads: 1362 MB in  3.00 seconds = 453.93 MB/sec
[root@machine user]# echo med_power_with_dipm > /sys/class/scsi_host/host0/link_power_management_policy
[root@machine user]# hdparm -t /dev/mapper/virtual 

/dev/mapper/virtual:
 Timing buffered disk reads: 1210 MB in  3.00 seconds = 403.28 MB/sec
[root@machine user]# hdparm -t /dev/mapper/virtual 

/dev/mapper/virtual:
 Timing buffered disk reads: 1208 MB in  3.00 seconds = 402.02 MB/sec
[root@machine user]# hdparm -t /dev/mapper/virtual 

/dev/mapper/virtual:
 Timing buffered disk reads: 1256 MB in  3.00 seconds = 418.42 MB/sec
[root@machine user]# hdparm -t /dev/mapper/virtual 

/dev/mapper/virtual:
 Timing buffered disk reads: 1196 MB in  3.00 seconds = 398.28 MB/sec
[root@machine user]# hdparm -t /dev/mapper/cryptroot

/dev/mapper/cryptroot:
 Timing buffered disk reads: 1274 MB in  3.00 seconds = 424.66 MB/sec
[root@machine user]# hdparm -t /dev/mapper/virtual 

/dev/mapper/virtual:
 Timing buffered disk reads: 1210 MB in  3.00 seconds = 403.32 MB/sec
[root@machine user]# lsblk /dev/sda7
NAME      MAJ:MIN RM SIZE RO TYPE  MOUNTPOINT
sda7        8:7    0  64G  0 part  
└─virtual 254:2    0  64G  0 crypt 
[root@machine user]# 

Here:
https://wiki.debianforum.de/Benchmark_f … %BCsselung
did I find some kind of benchmark configuration and encryption speed of different cpus.

here:
https://www.kernel.org/doc/Documentatio … -scsi_host
is some documentation.

Last edited by whitesnow (2018-09-02 09:17:38)

Offline

#2 2018-09-02 13:21:44

R00KIE
Forum Fellow
From: Between a computer and a chair
Registered: 2008-09-14
Posts: 4,734

Re: real speed of luks encryption hdd, noticable on ssd

You should be testing the block device itself, that is, test the speed of /dev/sdX and not have the encryption in the middle.

That said from what I understand of the tunable you mention it should control mostly idle power consumption, as in allowing the sata link to power down, the performance penalty should come in the time needed to take the ssd/hdd out of standby and get the sata link to come out of low power and have it up and running again at full speed/power.

The best compromise is most probably med_power_with_dipm as it mimics what the intel drivers do on windows so you also avoid weird problems because some ssd's firmware was not tested for anything else besides no power management or what windows does.


R00KIE
Tm90aGluZyB0byBzZWUgaGVyZSwgbW92ZSBhbG9uZy4K

Offline

#3 2018-09-09 07:59:55

whitesnow
Member
Registered: 2015-01-04
Posts: 52

Re: real speed of luks encryption hdd, noticable on ssd

You should be testing the block device itself, that is, test the speed of /dev/sdX and not have the encryption in the middle.

I did this.

[root@machine user]# hdparm -t /dev/sda4

/dev/sda4:
 Timing buffered disk reads: 1568 MB in  3.00 seconds = 522.37 MB/sec

Further two indications, that something changed.

1) the older smaller ssd was significantly faster reading the encrypted volume
(test not included in my post, after I found the solution with that parameter I did not worry about the different ssds that much)

It's many tests, I would need to create some graph with the results, put some more hours into that work, which I don't really have.

both ssds had about similar speeds reading the unencrypted device itself

2) the newer bigger ssd is significantly slower then the numbers shown in cryptsetup benchmark

And from this point on, I did not know where to look, what changed.

The best compromise is most probably med_power_with_dipm as it mimics what the intel drivers do on windows so you also avoid weird problems because some ssd's firmware was not tested for anything else besides no power management or what windows does.


It's only a speed problem. Nothing with functionality. And in real use, I usually never recognize it.

edit:
Maybe, as a summary:

I have two ssds, performing quite similar on the raw device, but noticable different on the encrypted device with different parameters in
/sys/class/scsi_host/host0/link_power_management_policy

Well, that still needs at least 4 numbers.

the following is copied from the typscript (command script) files, which I created, while looking for the reason of the different speeds.

before I found the parameter

/sys/class/scsi_host/host0/link_power_management_policy
sda
..
├─sda3         crypto_LUKS
├─sda4         vfat       
..
├─sda7         crypto_LUKS
│ └─ccache     zfs_member 
..
sdb
├─sdb1         ext4       
├─sdb2         vfat       
├─sdb3
├─sdb4         crypto_LUKS
│ └─cryptroot  zfs_member 
└─sdb5         crypto_LUKS


hdparm -t /dev/sdb4

/dev/sdb4:
 Timing buffered disk reads: 1576 MB in  3.00 seconds = 524.72 MB/sec
 
 hdparm -t /dev/mapper/cryptroot

/dev/mapper/cryptroot:
 Timing buffered disk reads: 728 MB in  3.00 seconds = 242.34 MB/sec

hdparm-t /dev/mapper/ccache

/dev/mapper/ccache:
 Timing buffered disk reads: 1024 MB in  3.00 seconds = 341.25 MB/sec
 
 
 
hdparm -t /dev/sda3

/dev/sda3:
 Timing buffered disk reads: 1580 MB in  3.00 seconds = 526.41 MB/sec

hdparm -t /dev/sda4
/dev/sda4:
 Timing buffered disk reads: 250 MB in  0.48 seconds = 519.31 MB/sec

hdparm -t /dev/sdb4

/dev/sdb4:
 Timing buffered disk reads: 1572 MB in  3.00 seconds = 523.74 MB/sec

hdparm -t /dev/sdb5

/dev/sdb5:
 Timing buffered disk reads: 1592 MB in  3.00 seconds = 530.36 MB/sec

The takeaway of that was:
I have two ssd's installed
sda (the older, smaller one - samsung 840 250GB)
sdb (the newer, Crucial BX300 480 GB)

they both perform similar, reading the raw device
see
sda4 519 MB/sec
sdb4 523 MB/sec

but significantly different on the encrypted volume

see
/dev/mapper/ccache  (on sda) 341 MB/sec
/dev/mapper/cryptroot (on sdb) 242 MB/sec

Last edited by whitesnow (2018-09-09 08:37:43)

Offline

#4 2018-09-09 09:27:12

R00KIE
Forum Fellow
From: Between a computer and a chair
Registered: 2008-09-14
Posts: 4,734

Re: real speed of luks encryption hdd, noticable on ssd

The only thing I can think of right now is to do a luksdump of both encrypted volumes and compare the encryption options, they must be different otherwise I don't see any reason why one would be slower than the other.


R00KIE
Tm90aGluZyB0byBzZWUgaGVyZSwgbW92ZSBhbG9uZy4K

Offline

#5 2018-09-16 18:15:49

whitesnow
Member
Registered: 2015-01-04
Posts: 52

Re: real speed of luks encryption hdd, noticable on ssd

Well, they are different. I wanted to know if I should change the block size. But on that, I could not find a noticeable speed difference.

Wanted to create some equal encrypted volumes on both disks, but don't know, when I will have time for that.
I'm mainly a user not a tester or developer.

sda             8:0    0 447.1G  0 disk  
..
├─sda4          8:4    0   128G  0 part  
│ └─cryptroot 254:0    0   128G  0 crypt 
..
└─sda7          8:7    0    64G  0 part  
  └─virtual   254:2    0    64G  0 crypt 
sdb             8:16   0 232.9G  0 disk  
..
├─sdb3          8:19   0    62G  0 part  
│ └─crypt-old 254:3    0    62G  0 crypt 
..
machine% sudo cryptsetup luksDump /dev/sda4
LUKS header information
Version:        2
Epoch:          3
Metadata area:  12288 bytes
UUID:           edit:deleted
Label:          (no label)
Subsystem:      (no subsystem)
Flags:          (no flags)

Data segments:
  0: crypt
        offset: 4194304 [bytes]
        length: (whole device)
        cipher: aes-xts-plain64
        sector: 4096 [bytes]

Keyslots:
  0: luks2
        Key:        512 bits
        Priority:   normal
        Cipher:     aes-xts-plain64
        PBKDF:      argon2i
        Time cost:  4
        Memory:     5775
        Threads:    4
        Salt:       edit:deleted
                    edit:deleted
        AF stripes: 4000
        Area offset:32768 [bytes]
        Area length:258048 [bytes]
        Digest ID:  0
Tokens:
Digests:
  0: pbkdf2
        Hash:       sha256
        Iterations: 66534
        Salt:       edit:deleted
                    edit:deleted
        Digest:     edit:deleted
                    edit:deleted
machine% sudo cryptsetup luksDump /dev/sda7
LUKS header information for /dev/sda7

Version:        1
Cipher name:    aes
Cipher mode:    xts-plain64
Hash spec:      sha512
Payload offset: 4096
MK bits:        256
MK digest:      edit:deleted 
MK salt:        edit:deleted 
                edit:deleted 
MK iterations:  100515
UUID:           edit:deleted

Key Slot 0: ENABLED
        Iterations:             8065960
        Salt:                   edit:deleted
                                edit:deleted 
        Key material offset:    8
        AF stripes:             4000
Key Slot 1: ENABLED
        Iterations:             2421652
        Salt:                   edit:deleted 
                                edit:deleted 
        Key material offset:    264
        AF stripes:             4000
Key Slot 2: DISABLED
Key Slot 3: DISABLED
Key Slot 4: DISABLED
Key Slot 5: DISABLED
Key Slot 6: DISABLED
Key Slot 7: DISABLED
machine% sudo cryptsetup luksDump /dev/sdb3
LUKS header information for /dev/sdb3

Version:        1
Cipher name:    aes
Cipher mode:    xts-plain64
Hash spec:      sha512
Payload offset: 4096
MK bits:        512
MK digest:      edit:deleted
MK salt:        edit:deleted
                edit:deleted
MK iterations:  133536
UUID:           edit:deleted

Key Slot 0: ENABLED
        Iterations:             1072026
        Salt:                   edit:deleted
                                edit:deleted
        Key material offset:    8
        AF stripes:             4000
Key Slot 1: DISABLED
Key Slot 2: DISABLED
Key Slot 3: DISABLED
Key Slot 4: DISABLED
Key Slot 5: DISABLED
Key Slot 6: DISABLED
Key Slot 7: DISABLED
machine% su
Password:
[root@machine user]# cat /sys/class/scsi_host/host*/link_power_management_policy
min_power
min_power
min_power
min_power
max_performance
max_performance
[root@machine user]# exit
exit
machine% sudo hdparm -T /dev/mapper/virtual

/dev/mapper/virtual:
 Timing cached reads:   7840 MB in  2.00 seconds = 3926.28 MB/sec
machine% sudo hdparm -t /dev/mapper/virtual

/dev/mapper/virtual:
 Timing buffered disk reads: 770 MB in  3.00 seconds = 256.60 MB/sec
machine% sudo cryptsetup luksOpen /dev/sdb3 crypt-old
Enter passphrase for /dev/sdb3:
No key available with this passphrase.
Enter passphrase for /dev/sdb3:
machine% sudo hdparm -t /dev/mapper/crypt-old

/dev/mapper/crypt-old:
 Timing buffered disk reads: 172 MB in  3.02 seconds =  57.00 MB/sec
machine% sudo hdparm -t /dev/mapper/crypt-old

/dev/mapper/crypt-old:
 Timing buffered disk reads: 212 MB in  3.00 seconds =  70.59 MB/sec
machine% su
Password:
[root@machine user]# cat /sys/class/scsi_host/host*/link_power_management_policy
min_power
min_power
min_power
min_power
max_performance
max_performance
[root@machine user]# echo max_performance > /sys/class/scsi_host/host2/link_power_management_policy
[root@machine user]# exit
exit
machine% sudo hdparm -t /dev/mapper/crypt-old

/dev/mapper/crypt-old:
 Timing buffered disk reads: 384 MB in  3.01 seconds = 127.39 MB/sec
machine% su
Password:
[root@machine user]# echo max_performance > /sys/class/scsi_host/host1/link_power_management_policy
[root@machine user]# exit
exit
machine% sudo hdparm -t /dev/mapper/crypt-old

/dev/mapper/crypt-old:
 Timing buffered disk reads: 762 MB in  3.01 seconds = 253.35 MB/sec
machine% sudo hdparm -t /dev/sdb3

/dev/sdb3:
 Timing buffered disk reads: 1068 MB in  3.02 seconds = 353.50 MB/sec
machine% su
Password:
[root@machine user]# echo min_power > /sys/class/scsi_host/host2/link_power_management_policy
[root@machine user]# exit
exit
machine% sudo hdparm -t /dev/sdb3

/dev/sdb3:
 Timing buffered disk reads: 1024 MB in  3.02 seconds = 338.73 MB/sec
machine% sudo hdparm -t /dev/mapper/crypt-old

/dev/mapper/crypt-old:
 Timing buffered disk reads: 1132 MB in  3.00 seconds = 376.88 MB/sec
machine% 

machine% sudo hdparm -t /dev/mapper/virtual
[sudo] password for user:

/dev/mapper/virtual:
 Timing buffered disk reads: 764 MB in  3.00 seconds = 254.31 MB/sec
machine% su
Password: 
[root@machine user]# echo max_performance > /sys/class/scsi_host/host0/link_power_management_policy
[root@machine user]# exit
exit
machine% sudo hdparm -t /dev/mapper/virtual

/dev/mapper/virtual:
 Timing buffered disk reads: 1258 MB in  3.00 seconds = 419.29 MB/sec
machine% su                                
Password: 
[root@machine user]# echo min_power > /sys/class/scsi_host/host0/link_power_management_policy
[root@machine user]# exit
exit
machine% sudo hdparm -t /dev/mapper/virtual

/dev/mapper/virtual:
 Timing buffered disk reads: 770 MB in  3.00 seconds = 256.30 MB/sec

This may all be not so much ordered. All that took me already a lot of time.
When I will have more time, I may do some more test, and put them in nicely order.

I don't really know what changed now.

That

/dev/mapper/crypt-old

is /dev/sdb3 (had been sda3 before moving the system to the new disk)
on the older Samsung SSD 840 EVO 250GB


Now having both links on "max_performance" the newer disk is faster

machine%sudo hdparm -t /dev/mapper/cryptroot

/dev/mapper/cryptroot:
 Timing buffered disk reads: 1342 MB in  3.00 seconds = 447.25 MB/sec
machine%sudo hdparm -t /dev/mapper/crypt-old 

/dev/mapper/crypt-old:
 Timing buffered disk reads: 1130 MB in  3.00 seconds = 376.16 MB/sec
machine%cat /sys/class/scsi_host/host*/link_power_management_policy
max_performance
max_performance
min_power
min_power
max_performance
max_performance
machine%

Having them both on "min_power"
the older one is faster

machine% cat /sys/class/scsi_host/host*/link_power_management_policy
min_power
min_power
min_power
min_power
max_performance
max_performance
machine% sudo hdparm -t /dev/mapper/crypt-old                       

/dev/mapper/crypt-old:
 Timing buffered disk reads: 1170 MB in  3.00 seconds = 389.71 MB/sec
machine% sudo hdparm -t /dev/mapper/cryptroot                       

/dev/mapper/cryptroot:
 Timing buffered disk reads: 824 MB in  3.00 seconds = 274.30 MB/sec

And when I put the disk first in (Laptop, put to standby, inserted the disk into the drive bay)
the older one was much slower opposite to previous results.
So there might even be something with the position of the disk. In the first test they had been placed the other way around - the older disk as a fixed disk and the newer disk in the drive bay.

But that speed behaviour did change after some time.

again the code as shown above already:

machine% sudo hdparm -t /dev/mapper/crypt-old

/dev/mapper/crypt-old:
 Timing buffered disk reads: 172 MB in  3.02 seconds =  57.00 MB/sec
machine% sudo hdparm -t /dev/mapper/crypt-old

Offline

#6 2018-10-03 20:29:37

barburger
Member
Registered: 2018-10-03
Posts: 1

Re: real speed of luks encryption hdd, noticable on ssd

I also noticed a recent speed difference in luks, although in decryption. Before it would take me around 2 seconds to decrypt my home partition, but since around two or three weeks, it jumped to 30 seconds to decrypt.

luksDump has shown me my usual key had around 3.3 million iterations, but when i removed it and added it back, it only had 700k iterations. And i could decrypt again in around 2 seconds.

I don't know if this is the same problem as op but it seemed relevant and recent enough.

Offline

Board footer

Powered by FluxBB