[SOLVED] Unable to decrypt drives correctly after latest update

pomekewe · 2024-03-05 08:58:02

Hello Good People,

After upgrading my main PC (Lenovo ThinkCentre) this morning, I've been unable to read any of my encrypted discs/arrays. This is following the latest updates to mkinitcpio, systemd, mdadm, lvm2 and cryptsetup.

All the discs affected are encrypted "plain", and despite adding the correct cipher, hash and keysize as command line parameters, after creating the mapped drive, it contains no valid partitions.

E.g:

myhost: ~# cryptsetup -v --type=plain open /dev/sdb sdb
Enter passphrase for /dev/sdb:
Command successful.
myhost: ~# fdisk -l /dev/mapper/sdb
Disk /dev/mapper/sdb: 465.76 GiB, 500107862016 bytes, 976773168 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 33553920 bytes
myhost: ~# cryptsetup status sdb
/dev/mapper/sdb is active.
type: PLAIN
cipher: aes-cbc-essiv:sha256
keysize: 256 bits
key location: dm-crypt
device: /dev/sdb
sector size: 512
offset: 0 sectors
size: 976773168 sectors
mode: read/write

The discs decrypt fine on my laptops which I haven't updated yet, for obvious reasons. The information shown by cryptsetup status is identical to that when I run it against the same disc on my laptop.

Is this a reportable bug, or am I totally missing something? Can I supply any more useful information?

Any advice gratefully received.

P.S. Yes, I know aes-cbc-essiv is now deprecated - one of the things I plan to do is move all the discs/arrays to LUKS instead of plain. I've been using these encrypted discs for several years without issue until now.

Last edited by pomekewe (2024-03-06 01:13:25)

frostschutz · 2024-03-05 10:27:55

Are you sure it's 100% identical? It's normal for `cryptsetup open --type=plain /dev/sdb crypt_sdb` not to work between systems/distros, because that uses default settings and defaults tend to change a lot (cryptsetup with no arguments shows some of them). It's the primary reason why plain mode is discouraged for anything other than temporary data (swap encrypted with random key). It's so easy to get settings wrong.

Instead of checking for partition tables (there can be other issues), check raw data by hexdump -C some of it? If the encryption parameters are wrong, it will be obviously different at first glance. Use a bit of an offset since data might deviate after 4K.

This checks 64 bytes at offset 1M, for raw sdb and crypt sdb. If you happen to have zeroes/insignificant data at this offset, pick another offset.

# hexdump -C --skip=1M --length=64 /dev/sdb
# hexdump -C --skip=1M --length=64 /dev/mapper/crypt_sdb

Run this on both of your systems.

/dev/sdb should look identical (if not, you're using the wrong device name).

/dev/mapper/crypt_sdb should look identical (if not, wrong passphrase, wrong cipher settings or bug in cipher itself).

You can also check `dmsetup table --showkeys`. for plain it should still show the master key it is using. For luks2 devices it just shows keyring entities unless you open with --disable-keyring.

Last edited by frostschutz (2024-03-05 10:38:02)

pomekewe · 2024-03-05 13:34:57

Thanks for your quick reply, frostschutz.

However, I find it puzzling. Firstly, when I said 'they' were identical, I was referring, as stated, to the cryptographic information reported by 'cryptsetup status' when the drives were set up, not the data as decrypted, which is what you assumed, I think.

Secondly, how is comparing the raw data to the decrypted data ever going to be the same? Or have I misunderstood your intention? Given that, I did this for both, on a working system (sdb) and the PC that fails to mount the drive (as sdc). You can see that the decrypted versions are different, even though the cryptographic information is the same, as shown by cryptsetup status.

Interestingly, and probably relevant, if I add the cipher, hash and key size options to cryptsetup, as shown in the status below, on the working laptop, it also fails to mount the decrypted drive, same as the mount of /dev/mapper/sdc below. So something is not working correctly when the parameters are explicitly passed to cryptsetup in either version (2.6.1-3 or 2.7.0-3).

I also tried the same with the LTS kernel, just in case, but got the same results.

Note that all my computers (two desktops and five laptops) are set up pretty much identically, with the proviso that my main PC is fully updated. The others are updated up to last weekend. I have not had any issues with encrypted drives until cryptsetup 2.7, which changed the default cryptographic settings. I had been running 2.6.1-3 on all my systems until today.

myhost: ~# cat hex.sdb.good 11:15PM
───────┬────────────────────────────────────────────────────────────────────────────────────────────────
│ File: hex.sdb.good
───────┼────────────────────────────────────────────────────────────────────────────────────────────────
1 │ 10000000 51 b0 15 22 cc bb aa ad af 7c 05 9d e2 88 ba cc |Q..".....|......|
2 │ 10000010 de 47 4f f1 93 e2 cc c8 3d ea 97 8a 7d af 87 2d |.GO.....=...}..-|
3 │ 10000020 19 bb bf a9 a0 fc 9b 57 7b 5d 1f e5 3b 74 df 26 |.......W{]..;t.&|
4 │ 10000030 38 37 d6 55 5e 8a 67 17 60 22 bb cf f2 84 c8 43 |87.U^.g.`".....C|
5 │ 10000040
───────┴────────────────────────────────────────────────────────────────────────────────────────────────
myhost: ~# cat hex.mapper.sdb.good 11:15PM
───────┬────────────────────────────────────────────────────────────────────────────────────────────────
│ File: hex.mapper.sdb.good
───────┼────────────────────────────────────────────────────────────────────────────────────────────────
1 │ 10000000 72 fa 01 00 00 08 01 72 fa 01 00 00 09 01 72 fa |r......r......r.|
2 │ 10000010 01 00 00 0a 01 72 fa 01 00 00 0b 01 72 fa 01 00 |.....r......r...|
3 │ 10000020 00 0c 01 72 fa 01 00 00 0d 01 72 fa 01 00 00 0e |...r......r.....|
4 │ 10000030 01 72 fa 01 00 00 0f 01 72 fa 01 00 00 10 01 72 |.r......r......r|
5 │ 10000040
───────┴────────────────────────────────────────────────────────────────────────────────────────────────

myhost: ~# cryptsetup -v --type=plain --cipher aes-cbc-essiv:sha256 --key-size 256 --hash sha256 open /dev/sdc sdc
Enter passphrase for /dev/sdc:
Command successful.

myhost: ~# cat hex.sdc.bad 11:15PM
───────┬────────────────────────────────────────────────────────────────────────────────────────────────
│ File: hex.sdc.bad
───────┼────────────────────────────────────────────────────────────────────────────────────────────────
1 │ 10000000 51 b0 15 22 cc bb aa ad af 7c 05 9d e2 88 ba cc |Q..".....|......|
2 │ 10000010 de 47 4f f1 93 e2 cc c8 3d ea 97 8a 7d af 87 2d |.GO.....=...}..-|
3 │ 10000020 19 bb bf a9 a0 fc 9b 57 7b 5d 1f e5 3b 74 df 26 |.......W{]..;t.&|
4 │ 10000030 38 37 d6 55 5e 8a 67 17 60 22 bb cf f2 84 c8 43 |87.U^.g.`".....C|
5 │ 10000040
───────┴────────────────────────────────────────────────────────────────────────────────────────────────
myhost: ~# cat hex.mapper.sdc.bad 11:16PM
───────┬────────────────────────────────────────────────────────────────────────────────────────────────
│ File: hex.mapper.sdc.bad
───────┼────────────────────────────────────────────────────────────────────────────────────────────────
1 │ 10000000 11 14 a2 29 5a a1 5f 98 b6 ea c0 27 82 de 1c db |...)Z._....'....|
2 │ 10000010 1a 38 7c e3 31 91 d9 e0 4c ec 8b a1 04 e7 86 a1 |.8|.1...L.......|
3 │ 10000020 f1 3d 46 57 5e 96 37 11 e8 40 4b 84 b1 3f 9e bc |.=FW^.7..@K..?..|
4 │ 10000030 e8 6d 0d dc eb a5 7b 96 9c 5e 16 e5 d2 fc 83 7f |.m....{..^......|
5 │ 10000040
───────┴────────────────────────────────────────────────────────────────────────────────────────────────

myhost: ~# mount /dev/mapper/sdc ext 11:16PM
mount: /root/ext: wrong fs type, bad option, bad superblock on /dev/mapper/sdc, missing codepage or helper program, or other error.
dmesg(1) may have more information after failed mount system call.

myhost: ~# cryptsetup status sdc 11:18PM
/dev/mapper/sdc is active.
type: PLAIN
cipher: aes-cbc-essiv:sha256
keysize: 256 bits
key location: dm-crypt
device: /dev/sdc
sector size: 512
offset: 0 sectors
size: 976773168 sectors
mode: read/write

frostschutz · 2024-03-05 13:49:05

pomekewe wrote:

However, I find it puzzling. Firstly, when I said 'they' were identical, I was referring, as stated, to the cryptographic information reported by 'cryptsetup status' when the drives were set up, not the data as decrypted, which is what you assumed, I think.
Secondly, how is comparing the raw data to the decrypted data ever going to be the same? Or have I misunderstood your intention?

Compare raw data on system #1 vs. #2, and decrypted data on system #1 vs #2. This way you can determine it's indeed the crypt layer that somehow produces the wrong data, from the same raw data.

This is just to rule out any other problem with the disk or device name.

So something is not working correctly when the parameters are explicitly passed to cryptsetup in either version (2.6.1-3 or 2.7.0-3).

Can you also compare `dmsetup table --showkeys`? (You should not post your key here, regardless whether its good or bad.)

pchau · 2024-03-05 21:26:30

Hi, I have the same issue.

My setup is like this: cryptsetup open /dev/mmcblk0p2 msd --type plain -c aes-cbc-essiv:sha256
I used to used crypsetup create which is deprecated.
In version 2.6.1 I can mount and fsck /dev/mapper/msd. But in version 2.7.0 I cannot mount it. The only solution after upgrade was that I extracted the binary of crypsetup in version 2.6.1 and put it in /usr/bin/cryptsetup.old for accessing my old encrypted disk.

Not sure where has gone wrong for 2.7.0.

pchau · 2024-03-05 21:32:58

As for dmsetup table --showkeys output:

Version 2.6.1

msd: 0 144877568 crypt aes-cbc-essiv:sha256 8331fac55dfe5b9e38a10c705435bfd071ad576bde701299a3c744231a08385f 0 179:2 0

Versionn 2.7.0

msd: 0 144877568 crypt aes-cbc-essiv:sha256 f409f6f01bbb7ced29873eac125120f1f60ac557dd62fa1337d6c578eea71139 0 179:2 0

Last edited by pchau (2024-03-05 23:32:14)

frostschutz · 2024-03-05 21:45:47

The key is different.

Note: if the above was your real key, anyone can decrypt it now without knowing your passphrase.

This happens when you change the passphrase (obviously), but also when you change the hash that's used to derive the key from your passphrase. Your command doesn't specify the hash.

What does cryptsetup --help show for password hashing? If it's not sha256, it could be ripemd160.

$ cryptsetup --help
cryptsetup 2.5.0 flags: UDEV BLKID KEYRING KERNEL_CAPI 
Usage: cryptsetup [OPTION...] <action> <action-specific>
[...]
Default compiled-in device cipher parameters:
	loop-AES: aes, Key 256 bits
»	plain: aes-cbc-essiv:sha256, Key: 256 bits, Password hashing: ripemd160«
	LUKS: aes-xts-plain64, Key: 256 bits, LUKS header hashing: sha256, RNG: /dev/urandom
	LUKS: Default keysize with XTS mode (two internal keys) will be doubled.

cryptsetup 2.6.1 flags: UDEV BLKID KEYRING KERNEL_CAPI 
Usage: cryptsetup [OPTION...] <action> <action-specific>
[...]
Default compiled-in device cipher parameters:
	loop-AES: aes, Key 256 bits
»	plain: aes-cbc-essiv:sha256, Key: 256 bits, Password hashing: ripemd160«
	LUKS: aes-xts-plain64, Key: 256 bits, LUKS header hashing: sha256, RNG: /dev/urandom
	LUKS: Default keysize with XTS mode (two internal keys) will be doubled.

$ cryptsetup --help
cryptsetup 2.7.0 flags: UDEV BLKID KEYRING KERNEL_CAPI HW_OPAL
Usage: cryptsetup [OPTION...] <action> <action-specific>
[...]
Default compiled-in device cipher parameters:
	loop-AES: aes, Key 256 bits
»	plain: aes-xts-plain64, Key: 256 bits, Password hashing: sha256«
	LUKS: aes-xts-plain64, Key: 256 bits, LUKS header hashing: sha256, RNG: /dev/urandom
	LUKS: Default keysize with XTS mode (two internal keys) will be doubled.

Last edited by frostschutz (2024-03-05 21:48:20)

pchau · 2024-03-05 22:16:37

frostschutz wrote:

The key is different.
Note: if the above was your real key, anyone can decrypt it now without knowing your passphrase.
This happens when you change the passphrase (obviously), but also when you change the hash that's used to derive the key from your passphrase. Your command doesn't specify the hash.
What does cryptsetup --help show for password hashing? If it's not sha256, it could be ripemd160.

Frostschutz you are spot on. Thanks for your great help. Now I can decrypt my partition. It would be nice if crypsetup status shows the hash used as well. Then we can see the differences.

frostschutz · 2024-03-05 22:21:49

pchau wrote:

It would be nice if crypsetup status shows the hash used as well. Then we can see the differences.

Password hashing happens before setting up the device, so it's not actually a property of the device itself.

Same with LUKS, from the opened device you can never tell which passphrase/key was used to open it (if you have multiple keyslots).

pchau · 2024-03-05 22:31:08

frostschutz wrote:

Password hashing happens before setting up the device, so it's not actually a property of the device itself.

Thanks for clarifying.

pomekewe · 2024-03-06 01:02:25

Thanks frostschutz, I've finally sorted out the problem. As you suggested, cryptsetup was using the wrong hash. Despite cryptsetup status showing the hash as sha256, it was actually ripemd160. Once I changed the --hash entry to that, it worked correctly. So it looks like a bug with cryptsetup status showing the incorrect hash algorithm.

Thanks for all your input. I'll now test this with cryptsetup 2.7.0 and see if that works now.

Will post the results of that soon.

pomekewe · 2024-03-06 01:09:46

Just fully updated my main PC for the latest cryptsetup, mkinitcpio, lvm2, mdadm and systemd, and decrypting with the correct has works perfectly.

Thanks again for your input, frostschutz. I'll look at filing a bug report against cryptsetup status.

frostschutz · 2024-03-06 09:12:09

It doesn't show the hash at all... the :sha256 in aes-cbc-essiv is part of the cipher. The default cipher, and hash, for plain mode is shown in cryptsetup --help.

This is what I was trying to get at in my first reply, but I didn't explain it well, and then followed with overthinking. Sorry about that :-)

pomekewe · 2024-03-06 11:52:22

No worries, all sorted in the end.

Arch Linux

#1 2024-03-05 08:58:02

[SOLVED] Unable to decrypt drives correctly after latest update

#2 2024-03-05 10:27:55

Re: [SOLVED] Unable to decrypt drives correctly after latest update

#3 2024-03-05 13:34:57

Re: [SOLVED] Unable to decrypt drives correctly after latest update

#4 2024-03-05 13:49:05

Re: [SOLVED] Unable to decrypt drives correctly after latest update

#5 2024-03-05 21:26:30

Re: [SOLVED] Unable to decrypt drives correctly after latest update

#6 2024-03-05 21:32:58

Re: [SOLVED] Unable to decrypt drives correctly after latest update

#7 2024-03-05 21:45:47

Re: [SOLVED] Unable to decrypt drives correctly after latest update

#8 2024-03-05 22:16:37

Re: [SOLVED] Unable to decrypt drives correctly after latest update

#9 2024-03-05 22:21:49

Re: [SOLVED] Unable to decrypt drives correctly after latest update

#10 2024-03-05 22:31:08

Re: [SOLVED] Unable to decrypt drives correctly after latest update

#11 2024-03-06 01:02:25

Re: [SOLVED] Unable to decrypt drives correctly after latest update

#12 2024-03-06 01:09:46

Re: [SOLVED] Unable to decrypt drives correctly after latest update

#13 2024-03-06 09:12:09

Re: [SOLVED] Unable to decrypt drives correctly after latest update

#14 2024-03-06 11:52:22

Re: [SOLVED] Unable to decrypt drives correctly after latest update

Board footer