[SOLVED] wiping dm-crypt/dd disk issue

tinh · 2022-09-14 10:17:30

Hello everyone,

In order to setup a crypted backup disk, I'm following these steps on the wiki Dm-crypt/Drive_preparation#dm-crypt_wipe_on_an_empty_disk_or_partition but I encounter errors with dd. It simply seems to die at a random time before ending process.

Below, the steps I follow: first, I created the encrypted container:

# cryptsetup open --type plain -d /dev/urandom /dev/sda1 to_be_wiped
WARNING: Device /dev/sda1 already contains a 'ext4' superblock signature.

WARNING!
========
Detected device signature(s) on /dev/sda1. Proceeding further may damage existing data.

Are you sure? (Type 'yes' in capital letters): YES
# lsblk
NAME            MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINTS
sda               8:0    0   1.8T  0 disk
└─sda1            8:1    0   1.8T  0 part
  └─to_be_wiped 254:0    0   1.8T  0 crypt
nvme0n1         259:0    0 238.5G  0 disk
[...]

And then, use dd to fill the disk:

# dd if=/dev/zero of=/dev/mapper/to_be_wiped status=progress
566387200 bytes (566 MB, 540 MiB) copied, 41 s, 13.8 MB/s

But, dd is not progressing anymore (I tested letting it be for the night),

ps aux | grep "dd "
root        6662  0.0  0.0   5840   948 pts/3    D+   Sep13   0:03 dd if=/dev/zero of=/dev/mapper/to_be_wiped status=progress

dd died... I retried many times the operation suspecting some connexion issues between my computeur and the disk, but I always got the same "result" : sometimes dd dies after 41seconds, sometimes more, anyway, it never finishes the filling operation.

What do I do wrong?

Maybe, worth mentionning I checked my disk using S.M.A.R.T methods, short and extended and nothing abnormal showed up:

# smartctl -l selftest /dev/sda
smartctl 7.3 2022-02-28 r5338 [x86_64-linux-5.19.7-arch1-1] (local build)
Copyright (C) 2002-22, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%     20767         -
# 2  Short offline       Completed without error       00%     20764         -

Last edited by tinh (2022-09-16 09:35:23)

frostschutz · 2022-09-14 11:00:01

anything in dmesg / journalctl?

if there are connection errors or disk errors, you should see some messages about it in dmesg when you write to it

schard · 2022-09-14 11:04:05

Please run:

# killall -USR1 dd

And post dd's reaction.
Also shred might me more suitable for the task at hand.

Last edited by schard (2022-09-14 11:08:48)

frostschutz · 2022-09-14 11:12:08

On another note, you could add bs=1M so dd won't copy in 512 byte blocks with tons of syscall overhead.

And if you don't intend to verify the data you have written (`cmp /dev/zero /dev/mapper/to_be_wiped` after zeroing it) you can also use /dev/urandom directly... it used to be slow in the past but it's plenty fast enough for disk wipes now.

$ timeout 11 dd bs=1M if=/dev/urandom of=/dev/null status=progress
6182404096 bytes (6.2 GB, 5.8 GiB) copied, 10 s, 618 MB/s

On older systems without aes-ni, it's possible for urandom to be faster than crypt now. (Compare with 'cryptsetup benchmark') It used to be the other way around, on some systems urandom was so slow that crypt was faster even without aes acceleration. That's why every never-since-updated wiki jumps through hoops with using crypt or shred instead of urandom. (crypt still makes sense for the verification pass, if you want to make sure the overwrite worked)

However none of that will solve your issue if the disk can't be written to.

Last edited by frostschutz (2022-09-14 11:15:46)

tinh · 2022-09-14 12:22:59

Thank you for your time it helps!

frostschutz wrote:

anything in dmesg / journalctl?

So, dmseg with disk related topic (at the very beginning of the log, I unplug and plug back the disk).

# dmesg -H
[...]
[ +11.495081] usb 1-3.4: USB disconnect, device number 11
[  +9.777814] usb 1-3.4: new high-speed USB device number 12 using xhci_hcd
[  +0.215165] usb 1-3.4: New USB device found, idVendor=067b, idProduct=2773, bcdDevice= 1.00
[  +0.000014] usb 1-3.4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[  +0.000007] usb 1-3.4: Product: USB-SATA Bridge
[  +0.000011] usb 1-3.4: Manufacturer: Prolific Technology Inc.
[  +0.000005] usb 1-3.4: SerialNumber: 0000000000000000015
[  +0.004238] usb-storage 1-3.4:1.0: USB Mass Storage device detected
[  +0.000314] scsi host1: usb-storage 1-3.4:1.0
[  +1.007950] scsi 1:0:0:0: Direct-Access     ST2000DM 008-2FR102       0001 PQ: 0 ANSI: 0
[  +0.001023] sd 1:0:0:0: [sdb] 3907029168 512-byte logical blocks: (2.00 TB/1.82 TiB)
[  +0.000357] sd 1:0:0:0: [sdb] Write Protect is off
[  +0.000009] sd 1:0:0:0: [sdb] Mode Sense: 03 00 00 00
[  +0.000329] sd 1:0:0:0: [sdb] No Caching mode page found
[  +0.000006] sd 1:0:0:0: [sdb] Assuming drive cache: write through
[  +0.076257]  sdb: sdb1
[  +0.000117] sd 1:0:0:0: [sdb] Attached SCSI disk
[Sep14 14:02] usb 1-3.4: reset high-speed USB device number 12 using xhci_hcd
[...]
[ +18.438667] usb 1-3.4: reset high-speed USB device number 12 using xhci_hcd
[Sep14 14:04] usb 1-3.4: reset high-speed USB device number 12 using xhci_hcd

the last line

usb 1-3.4: reset high-speed USB device number 12 using xhci_hcd

is repeated each time I (try to) kill dd using kill technique Schard mentionned (but happen also without touching anything). Also same log w/ journalctl.

Finally, interestingly enough, killing dd seems to "wake him up", see below:

# dd if=/dev/urandom of=/dev/sdb status=progress                                                           :(
410546688 bytes (411 MB, 392 MiB) copied, 66 s, 6.3 MB/s
// dd seems dead, nothing happens
// ... killall -USR1dd, from another pts
// few seconds later
801849+0 records in
801849+0 records out
410546688 bytes (411 MB, 392 MiB) copied, 65.5868 s, 6.3 MB/s
1523081728 bytes (1.5 GB, 1.4 GiB) copied, 456 s, 3.3 MB/s
// dd seems dead, nothing happens
// ... killall -USR1dd, from another pts
// few seconds later
2974769+0 records in
2974769+0 records out
1523081728 bytes (1.5 GB, 1.4 GiB) copied, 456.411 s, 3.3 MB/s
1526379008 bytes (1.5 GB, 1.4 GiB) copied, 457 s, 3.3 MB/s

Last edited by tinh (2022-09-14 12:37:27)

schard · 2022-09-15 08:34:53

So dd is not dead, but still copying data. Only the progress monitoring is hung up.
What does iotop tell you during the dd operation?

Last edited by schard (2022-09-15 08:35:30)

tinh · 2022-09-16 09:34:47

tinh wrote:

the last line
usb 1-3.4: reset high-speed USB device number 12 using xhci_hcd
is repeated each time I (try to) kill dd using kill technique Schard mentionned (but happen also without touching anything). Also same log w/ journalctl.

Few researches on this error showed me something wrong was happening with the hardware. Not sure what precisely cause I'm missing some prior knowlegde. But anyway, I plugged the disk directly to the computer, instead of using it through a hub as I did. It worked just fine and I could finish the initialization!

# dd if=/dev/urandom of=/dev/sdb status=progress
2000394811392 bytes (2.0 TB, 1.8 TiB) copied, 101789 s, 19.7 MB/s
dd: writing to '/dev/sdb': No space left on device
3907029169+0 records in
3907029168+0 records out
2000398934016 bytes (2.0 TB, 1.8 TiB) copied, 101791 s, 19.7 MB/s
dd if=/dev/urandom of=/dev/sdb status=progress  2269.82s user 19431.18s system 21% cpu 28:16:31.10 total

So glad! Thank you for your help and time investigating the issue. Also, I learned quite a few things in a way, thanks for that as well.

Last edited by tinh (2022-09-16 09:41:02)

Arch Linux

#1 2022-09-14 10:17:30

[SOLVED] wiping dm-crypt/dd disk issue

#2 2022-09-14 11:00:01

Re: [SOLVED] wiping dm-crypt/dd disk issue

#3 2022-09-14 11:04:05

Re: [SOLVED] wiping dm-crypt/dd disk issue

#4 2022-09-14 11:12:08

Re: [SOLVED] wiping dm-crypt/dd disk issue

#5 2022-09-14 12:22:59

Re: [SOLVED] wiping dm-crypt/dd disk issue

#6 2022-09-15 08:34:53

Re: [SOLVED] wiping dm-crypt/dd disk issue

#7 2022-09-16 09:34:47

Re: [SOLVED] wiping dm-crypt/dd disk issue

Board footer