You are not logged in.

#1 2015-12-27 19:00:36

eldarion
Member
From: Santarém - Portugal
Registered: 2006-08-01
Posts: 71

[SOLVED] dd where source and destination are both USB

Today, archlinux let me down.

This is more of a "warning, don't do this" post that a help request, because i know i know there is nothing i can do now.

So, here is the story....

Today, i was thinking on creating an USB book disk from a distro, so i could try it out. Since i have very low hdd space, i downloaded the iso to my 500 GB  usb disk.
Than, as always, i did two things:

1) lsblk
    to see what is the name of the usb disk to copy the iso content to. It turns out to be sdc.

2)  dd bs=4M if=./isoimage.iso of=/dev/sdc && sync
     to create the usb boot disk
 
What went wrong? "./isoimage.iso" was actually /run/media/500GB DISK/isoimage.iso.
After the "dd bs=4M if=./isoimage.iso of=/dev/sdc && sync" ended, i had my sdb disk (the 500 GB disk) flashed with the contents of isoimage.iso.
It was not a sdx mix mistake because after i noticed this, i did an "history" and an "lsblk" again, and i did not screw up the drives naming.

So, instead of flashing sdc, it flashed sdb. All my data lost. Almost 500Gb of it....

Moral of the story: don't even think on using dd with both source and destination drives an usb disk. NEVER!

This is a sad tale.
If someone have any ideas on how to recover data after using dd, please, feel free to post.

Thank you in advance.

Last edited by eldarion (2015-12-28 01:28:18)

Offline

#2 2015-12-27 19:14:27

frostschutz
Member
Registered: 2013-11-15
Posts: 1,409

Re: [SOLVED] dd where source and destination are both USB

I had my sdb disk (the 500 GB disk) flashed with the contens of isoimage.iso.

The only way that can happen is if you yourself gave the wrong device name. Even if the USB has some issue due to bad cable or whatever, which might cause devices to be disconnected, and subsequently re-detected under a different name, as long as a device name is still in use (because dd has an open filehandle on it), it would not be able to replace one device name with another. It might be different if dd errored out and you just re-started the dd without verifiyng that the device names are still correct... in either case, you should check your dmesg / syslog.

If someone have any ideas on how to recover data after using dd, please, feel free to post.

You can not recover what has been overwritten. So if the ISO was 4GB is size and completely written to that disk, the first 4GB are gone. Recovery now depends on what was on the disk; if you had multiple partitions, you should be able to recover partitions that started beyond the ISO size; if it was one big filesystem it depends on the type of filesystem; most filesystems have their important metadata at the beginning so it's not easy to repair with that gone.

You might have to resort to photorec. But photorec might be confused by the ISO and scan the entire disk like you would handle an ISO type filesystem, which is wrong for regular disk systems; so you might have to zero out the ISO size or create a loop device with offset for photorec to scan properly as raw data mode.

Last edited by frostschutz (2015-12-27 19:22:44)

Offline

#3 2015-12-27 20:52:51

eldarion
Member
From: Santarém - Portugal
Registered: 2006-08-01
Posts: 71

Re: [SOLVED] dd where source and destination are both USB

Thank you for your reply, frostschutz.

I really don't know how to explain this other than a dd "screw up". I ran dd one time only and i am certain that sdb was my 500 GB disk and sdc was my destination disk. Like you said, even if a redetection occurred after i run "dd", sdb was in use (mounted), so dd theoretically would give me an error.

Thanks for the suggestion. I'll give photorec a try. Right now, i'm trying Minitool Partition wizard to try to recover the partition table.Lets see how it goes.

Offline

#4 2015-12-27 21:07:59

frostschutz
Member
Registered: 2013-11-15
Posts: 1,409

Re: [SOLVED] dd where source and destination are both USB

If sdb is mounted, and you unplug the disk and replug it (or have a cable problem or whatever to the same effect), the same disk will now be sdc, sdd or whatever is free at the time.

sdb will still be mounted but it will be a filesystem that got the rug pulled from under its feet. Both read and write will fail for this mount, all it will do is occupy the sdb name.

Offline

#5 2015-12-27 21:09:34

TheChickenMan
Member
From: United States
Registered: 2015-07-25
Posts: 354

Re: [SOLVED] dd where source and destination are both USB

I hope this isn't a hijack but using dd has always kind of scared me. I have been looking for a way (udev perhaps?) to make my permanent drives sd[a-h] be recognized as something separate and higher in the alphabet which would be less likely to be confused. Is there someway to, for safety, map my drives so that the kernel will assign them to sd[q-x]?


If quantum mechanics hasn't profoundly shocked you, you haven't understood it yet.
Niels Bohr

Offline

#6 2015-12-27 21:13:29

frostschutz
Member
Registered: 2013-11-15
Posts: 1,409

Re: [SOLVED] dd where source and destination are both USB

You can use one of the /dev/disk/by-*/ names instead, for example with GPT partitions I quite like the partlabels. I'm using RAID and I've labeled my partitions according to which md and role they have in the RAID. So I have /dev/disk/by-partlabel/md1-{0,1,2,3,4,5,6}. Should my RAID metadata ever be damaged, I still have the relevant data [disk order, in particular] in the partlabel. However, for these /dev/disk/by-*, you are relying on udev working flawlessly. If udev ever is confused (which may happen if you experience USB errors, i.e. faulty hardware) those links may still point to the wrong devices. What you'd need is a dd that interactively prompts you: write to this disk which has serial number so-and-so etc. etc.? I.e. it should display a few details about the device you're about to write to and ask for confirmation.

Last edited by frostschutz (2015-12-28 00:48:40)

Offline

#7 2015-12-27 22:01:02

TheChickenMan
Member
From: United States
Registered: 2015-07-25
Posts: 354

Re: [SOLVED] dd where source and destination are both USB

Thanks for the suggestion. I guess I should try using the disk labels when I need to work with dd. I do have my disks in fstab and other config files listed by UUID. When referring to them in terminal for tasks such as dd I always just used the device letter though. Less to type and old habits die hard. I still think it would be really nice to be able to a static /dev/sd[letter] to a specific disk though.


If quantum mechanics hasn't profoundly shocked you, you haven't understood it yet.
Niels Bohr

Offline

#8 2015-12-28 01:25:37

eldarion
Member
From: Santarém - Portugal
Registered: 2006-08-01
Posts: 71

Re: [SOLVED] dd where source and destination are both USB

Ok, just to let you know, i recovered all (or probably 99%) of my data. I ended up using TestDisk and managed to recover the NTFS partition of the disk and recreate the NTFS boot sector.

I followed the instructions from the official page with two small changes:

1) on "Partition table type selection", testdisk detected an "non partitioned media". So i selected "Intel" insted of the default "None"
2) after the "quick search", i chose the "lost" ntfs partition and made an "rebuild BS".

I have probably some 4GB of corrupted data, but that is nothing compared with almost 500 GB of data recovered.

Thanks again for helping.

Last edited by eldarion (2015-12-28 01:29:15)

Offline

#9 2015-12-28 01:32:40

TheChickenMan
Member
From: United States
Registered: 2015-07-25
Posts: 354

Re: [SOLVED] dd where source and destination are both USB

Great to hear that you got your data back. I was thinking more about what frost said. I think the best solution would be for me to write a wrapper for dd that checks for disk information and asks for confirmation. Not sure when I'll have the time to get that finished but I think it would help allay my fears about using the command all the time and might be a good addition to the aur at some point.


If quantum mechanics hasn't profoundly shocked you, you haven't understood it yet.
Niels Bohr

Offline

#10 2015-12-28 16:25:11

eldarion
Member
From: Santarém - Portugal
Registered: 2006-08-01
Posts: 71

Re: [SOLVED] dd where source and destination are both USB

TheChickenMan wrote:

Great to hear that you got your data back. I was thinking more about what frost said. I think the best solution would be for me to write a wrapper for dd that checks for disk information and asks for confirmation. Not sure when I'll have the time to get that finished but I think it would help allay my fears about using the command all the time and might be a good addition to the aur at some point.

Hello TheChickenMan.
Since i still don't know how it happened i can't say if a wrapper is the solution. That said, i agree with you. For minimizing the "dd from" and "dd to" disk mistakes, a wrapper would help. A simple shell script that would run before dd showing the source and destination info and asking for confirmation (as well as checking the "integrity" of the user choice immediately before executing dd) would help. But again, nothing is guaranteed so i don't think it's worth the work. The best way to avoid this would be not to make dd from a removable source and to unmount all the removable disks exept the one we want to "dd to" (to minimize the drive designation to changes).

Offline

#11 2016-01-22 09:56:14

kazimir
Member
Registered: 2012-01-20
Posts: 79

Re: [SOLVED] dd where source and destination are both USB

As an alternative to dd you could use OpenSUSE imagewriter or isousb (available in AUR) or rufus.

For a wrapper, you could have a look at the script used in system rescue cd

Offline

Board footer

Powered by FluxBB