how to do a perfect copy from internal hdd to external hdd

nsm_nikhil · 2008-07-11 04:07:58

I'll be moving soon to university, and would like to take my entire Arch installation AS IT IS. Can I acheive this by just copying the / Root partition and /home onto the right partitions on the external USB hdd (I've formatted them as ext3 ) and tweaking the boot parameters and /etc/fstab and reinstalling GRUB on the MBR of the external, or will I need to do a reinstall?

Also is a swap partition recommended on an external hdd?

Thanks

Nikhil

bbs · 2008-07-11 21:00:15

i would actually recommend booting from a live cd or some such since you will not want the machine to be running since some files may be sensitive to access -- boot a live cd and make sure both drives have the same partition scheme -- then you can copy (assuming you have default) -- / to / -- boot to /boot -- /home -- so on and so forth

-jt

rson451 · 2008-07-11 22:05:57

if you want to take it as is, take a look at the program 'dd'.

Last edited by rson451 (2008-07-11 22:06:12)

lldmer · 2008-07-11 22:23:55

Maybe you will find this usefull. You can use

sfdisk -d /dev/sda | sfdisk /dev/sdb

to copy the partition table from sda to sdb.

lloeki · 2008-07-11 22:56:12

dd if=/dev/sda of=/dev/sdb bs=1048576 worked very nicely for me, then followed by fdisk+resize2fs to grow the last partition.

nsm_nikhil · 2008-07-12 03:59:01

ok, i'll try what you guys suggested in a few days. Thanks a lot.

@lloeki
I didn't understand what you said about resizing the last partition. Why would I wand to do that?

Here is my partition table for /dev/sdb which is my main hdd.

               First       Last
 # Type       Sector      Sector   Offset    Length   Filesystem Type (ID) Flag
-- ------- ----------- ----------- ------ ----------- -------------------- ----
 1 Primary           0     1992059     63     1992060 Linux swap / So (82) None
 2 Primary     1992060    29334689      0    27342630 Linux (83)           Boot
 3 Primary    29334690    58637249      0    29302560 Linux (83)           None
 4 Primary    58637250    78156224      0    19518975 Linux (83)           Boot

Partition #4 is not used, it is spare space to try out other linux distributions whenever I want to

Will the dd command do a perfect copy. As far as I know it doesn't care about the data, it just copies blind, am I correct?

Thanks for the help

Nikhil

lilsirecho · 2008-07-12 16:49:44

Gents;

Reading posts with interest and have an allied problem area to be addressed.

If one uses lilo as the boot system, what needs to be done if transferring the archlinux from one drive to another since a new lilo install has to be made in the new drive.

I assume a /sbin/lilo -v is required on the new HDD. How is that accomplished?

lloeki · 2008-07-14 10:44:00

I didn't understand what you said about resizing the last partition. Why would I wand to do that?

that was applicable to my case. I migrated a debian system from an old 80Gb disk to a 320Gb one.
steps were:
- dd to byte-copy everything as-is (this is what is interesting for you)
- fdisk to resize the last partition to fill the remaining free space
- resize2fs to grow the filesystem to fill the resized partition

'bs' parameter is useful as it tells dd to read by chunks of 1Mb, which results in a faster transfer.
also, you may want to do 'kill -USR1 $(pidof dd)' from time to time to monitor copy progress.
you will want to do the copy off the running system (e.g with a livecd), else you'll get dirty filesystem/mysql databases/etc...

dd copies the system at the block device level, but you could copy it at the (mounted) filesystem level e.g with 'rsync -avH --numeric-ids /mnt/src /mnt/dest' or something like that. useful as it allows you to do that over ssh (I'm using a variation of it for remote backups)

lloeki · 2008-07-14 10:48:55

lilsirecho, since dd copies what you tell him to copy (here the whole block device) it will copy the mbr and thus the bootloader too.

only reason to reinstall bootlaoder is because partition indexes changed if the hard disk order has changed, eg (hd0,0) becomes hd(1,0) in grub case, else grub won't find its config file (which has to be updated too, along with fstab). I guess the same would go for lilo. but if it's just a swap, then nothing to do. YMMV.

nsm_nikhil · 2008-07-14 12:47:02

thanks a lot guys!

DaveCode · 2008-08-26 09:19:08

GNU ddrescue for this task, "ddrescue --help". Either way, the trick is source/destination partitions of exact size. If partition sizes differ, then go from partition to file. The bigger partition holds the file of the smaller partition. Going in reverse, file to partition, works fine. Make up any file name you like, I use .ddrescue suffix.

For precise disk partitioning, GNU cfdisk (ncurses interface) includes raw byte units.
http://www.gnu.org/software/fdisk/
It links dynamically to
http://www.gnu.org/software/parted/index.shtml

But GNU coreutils replaces too much stuff. It's best to build the GNU cfdisk yourself, or pull it and libparted from the Arch coreutils package.

DaveCode · 2008-08-26 09:26:56

About the partition table and MBR; this technique automatically captures them, assuming you use the full disk device as source/target instead of just one partition. So, /dev/hda not /dev/hda1, or exploiting udev, /dev/disk/by-path/... avoiding the partN listings.

For individual partitions (not the whole device) you can use /dev/disk/by-label/....
but that of course will not capture the table or MBR, which I often backup separately.

Profjim · 2008-08-26 22:44:44

Another trick for full-disk copying is:
# assume you want to copy entire partition mounted as /source (e.g.. / or /home) to a target partition mounted as /mnt/target
sudo -s # or su
cd /source
find . -xdev -depth -print0 | cpio --null --sparse -pvdm /mnt/target

That's faster than dd because it only copies sectors in use. As other posters recommended, I'd do this while booted from a separate install, i.e., don't let source be /.

Another advantage to this is that it allows /target to already be a different size from /source (so long as it's big enough to handle the data).

Symbolic links should work fine, and mod times and permissions will be preserved. I don't know what will happen to hard links. If you want to go this way, and need to preserve any hard links, check on that or test it. Possibly you'll just get multiple (de-linked) copies of the original files.

Once you've copied, the main things to worry about are your /boot/grub/menu.lst and /etc/fstab files. The partition numbers in the grub file may need changing. If either the grub file or the fstab file rely on /dev/sdXX references or UUID references, those will need to be corrected too. If they just rely on labels, your work is easier because you can just change the labels on your partitions and be done.

DaveCode · 2008-08-26 23:08:16

It's best to keep file systems dismounted for backup and copying. That is safest and also most universal (covers all filesystems). Not everyone shares that viewpoint, of course. Each way has pros and cons. We might quibble about speed.

Myself, I run from live CD so that no system disks are running at all when I want to image them. Thus, I get max throughput, as no system tasks need to access the hard drive except ddrescue.

Profjim · 2008-08-26 23:37:30

DaveCode, I appreciate what you're saying. Your way is safer. I posted the other instructions so that searchers could find the different options collected in one place.

I've been in situations where I didn't have the time / free disk space to do a full disk or even full byte-for-byte partition copy, but I did want to have as faithful a backup as I could. The find/cpio line was recommended and has worked perfectly for me.

As to filesystems, I've only tried this with ext2/ext3.

nsm_nikhil · 2008-08-27 05:56:33

DaveCode, I appreciate your help, but what I wanted to do was just install from my existing installation.
So I just followed this wiki guide -> http://wiki.archlinux.org/index.php/Ins … ting_Linux

Thanks

DaveCode · 2008-09-09 10:31:27

Um, how to put this? that way is harder and slower than dd_rescue, and less "perfect."

I clone systems with dd_rescue. The only files to tweak are fstab, xorg.conf, and network. That's all. Udev does the rest. DHCP lets you skip network tweaks.

The files are common across the clones. The only differences are what gets commented out, like:

# config file
# ------------
# machine A
# blah blah foo
# ------------
machine B
blah blah bar
# ------------
# machine C
# blah blah boo
# ------------

So it's brainless once configs are defined. xorg.conf has small, alternate ServerLayout sections pointing to the right devices (about 5 lines) which is all that needs to change.

Arch Linux

#1 2008-07-11 04:07:58

how to do a perfect copy from internal hdd to external hdd

#2 2008-07-11 21:00:15

Re: how to do a perfect copy from internal hdd to external hdd

#3 2008-07-11 22:05:57

Re: how to do a perfect copy from internal hdd to external hdd

#4 2008-07-11 22:23:55

Re: how to do a perfect copy from internal hdd to external hdd

#5 2008-07-11 22:56:12

Re: how to do a perfect copy from internal hdd to external hdd

#6 2008-07-12 03:59:01

Re: how to do a perfect copy from internal hdd to external hdd

#7 2008-07-12 16:49:44

Re: how to do a perfect copy from internal hdd to external hdd

#8 2008-07-14 10:44:00

Re: how to do a perfect copy from internal hdd to external hdd

#9 2008-07-14 10:48:55

Re: how to do a perfect copy from internal hdd to external hdd

#10 2008-07-14 12:47:02

Re: how to do a perfect copy from internal hdd to external hdd

#11 2008-08-26 09:19:08

Re: how to do a perfect copy from internal hdd to external hdd

#12 2008-08-26 09:26:56

Re: how to do a perfect copy from internal hdd to external hdd

#13 2008-08-26 22:44:44

Re: how to do a perfect copy from internal hdd to external hdd

#14 2008-08-26 23:08:16

Re: how to do a perfect copy from internal hdd to external hdd

#15 2008-08-26 23:37:30

Re: how to do a perfect copy from internal hdd to external hdd

#16 2008-08-27 05:56:33

Re: how to do a perfect copy from internal hdd to external hdd

#17 2008-09-09 10:31:27

Re: how to do a perfect copy from internal hdd to external hdd

Board footer