Img-->hdd

tony5429 · 2009-05-05 13:08:32

I recently used the dd command on a computer to backup its entire 12GB hard drive (with partitions for Windows and Arch32) to a file, backup.img. Now I've got a 300GB hard drive laying around and an Arch64 system here. How can I get the 12GB image on the 300GB hard drive? And is there a way I can do it so that I can make the Windows and Arch32 partitions larger (since I'll have more space to work with)? Thanks!

perbh · 2009-05-05 14:56:42

Generally, it's a bad idea to dd an entire disk - far better to dd each partition in which case you can loop-mount the image and then copy to your new disk.
An even better method is to make a tar-file out of each partition.

However, seeing that done-is-done, I believe you need some c-moment gymnastics to make it work, and in the end you will need to end up with an image of each partition.
Your first sector (512 bytes) of the image will contain the mbr of which we are only interested in the partition table, ie bytes #447 through #510 (starting at '1'). You need to google for 'mbr' to figure out out exactly how the partition table is made up ... each entry is 16 bytes and I know it contains 'start sector' and 'no of sectors' but I cant remember exactly how it is laid out ...

So in order to pick out - say partition #1 and you have found that start sector is sector #65 (starting with '1') and no of sectors is 16,000,000 (roughly 8 giga), you need something like this:

dd if=backup.img bs=512 skip=64 count=16000000 of=disk-part1.img

Then you will do the same for the other partitions on the disk
In order to access the files, you then need to mount it - thus (assume that the first partition was windows with the ntfs filesystem):

mkdir /olddisk
mount -t ntfs -o ro,loop disk-part1.img /olddisk

Now you can copy from /olddisk to your new disk

Not easy, I know - but in the future rather make a tarball out of a complete filesystem. That way you will not have to save unused space.

Good luck!

tony5429 · 2009-05-05 15:39:18

Thank you perbh! This is a very complicated process - wow! Actually, I've got QEMU on this computer and I just backed up the image and tried booting into it and found that the windows partition at least boots fine, so I know the data's there. What would happen if I just removed all the partitions from the 300GB drive, mounted it in this system as /dev/sda5, and then ran this...

dd if=backup.img of=/dev/sda5

Do you know? Do you think I would get the 12GB of 4 partitions that were in the original hard drive, but would just have 288 GB of unformatted space on top of it? And do you think I'd be able to boot into it?

perbh · 2009-05-05 15:47:18

no, no, no!!
Not a good idea!
You will screw up the 'root'-sector of sda5!
That's why I said it was a bad, bad idea to dd an entire disk because you also get the mbr - which in itself contains a partition table.

If you have a spare computer, put the new disk in that, use a live-cd to boot it and network the two computers against each other - and copy from one to the other.
Otherwise, you reap as you sow. You dd'd an entire disk instead of making a tarball of each partition, and now you have to live with it. At least - if you are in the same situation another time, you will not do it the same way again - it's how we live and learn. Personally I have been in the root directory (as user 'root') and done a 'rm -r *' - not fun!! But I've only done it once *chuckles*

[edit]
sorry, I didn't read properly what you said - and yes, that _should_ work ... but you have to do a

dd if=backup.img bs=4M of=/dev/sda

you would end up with 288 gigs unallocated at the end of the drive and have your original 'setup' in the first 12 gigs - and, it should be bootable.
[/edit]

Last edited by perbh (2009-05-05 15:52:28)

tony5429 · 2009-05-15 01:27:01

It worked. Thanks! Now... is there any way to use that extra 288 GB?

Wittfella · 2009-05-15 06:50:48

Try running gparted or some other partition manager from a livecd and you should be able to resize the partitiions

R00KIE · 2009-05-15 11:11:49

You can still mount partitions inside a full disk backup I do it all the time.

Mount the image file in a loop device

losetup -r /dev/loopX image_file

Don't use -r if you want write access.

Then find the start of each partition with

parted /dev/loopX unit B print

Then mount the partition you want with

mount /dev/loopX mountpoint -o offset=start_of_partition

And thats it, you can read anything you want from there.

tony5429 · 2009-05-27 18:28:26

Everything works now, but I'd like to go ahead and make a backup of the system as it currently stands. It is one hard drive with partitions sda1, sda2, sda3, and sda4. For each partition, I've used dd if=sda(1-4) of=/media/disk/sda(1-4).img

Now the only thing I am missing is the MBR, correct? And how do I dd just the MBR? Thanks.

perbh · 2009-05-27 18:36:06

dd if=/dev/sda bs=446 count=1 of=/some/path/mbr.img

If you want the _entire_ mbr - including primary partitions, you use bs=512 instead of bs=446

@rookie:
Thanks a lot - I didn't think that was possible!! One lives and learns ...

tony5429 · 2009-05-27 21:13:16

Thanks. I am using this script to backup everything to a bzipped tarball:

dd if=/dev/sda1 of=sda1.img
dd if=/dev/sda2 of=sda2.img
dd if=/dev/sda3 of=sda3.img
dd if=/dev/sda4 of=sda4.img
dd if=/dev/sda bs=446 count=1 of=mbr_minimal.img
dd if=/dev/sda bs=512 count=1 of=mbr_full.img
tar cjf backup.tar.bz2 *.img --remove-files

So that includes everything I would need to restore it in a worst-case scenario. Correct?

tony5429 · 2009-05-27 21:14:02

I should make it clear also that I am running that from an external hard drive with plenty of free space.

karol · 2009-05-27 21:49:30

Why bzip2? I takes ages to decompress. I suggest gzip (you say you've got plenty of space) or lzma w/ xztools for better compression. Just be sure to check whether it works as expected in your case - maybe you'll be better of w/ no compression at all?

Edit: Any chance one of the aforementioned partitions is swap?

Last edited by karol (2009-05-27 21:52:35)

tony5429 · 2009-05-28 00:56:50

Yeah; I noticed bzip2 takes a long time - I'll probably just tar it with no compression.

Yes - sda2 is swap...

sda1 - NTFS
sda2 - swap
sda3 - boot (XFS)
sda4 - / (XFS)

Is there something wrong with backing up swap? Or are you just saying it is pointless?

perbh · 2009-05-28 02:55:19

absolutely pointless!!
I would also strongly suggest that you make a tarball out of each partition (saves a l-o-t of space).
Something like this:

curdir=`pwd`
mkdir /mnt/tmp
for i in 1 3 4; do
  mount /dev/sda$i /mnt/tmp
  cd /mnt/tmp && tar cvzf $curdir/backup-sda$i.tar.gz
  cd $curdir
  umount /mnt/tmp
done

karol · 2009-05-28 09:40:50

Swap is just scratch space that gets written to if your system runs out of RAM. It gets cleared just like RAM and there's nothing valuable there :-)

Do you run that OS you're trying to backup *while* making the backup? If so, you get lots of garbage in your .imgs. As you're running XFS I suggest you try xfsdump, it's very easy to use, somewhat similar to 'dd if=/dev/sda4 of=sda4.img' but smarter: 'xfsdump -f /dev/tape /'. It means you want to backup your root (/) to /dev/tape. 'xfsrestore -f /dev/tape /' means you want to restore the contents of root from /dev/tape.
There are many many ways to efficiently and rather effortlessly backup your system, pick the one that suits you. It's like the one about a camera: the best camera is the one you actually carry.

Two things wrt to your dd use:
- You could specify 'bs' (stands for 'block sector' I believe) to sth like bs=4M (like perbh wrote). I don't know of any adverse effects, it just uses bigger chunks to copy, so it gets copied faster (computers don't like to deal w/ lots of small files, they prefer few bigger ones). This of course isn't the case when you backup your MBR :-)
- Where do you put your .img files? If you put sda1.img and sda3.img on /dev/sda4 - guess what, they'll end up inside sda4.img, so you'll have actually two copies of your sda1 and sda3 partitions. You should put them on your external drive to avoid this.

Last edited by karol (2009-05-28 09:43:29)

tony5429 · 2009-06-12 15:55:50

Thanks for all the tips everyone! This is hugely helpful. Since I'm backing up partitions with different filesystems, and because I want to have the swap placeholder for future reference (even though it will be junk data), I'm planning on doing the following. I'm also using dd rather than tar because I am more comfortable with it, and just to be safe I'm not changing the block size.

1. Boot off arch cd
2. Mount external hard drive to /mnt/disk
3. cd /mnt/disk
3. dd if=/dev/sda1 of=sda1.img
4. dd if=/dev/sda2 of=sda2.img
5. dd if=/dev/sda3 of=sda3.img
6. dd if=/dev/sda4 of=sda4.img
7. dd if=/dev/sda bs=446 count=1 of=mbr_minimal.img
8. dd if=/dev/sda bs=512 count=1 of=mbr_full.img
9. tar cf backup.tar *.img --remove-files

I just want to make sure... will that definitely back up everything I could ever need to restore the system?

tony5429 · 2009-06-14 07:12:52

Does anyone know if that will work for me? Thanks in advance...

tony5429 · 2009-06-15 19:30:21

Anyone?...

R00KIE · 2009-06-16 09:31:23

As I see it, step 7 is pointless because you are storing the part where the boot code lies and not the partition table.

tony5429 · 2009-06-16 21:02:46

Hrm.. Okay - I have taken out step 7. Now I've got this...

1. Boot off arch cd
2. Mount external hard drive to /mnt/disk
3. cd /mnt/disk
3. dd if=/dev/sda1 of=sda1.img
4. dd if=/dev/sda2 of=sda2.img
5. dd if=/dev/sda3 of=sda3.img
6. dd if=/dev/sda4 of=sda4.img
7. dd if=/dev/sda bs=512 count=1 of=mbr.img
8. tar cf backup.tar *.img --remove-files

So will that backup everything I need to completely restore the system?

R00KIE · 2009-06-16 21:46:01

That should be it.
If you are planning to use the partitions only as a backup and don't plan to mount the images to get something out then consider using gzip to compress the images.
http://www.debianhelp.co.uk/ddcommand.htm

Also you can safely add bs=4M or bs=8M to the dd lines, it will most probably increase the speed because dd will copy the block size you specify at once instead of the default 512 bytes at a time. Higher values will most probably make no difference but feel free to try a few values and see what works better in your setup.

lamnk · 2009-06-16 22:12:01

Hi tony, i just have had the same situation as yours: migrating the old 32bit system to 64bit on a new hard drive.

What i did:
1. Create tar balls for /etc and /home of the old drive to external one:

tar -cf /media/external/home.tar /etc/

2. Install 64bit Arch and needed packages (i saved a list of packages i need to install from last setup)
3. Copy the tarball over the new drive (not directly to /home and /etc)
4. Selectively copy config files and .files to their location

This approach takes times (one afternoon for me) to setup but in the end you get a cleaner system.

About your approach to make images of partitions:
1. Why backup / ? You will have 64bit system, the old system just dont work
2. Why make a tar ball of all images ?

Arch Linux

#1 2009-05-05 13:08:32

Img-->hdd

#2 2009-05-05 14:56:42

Re: Img-->hdd

#3 2009-05-05 15:39:18

Re: Img-->hdd

#4 2009-05-05 15:47:18

Re: Img-->hdd

#5 2009-05-15 01:27:01

Re: Img-->hdd

#6 2009-05-15 06:50:48

Re: Img-->hdd

#7 2009-05-15 11:11:49

Re: Img-->hdd

#8 2009-05-27 18:28:26

Re: Img-->hdd

#9 2009-05-27 18:36:06

Re: Img-->hdd

#10 2009-05-27 21:13:16

Re: Img-->hdd

#11 2009-05-27 21:14:02

Re: Img-->hdd

#12 2009-05-27 21:49:30

Re: Img-->hdd

#13 2009-05-28 00:56:50

Re: Img-->hdd

#14 2009-05-28 02:55:19

Re: Img-->hdd

#15 2009-05-28 09:40:50

Re: Img-->hdd

#16 2009-06-12 15:55:50

Re: Img-->hdd

#17 2009-06-14 07:12:52

Re: Img-->hdd

#18 2009-06-15 19:30:21

Re: Img-->hdd

#19 2009-06-16 09:31:23

Re: Img-->hdd

#20 2009-06-16 21:02:46

Re: Img-->hdd

#21 2009-06-16 21:46:01

Re: Img-->hdd

#22 2009-06-16 22:12:01

Re: Img-->hdd

Board footer