You are not logged in.

#1 2010-08-24 14:30:39

DarksideEE7
Member
From: Arkansas, United States
Registered: 2009-06-06
Posts: 356

Cloning a current working install to a RAID0 array using dd

So I wiped my 3x500GB RAID0 array that contained Windows 7 last night so I can format it ext4 (or whatever I find performs best for RAID0) in order to clone my current esata 500GB drive that contains my Arch x86_64 installation.  So my question is while I've booted Arch x86_64, which would be best:

sudo dd if=/dev/sda1 of=/dev/sdb1                       (Skipping swap)
sudo dd if=/dev/sda3 of=/dev/sdb3
sudo dd if=/dev/sda4 of=/dev/sdb4

or

boot into a live usb and do the same


or

Start a usb install and after finishing dd the contents of each partition over.

Offline

#2 2010-08-24 15:02:52

lilsirecho
Veteran
Registered: 2003-10-24
Posts: 5,000

Re: Cloning a current working install to a RAID0 array using dd

Perhaps you may find that you cannot boot on a raid0 array.


Prediction...This year will be a very odd year!
Hard work does not kill people but why risk it: Charlie Mccarthy
A man is not complete until he is married..then..he is finished.
When ALL is lost, what can be found? Even bytes get lonely for a little bit!     X-ray confirms Iam spineless!

Offline

#3 2010-08-24 15:13:15

.:B:.
Forum Fellow
Registered: 2006-11-26
Posts: 5,819
Website

Re: Cloning a current working install to a RAID0 array using dd

RAID 0 is striping, RAID 1 is mirroring. I assume you mean RAID 1?


Got Leenucks? :: Arch: Power in simplicity :: Get Counted! Registered Linux User #392717 :: Blog thingy

Offline

#4 2010-08-24 15:30:38

DarksideEE7
Member
From: Arkansas, United States
Registered: 2009-06-06
Posts: 356

Re: Cloning a current working install to a RAID0 array using dd

lilsirecho wrote:

Perhaps you may find that you cannot boot on a raid0 array.

You can't boot on a RAID0 array?  Why not?  I have a RAID0 array that had Windows 7 installed and a RAID5 array for media.......so you're telling me the boot partition can't be located on the RAID0 array?

.:B:. wrote:

RAID 0 is striping, RAID 1 is mirroring. I assume you mean RAID 1?

No I meant RAID0.  I had RAID0 for added performance.  All of my important data is on a RAID5 array.  This is all on a dedicated LSI PCI-E controller.

Last edited by DarksideEE7 (2010-08-24 15:31:54)

Offline

#5 2010-08-24 16:14:17

gazj
Member
From: /home/gazj -> /uk/cambs
Registered: 2007-02-09
Posts: 681
Website

Re: Cloning a current working install to a RAID0 array using dd

You can boot to hardware or fakeraid raid 0, but not sofware raid (or so I seem to remember, don't flame me if this is wrong).  If your using software raid.  I reccomend have a 256mb raid 1 /boot and a raid 0 / and raid 5 for your data.

I presume that as your using of=/dev/sdb your using hardware or fakeraid.  Software raid would be /dev/md*

I personally would use cp or rsync or even tar to copy the contents to predefined partitions.  dd copys everything filesystem partition size etc.  If your old partition size happens to be bigger than the new you would run into trouble with dd.

Oh and I would do it from a livecd personally

Last edited by gazj (2010-08-24 16:21:50)

Offline

#6 2010-08-24 16:45:00

.:B:.
Forum Fellow
Registered: 2006-11-26
Posts: 5,819
Website

Re: Cloning a current working install to a RAID0 array using dd

DarksideEE7 wrote:
.:B:. wrote:

RAID 0 is striping, RAID 1 is mirroring. I assume you mean RAID 1?

No I meant RAID0.  I had RAID0 for added performance.  All of my important data is on a RAID5 array.  This is all on a dedicated LSI PCI-E controller.

If you'd dd your partitions form sda to sdb, you have a de facto mirror, no? No striping, which would be what you're looking for.

gazj wrote:

You can boot to hardware or fakeraid raid 0, but not sofware raid (or so I seem to remember, don't flame me if this is wrong).

You mean:
* Harware RAID? Good
* Software RAID? Good
* FakeRAID? Be gone ye devil!


Got Leenucks? :: Arch: Power in simplicity :: Get Counted! Registered Linux User #392717 :: Blog thingy

Offline

#7 2010-08-24 17:20:30

DarksideEE7
Member
From: Arkansas, United States
Registered: 2009-06-06
Posts: 356

Re: Cloning a current working install to a RAID0 array using dd

gazj wrote:

I presume that as your using of=/dev/sdb your using hardware or fakeraid.  Software raid would be /dev/md*

Yes it is hardware RAID.  It's a Dell Perc 5/i controller (which is essentially an LSI 8480E).  I planned on partitioning the RAID0 array to something like:
/dev/sda1     boot     150MB (give or take, can't remember what is normally suggested)
/dev/sda2     swap     256MB
/dev/sda3     root     15GB
/dev/sda4     home     rest

I may partition /var/ as a reiserFS as well just for the hell of it. 

I guess my question isn't necessarily about RAID, since I've used it personally and professionally for years in all different formats and thus am very experienced in it's configuration, but I'm less experienced at cloning Linux file systems.

So you think rsync would be the way to go, eh?  I have a nice front end to it called lucky backup, but I just figured it would be easier to use dd because of the fact that it exactly will copied each partition (from what I understand).  I didn't know that dd actually retained the partition size if you set the of to a partition instead of a drive.  I'll probably boot into a live USB and then just run rsync for each partition.

If you'd dd your partitions form sda to sdb, you have a de facto mirror, no? No striping, which would be what you're looking for.

Yes, but this is going to be a one time thing, not something I'll be doing regularly as a means of backup.  I basically want to transfer my current working Arch install to a RAID0 array for better performance since I only boot into windows for gaming.  All of my disk intensive activity happens in Linux.


At first I thought you guys were saying that Arch Linux isn't capable of bootstrapping if the boot partition is located on a RAID0 array....I was thinking no way that's right tongue

Last edited by DarksideEE7 (2010-08-24 17:37:50)

Offline

#8 2010-08-24 17:37:28

gazj
Member
From: /home/gazj -> /uk/cambs
Registered: 2007-02-09
Posts: 681
Website

Re: Cloning a current working install to a RAID0 array using dd

gazj wrote:

You can boot to hardware or fakeraid raid 0, but not sofware raid (or so I seem to remember, don't flame me if this is wrong).

.:B:. wrote:

You mean:
* Harware RAID? Good
* Software RAID? Good
* FakeRAID? Be gone ye devil!

Well fakeraid is the only raid I have never used, but from comments like yours from various sources keeps me away from it.  People frown on software raid.  I don't know why? It's reliable and its extremely diverse, how else can you have raid 0,1 and 5 on one physical disk (at least I don't know of doing this with hardware raid).  Have you ever tried replacing an old raid hardware device when it fails.  I like software raid alot smile

Offline

#9 2010-08-24 17:40:47

gazj
Member
From: /home/gazj -> /uk/cambs
Registered: 2007-02-09
Posts: 681
Website

Re: Cloning a current working install to a RAID0 array using dd

Darkside, as far as I can see this would be fine.  If you wanted to change filesystem sizes or types obvioulsy now is the time (i.e. rsync cp or tar).  If your happy then dd until your hearts content wink

Offline

#10 2010-08-24 17:41:05

DarksideEE7
Member
From: Arkansas, United States
Registered: 2009-06-06
Posts: 356

Re: Cloning a current working install to a RAID0 array using dd

gazj wrote:
gazj wrote:

You can boot to hardware or fakeraid raid 0, but not sofware raid (or so I seem to remember, don't flame me if this is wrong).

.:B:. wrote:

You mean:
* Harware RAID? Good
* Software RAID? Good
* FakeRAID? Be gone ye devil!

Well fakeraid is the only raid I have never used, but from comments like yours from various sources keeps me away from it.  People frown on software raid.  I don't know why? It's reliable and its extremely diverse, how else can you have raid 0,1 and 5 on one physical disk (at least I don't know of doing this with hardware raid).  Have you ever tried replacing an old raid hardware device when it fails.  I like software raid alot smile

There's nothing wrong with software RAID.  I've used mdadm for software RAID in Linux at work and it's incredibly easy. 

Just for fun as proof of concept (while showing some coworkers how to configure mdadm) I set up a RAID1 array between an SD card and a USB flash drive.  It wasn't as slow as I thought it would be.......

Offline

#11 2010-08-24 17:45:16

gazj
Member
From: /home/gazj -> /uk/cambs
Registered: 2007-02-09
Posts: 681
Website

Re: Cloning a current working install to a RAID0 array using dd

Thats is pretty cool.  I haven't got anything against hardware raid.  It is good.  I had 7 servers using it when I used to work in IT.  But when it fails and you haven't got backup hardware (well we did but on our other site 25 miles away) the headaches and the moaning directors begin wink

Last edited by gazj (2010-08-24 17:45:53)

Offline

#12 2010-08-24 17:48:09

DarksideEE7
Member
From: Arkansas, United States
Registered: 2009-06-06
Posts: 356

Re: Cloning a current working install to a RAID0 array using dd

gazj wrote:

Thats is pretty cool.  I haven't got anything against hardware raid.  It is good.  I had 7 servers using it when I used to work in IT.  But when it fails and you haven't got backup hardware (well we did but on our other site 25 miles away) the headaches and the moaning directors begin wink

That's exactly why we didn't use hardware RAID on all of our systems.  It's much easier to use Linux software RAID. 

You may have heard people griping about software RAID in Windows using Intel Matrix Storage Manager.  From what I understand if you have an array that fails you have to use the same Intel southbridge (ICH9R won't work with ICH7R, etc.).  That means when you replace the motherboard you better be able to find the same or similar model.

I'm guessing the only people running that sort of RAID are people without anything important that just want faster throughput. 

Hardware RAID is awesome for RAID5 though.  I have a 5x500GB array and when I replace a driver reconstruction only takes around an hour.  I don't even want to know how long software RAID would take.  Plus I have a battery back up attachment that protects the 256MB ECC DIMM that provides write through cache for the arrays.

Last edited by DarksideEE7 (2010-08-24 17:50:10)

Offline

#13 2010-08-24 21:02:52

DarksideEE7
Member
From: Arkansas, United States
Registered: 2009-06-06
Posts: 356

Re: Cloning a current working install to a RAID0 array using dd

Well I copied the boot and root partitions so far....the home partition got a kernel panic in the middle of mkfs.ext4 when on a liveUSB, so I went ahead and rebooted and I'm doing it with a live system.  home is probably okay anyhow.  If this wasn't a home system I would unmount the source home partition first, but it's not that big of a deal to go through the trouble.

Offline

#14 2010-08-26 00:23:55

DarksideEE7
Member
From: Arkansas, United States
Registered: 2009-06-06
Posts: 356

Re: Cloning a current working install to a RAID0 array using dd

Well manually partitioning the RAID0 array and setting the bootable flag in fdisk, then copying the files over and attempting to boot didn't seem to work.  I may try again, or I may try it gentoo style and setup the OS, partition everything, then just chroot into the new filesystem and see if everything works.  From there I can see if I can get it to boot.  Maybe the kernel isn't able to bootstrap my system in RAID after all......

Any ideas/suggestions?  Essentially I just want to get all of the filesystem contents from my current setup to a hardware RAID array and use that instead of the external drive.  I mean if Windows 7 can do it you would think Linux could do it.  Especially considering that this RAID controller is meant for servers that run RHEL.

Offline

#15 2010-08-26 13:01:13

R00KIE
Forum Fellow
From: Between a computer and a chair
Registered: 2008-09-14
Posts: 4,734

Re: Cloning a current working install to a RAID0 array using dd

DarksideEE7 wrote:

Hardware RAID is awesome for RAID5 though.  I have a 5x500GB array and when I replace a driver reconstruction only takes around an hour.  I don't even want to know how long software RAID would take.  Plus I have a battery back up attachment that protects the 256MB ECC DIMM that provides write through cache for the arrays.

From my experience with a machine with a core i7 870 using software raid5 it takes a little over 2 hours to rebuild an array of 3 x 950GB partitions. This seems to be limited by the disks performance and not by the cpu performance as would be the case with any fairly recent machine I believe.

The machine does have ups backup power so it can shutdown cleanly. I was asked about the possibility of using an hardware raid card and I said no thanks, if something breaks or the driver support is discontinued then I'd be stuck if anything broke.


R00KIE
Tm90aGluZyB0byBzZWUgaGVyZSwgbW92ZSBhbG9uZy4K

Offline

#16 2010-08-26 19:31:59

DarksideEE7
Member
From: Arkansas, United States
Registered: 2009-06-06
Posts: 356

Re: Cloning a current working install to a RAID0 array using dd

R00KIE wrote:
DarksideEE7 wrote:

Hardware RAID is awesome for RAID5 though.  I have a 5x500GB array and when I replace a driver reconstruction only takes around an hour.  I don't even want to know how long software RAID would take.  Plus I have a battery back up attachment that protects the 256MB ECC DIMM that provides write through cache for the arrays.

From my experience with a machine with a core i7 870 using software raid5 it takes a little over 2 hours to rebuild an array of 3 x 950GB partitions. This seems to be limited by the disks performance and not by the cpu performance as would be the case with any fairly recent machine I believe.

The machine does have ups backup power so it can shutdown cleanly. I was asked about the possibility of using an hardware raid card and I said no thanks, if something breaks or the driver support is discontinued then I'd be stuck if anything broke.

Yea an i7 is more than powerful enough to hand a RAID5 array.  I'll end up using my current Q6600 system to run a RAID5 array eventually when I retire it from my gaming rig. 


So nobody has any comments on installing via chroot with arch?  I've done it with Gentoo but never with Arch.

Offline

#17 2010-08-27 08:35:33

DarksideEE7
Member
From: Arkansas, United States
Registered: 2009-06-06
Posts: 356

Re: Cloning a current working install to a RAID0 array using dd

Well I thought I would give a final update for anyone else arriving here via search for similar questions because the current Arch partitioning scripts do not like RAID0 arrays, but they are supported.  It just kept failing when trying to write the partition table whether I did auto-prepare, manual partition, cfdisk, etc. 

In the end I just did it manually in the command line using fdisk and manually setting the the bootable flag,
creating and enabling swap space with mkswap and swapon
creating the filesystems using mkfs.ext4 and mkfs.ext2

I then entered the menu and selected 'manually prepare and partition hard disks and mount points'.  It will ask if you want to keep your current setup, so answer yes.

Offline

Board footer

Powered by FluxBB