ArchLinux, 4xHDD and RAID10

JakeStyles · 2009-05-19 06:56:41

Howdy, folks.

I just started learning ArchLinux and my first PC would be nice 4xSATA 320Gb RAID10 setup.

I do just about everyhting I can do, but cant get grub installed. It cant get 'setup (hd0)' done....

Story so far...
1) boot with liveCD

2) put kbd on and load modules
modprobe raid0
modprobe raid1
modprobe raid5

3) Make partitions with cfdisk (all primary)
/dev/sda1 bootable linux raid autodetect 200Mb
/dev/sda2 linux raid autodetect 2048Mb
/dev/sda3 linux raid autodetect 20000Mb
/dev/sda4 linux raid autodetect xxxxxxMb

4) repeat for sdb, sdc and sdd

5) reboot (just in case) and check that all parts are in order

6) make raid
mdadm --create --verbose /dev/md1 --level=10 --raid-devices=4 /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1
mdadm --create --verbose /dev/md2 --level=10 --raid-devices=4 /dev/sda2 /dev/sdb2 /dev/sdc2 /dev/sdd2
.....

7) Make filesystems
mkfs.ext2 /dev/md1
mkreiserfs /dev/md3
mkreiserfs /dev/md4

8) Swap on, baby Swap on!
mkswap /dev/md2
swapon /dev/md2

9) Mounted filesystems
mount /dev/md3 /mnt
cd /mnt
mkdir home
mkdir boot
mount /dev/md1 /mnt/boot
mount //dev/md4 /mnt/home

10) run installation EXCEPT grub
/arch/setup

11) tried to install grub
grub
device (hd0) /dev/sda
root (hd0,0)
setup (hd0)

and it does not find /boot/grub/stage1 there....

I have few ideas but better not try before asking ppl who might know...

I used http://www.uplinkzero.com/howto/arch-so … tion-guide and http://wiki.archlinux.org/index.php/Ins … AID_or_LVM as reference.

I would appreciate any help here.

Jake and the Fatman

fukawi2 · 2009-05-19 07:37:49

I'm not sure if grub supports RAID 10... Do you really need the performance benefits of RAID 10 on your /boot? Can you try doing RAID-1 (across 2 or 4 partitions) instead?

Also, there's no need to create an MD array for your swap:
http://unthought.net/Software-RAID.HOWT … html#ss2.3

JakeStyles · 2009-05-19 08:11:56

fukawi2 wrote:

I'm not sure if grub supports RAID 10... Do you really need the performance benefits of RAID 10 on your /boot? Can you try doing RAID-1 (across 2 or 4 partitions) instead?

I just want to create fast fileserver and use RAID10. If needed, I just put one more HDD and make one partition from 4 others, like /data. Then I have 1 HDD for basic system and run others in RAID10. I just need to make fast file services.

Think it like I need to share multiple ISO files for LAN use. max 20 clients download dvd-size image files every now and then.

Next step is to link my nic's to one. Like Network Load Balancing.

fukawi2 · 2009-05-19 09:50:15

JakeStyles wrote:

Think it like I need to share multiple ISO files for LAN use. max 20 clients download dvd-size image files every now and then.

I'd install your system on RAID-1 and then store your data on RAID-0 or RAID-10 depending on how secure you want your data. Having the system on RAID-1 will tolerate a hardware failure of a drive, and as long as you have enough RAM, the programs that are often used will be stored in cache after they're first loaded so should still be nice and fast.
sdX1 = /boot (RAID-1)
sdX2 = / (RAID-1)
sdX3 = Swap
sdX4 = /data (RAID-0 or RAID-10)

JakeStyles wrote:

Next step is to link my nic's to one. Like Network Load Balancing.

Bonding is what you're after -- there's information in the wiki.

JakeStyles · 2009-05-20 06:17:07

I'd install your system on RAID-1 and then store your data on RAID-0 or RAID-10 depending on how secure you want your data. Having the system on RAID-1 will tolerate a hardware failure of a drive, and as long as you have enough RAM, the programs that are often used will be stored in cache after they're first loaded so should still be nice and fast.
sdX1 = /boot (RAID-1)
sdX2 = / (RAID-1)
sdX3 = Swap
sdX4 = /data (RAID-0 or RAID-10)

So putting this all together, I would need
4xHDD RAID10 for /data
2xHDD RAID1 for rest

I think I stick with 1 system drive /sda and make 4 others as /data (sdb, sdc, sdd sde).

Should I make swap partition in RAID10? Some people suggest it. It means I should make separate partitions on each drive but not make them RAID10. Did I understand it correctly? (sdb2 - sde2 would be RAID10 /data)

fukawi2 · 2009-05-20 07:29:01

JakeStyles wrote:

So putting this all together, I would need
4xHDD RAID10 for /data
2xHDD RAID1 for rest

No, I was suggesting still partitioning your drives the same, just arranging the RAID differently. (Using RAID-1 in place of RAID-10 for your system partitions/mount points).

JakeStyles · 2009-05-25 05:34:57

ok, now I have a slight problem

boot is /dev/sda1
swap is /dev/sda2 & /dev/sdb1 & dev/sdc1 & /dev/sdd1

device /dev/md1 consists of /dev/sda3, /dev/sdb3, /dev/sdc3 and /dev/sdd3

I modified mkinitcpio.conf:

MODULES="radi0 raid1 raid5"

HOOKS="base udev autodetect pata scsi sata mdadm filesystems"

Then I

mdadm -D --scan >>mdadm.conf

And 2 ARRAYs were added in file

Then I modified menu.lst for grub

kernel /vmlinuz26 root=/dev/md1 ro md=1,/dev/sda3,/dev/sdb3,/dev/sdc3,/dev/sdd3

When I boot, I get this:

...
:: Running Hook [filesystems]
:: Loading root filesystem module...
Attempting to create root device '/dev/md1'
ERROR: Failed to parse block device name for '/dev/md1'
unknown
ERROR: root fs cannot be detected. Try using the rootfstype= kernel parameter.
Waiting for devices to settle... done.

Root device '/dev/md1' doesn't exist, attempting to create it
ERROR: Failed to parse block device name for '/dev/md1'
ERROR: Unable to create/detect root device '/dev/md1'
Dropping to a recovery shell...

It seems like hooks does sata and then filesystems but mdadm is passed. I tried using 'raid' instead but same results.

fukawi2 · 2009-05-25 06:13:38

JakeStyles wrote:

I modified mkinitcpio.conf:

MODULES="radi0 raid1 raid5"

HOOKS="base udev autodetect pata scsi sata mdadm filesystems"

You know you made a typo in raid0? Your 'd' and 'i' are swapped
Also, I don't think you actually even need to add the MODULES part. My system is running (and booting) from a RAID-1 setup, and my MODULES is empty.

JakeStyles wrote:

Then I modified menu.lst for grub

kernel /vmlinuz26 root=/dev/md1 ro md=1,/dev/sda3,/dev/sdb3,/dev/sdc3,/dev/sdd3

You don't need the md arguments when using the mdadm hook in mkinitcpio. Remove them and leave the 'root=/dev/md1' and 'ro' parts.

Last edited by fukawi2 (2009-05-25 06:15:01)

sand_man · 2009-05-25 06:39:55

fukawi2, maybe he wanted a radi0 module!

JakeStyles · 2009-05-25 07:00:11

fukawi2 wrote:
JakeStyles wrote:
I modified mkinitcpio.conf:
MODULES="radi0 raid1 raid5"

HOOKS="base udev autodetect pata scsi sata mdadm filesystems"
You know you made a typo in raid0? Your 'd' and 'i' are swapped
Also, I don't think you actually even need to add the MODULES part. My system is running (and booting) from a RAID-1 setup, and my MODULES is empty.
MODULES was right, I just wrote it here wrong. Its 'raid0'. However I cleared it.
JakeStyles wrote:
Then I modified menu.lst for grub
kernel /vmlinuz26 root=/dev/md1 ro md=1,/dev/sda3,/dev/sdb3,/dev/sdc3,/dev/sdd3
You don't need the md arguments when using the mdadm hook in mkinitcpio. Remove them and leave the 'root=/dev/md1' and 'ro' parts.

Ok, now fixed it

kernel /vmlinuz26 root=/dev/md1 ro

Still same error.

Is there anything special to do in fstab?
I just put in

/dev/md1 / reiserfs defaults 0 1

I was wondering, if my root / is in /dev/md1, how can it be read before mounting it? I mean /etc/mdadm.conf etc.

Last edited by JakeStyles (2009-05-25 07:02:51)

fukawi2 · 2009-05-25 10:01:02

JakeStyles wrote:

kernel /vmlinuz26 root=/dev/md1 ro
Still same error.

Exactly the same error? Is /dev/md1 RAID-0 or RAID-1? If it's RAID-0, then maybe you do need the RAID0 module in mkinitcpio.conf
I'm kinda making semi-educated guesses here since I've only ever working with Software RAID-1, not RAID-0.

JakeStyles wrote:

Is there anything special to do in fstab?
I just put in
/dev/md1 / reiserfs defaults 0 1
I was wondering, if my root / is in /dev/md1, how can it be read before mounting it? I mean /etc/mdadm.conf etc.

That is something I've never really understood -- all I know is that it does manage it

nowahn · 2009-05-25 11:01:52

JakeStyles wrote:

HOOKS="base udev autodetect pata scsi sata mdadm filesystems"

I use the raid hook, not the mdadm hook, and it works fine with raid0, raid1 and raid5 (the md=1,... argument is required). I've never tried the mdadm hook.

To use the raid hook, you should only have to change mdadm to raid in mkinitcpio.conf and give the md=1,... argument to the kernel.

fukawi2 · 2009-05-25 11:55:57

nowahn wrote:

I use the raid hook, not the mdadm hook, and it works fine with raid0, raid1 and raid5 (the md=1,... argument is required). I've never tried the mdadm hook.

AFAIK, the mdadm hook is a recent addition aimed at replacing the raid hook. IME it's more reliable and stable -- but yeah, it's worth going back and trying the old way. Like I say, I've only used RAID-1 before, so if you're using RAID-0 then perhaps the mdadm hook isn't quite as good with that as it is with RAID-1.

JakeStyles · 2009-05-26 05:30:30

nowahn wrote:
JakeStyles wrote:
HOOKS="base udev autodetect pata scsi sata mdadm filesystems"
I use the raid hook, not the mdadm hook, and it works fine with raid0, raid1 and raid5 (the md=1,... argument is required). I've never tried the mdadm hook.
To use the raid hook, you should only have to change mdadm to raid in mkinitcpio.conf and give the md=1,... argument to the kernel.

I already tried that with raid hook, no change.

Is there any way that mdadm.conf is loaded from somewhere else, because it seems after sata hook it does filesystems.

During installation I have to boot from CD and mount root /mnt first and then edit /mnt/etc/mdadm.conf

Last edited by JakeStyles (2009-05-26 05:42:20)

fukawi2 · 2009-05-26 06:31:24

Are you chroot'ing to /mnt and running `mkinitcpio -p kernel26` after you modify mkinitcpio.conf?

mount -o bind /proc /mnt/proc
mount -o bind /dev /mnt/dev
mount -o bind /sys /mnt/sys
chroot /mnt /bin/bash

Last edited by fukawi2 (2009-05-26 06:32:11)

JakeStyles · 2009-05-26 09:52:05

I changed to RAID1 and did everything again.

menu.lst

kernel /vmlinuz26 root=/dev/md1 ro md=1,/dev/sda3,/dev/sdb3

mkinitcpio.conf:

HOOKS="base udev autodetect pata scsi sata raid filesystems"

mdadm.conf was changed to include new raid1 and I deleted all other modifications (RAID10)

After chroot'ing I did 'mkinitcpio -p kernel26' and everything goes well but when I boot, problem is still there.

......
:: Running Hook [raid]
:: Running Hook [filesystems]
:: Loading root filesystem module...
Attempting to create root device '/dev/md1'
ERROR: Failed to parse block device name for '/dev/md1'
unknown
ERROR: root fs cannot be detected. Try using the rootfstype= kernel parameter.
Waiting for devices to settle... done.

Root device '/dev/md1' doesn't exist, attempting to create it
ERROR: Failed to parse block device name for '/dev/md1'
ERROR: Unable to create/detect root device '/dev/md1'
Dropping to a recovery shell...

Between [raid] and [filesystems] were no lines.

I trust ReiserFS is no problem here?

Last edited by JakeStyles (2009-05-26 09:53:23)

fukawi2 · 2009-05-26 23:07:12

JakeStyles wrote:

I trust ReiserFS is no problem here?

GRUB handles Reiser just fine -- besides, mdadm doesn't give a rats what the filesystem is as it's just assembling the RAID underneath it.

So you're back to using *only* RAID-1 now? Try using the 'mdadm' hook in mkinitcpio.conf and regenerate your init image and see what happens. Don't forget to save your mdadm.conf file first.

As a side note -- we've done tests at my work, and Software RAID-1 with LVM over the top achieves the same thing as RAID-10 (mirroring + striping), but is actually somewhat faster.

DREMA · 2009-06-26 04:06:33

Hello!
Do you solve your problem? I'm in the exact situation that you were.
I get that error when trying to boot.
Any advice would be appreciated, thanks.

DREMA · 2009-06-26 18:47:24

Ok, I doubled checked everything again, re ran "mkinitcpio -p kernel26" and now everything is running fine.
Thanks anyway.

Arch Linux

#1 2009-05-19 06:56:41

ArchLinux, 4xHDD and RAID10

#2 2009-05-19 07:37:49

Re: ArchLinux, 4xHDD and RAID10

#3 2009-05-19 08:11:56

Re: ArchLinux, 4xHDD and RAID10

#4 2009-05-19 09:50:15

Re: ArchLinux, 4xHDD and RAID10

#5 2009-05-20 06:17:07

Re: ArchLinux, 4xHDD and RAID10

#6 2009-05-20 07:29:01

Re: ArchLinux, 4xHDD and RAID10

#7 2009-05-25 05:34:57

Re: ArchLinux, 4xHDD and RAID10

#8 2009-05-25 06:13:38

Re: ArchLinux, 4xHDD and RAID10

#9 2009-05-25 06:39:55

Re: ArchLinux, 4xHDD and RAID10

#10 2009-05-25 07:00:11

Re: ArchLinux, 4xHDD and RAID10

#11 2009-05-25 10:01:02

Re: ArchLinux, 4xHDD and RAID10

#12 2009-05-25 11:01:52

Re: ArchLinux, 4xHDD and RAID10

#13 2009-05-25 11:55:57

Re: ArchLinux, 4xHDD and RAID10

#14 2009-05-26 05:30:30

Re: ArchLinux, 4xHDD and RAID10

#15 2009-05-26 06:31:24

Re: ArchLinux, 4xHDD and RAID10

#16 2009-05-26 09:52:05

Re: ArchLinux, 4xHDD and RAID10

#17 2009-05-26 23:07:12

Re: ArchLinux, 4xHDD and RAID10

#18 2009-06-26 04:06:33

Re: ArchLinux, 4xHDD and RAID10

#19 2009-06-26 18:47:24

Re: ArchLinux, 4xHDD and RAID10

Board footer