You are not logged in.
Hi folks,
i have a raid0 setup consisting of 2 partitions, each on its own physical drive. The problem is that udev uses whole disks as parts of the raid0 setup, instead of the partitions.
My setup has worked for 5 years, however since i restarted my machine after it was running for a month straight it's broken now. Basically i have an up to date system which is unable to assemble my raid0 setup correctly as it used to be.
To be more specific, these are the two disks:
# fdisk -l /dev/sdd
Device Boot Start End Blocks Id System
/dev/sdd1 * 63 479909744 239954841 fd Linux raid autodetect
/dev/sdd2 479909745 488392064 4241160 82 Linux swap / Solaris
# fdisk -l /dev/sde
Device Boot Start End Blocks Id System
/dev/sde1 * 63 479909744 239954841 fd Linux raid autodetect
/dev/sde2 479909745 488392064 4241160 82 Linux swap / SolarisMy mdadm.conf is:
ARRAY /dev/md0 level=raid0 num-devices=2 UUID=98043246:35472a47:129167bc:dd4f370fHowever it does not seem to have any effect, since it is now using md127 instead of md0. In fact when i comment out the above line it does exactly the same.
This is what it does at bootup:
[ 5.007914] md: bind<sdd>
[ 5.017113] md: bind<sde>
[ 5.048870] mdadm: sending ioctl 800c0910 to a partition!
[ 5.048876] mdadm: sending ioctl 800c0910 to a partition!
[ 5.048896] mdadm: sending ioctl 1261 to a partition!
[ 5.048897] mdadm: sending ioctl 1261 to a partition!
[ 5.049192] mdadm: sending ioctl 1261 to a partition!
[ 5.049194] mdadm: sending ioctl 1261 to a partition!
[ 5.049435] mdadm: sending ioctl 1261 to a partition!
[ 5.049436] mdadm: sending ioctl 1261 to a partition!
[ 5.049704] mdadm: sending ioctl 1261 to a partition!
[ 5.049706] mdadm: sending ioctl 1261 to a partition!
[ 5.050651] md: raid0 personality registered for level 0
[ 5.050987] md/raid0:md127: md_size is 976793600 sectors.
[ 5.050989] md: RAID0 configuration for md127 - 1 zone
[ 5.050990] md: zone0=[sdd/sde]
[ 5.050993] zone-offset= 0KB, device-offset= 0KB, size= 488396800KB
[ 5.050994]
[ 5.051003] md127: detected capacity change from 0 to 500118323200You can see how it uses sdd and sde as whole disks instead of sdd1 and sde1.
As a secondary result of that the swap partitions on these disks do not work either.
I dont care if it uses md0 or md127, however i do care that it does not use my partition tables as part of the filesystem!!!
Help is much appreciated, i already spent lots of hours on this, thank you.
EDIT: More information:
# mdadm --examine /dev/sdd1
/dev/sdd1:
Magic : a92b4efc
Version : 0.90.00
UUID : 00000000:00000000:00000000:00000000
Creation Time : Thu May 31 19:47:34 2012
Raid Level : -unknown-
Raid Devices : 0
Total Devices : 1
Preferred Minor : 0
Update Time : Thu May 31 19:48:04 2012
State : active
Active Devices : 0
Working Devices : 1
Failed Devices : 0
Spare Devices : 1
Checksum : 56f8d639 - correct
Events : 1
Number Major Minor RaidDevice State
this 0 8 33 0 spare /dev/sdc1
0 0 8 33 0 spare /dev/sdc1# mdadm --examine /dev/sde1
/dev/sde1:
Magic : a92b4efc
Version : 0.90.00
UUID : 00000000:00000000:00000000:00000000
Creation Time : Fri Jun 1 01:47:22 2012
Raid Level : -unknown-
Raid Devices : 0
Total Devices : 1
Preferred Minor : 0
Update Time : Fri Jun 1 02:07:46 2012
State : active
Active Devices : 0
Working Devices : 1
Failed Devices : 0
Spare Devices : 1
Checksum : 56f983ab - correct
Events : 1
Number Major Minor RaidDevice State
this 0 8 49 0 spare /dev/sdd1
0 0 8 49 0 spare /dev/sdd1# mdadm --examine /dev/sdd
/dev/sdd:
Magic : a92b4efc
Version : 0.90.00
UUID : 1f8177a5:666fc527:7ad9fd8c:70e05756
Creation Time : Fri Aug 3 18:33:04 2007
Raid Level : raid0
Raid Devices : 2
Total Devices : 2
Preferred Minor : 0
Update Time : Fri Aug 3 18:33:04 2007
State : active
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Checksum : a83f932b - correct
Events : 1
Chunk Size : 128K
Number Major Minor RaidDevice State
this 0 8 32 0 active sync /dev/sdc
0 0 8 32 0 active sync /dev/sdc
1 1 8 48 1 active sync /dev/sdd# mdadm --examine /dev/sde
/dev/sde:
Magic : a92b4efc
Version : 0.90.00
UUID : 1f8177a5:666fc527:7ad9fd8c:70e05756
Creation Time : Fri Aug 3 18:33:04 2007
Raid Level : raid0
Raid Devices : 2
Total Devices : 2
Preferred Minor : 0
Update Time : Fri Aug 3 18:33:04 2007
State : active
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Checksum : a83f933d - correct
Events : 1
Chunk Size : 128K
Number Major Minor RaidDevice State
this 1 8 48 1 active sync /dev/sdd
0 0 8 32 0 active sync /dev/sdc
1 1 8 48 1 active sync /dev/sddSomething is not right here... sdd used to be sdc and sde used to be sdd. But here mdadm shows the old device names...
Last edited by madMAx (2012-06-02 10:35:34)
Offline
Perhaps my post about grub2 and raid0 bootable will give you some help.
I am not clear as to whether you boot on the raid or establish a raid0 storage.
The /dev/md127 is prevalent with latest kernels but it can be stopped with ...mdadm --stop /dev/md127...
You can follow that with ....(for example)
mdadm --assemble /dev/md0 /dev/sda1 /dev/sdb1This may give you more insight into your difficulty.
Dunno if you are grub or grub2.............
Prediction...This year will be a very odd year!
Hard work does not kill people but why risk it: Charlie Mccarthy
A man is not complete until he is married..then..he is finished.
When ALL is lost, what can be found? Even bytes get lonely for a little bit! X-ray confirms Iam spineless!
Offline
Thanks for your reply.
No, i do not boot from it, it's just for storage.
I already tried assembling it by hand, but it doesnt do anything, nor does it show any error:
[root@mg1 ~]# mdadm -A -v /dev/md0 /dev/sdc1 /dev/sdd1
mdadm: looking for devices for /dev/md0
[root@mg1 ~]# However there are some kernel messages generated when issuing the above command:
[ 1542.603404] scsi_verify_blk_ioctl: 12 callbacks suppressed
[ 1542.603407] mdadm: sending ioctl 800c0910 to a partition!
[ 1542.603409] mdadm: sending ioctl 800c0910 to a partition!
[ 1542.603417] mdadm: sending ioctl 1261 to a partition!
[ 1542.603419] mdadm: sending ioctl 1261 to a partition!
[ 1542.626072] mdadm: sending ioctl 1261 to a partition!
[ 1542.626074] mdadm: sending ioctl 1261 to a partition!
[ 1542.627612] mdadm: sending ioctl 1261 to a partition!
[ 1542.627615] mdadm: sending ioctl 1261 to a partition!
[ 1542.643584] mdadm: sending ioctl 1261 to a partition!
[ 1542.643587] mdadm: sending ioctl 1261 to a partition!"12 callbacks suppressed" -> maybe thats why mdadm doesnt show any activity?
EDIT: When i use whole disks it works:
[root@mg1 ~]# mdadm -A -v /dev/md0 /dev/sdc /dev/sdd
mdadm: looking for devices for /dev/md0
mdadm: /dev/sdc is identified as a member of /dev/md0, slot 0.
mdadm: /dev/sdd is identified as a member of /dev/md0, slot 1.
mdadm: added /dev/sdd to /dev/md0 as 1
mdadm: added /dev/sdc to /dev/md0 as 0
mdadm: /dev/md0 has been started with 2 drives.But this is wrong, there are partitions on the drives:
[root@mg1 ~]# fdisk -l /dev/sdc
Device Boot Start End Blocks Id System
/dev/sdc1 * 63 479909744 239954841 fd Linux raid autodetect
/dev/sdc2 479909745 488392064 4241160 82 Linux swap / Solaris
[root@mg1 ~]# fdisk -l /dev/sdd
Device Boot Start End Blocks Id System
/dev/sdd1 * 63 479909744 239954841 fd Linux raid autodetect
/dev/sdd2 479909745 488392064 4241160 82 Linux swap / Solaris
[root@mg1 ~]# And when i try to mount it, it does not work obviously:
[root@mg1 ~]# mount /dev/md0 /mnt/kk/
mount: wrong fs type, bad option, bad superblock on /dev/md0,
missing codepage or helper program, or other error
In some cases useful info is found in syslog - try
dmesg | tail or so[ 314.344972] EXT4-fs (md0): mounting ext3 file system using the ext4 subsystem
[ 314.348623] EXT4-fs (md0): ext4_check_descriptors: Block bitmap for group 880 not in group (block 0)!
[ 314.348626] EXT4-fs (md0): group descriptors corrupted!How can it use the whole disk as part of a raid0 when there are obviously multiple partitions on the drives??
Last edited by madMAx (2012-06-01 21:57:56)
Offline
Interesting things are happening!
My sojurn into raid0 is the outcome from the referenced blog in my opening post.
It allows booting from a partition in grub2 which is according to the devs verboten.
Perhaps your case can be solved by utilizing a chroot and performing the steps to install grub2 as well as defining the raid array with mdadm. My case is different than yours, however because I was making a new install and not running an established system.
If you are using grub2, it would seem that the chroot could still be applied. If you do try it, make sure all other arrays are stopped and your raid0 is assembled.
As to the partition statements, my dmesg makes the same reference to partition but it is different because I am booting from it.
Cat /proc/mdstat will also provide array data and may indeed aid in deciphering the problem.
Cfdisk always has raid array notation for partitions that are in an array.
My system uses uuid for all devices in the raid system. I have five such raid0 pairs all running CF cards and all upgrade just fine. I have different packages on each array pair and the devices are of three types, 266x, 400x and supposedly 600x. The 600x devices don't measure up. They produce 141mb/s whereas the 400x make 180mb's. Differences in the algorithms and the udma assigned I assume.
Wish I had more to aid you but raid has always been a bug-a-boo. I wish arch had a raid forum for all the raid problem discussions.
Your raid running in full device mode is strange and refusing to assemble them as partitions is a newone.
I wonder at this juncture whether the raid was initially set up as full devices even though you state otherwise. Recent changes in archlinux frown on partitions for booting but probably not for raid devices.
What does cfdisk have to say about the raid?
Last edited by lilsirecho (2012-06-02 05:11:59)
Prediction...This year will be a very odd year!
Hard work does not kill people but why risk it: Charlie Mccarthy
A man is not complete until he is married..then..he is finished.
When ALL is lost, what can be found? Even bytes get lonely for a little bit! X-ray confirms Iam spineless!
Offline
After studying the large amount of data, I observe that the listing of spares indicates that the designations for the devices are not part of an array. Using sd nomenclature does pose ID problems.
Make a check on mdadm.conf and cat/proc/mdstat for the status of md0.
Md0 will be supplanted by /dev/md127 or some such every time you boot in the latest kernels. It happens in my system and I have to use the stop command to address my desired /dev/md0.
You may have to clear out old array data in mdadm.conf.
Mostly, it is likely to be the ID problem when using sd.
Last edited by lilsirecho (2012-06-02 05:56:52)
Prediction...This year will be a very odd year!
Hard work does not kill people but why risk it: Charlie Mccarthy
A man is not complete until he is married..then..he is finished.
When ALL is lost, what can be found? Even bytes get lonely for a little bit! X-ray confirms Iam spineless!
Offline
Ok i got it back now.
I had to recreate the array with the matching chunk size:
mdadm --create /dev/md0 -v -f -e 0 -c 128 -l 0 -n 2 /dev/sdc1 /dev/sdd1Fsck cleaned up the remaing mess.
It showed that every free block, free inode and directory count for every group was wrong:
...
Directories count wrong for group #3651 (0, counted=195).
...
Free inodes count wrong for group #3652 (16384, counted=16370).
....I have no idea what had happend here. My raid superblocks somehow got overwritten with a correct timestamp but invalid data otherwise. And those filesystem inconsistencies...
Anyhow i have my data back.
Last edited by madMAx (2012-06-02 15:30:40)
Offline
Well done and best of luck with your system.
Prediction...This year will be a very odd year!
Hard work does not kill people but why risk it: Charlie Mccarthy
A man is not complete until he is married..then..he is finished.
When ALL is lost, what can be found? Even bytes get lonely for a little bit! X-ray confirms Iam spineless!
Offline