[solved] mdadm raid5: Second drive crashed while recovering.

clvn · 2011-02-07 16:15:32

Hi

I have a server with 6 sata-drives in a raid5 configuration. One of the drives dropped from the array a few weeks ago, rendering it working, but without any security (e.g. U_UUUU). I rebooted the server, added the drive that dropped from the raid, and the rebuilding went fint. Until it reached around 60-80% recovery. At thgat point, another drive dropped from the array, rendering the whole array useless (U_UUU_)

Luckily, i know exactly which drives that contained the correct datasets before i started to rebuild the raid.

Some commands i ran before attempting to rebuild the array:

# clvn@enigma: cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sdh1[2] sdc1[5] sdd1[3] sdg1[6](F) sdb1[0] sde1[4]
7325679680 blocks level 5, 64k chunk, algorithm 2 [6/5] [U_UUUU]

unused devices: <none>

# clvn@enigma: for i in {c,d,h,b,e}; do echo /dev/sd"$i"1:; sudo smartctl -a /dev/sd"$i"1 | grep Serial; done
/dev/sdc1:
Serial Number: WD-WCAVY2742388
/dev/sdd1:
Serial Number: 6XW059BH
/dev/sdh1:
Serial Number: WD-WMAVU1395536
/dev/sdb1:
Serial Number: S1XWJ9CZ805937
/dev/sde1:
Serial Number: WD-WMAVU1398710

Anyone have any suggestions? I haven't tried anything yet, except a reboot to see if that would help. And some googleing

Last edited by clvn (2011-02-10 16:29:58)

briest · 2011-02-08 21:05:06

Exactly my story; last year a disk failed in rebuilding raid5 array. Unfortunately, we've lost the data (modulo backups); data recovery company worked on the array for a month and presented us with incomplete file listing and price tag so high, that company did not decide to buy full recovery. One advice: if you want to do anything by yourself, make images of all drives, then put drives in safe, quiet place and work on copy of the images.

clvn · 2011-02-08 21:42:21

Thanks for the info briest, even though its not promising. I'll keep this thread updated.

clvn · 2011-02-10 01:11:58

This is great. I forgot how much i love mdadm.

sudo mdadm --assemble /dev/md0 --force /dev/sdb1 /dev/sde1 /dev/sdg1 /dev/sdc1 /dev/sdd1 (the /dev/sd*-devices are the five out of six drives that i knew contained the whole dataset)

*mdadm's magic powers goes here, schwop*

And everything seems fine.

# clvn@enigma: cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 sdb1[0] sdc1[5] sde1[4] sdd1[3] sdg1[2]
7325679680 blocks level 5, 64k chunk, algorithm 2 [6/5] [U_UUUU]

unused devices: <none>

Arch Linux

#1 2011-02-07 16:15:32

[solved] mdadm raid5: Second drive crashed while recovering.

#2 2011-02-08 21:05:06

Re: [solved] mdadm raid5: Second drive crashed while recovering.

#3 2011-02-08 21:42:21

Re: [solved] mdadm raid5: Second drive crashed while recovering.

#4 2011-02-10 01:11:58

Re: [solved] mdadm raid5: Second drive crashed while recovering.

Board footer