You are not logged in.
Help, my raid5 array is acting up.
I added a new drive, following the arch wiki and everything seemed to have worked fine.
Old layout was 3 drives, new one 4.
The basic steps were:
- copied the partition table
- added the drive (mdadm --add)
- grew the array from 3 to 4 drives (mdadm --grow -n 4 /dev/md0)
After that was finished (20hrs or so) the array appeared ot be working fine and I resized my physical volume (pvresize /dev/md0)
mdadm --examine --detail showed the right size, etc. Same for pvdisplay. I haven't increased the size of any lv's.
BUT
it did not survive a reboot. System went into recovery console without the array being assembled.
sudo mdadm --misc --detail /dev/md0
mdadm: md device /dev/md0 does not appear to be active.sudo mdadm --examine --scan
ARRAY /dev/md/0 metadata=1.2 UUID=c824d8f6:30d0f7a6:90ae6950:d7cc6574 name=(none):0
ARRAY /dev/md/0 metadata=1.2 UUID=c824d8f6:30d0f7a6:90ae6950:d7cc6574 name=(none):0
spares=3sudo mdadm --assemble /dev/md0
mdadm: superblock on /dev/sde1 doesn't match others - assembly abortedsde1 is the new drive.
sudo mdadm --assemble /dev/md0 /dev/sdb1 /dev/sdc1 /dev/sdd1
mdadm: /dev/md0 assembled from 0 drives and 3 spares - not enough to start the array.sudo mdadm --examine /dev/sdb1
/dev/sdb1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : c824d8f6:30d0f7a6:90ae6950:d7cc6574
Name : (none):0
Creation Time : Thu Feb 16 16:02:49 2012
Raid Level : -unknown-
Raid Devices : 0
Avail Dev Size : 3906820096 (1862.92 GiB 2000.29 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : active
Device UUID : a06ccdf9:6b0f014c:527d3ce8:5d0ffb11
Update Time : Fri Sep 14 09:05:30 2012
Checksum : 15766d05 - correct
Events : 1
Device Role : spare
Array State : ('A' == active, '.' == missing)This is essentially the same for all the "old" drives (sdb1-sdd1), but the newly added one (sde1) shows this:
sudo mdadm --examine /dev/sde1
/dev/sde1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : c824d8f6:30d0f7a6:90ae6950:d7cc6574
Name : (none):0
Creation Time : Thu Feb 16 16:02:49 2012
Raid Level : raid5
Raid Devices : 4
Avail Dev Size : 3906820096 (1862.92 GiB 2000.29 GB)
Array Size : 11720457216 (5588.75 GiB 6000.87 GB)
Used Dev Size : 3906819072 (1862.92 GiB 2000.29 GB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
State : clean
Device UUID : ad036aac:accd167c:e24283e0:b81e9dfa
Update Time : Thu Sep 13 21:56:16 2012
Checksum : d924f928 - correct
Events : 113780
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 3
Array State : AAAA ('A' == active, '.' == missing)Any help will be much appreciated !
Offline
OK, I think I am up and running again without any data loss.
With the help of this post, the thread it is referencing and my newborn daughter on my chest for good luck I did the following:
sudo mdadm --zero-superblock /dev/sdd1 /dev/sdb1 /dev/sdc1 /dev/sde1I then recreated the array:
sudo mdadm --create -v -l 5 -n 4 --assume-clean /dev/md0 /dev/sdd1 /dev/sdb1 /dev/sdc1 /dev/sde1
mdadm: layout defaults to left-symmetric
mdadm: layout defaults to left-symmetric
mdadm: chunk size defaults to 512K
mdadm: layout defaults to left-symmetric
mdadm: /dev/sdb1 appears to contain an ext2fs file system
size=209715200K mtime=Thu Jan 1 09:30:00 1970
mdadm: layout defaults to left-symmetric
mdadm: layout defaults to left-symmetric
mdadm: size set to 1953409536K
Continue creating array? y
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.That seemed to have gone well, if anything a bit to quickly, I was expecting some lengthy resyng or who nows what operation.
Quick status check:
sudo mdadm --detail --misc /dev/md0
/dev/md0:
Version : 1.2
Creation Time : Fri Sep 14 14:24:13 2012
Raid Level : raid5
Array Size : 5860228608 (5588.75 GiB 6000.87 GB)
Used Dev Size : 1953409536 (1862.92 GiB 2000.29 GB)
Raid Devices : 4
Total Devices : 4
Persistence : Superblock is persistent
Update Time : Fri Sep 14 14:24:13 2012
State : clean
Active Devices : 4
Working Devices : 4
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 512K
Name : mythbackend:0 (local to host mythbackend)
UUID : e815408b:2cf42209:67878b17:ff209ec5
Events : 0
Number Major Minor RaidDevice State
0 8 49 0 active sync /dev/sdd1
1 8 17 1 active sync /dev/sdb1
2 8 33 2 active sync /dev/sdc1
3 8 65 3 active sync /dev/sde1Before a reboot I made sure my mdadm.conf was up to date:
sudo mdadm --examine --scan > /etc/mdadm.conf
-bash: /etc/mdadm.conf: Permission deniedWeird. I created a copy and updated it manually. Only changed were a new UUID for /dev/md0 and I added /dev/sde1 to the DEVICE line.
Then the reboot...and it seems to work ok. lvm stuff is there and I can access my data.
Some questions:
1. Why did this happen ?
I realised that after adding the drive etc. I did not update the mdadm.conf file, specifically I did not add /dev/sde1 to the DEVICE line. Did this screw things up during the next boot ? I remember somebody mentioning that mdadm does not rely on mdadm.conf for assembling the array, but I do seem to recall that I had problems with an outdated mdadm.conf file before
2. After this recovery, what can I do to check the array ? Should I do a resync ? If so, how ? Or is there another check ?
Offline