Arch Linux

daemoon

Member

Registered: 2013-02-21

Posts: 7

[SOLVED] mdadm RAID1 with members' UUID being different

I got two identical hard drives composed to a RAID1 on which I created a LUKS container. The RAID1 seems to work but the two member disks /dev/sdb and /dev/sdc don't have the same UUID as you would expect.

# cat /proc/mdstat
Personalities : [raid1] 
md0 : active raid1 sdc[2] sdb[1]
      2930135360 blocks super 1.2 [2/2] [UU]
      
unused devices: <none>

# blkid
/dev/sdb: UUID="acb3b59a-ef33-46a2-a801-6c679f9c8abc" TYPE="crypto_LUKS" 
/dev/sdc: UUID="b69b5e02-7535-1132-e4f1-a2e835d29862" UUID_SUB="37ffb639-febd-9e96-2f21-ca715643a924" LABEL="server-cm:0" TYPE="linux_raid_member" 
/dev/md0: UUID="63ec4364-7e8e-4e16-ba43-d7a63a09af44" TYPE="crypto_LUKS" 

# mdadm --detail /dev/md0
/dev/md0:
        Version : 1.2
  Creation Time : Sat Nov 16 15:10:07 2013
     Raid Level : raid1
     Array Size : 2930135360 (2794.39 GiB 3000.46 GB)
  Used Dev Size : 2930135360 (2794.39 GiB 3000.46 GB)
   Raid Devices : 2
  Total Devices : 2
    Persistence : Superblock is persistent

    Update Time : Sun Nov 17 21:25:36 2013
          State : clean 
 Active Devices : 2
Working Devices : 2
 Failed Devices : 0
  Spare Devices : 0

           Name : server-cm:0  (local to host server-cm)
           UUID : b69b5e02:75351132:e4f1a2e8:35d29862
         Events : 36788

    Number   Major   Minor   RaidDevice State
       2       8       32        0      active sync   /dev/sdc
       1       8       16        1      active sync   /dev/sdb

Since I had to migrate from a single disk (/dev/sdb being the first disk) I first created the RAID using only one disk (/dev/sdc) and afterwards adding /dev/sdb. It took a few hours to sync but now they both have the same data, which I tested by simulating a disk failure by switching off the computer and unplugging /dev/sdc. Afterwards, having plugged in the disk again, the whole disk had to be synced again, although there was no write access; the RAID hasn't even been mounted. I think that the different UUID are caused from first creating the array (and with it the superblock) on /dev/sdc only and that this might be the reason for the groundless sync described earlier.

Do I have to copy the superblock somehow from /dev/sdc to /dev/sdb? What else could be the reason?

Last edited by daemoon (2013-11-18 03:52:32)

frostschutz

Member

Registered: 2013-11-15

Posts: 1,487

Re: [SOLVED] mdadm RAID1 with members' UUID being different

The decision whether to sync or not is usually made based on the mdadm metadata alone. It does not start actually comparing data on drives unless you tell it to (sync action check repair).

Disks shouldn't be missing from a RAID ever; if this happens for you for some reason, maybe consider using a write-intent bitmap. It costs performance (lowering the resolution helps, i.e. larger regions per bit) but prevents full resyncs if they aren't necessary.

As for your UUID issue I think you are looking at the wrong thing altogether. mdadm 1.2 metadata starts 4k into the disk, what you're seeing there is probably old data that lives in those first 4k that are left alone by mdadm.

# cryptsetup luksFormat /dev/loop0
# blkid /dev/loop0
/dev/loop0: UUID="d8f4c8bc-f992-4c68-ae40-5faaf3a474d5" TYPE="crypto_LUKS" 
# mdadm /dev/md99 --create --level=1 --raid-devices=2 /dev/loop0 missing
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md99 started.
# mdadm --examine /dev/loop0
/dev/loop0:
          Magic : a92b4efc
        Version : 1.2
[...]
# file -s /dev/loop0
/dev/loop0: LUKS encrypted file, ver 1 [aes, xts-plain64, sha1] UUID: d8f4c8bc-f992-4c68-ae40-5faaf3a474d5

So even though it was made into a RAID by mdadm, the old LUKS header (parts of it, not servicable anymore) is still there including the old UUID.

You could zero the first 4k with dd (if=/dev/zero bs=4k count=1 of=sdx_the_raidmember) which should be safe for 1.2 metadata. better safe than sorry, so make sure the raid is in sync, and stop before / start after and see if everything works, otherwise readd/resync the member.

Last edited by frostschutz (2013-11-17 23:41:36)

daemoon

Member

Registered: 2013-02-21

Posts: 7

Re: [SOLVED] mdadm RAID1 with members' UUID being different

Having overwritten the first 4k on /dev/sdb, the two disks now have the same UUID which makes it clear that they store the same data. Thank you for your advice!