Arch Linux

msoltyspl

Member

From: Poland

Registered: 2007-07-09

Posts: 11

mdadm / udev conflicts

[I wasn't really sure if it should go into workstation section or this one, but it seemed more appropriate here.]

Scenario:

partitionable raid-5 array created with mdadm, i.e.
mdadm -C /dev/md/d0 -l5 -n4 -x1 -e1 /dev/sd{a,b,c,d,e}4 -z8388608 --auto=p6

On the array we create 2 partitions - nevermind the size/id/etc., just be sure they're created. Then we update mdadm.conf, automatically ot not, to:

ARRAY /dev/md/d0 UUID=<valid uid>
CREATE auto=p6 mode=0660 group=disk owner=root

Next, to see what is happening on udev side, let's start udevd manually, with debug options (--debug-trace --verbose).

Under standard arch's udev rules, and fully updated arch64 (inc. mdadm 2.6.2-1), following happens, starting
with clean, unassembled array:

1) mdadm -A /dev/md/d0

no rules in arch for md_d* case at all - so only standard /dev/md_d0 is created, whereas mdadm tries to create md/d0 and symlink it as /dev/md_d0.

No big deal yet, but...

2) mdadm -S /dev/md/d0

For some reason, "add" (!) uevent for every existing partition is generated; looking at udevd trace, /dev/md_d0p1 and /dev/md_d0p2 are created, replacing already existing symlinks created by mdadm in step 1)

3) mdadm -A /dev/md/d0

Here - "remove" (!) uevent for every created partition; causing in most cases removal of symlinks created by mdadm. Next - proper "change" for md_d0.

If we called rm -r /dev/md* between 2) and 3), the devices referencing existing partitions after 3) would have inappropriate mode and group (and no symlinks to the either). Probably a reason of conflict with udev (?).

If we repeated 2) after 2) - "remove" would happen in the 2nd one.

4) ... using other comamnds, there were similar nodes/links issues, coming from what udev and mdadm think

I don't know what exactly is at fault here - looks like mdadm, but....
I have spent helathy amount of time today wondering "wth is happening to those device nodes", while testing raid in different setups / situations - thus plenty of -S / -A / --fail / etc. commands. So heads up

In my case, ignore_device option for both md and md_d solved every problem I've had (also, I don't need any by-uuid / by-label links). And of course - there're no rules for md_d by default either. Overally - as mdadm manages nodes by itself - wouldn't it be better default for the rules ?

On the related subject - as I'll modify a bit my setup, I'm planning on changing raid / raid-paritions hooks a bit - merge them, rely only on mdadm which is already included in the latter case, add a bit different kernel options, so array can be assembled by i.e. name. etc.

udev trace, for reference:

1)

[8222] udev_event_run: seq 2176 forked, pid [8284], 'change' 'block', 0 seconds old
[8284] udev_rules_get_name: no node name set, will use kernel name 'md_d0'
[8284] udev_db_get_device: found a symlink as db file
[8284] udev_device_event: device '/block/md_d0' already in database, cleanup
[8284] name_index: removing index: '/dev/.udev/names/md_d0/\x2fblock\x2fmd_d0'
[8284] udev_node_add: creating device node '/dev/md_d0', major=254, minor=0, mode=0660, uid=0, gid=6
[8284] udev_node_mknod: atomically replace '/dev/md_d0'
[8284] name_index: creating index: '/dev/.udev/names/md_d0/\x2fblock\x2fmd_d0'
[8284] run_program: '/lib/udev/root-link.sh'
[8284] run_program: '/lib/udev/root-link.sh' returned with status 0
[8284] pass_env_to_socket: passed -1 bytes to socket '/org/kernel/udev/monitor',
[8284] udev_event_run: seq 2176 finished

2)

[8222] udev_event_run: seq 2177 forked, pid [8288], 'add' 'block', 0 seconds old
[8288] udev_rules_get_name: no node name set, will use kernel name 'md_d0p1'
[8288] udev_db_get_device: no db file to read /dev/.udev/db/\x2fblock\x2fmd_d0\x2fmd_d0p1: No such file or directory
[8288] udev_node_add: creating device node '/dev/md_d0p1', major=254, minor=1, mode=0660, uid=0, gid=6
[8288] udev_node_mknod: atomically replace '/dev/md_d0p1'
[8288] name_index: creating index: '/dev/.udev/names/md_d0p1/\x2fblock\x2fmd_d0\x2fmd_d0p1'
[8288] run_program: '/lib/udev/root-link.sh'
[8288] run_program: '/lib/udev/root-link.sh' returned with status 0
[8288] pass_env_to_socket: passed -1 bytes to socket '/org/kernel/udev/monitor',
[8288] udev_event_run: seq 2177 finished
[8222] udev_event_run: seq 2178 forked, pid [8290], 'add' 'block', 0 seconds old
[8290] udev_rules_get_name: no node name set, will use kernel name 'md_d0p2'
[8290] udev_db_get_device: no db file to read /dev/.udev/db/\x2fblock\x2fmd_d0\x2fmd_d0p2: No such file or directory
[8290] udev_node_add: creating device node '/dev/md_d0p2', major=254, minor=2, mode=0660, uid=0, gid=6
[8290] udev_node_mknod: atomically replace '/dev/md_d0p2'
[8290] name_index: creating index: '/dev/.udev/names/md_d0p2/\x2fblock\x2fmd_d0\x2fmd_d0p2'
[8290] run_program: '/lib/udev/root-link.sh'
[8290] run_program: '/lib/udev/root-link.sh' returned with status 0
[8290] pass_env_to_socket: passed -1 bytes to socket '/org/kernel/udev/monitor',
[8290] udev_event_run: seq 2178 finished

3)

[8222] udev_event_run: seq 2179 forked, pid [8294], 'remove' 'block', 0 seconds old
[8294] udev_db_get_device: found a symlink as db file
[8294] name_index: removing index: '/dev/.udev/names/md_d0p1/\x2fblock\x2fmd_d0\x2fmd_d0p1'
[8294] udev_node_remove: removing device node '/dev/md_d0p1'
[8294] run_program: '/lib/udev/root-link.sh'
[8294] run_program: '/lib/udev/root-link.sh' returned with status 0
[8294] pass_env_to_socket: passed -1 bytes to socket '/org/kernel/udev/monitor',
[8294] udev_event_run: seq 2179 finished
[8222] udev_event_run: seq 2180 forked, pid [8296], 'remove' 'block', 0 seconds old
[8296] udev_db_get_device: found a symlink as db file
[8296] name_index: removing index: '/dev/.udev/names/md_d0p2/\x2fblock\x2fmd_d0\x2fmd_d0p2'
[8296] udev_node_remove: removing device node '/dev/md_d0p2'
[8296] run_program: '/lib/udev/root-link.sh'
[8296] run_program: '/lib/udev/root-link.sh' returned with status 0
[8296] pass_env_to_socket: passed -1 bytes to socket '/org/kernel/udev/monitor',
[8296] udev_event_run: seq 2180 finished
[8222] udev_event_run: seq 2181 forked, pid [8301], 'change' 'block', 0 seconds old
[8301] udev_rules_get_name: no node name set, will use kernel name 'md_d0'
[8301] udev_db_get_device: found a symlink as db file
[8301] udev_device_event: device '/block/md_d0' already in database, cleanup
[8301] name_index: removing index: '/dev/.udev/names/md_d0/\x2fblock\x2fmd_d0'
[8301] udev_node_add: creating device node '/dev/md_d0', major=254, minor=0, mode=0660, uid=0, gid=6
[8301] udev_node_mknod: preserve file '/dev/md_d0', because it has correct dev_t
[8301] name_index: creating index: '/dev/.udev/names/md_d0/\x2fblock\x2fmd_d0'
[8301] run_program: '/lib/udev/root-link.sh'
[8301] run_program: '/lib/udev/root-link.sh' returned with status 0
[8301] pass_env_to_socket: passed -1 bytes to socket '/org/kernel/udev/monitor',
[8301] udev_event_run: seq 2181 finished

msoltyspl

Member

From: Poland

Registered: 2007-07-09

Posts: 11

Re: mdadm / udev conflicts

Updated with some fixes and reported: http://bugs.archlinux.org/task/7699

FYI