Bug#594418: new mdadm version causes high I/O during boot, boot fails, perhaps issue with udev rules

[Alexander Kurtz]

Package: mdadm
Version: 3.1.2-2.1
Severity: serious
Justification: renders machine unbootable

Hi,

=== Background ===

My setup looks like this:

	$ cat /proc/mdstat 
	Personalities : [raid0] [raid1] [raid6] [raid5] [raid4] 
	md0 : active raid6 sdc1[0] md2[3] sde1[2] sdd1[1]
	      1953134208 blocks level 6, 64k chunk, algorithm 2 [4/4] [UUUU]
	      
	md1 : active raid1 sdc2[0] sda2[4] sdb2[3] sde2[2] sdd2[1]
	      56128 blocks [5/5] [UUUUU]
	      
	md2 : active raid0 sda1[0] sdb1[1]
	      976574976 blocks 64k chunks
	      
	unused devices: <none>

/dev/md0 contains a LUKS-volume which contains the root fs (btrfs).
/dev/md1 contains /boot (ext4)

=== Problem ===

Since todays mdadm upgrade, my machine fails to boot:

a) After passing GRUB2 the arrays are correctly assembled by the 
   initramfs, but nothing happens for about 30 seconds execpt the HDD 
   LED being on all the time (see `screen1.jpg').

b) After that, a lot of (error?) messages fill the screen and I can 
   finally enter my password. However, after that nothing happens 
   anymore (see `screen 2.jpg').

c) For diagnosing I then restarted and booted the kernel with 
   `init=/bin/sh' being the only option. After waiting some time again 
   I was asked to enter my password just like before. I hit `Enter' 
   three times and after another 30 seconds (the HDD LED was on all the 
   time) and some other error messages I was thrown into a rescue shell 
   (see `screen3.jpg'):

d) Being alerted by those "error messages" I checked wether udev was 
   running: It wasn't. As soon as I restarted it, the HDD LED went on 
   again and the system became unusable (see `screen4.jpg').

e) I then repeated step c) but this time I stopped all MD arrays
   (using `mdadm --stop') before restarting udev. Now everything went 
   fine. But as soon as I restarted the MD arrays again (using
   `/scripts/local-top/mdadm') the system load went up and the HDD LED
   went on again just like before (see `screen{5,6}.jpg').

f) However this time I managed to fire up `top'. See `screen7.jpg' for 
   the result.

After that I used a Live USB to chroot into the system and commented out
everything in `/lib/udev/rules.d/64-md-raid.rules' (see
`64-md.rules.edited'). After rebuilding the initramfs the system booted
fine again, except that (of course) the symlinks under /dev/disk/* were
missing for the MD arrays.

I don't know exactly what happened, but something in that file seems
broken. I am happy if I can help debugging this, just tell me what to
do!

Best regards

Alexander Kurtz 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: screen1.jpg
Type: image/jpeg
Size: 89689 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/pkg-mdadm-devel/attachments/20100825/0f1ba6f7/attachment-0007.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: screen2.jpg
Type: image/jpeg
Size: 167289 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/pkg-mdadm-devel/attachments/20100825/0f1ba6f7/attachment-0008.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: screen3.jpg
Type: image/jpeg
Size: 168588 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/pkg-mdadm-devel/attachments/20100825/0f1ba6f7/attachment-0009.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: screen4.jpg
Type: image/jpeg
Size: 174332 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/pkg-mdadm-devel/attachments/20100825/0f1ba6f7/attachment-0010.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: screen5.jpg
Type: image/jpeg
Size: 171080 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/pkg-mdadm-devel/attachments/20100825/0f1ba6f7/attachment-0011.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: screen6.jpg
Type: image/jpeg
Size: 190135 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/pkg-mdadm-devel/attachments/20100825/0f1ba6f7/attachment-0012.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: screen7.jpg
Type: image/jpeg
Size: 175021 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/pkg-mdadm-devel/attachments/20100825/0f1ba6f7/attachment-0013.jpg>
-------------- next part --------------
# do not edit this file, it will be overwritten on update

###SUBSYSTEM!="block", GOTO="md_end"
###ACTION!="add|change", GOTO="md_end"
###ACTION=="change", GOTO="md_no_incr"

# import data from a raid member and activate it
#ENV{ID_FS_TYPE}=="linux_raid_member", IMPORT{program}="/sbin/mdadm --examine --export $tempnode", RUN+="/sbin/mdadm --incremental $env{DEVNAME}"
# import data from a raid set
###LABEL="md_no_incr"
###KERNEL!="md*", GOTO="md_end"

# partitions have no md/{array_state,metadata_version}, but should not
# for that reason be ignored.
###ENV{DEVTYPE}=="partition", GOTO="md_ignore_state"

# container devices have a metadata version of e.g. 'external:ddf' and
# never leave state 'inactive'
###ATTR{md/metadata_version}=="external:[A-Za-z]*", ATTR{md/array_state}=="inactive", GOTO="md_ignore_state"
###TEST!="md/array_state", GOTO="md_end"
###ATTR{md/array_state}=="|clear|inactive", GOTO="md_end"
###LABEL="md_ignore_state"

###IMPORT{program}="/sbin/mdadm --detail --export $tempnode"
###ENV{DEVTYPE}=="disk", ENV{MD_NAME}=="?*", SYMLINK+="disk/by-id/md-name-$env{MD_NAME}", OPTIONS+="string_escape=replace"
###ENV{DEVTYPE}=="disk", ENV{MD_UUID}=="?*", SYMLINK+="disk/by-id/md-uuid-$env{MD_UUID}"
###ENV{DEVTYPE}=="disk", ENV{MD_DEVNAME}=="?*", SYMLINK+="md/$env{MD_DEVNAME}"
###ENV{DEVTYPE}=="partition", ENV{MD_NAME}=="?*", SYMLINK+="disk/by-id/md-name-$env{MD_NAME}-part%n", OPTIONS+="string_escape=replace"
###ENV{DEVTYPE}=="partition", ENV{MD_UUID}=="?*", SYMLINK+="disk/by-id/md-uuid-$env{MD_UUID}-part%n"
###ENV{DEVTYPE}=="partition", ENV{MD_DEVNAME}=="*[^0-9]", SYMLINK+="md/$env{MD_DEVNAME}%n"
###ENV{DEVTYPE}=="partition", ENV{MD_DEVNAME}=="*[0-9]", SYMLINK+="md/$env{MD_DEVNAME}p%n"

###IMPORT{program}="/sbin/blkid -o udev -p $tempnode"
###OPTIONS+="link_priority=100"
###ENV{ID_FS_USAGE}=="filesystem|other|crypto", ENV{ID_FS_UUID_ENC}=="?*", SYMLINK+="disk/by-uuid/$env{ID_FS_UUID_ENC}"
###ENV{ID_FS_USAGE}=="filesystem|other", ENV{ID_FS_LABEL_ENC}=="?*", SYMLINK+="disk/by-label/$env{ID_FS_LABEL_ENC}"

###LABEL="md_end"
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: This is a digitally signed message part
URL: <http://lists.alioth.debian.org/pipermail/pkg-mdadm-devel/attachments/20100825/0f1ba6f7/attachment-0001.pgp>