Bug#534470: Using mdadm 2.6.7.2 to assemble a raid array created with mdadm 1.9.0 will corrupt it making mdadm 1.9.0 to crash when trying to reassemble

[RUSSOTTO François-Xavier 200103]

Package: mdadm
Version: 2.6.7.2
Severity: critical

After booting a Debian 5.0.1 - Lenny install cdrom in rescue mode (debian-501-ia64-netinst.iso) on an Itanium 2 server with Debian 3.1 - Sarge (ia64) installed on a software raid 5 root partition, opening a console in the root partition mounted from the raid array (auto-assembled) corrupts the raid array, leading to a kernel panic at server reboot, and preventing from manual reassembly using mdadm 1.9.0 (Sarge).

The raid 5 array containing the root partition is made of 3 partitions on 3 scsi disks (sda2, sdb2, sdc2) which ran fluently for years. Here is the output at server reboot:

	md: invalid superblock checksum on sdb2
	md: sdb2 has invalid sb, not importing!
	md: md_import_device returned -22
	mdadm: failed to add /dev/sdb2 to /dev/md0: Invalid argument
	md: invalid superblock checksum on sdc2
	md: sdc2 has invalid sb, not importing!
	md: md_import_device returned -22
	mdadm: failed to add /dev/sdc2 to /dev/md0: Invalid argument
	md: invalid superblock checksum on sda2
	md: sda2 has invalid sb, not importing!
	md: md_import_device returned -22
	mdadm: failed to add /dev/sda2 to /dev/md0: Invalid argument
	md: bug in file drivers/md/md.c, line 1513
	
	md:o**********************************
	md:o* <COMPLETE RAID STATE PRINTOUT> *
	md:o**********************************
	md0:
	md:o**********************************
	
	mdadm: failed to RUN_ARRAY /dev/md0: Invalid argument
	EXT3-fs: unable to read superblock
	pivot_root: No such file or directory
	/sbin/init: 432: cannot open /dev/console: No such file
	Kernel panic: Attempted to kill init!

When I boot on the Lenny cdrom again in rescue mode, I can successfully reassemble the array (mdadm 2.6.7.2) and mount the root partition on it with no problem at all: "mdadm --detail /dev/md0" reports no error (all 3 devices active, none failed) and the data on partition is accessible.

However, if I boot a Sarge install cdrom (debian-31r8-ia64-netinst.iso) and tries to reassemble the raid manually in a console, mdadm (1.9.0) fails:

	$ mdadm --assemble /dev/md0 /dev/sda2 /dev/sdb2 /dev/sdc2
	mdadm: failed to add /dev/sdb2 to /dev/md0: Invalid argument
	mdadm: failed to add /dev/sdc2 to /dev/md0: Invalid argument
	mdadm: failed to add /dev/sda2 to /dev/md0: Invalid argument

The output of "mdadm --examine" on sda2, sdb2, sdc2 is correct though: all partitions the array is made from are listed correctly, no failure is reported (all 3 devices active, none failed), checksum is reported correct on every partition.  I tried to force reassembling in several ways using options --force, --update=resync, --update=summaries with no success.

The server is using Debian 3.1 Sarge ia64, kernel 2.6.8-2-mckinley-smp

I previously posted this bug report for package "cdrom" but I'm mistaken because it is clearly related to mdadm 2.6.7.2 which corrupted the raid array of my Sarge server preventing mdadm 1.9.0 from working properly afterwards.

Apart from fixing this bug, I would be grateful that you suggest me a safe way to make the server bootable again. I was thinking about booting on a Sarge install cdrom and try to re-create the raid array with option "--assume-clean" or, if that fails, re-create the array and restore content from a tar backup.

Thank you very much.