Bug#534470: Using mdadm 2.6.7.2 to assemble a raid array created withmdadm 1.9.0 will corrupt it making mdadm 1.9.0 to crash whentrying to reassemble

[RUSSOTTO François-Xavier 200103]

Thank you for your answer Neil.

I assumed that an update to the superblock checksumming routine was the reason though I didn't find very much information about this update and potential consequences on older systems. But, above all, I am very surprised that the new mdadm does not even warn the user about potential raid corruption when used on an old one.

As you suggested, I tried recreating the array using option --assume-clean. So, I booted a Sarge cdrom, checked that chunksize and layout was default ones then I typed:

$ mdadm --create /dev/md0 --level=raid5 --raid-devices=3 --auto=yes --assume-clean /dev/sda2 /dev/sdb2 /dev/sdc2

And that worked! recreating the array without loss of data: I ran fsck on /dev/md0, no error were reported, then mounted it to check that all data was there; no pb.

Then, I rebooted the server and... GOT A KERNEL PANIC AGAIN... this time for: invalid uuid ! I patiently rebooted the Sarge cdrom again and tried to reassemble the raid updating the uuid (which I fortunately backed up), using:

$ mdadm --assemble /dev/md0 --auto=yes --uuid=#old#:#uuid#:#back#:#up#  --update=uuid /dev/sda2 /dev/sdb2 /dev/sdc2

Then, I got an error from mdadm saying that option "uuid" for --update was not valid! Only "sparc2.2", "summaries" and "resync" seams supported on that version of mdadm (1.9.0).

Now, I don't know what to do and would be grateful for any suggestion to make my system boot again.

Thanks by advance for your help,

Regards


> -----Message d'origine-----
> De : Neil Brown [mailto:neilb at suse.de]
> Envoyé : vendredi 26 juin 2009 03:56
> À : RUSSOTTO François-Xavier 200103; 534470 at bugs.debian.org
> Objet : Re: Bug#534470: Using mdadm 2.6.7.2 to assemble a raid array
> created withmdadm 1.9.0 will corrupt it making mdadm 1.9.0 to crash
> whentrying to reassemble
> 
> On Wednesday June 24, francois-xavier.russotto at cea.fr wrote:
> > Package: mdadm
> > Version: 2.6.7.2
> > Severity: critical
> >
> > After booting a Debian 5.0.1 - Lenny install cdrom in rescue mode
> > (debian-501-ia64-netinst.iso) on an Itanium 2 server with Debian 3.1
> > - Sarge (ia64) installed on a software raid 5 root partition,
> > opening a console in the root partition mounted from the raid array
> > (auto-assembled) corrupts the raid array, leading to a kernel panic
> > at server reboot, and preventing from manual reassembly using mdadm
> > 1.9.0 (Sarge).
> >
> > The raid 5 array containing the root partition is made of 3
> > partitions on 3 scsi disks (sda2, sdb2, sdc2) which ran fluently for
> > years. Here is the output at server reboot:
> >
> > 	md: invalid superblock checksum on sdb2
> ....
> >
> > Apart from fixing this bug, I would be grateful that you suggest me
> > a safe way to make the server bootable again. I was thinking about
> > booting on a Sarge install cdrom and try to re-create the raid array
> > with option "--assume-clean" or, if that fails, re-create the array
> > and restore content from a tar backup.
> 
> This problem is due to the fact that the superblock checksumming
> routine was changed since 2.6.8.
> It was changed because it used code that was different on different
> architectures, and was basically unmaintainable.
> New kernels can accept most old checksums, but old kernels cannot
> necessarily accept the new ones, and mdadm does not know enough about
> kernel internals to always create the correct old one.
> 
> So it isn't really fixable.
> 
> If you boot with the Sarge install cdrom and reacreate the array with
> --assume-clean as you suggest it should work fine.
> Check the "--examine" values for chunksize and layout, and the order
> of the drives, and make sure you preserve all of those.
> 
> NeilBrown