Bug#569359: mdadm -Ds insists on /dev/md/X device presence

[Neil Brown]

On Mon, 7 Jun 2010 16:25:47 +0100
Martin Michlmayr <tbm at cyrius.com> wrote:

> * martin f krafft <madduck at debian.org> [2010-05-26 10:31]:
> > also sprach Neil Brown <neilb at suse.de> [2010.05.26.1016 +0200]:
> > > The most likely explanation for this is that /var/run/mdadm/map existed and
> > > contained and entry for 'md0' declaring that it could be found at /dev/md/0.
> > > 
> > > Normally mdadm will write such a name there when auto-assembling an array,
> > > then udev will notice that the array has appeared, will use mdadm to find the
> > > path name in /var/run/mdadm/map and will create the node in /dev.
> > > 
> > > So it would seem that udev is not running, but /dev/.udev exists.
> > > If udev was running, it would have created /dev/md/0.
> > > If /dev/.udev didn't exist, then mdadm would have created it when assembling
> > > the array.
> > > 
> > > If this is correct, udev is not running but /dev/.udev exists, then you can
> > > force mdadm to still create the device with
> > >   export MDADM_NO_UDEV=1
> > 
> > Martin, could you please investigate this further? I don't have much
> > experience with d-i, nor really the time to attain that right now.
> 
> Thanks for your explanation, Neal.  So here's a description of what's
> going on:
> 
> We boot into Debian installer.  The installer creates a RAID1 device with:
>     mdadm --create /dev/md0 --auto=yes [...]
> udev is running.  /dev/md0 exists.  /dev/md/0 does not exist.
> 
> There's a file /var/run/map (not /var/run/mdadm/map) which contains
> this:
>     md0 1.2 dcfdb5af:3b385a1e:def86b24:09a2527f /dev/md0

When I compile mdadm from source, it creates /var/run/mdadm/map.  When I use
Debian's mdadm, it creates /var/run/map as you say. Something is broken
there.
Otherwise everything seems OK so far.

> 
> When I run:
>     mdadm --examine --scan --config=partitions
> I get:
>     ARRAY /dev/md/0 metadata=1.2 UUID=afb5fddc:1e5a383b:246bf8de:7f52a209 name=debian:0
> i.e. the /dev/md/0 form.  (Also note that that the UUID in the map
> file is different to that reported by --examine --scan; not sure
> why.)

You use --examine like this?  You know the array you have just created.  You
know the name you want to call it (/dev/md0).  Why not
   mdadm --detail --brief /dev/md0
??
It gives a name like /dev/md/foo because it is using 1.x (1.2 in this case)
metadata.  1.x metadata stores and array name, not a number.  In that case
the name just happens to be numerical.
If the name was actually 'foo', it would be wrong to report /dev/mdfoo, but
correct to report /dev/md/foo.
With 1.x metadata you get /dev/md/name
With 0.90 metadata you get /dev/mdNUM


> 
> After creating the RAID0 and formating it, Debian installer will then
> install Debian to disk (in a chroot).  At some point, mdadm is
> installed in the chroot.  /dev from the real system is bind mounted
> in the chroot.
> 
> Debian's mdadm sees that there's no mdadm config file and tries to
> generate one.  It runs:
>     mdadm --examine --scan --config=partitions
> and writes this to /etc/mdadm/mdadm.conf.  The output is:
>     ARRAY /dev/md/0 metadata=1.2 UUID=afb5fddc:1e5a383b:246bf8de:7f52a209 name=debian:0
> There's no map file in /var/run at this point.
> 
> /dev/md0 exists but /dev/md/0 does not.
> 
> Debian then generates a ramdisk.  It looks at /etc/mdadm/mdadm.conf
> and sees that the RAID device is /dev/md/0 and then does:
>     mdadm --detail /dev/md/0
> to determine the RAID level.  This fails because /dev/md/0 does not
> exist.  As a consequence, the ramdisk won't contain the RAID modules
> and will fail to boot.

Maybe we need to get "mdadm --assemble", if it finds that the array is
already assembled, to create the device named in mdadm.conf anyway.
i.e. if the array mentioned in mdadm.conf already exists in 'map' with
a different name, mdadm --assemble just creates the new name.  There is
another context that came up recently where that would be helpful.

Alternately, you could just
  mdadm -Ir

this rebuilds the 'map' file using the same name that mdadm -E would use, and
then tickles udev so that it creates the right names in /dev...
Almost.
I'm not sure it uses exactly the same names as -E.

It would be safer to use
   mdadm -As
to ensure all arrays are assembled, then
   mdadm -Ds
to create mdadm.conf

... but why do you even want to create mdadm.conf ???

Maybe a deeper rethink is needed here.
 
> 
> So my questions (the first one primarily for Neil, the second one
> for madduck):
> 
>  - Why does mdadm --examine --scan output the /dev/md/X form rather
>    than /dev/mdX when no config and map file exists.  Is /dev/md/X
>    prefered over /dev/mdX?  If so, maybe Debian installer should
>    use
>        mdadm --create /dev/md/X
>    instead of
>        mdadm --create /dev/mdX
>    ?

  I think I have explained above. It is a metadata version difference.


> 
>  - Why is /dev/md/X not created?  Well, I guess it's not generated
>    because we use /dev/mdX and not even the map file mentions
>    /dev/md/X but given that "mdadm --examine --scan" prefers /dev/md/X
>    maybe we should add a udev rule to the installer that always
>    ensures that /dev/md/X is created?
> 

Something like that, yes.

NeilBrown