Bug#510911: multipath-tools: bad side effects with FC devices

[Vincent.McIntyre at csiro.au]

Finally got back to this.

> [..snip..]
>> I expected multipath-tools to leave the device alone, or at least
>> allow the system to mount and be able to access the data as before,
>> since I had not 'told' the package about these devices.
>> I had not configured /etc/multipath.conf, that file did not exist.
>> Nor was there an /etc/default/multipath-tools.
> This is all automatically being taken care off by
> /etc/init.d/multipath-tools.

I understand it does, I was just emphasising that I had not told the
package about these devices.

>> On a system where we _do_ have multipathed storage, the devices are
>> referred to as /dev/mapper/mpath0p1, /dev/mapper/mpath1p1, etc.
> You only get the mpath* devices if you use user_friendly_names=yes in
> your multipath.conf, otherwise the WWID is used.

Ah. I wasn't deeply involved in the setup of that system, otherwise I'd
have known that.

>> The behaviour of 'mount' and 'umount' was also extremely confusing.
>> The device is not mounted and cannot be umounted but is 'busy'.
> Because it's part of a multipath map.

Right. I suppose multipath -l would have shown me this.

>> Why does 'lsof' not show what is keeping the device 'busy'?
> Because there's no file descriptor involved.

Thought so. Thanks.

>>  * if the package is able (at installation time) to detect devices that
>>    it will start to manage at the next reboot, it would be helpful to
>>    give a debconf warning about this and point people to /usr/share/doc.
>>    I don't think there is such a warning now.
> No there's no such warning and I don't want to add a pointer to avoid
> unnecessary prompting. Pointing to
> /usr/share/doc/multipath-tools/README.Debian in the package description
> would be fine though.

Good idea. That would have helped in my situation.

>>  * provide your paragraph in /usr/share/doc:
>>      Once you use multipath you have to access multipathed devices via
>>      /dev/mapper/<wwid> or via /dev/disk/by-id/.
>>      For example, in /etc/fstab one could write:
>>       /dev/mapper/<wwid>  /data/foo_1  ext3  defaults  0  0
> We should better point people to /dev/disk/by-id/... since this way
> mounting works with and without multipath.

I agree.

>>  * it would help to explain what the WWN-named files in /dev/mapper are.
>>    I've had a go at a patch for this and the point above.
> Thanks a lot for that. Would you mind extending README.Debian a bit
> (instead of the FAQ) with the above suggestions and update the package
> description to point to that file?

Attached, fairly invasive patch against 0.4.8-13.

>>  * alternatively, or as well, provide a default /etc/multipath.conf that
>>    ignores/blacklists every device on the system, so multipath-tools won't
>>    attempt to manage anything until the system owner takes a positive
>>    action and configures the package.
> I'd rather not. This would mean extra configuration for people that want
> to use it.

Fair enough. Perhaps an example config fragment then? See the patch.
I didn't break it out into a separate file, perhaps it should be?

>> You may well ask 'wtf did you install the package in the first place?'.
>> I was having a look-see to see what the package did when it was
>> installed, before trying it out on a system that does have multipathed
>> storage.
>> I neglected to uninstall it again and was penalized heavily for this.
> I think a pointer in the package description would have helped?

Yes.

Thanks again
Vince
-------------- next part --------------
diff -urN -ruN multipath-tools-0.4.8.orig/debian/control multipath-tools-0.4.8/debian/control
--- multipath-tools-0.4.8.orig/debian/control	2009-01-12 08:32:37.060968000 +1100
+++ multipath-tools-0.4.8/debian/control	2009-01-12 09:09:27.306701000 +1100
@@ -17,6 +17,9 @@
 Description: maintain multipath block device access
  These tools are in charge of maintaining the disk multipath device maps and
  react to path and map events.
+ .
+ If you install this package you may have to change the way you address
+ some of the devices connected to your system. See README.Debian for details.
 
 Package: kpartx
 Architecture: any
@@ -39,7 +42,7 @@
   * load the necessary kernel modules
   * detect multipath block devices
  .
- Don't install this package if you're not booting from a multipath device
+ Don't install this package if you're not booting from a multipath device.
 
 Package: multipath-udeb
 XC-Package-Type: udeb
diff -urN -ruN multipath-tools-0.4.8.orig/debian/multipath-tools.README.Debian multipath-tools-0.4.8/debian/multipath-tools.README.Debian
--- multipath-tools-0.4.8.orig/debian/multipath-tools.README.Debian	2009-01-12 08:32:37.197654000 +1100
+++ multipath-tools-0.4.8/debian/multipath-tools.README.Debian	2009-01-21 10:40:19.417523000 +1100
@@ -1,16 +1,65 @@
-To enable booting from multipathed devices install multipath-tools-boot
+Additional information for users of multipath-tools from Debian.
+
+What are these weird numbers in /dev/mapper?
+============================================
+Modern Fibre-Channel connected storage devices are identified by
+World-Wide-Numbers. If multipath-tools detects one of these devices,
+it uses that as the default identifier in /dev/mapper.
+If you want multipath-tools to use easier-to-read names, set
+  user_friendly_names = yes
+in the 'defaults' section of /etc/multipath.conf.
+
+Where did my FC-connected filesystem go?
+========================================
+If you were previously mounting a device connected to your system by
+Fibre Channel and then installed multipath-tools, you need to change
+the way you mount the device. The device must now be accessed using
+the identifier by which device-mapper knows it.
+
+For example if you have in /dev/mapper a file like this:
+   brw-rw---- 1 root disk 254,  8 2009-01-05 14:35 /dev/mapper/36000393000007d3901000000fef00a2d
+
+then you can mount the filesystem like this:
+  mount /dev/mapper/36000393000007d3901000000fef00a2d /mnt
+or this
+  mount /dev/disk/by-id/scsi-36000393000007d3901000000fef00a2d /mnt
+You should prefer the latter, as this will work whether or not 
+multipath-tools is installed.
+
+Note that with multipath-tools installed you cannot use the device's node
+in /dev, e.g.
+  mount /dev/sdc1 /mnt
+  mount: /dev/sdc1 already mounted or /mnt busy
+The device is 'busy' because it is part of a multipath map.
+See the output of 'multipath -l' to confirm this.
+
+Preventing multipath-tools mapping any devices
+==============================================
+You may wish to prevent multipath-tools mapping any of your storage devices.
+To do this replace the contents of /etc/multipath.conf with:
+
+  blacklist {
+         devnode "*"
+  }
+
+Booting from multipathed devices
+================================
+To enable booting from multipathed devices install the multipath-tools-boot
 package. Be sure to update your bootloader configuration (e.g.
-/boot/grub/menu.lst) to point to the multipathed root device afterwards. This
-is either
+/boot/grub/menu.lst) to point to the multipathed root device afterwards.
+
+By default this will be
+
+  /dev/mapper/<WWID>-part<N>
 
-/dev/mapper/<WWID>-part<N>
+unless you use an alias in /etc/multipath.conf.
 
-if you don't use an alias in /etc/multipath.conf or
+If you set user_friendly_names = yes in /etc/multipath.conf, the path
+to the device will be
 
-/dev/mapper/mpath<X>-part<N>
+  /dev/mapper/mpath<X>-part<N>
 
-when using user_friendly_names = yes in /etc/multipath.conf. Also remember to
-run update-initramfs after making changes in multipath.conf so the updated
-version get's included into the initramfs.
+If you make changes in multipath.conf, remember to run update-initramfs
+so the updated version of the configuration gets included in the initramfs.
 
  -- Guido G??nther <agx at sigxcpu.org>  Tue, 20 Feb 2008 13:49:18 +0100