Bug#821049: multipath-tools: reloading devmap results in block device being created incorrectly

[Mike Bacco]

Package: multipath-tools
Version: 0.5.0+git1.656f8865-9
Severity: normal

Dear Maintainer,

When executing 'multipath -r' the existing symlinks to dm-X devices 
which are present in /dev/mapper are overwritten and replaced with block 
devices as demonstrated below:

mike at dl380gen9-01:~$ sudo ls -l /dev/mapper
total 0
crw------- 1 root root 10, 236 Apr 14 14:31 control
lrwxrwxrwx 1 root root 7 Apr 14 14:31 mpatha -> ../dm-0
lrwxrwxrwx 1 root root 7 Apr 14 14:31 mpathb -> ../dm-1
mike at dl380gen9-01:~$ sudo multipath -r
reload: mpatha (360002ac0000000000000001d000028be) undef 3PARdata,VV
size=10G features='1 queue_if_no_path' hwhandler='1 alua' wp=undef
`-+- policy='queue-length 0' prio=50 status=undef
|- 1:0:0:1 sdb 8:16 active ready running
`- 1:0:1:1 sdd 8:48 active ready running
reload: mpathb (360002ac0000000000000001e000028be) undef 3PARdata,VV
size=20G features='1 queue_if_no_path' hwhandler='1 alua' wp=undef
`-+- policy='queue-length 0' prio=50 status=undef
|- 1:0:0:2 sdc 8:32 active ready running
`- 1:0:1:2 sde 8:64 active ready running
mike at dl380gen9-01:~$ sudo ls -l /dev/mapper
total 0
crw------- 1 root root 10, 236 Apr 14 14:31 control
brw-rw---- 1 root disk 254, 0 Apr 14 14:40 mpatha
brw-rw---- 1 root disk 254, 1 Apr 14 14:40 mpathb

Additionally is has been observed that if the map is flushed with 
multipath -f and then the backend storage removed a subsequent execution 
of multipath -r results in the block device entry being recreated and 
the map being loaded with failed paths

This problem appears to have been introduced with 0.5.0+git0.770e6d0d-1. 
Testing with 0.5.0-7 from snapshots results in the expected behavior of 
the devmap reload resulting in symlink entries to dm-X devices not being 
changed to block device types, and new entries being created as symlinks 
to dm-X devices and not as block devices

When tracing the multipath process I have been able to observe the 
creation of the block device entries as the device files are missing

--
stat("/dev/mapper/mpatha", 0x7ffeae26abc0) = -1 ENOENT (No such file or 
directory)
umask(0)                                = 022
mknod("/dev/mapper/mpatha", S_IFBLK|0660, makedev(254, 0)) = 0
umask(022)                              = 0
chown("/dev/mapper/mpatha", 0, 6)       = 0
stat("/dev/mapper/mpathb", 0x7ffeae26abc0) = -1 ENOENT (No such file or 
directory)
umask(0)                                = 022
mknod("/dev/mapper/mpathb", S_IFBLK|0660, makedev(254, 4)) = 0
umask(022)                              = 0
chown("/dev/mapper/mpathb", 0, 6)       = 0
--

It has also been observed that if the maps are flushed and multipath is 
re-run the resulting symlinks to dm-X devices are created in 
/dev/mapper. Its only when mulitpath -r is executed that this behavior 
is observed.

Please let me know if any additional details are needed or if there is 
additional testing that can be done. We will continue working as well to 
try and identify the specific commit which introduced this change in 
behavior.

-- Package-specific info:
Contents of /etc/multipath.conf:
defaults {
         polling_interval        10
         max_fds                 8192
         user_friendly_names     yes
}
blacklist {
        wwid 3600508b1001c4327dc097cb0a82395d5
        devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
        devnode "^hd[a-z][[0-9]*]"
        device {
                vendor HP
                product 'LOGICAL_VOLUME'
        }
}
devices {
         device {
                 vendor                  "3PARdata"
                 product                 "VV"
                 path_grouping_policy    group_by_prio
                 path_selector           "queue-length 0"
                 path_checker            tur
                 checker                 tur
                 features                "0"
                 hardware_handler        "1 alua"
                 prio                    alua
                 failback                immediate
                 no_path_retry           18
                 rr_min_io               1
         }
}


-- System Information:
Debian Release: stretch/sid
   APT prefers testing
   APT policy: (500, 'testing')
Architecture: amd64 (x86_64)

Kernel: Linux 4.4.0-1-amd64 (SMP w/40 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages multipath-tools depends on:
ii  init-system-helpers  1.29
ii  kpartx               0.5.0+git1.656f8865-9
ii  libaio1              0.3.110-2
ii  libc6                2.22-5
ii  libdevmapper1.02.1   2:1.02.120-1
ii  libgcc1              1:5.3.1-13
ii  libreadline6         6.3-8+b4
ii  libsystemd0          229-4
ii  libudev1             229-4
ii  lsb-base             9.20160110
ii  sg3-utils-udev       1.42-2
ii  udev                 229-4

multipath-tools recommends no packages.

Versions of packages multipath-tools suggests:
ii  multipath-tools-boot  0.5.0+git1.656f8865-9

-- no debconf informatio