Bug#586182: multipath-tools: sometimes fails to create /dev/mapper/mpathN devices
Vincent McIntyre
Vince.McIntyre at atnf.csiro.au
Thu Jun 17 07:19:00 UTC 2010
Package: multipath-tools
Version: 0.4.8-14+lenny2
Severity: normal
*** Please type your report below this line ***
Sometimes, multipath-tools does not create all the /dev/mapper/mpathN
devices one is expecting it to create. This seems to occur if there
more than two multipathed devices.
I have a system with a Promise E-Class 610f storage unit and two
J-Class expansion units, attached in a SAS daisy-chain.
The E-class is attached to a Dell 2950 server via a Cisco FC switch.
There are four FC links, two from each controller on the E-Class.
The FC card is a 4-port LSI/Symbios FC949ES,
pciid is 1000:0646 with subsystem id 1000:1260.
Each storage unit is configured as one logical disk, containing 16
physical disks in a RAID-6 raid set.
These are presented to the Dell 2950 as 3 distinct LUNs,
one for each logical disk.
We started with just the E-Class attached (mpath0).
Then, with the Dell 2950 shut down we added the J-classes.
Neither J-class had a partition label or filesystem on them.
On boot, all three storage units were detected correctly
(a total of 12 sdX devices because of the multiple paths).
However only two /dev/mapper/mpathN devices were created:
# ls -l /dev/mapper
total 0
crw-rw---- 1 root root 10, 59 2010-06-17 11:51 control
brw-rw---- 1 root disk 254, 14 2010-06-17 11:51 install_vg0-data
brw-rw---- 1 root disk 254, 13 2010-06-17 11:51 install_vg0-local
brw-rw---- 1 root disk 254, 10 2010-06-17 11:51 install_vg0-opt
brw-rw---- 1 root disk 254, 12 2010-06-17 11:51 install_vg0-srv
brw-rw---- 1 root disk 254, 11 2010-06-17 11:51 install_vg0-tmp
brw-rw---- 1 root disk 254, 8 2010-06-17 11:51 install_vg0-usr
brw-rw---- 1 root disk 254, 9 2010-06-17 11:51 install_vg0-usr+local
brw-rw---- 1 root disk 254, 6 2010-06-17 11:51 install_vg0-var
brw-rw---- 1 root disk 254, 7 2010-06-17 11:51 install_vg0-var+log
brw-rw---- 1 root disk 254, 5 2010-06-17 11:51 install_vg1-srv+backup
brw-rw---- 1 root disk 254, 4 2010-06-17 11:51 install_vg1-srv+mysql
brw-rw---- 1 root disk 254, 0 2010-06-17 11:51 mpath0
brw-rw---- 1 root disk 254, 2 2010-06-17 11:51 mpath0-part1
brw-rw---- 1 root disk 254, 1 2010-06-17 11:51 mpath1
Yet /var/lib/multipath/bindings showed the right WWNs for all three units:
# cat /var/lib/multipath/bindings
# Multipath bindings, Version : 1.0
# NOTE: this file is automatically maintained by the multipath program.
# You should not need to edit this file in normal circumstances.
#
# Format:
# alias wwid
#
mpath0 222810001550c3cb3
mpath1 2220e0001558b5168
mpath2 222de000155468f10
# multipath -v 3 -l
ram0: device node name blacklisted
ram1: device node name blacklisted
ram2: device node name blacklisted
ram3: device node name blacklisted
ram4: device node name blacklisted
ram5: device node name blacklisted
ram6: device node name blacklisted
ram7: device node name blacklisted
ram8: device node name blacklisted
ram9: device node name blacklisted
ram10: device node name blacklisted
ram11: device node name blacklisted
ram12: device node name blacklisted
ram13: device node name blacklisted
ram14: device node name blacklisted
ram15: device node name blacklisted
sda: not found in pathvec
sda: mask = 0x1
sda: dev_t = 8:0
sda: size = 1465149168
sda: subsystem = scsi
sda: vendor = ATA
sda: product = Hitachi HUA72107
sda: rev = A74A
sda: h:b:t:l = 1:0:0:0
sdb: not found in pathvec
sdb: mask = 0x1
sdb: dev_t = 8:16
sdb: size = 1465149168
sdb: subsystem = scsi
sdb: vendor = ATA
sdb: product = Hitachi HUA72107
sdb: rev = A74A
sdb: h:b:t:l = 1:0:1:0
sdc: not found in pathvec
sdc: mask = 0x1
sdc: dev_t = 8:32
sdc: size = 1465149168
sdc: subsystem = scsi
sdc: vendor = ATA
sdc: product = Hitachi HUA72107
sdc: rev = A74A
sdc: h:b:t:l = 1:0:2:0
hda: device node name blacklisted
sdd: not found in pathvec
sdd: mask = 0x1
sdd: dev_t = 8:48
sdd: size = 1465149168
sdd: subsystem = scsi
sdd: vendor = ATA
sdd: product = Hitachi HUA72107
sdd: rev = A74A
sdd: h:b:t:l = 1:0:3:0
sde: not found in pathvec
sde: mask = 0x1
sde: dev_t = 8:64
sde: size = 27343744512
sde: subsystem = scsi
sde: vendor = Promise
sde: product = VTrak E610f
sde: rev = 1006
sde: h:b:t:l = 0:0:0:0
sde: tgt_node_name = 0x2501000155d2394
sdf: not found in pathvec
sdf: mask = 0x1
sdf: dev_t = 8:80
sdf: size = 27343744512
sdf: subsystem = scsi
sdf: vendor = Promise
sdf: product = VTrak E610f
sdf: rev = 1006
sdf: h:b:t:l = 0:0:0:1
sdf: tgt_node_name = 0x2501000155d2394
sdg: not found in pathvec
sdg: mask = 0x1
sdg: dev_t = 8:96
sdg: size = 27343744512
sdg: subsystem = scsi
sdg: vendor = Promise
sdg: product = VTrak E610f
sdg: rev = 1006
sdg: h:b:t:l = 0:0:0:2
sdg: tgt_node_name = 0x2501000155d2394
sdh: not found in pathvec
sdh: mask = 0x1
sdh: dev_t = 8:112
sdh: size = 27343744512
sdh: subsystem = scsi
sdh: vendor = Promise
sdh: product = VTrak E610f
sdh: rev = 1006
sdh: h:b:t:l = 0:0:1:0
sdh: tgt_node_name = 0x2501000155d2394
sdi: not found in pathvec
sdi: mask = 0x1
sdi: dev_t = 8:128
sdi: size = 27343744512
sdi: subsystem = scsi
sdi: vendor = Promise
sdi: product = VTrak E610f
sdi: rev = 1006
sdi: h:b:t:l = 0:0:1:1
sdi: tgt_node_name = 0x2501000155d2394
sdj: not found in pathvec
sdj: mask = 0x1
sdj: dev_t = 8:144
sdj: size = 27343744512
sdj: subsystem = scsi
sdj: vendor = Promise
sdj: product = VTrak E610f
sdj: rev = 1006
sdj: h:b:t:l = 0:0:1:2
sdj: tgt_node_name = 0x2501000155d2394
sdk: not found in pathvec
sdk: mask = 0x1
sdk: dev_t = 8:160
sdk: size = 27343744512
sdk: subsystem = scsi
sdk: vendor = Promise
sdk: product = VTrak E610f
sdk: rev = 1006
sdk: h:b:t:l = 0:0:2:0
sdk: tgt_node_name = 0x2500000155d2394
sdl: not found in pathvec
sdl: mask = 0x1
sdl: dev_t = 8:176
sdl: size = 27343744512
sdl: subsystem = scsi
sdl: vendor = Promise
sdl: product = VTrak E610f
sdl: rev = 1006
sdl: h:b:t:l = 0:0:2:1
sdl: tgt_node_name = 0x2500000155d2394
sdm: not found in pathvec
sdm: mask = 0x1
sdm: dev_t = 8:192
sdm: size = 27343744512
sdm: subsystem = scsi
sdm: vendor = Promise
sdm: product = VTrak E610f
sdm: rev = 1006
sdm: h:b:t:l = 0:0:2:2
sdm: tgt_node_name = 0x2500000155d2394
sdn: not found in pathvec
sdn: mask = 0x1
sdn: dev_t = 8:208
sdn: size = 27343744512
sdn: subsystem = scsi
sdn: vendor = Promise
sdn: product = VTrak E610f
sdn: rev = 1006
sdn: h:b:t:l = 0:0:3:0
sdn: tgt_node_name = 0x2500000155d2394
sdo: not found in pathvec
sdo: mask = 0x1
sdo: dev_t = 8:224
sdo: size = 27343744512
sdo: subsystem = scsi
sdo: vendor = Promise
sdo: product = VTrak E610f
sdo: rev = 1006
sdo: h:b:t:l = 0:0:3:1
sdo: tgt_node_name = 0x2500000155d2394
sdp: not found in pathvec
sdp: mask = 0x1
sdp: dev_t = 8:240
sdp: size = 27343744512
sdp: subsystem = scsi
sdp: vendor = Promise
sdp: product = VTrak E610f
sdp: rev = 1006
sdp: h:b:t:l = 0:0:3:2
sdp: tgt_node_name = 0x2500000155d2394
loop0: device node name blacklisted
loop1: device node name blacklisted
loop2: device node name blacklisted
loop3: device node name blacklisted
loop4: device node name blacklisted
loop5: device node name blacklisted
loop6: device node name blacklisted
loop7: device node name blacklisted
dm-0: device node name blacklisted
dm-1: device node name blacklisted
dm-2: device node name blacklisted
dm-4: device node name blacklisted
dm-5: device node name blacklisted
dm-6: device node name blacklisted
dm-7: device node name blacklisted
dm-8: device node name blacklisted
dm-9: device node name blacklisted
dm-10: device node name blacklisted
dm-11: device node name blacklisted
dm-12: device node name blacklisted
dm-13: device node name blacklisted
dm-14: device node name blacklisted
===== paths list =====
uuid hcil dev dev_t pri dm_st chk_st vend/prod/rev
1:0:0:0 sda 8:0 -1 [undef][undef] ATA ,Hitachi HUA72107
1:0:1:0 sdb 8:16 -1 [undef][undef] ATA ,Hitachi HUA72107
1:0:2:0 sdc 8:32 -1 [undef][undef] ATA ,Hitachi HUA72107
1:0:3:0 sdd 8:48 -1 [undef][undef] ATA ,Hitachi HUA72107
0:0:0:0 sde 8:64 -1 [undef][undef] Promise ,VTrak E610f
0:0:0:1 sdf 8:80 -1 [undef][undef] Promise ,VTrak E610f
0:0:0:2 sdg 8:96 -1 [undef][undef] Promise ,VTrak E610f
0:0:1:0 sdh 8:112 -1 [undef][undef] Promise ,VTrak E610f
0:0:1:1 sdi 8:128 -1 [undef][undef] Promise ,VTrak E610f
0:0:1:2 sdj 8:144 -1 [undef][undef] Promise ,VTrak E610f
0:0:2:0 sdk 8:160 -1 [undef][undef] Promise ,VTrak E610f
0:0:2:1 sdl 8:176 -1 [undef][undef] Promise ,VTrak E610f
0:0:2:2 sdm 8:192 -1 [undef][undef] Promise ,VTrak E610f
0:0:3:0 sdn 8:208 -1 [undef][undef] Promise ,VTrak E610f
0:0:3:1 sdo 8:224 -1 [undef][undef] Promise ,VTrak E610f
0:0:3:2 sdp 8:240 -1 [undef][undef] Promise ,VTrak E610f
params = 1 queue_if_no_path 0 1 1 round-robin 0 4 1 8:80 100 8:128 100 8:176 100 8:224 100
status = 2 0 0 0 1 1 A 0 4 0 8:80 A 0 8:128 A 0 8:176 A 0 8:224 A 0
mpath1 (2220e0001558b5168) dm-1 Promise ,VTrak E610f
[size=13T][features=1 queue_if_no_path][hwhandler=0]
\_ round-robin 0 [prio=0][active]
\_ 0:0:0:1 sdf 8:80 [active][undef]
\_ 0:0:1:1 sdi 8:128 [active][undef]
\_ 0:0:2:1 sdl 8:176 [active][undef]
\_ 0:0:3:1 sdo 8:224 [active][undef]
params = 1 queue_if_no_path 0 1 1 round-robin 0 4 1 8:64 100 8:112 100 8:160 100 8:208 100
status = 2 0 0 0 1 1 A 0 4 0 8:64 A 0 8:112 A 0 8:160 A 0 8:208 A 0
mpath0 (222810001550c3cb3) dm-0 Promise ,VTrak E610f
[size=13T][features=1 queue_if_no_path][hwhandler=0]
\_ round-robin 0 [prio=0][active]
\_ 0:0:0:0 sde 8:64 [active][undef]
\_ 0:0:1:0 sdh 8:112 [active][undef]
\_ 0:0:2:0 sdk 8:160 [active][undef]
\_ 0:0:3:0 sdn 8:208 [active][undef]
The boot log shows these events related to multipath:
# grep -e multipath -e udev -e kpartx /var/log/syslog
Jun 17 11:51:56 install kernel: [ 18.019074] udevd version 125 started
Jun 17 11:51:56 install kernel: [ 25.008456] device-mapper: multipath: version 1.0.5 loaded
Jun 17 11:51:56 install kernel: [ 25.070222] device-mapper: multipath round-robin: version 1.0.0 loaded
If we stop multipath-tools and start it again, the truant device appears.
# /etc/init.d/multipath-tools stop
# ls -l /dev/mapper
total 0
crw-rw---- 1 root root 10, 59 2010-06-17 11:51 control
brw-rw---- 1 root disk 254, 14 2010-06-17 11:51 install_vg0-data
brw-rw---- 1 root disk 254, 13 2010-06-17 11:51 install_vg0-local
brw-rw---- 1 root disk 254, 10 2010-06-17 11:51 install_vg0-opt
brw-rw---- 1 root disk 254, 12 2010-06-17 11:51 install_vg0-srv
brw-rw---- 1 root disk 254, 11 2010-06-17 11:51 install_vg0-tmp
brw-rw---- 1 root disk 254, 8 2010-06-17 11:51 install_vg0-usr
brw-rw---- 1 root disk 254, 9 2010-06-17 11:51 install_vg0-usr+local
brw-rw---- 1 root disk 254, 6 2010-06-17 11:51 install_vg0-var
brw-rw---- 1 root disk 254, 7 2010-06-17 11:51 install_vg0-var+log
brw-rw---- 1 root disk 254, 5 2010-06-17 11:51 install_vg1-srv+backup
brw-rw---- 1 root disk 254, 4 2010-06-17 11:51 install_vg1-srv+mysql
brw-rw---- 1 root disk 254, 0 2010-06-17 11:51 mpath0
brw-rw---- 1 root disk 254, 2 2010-06-17 11:51 mpath0-part1
brw-rw---- 1 root disk 254, 1 2010-06-17 11:51 mpath1
# /etc/init.d/multipath-tools start
# ls -l /dev/mapper
total 0
crw-rw---- 1 root root 10, 59 2010-06-17 11:51 control
brw-rw---- 1 root disk 254, 14 2010-06-17 11:51 install_vg0-data
brw-rw---- 1 root disk 254, 13 2010-06-17 11:51 install_vg0-local
brw-rw---- 1 root disk 254, 10 2010-06-17 11:51 install_vg0-opt
brw-rw---- 1 root disk 254, 12 2010-06-17 11:51 install_vg0-srv
brw-rw---- 1 root disk 254, 11 2010-06-17 11:51 install_vg0-tmp
brw-rw---- 1 root disk 254, 8 2010-06-17 11:51 install_vg0-usr
brw-rw---- 1 root disk 254, 9 2010-06-17 11:51 install_vg0-usr+local
brw-rw---- 1 root disk 254, 6 2010-06-17 11:51 install_vg0-var
brw-rw---- 1 root disk 254, 7 2010-06-17 11:51 install_vg0-var+log
brw-rw---- 1 root disk 254, 5 2010-06-17 11:51 install_vg1-srv+backup
brw-rw---- 1 root disk 254, 4 2010-06-17 11:51 install_vg1-srv+mysql
brw-rw---- 1 root disk 254, 0 2010-06-17 11:51 mpath0
brw-rw---- 1 root disk 254, 2 2010-06-17 11:51 mpath0-part1
brw-rw---- 1 root disk 254, 1 2010-06-17 11:51 mpath1
brw-rw---- 1 root disk 254, 3 2010-06-17 12:02 mpath2
and in syslog there appears:
Jun 17 12:02:03 install multipathd: mpath1: Entering recovery mode: max_retries=20
Jun 17 12:02:03 install multipathd: mpath2: load table [0 27343744512 multipath 1 queue_if_no_path 0 1 1 round-robin 0 4 1 8:96 100 8:144 100 8:192 100 8:240 100]
Jun 17 12:02:03 install multipathd: mpath0: event checker started
Jun 17 12:02:03 install multipathd: mpath1: event checker started
Jun 17 12:02:03 install multipathd: mpath2: event checker started
Jun 17 12:02:03 install multipathd: dm-3: add map (uevent)
At this point I was able to make partition labels and partition the disks
(/dev/mapper/mpath{1,2}), they behaved normally.
On rebooting, the same thing happens.
The first two mpathN appear normally:
# ls -l /dev/mapper
total 0
crw-rw---- 1 root root 10, 59 2010-06-17 16:58 control
brw-rw---- 1 root disk 254, 16 2010-06-17 16:58 horus_vg0-data
brw-rw---- 1 root disk 254, 15 2010-06-17 16:58 horus_vg0-local
brw-rw---- 1 root disk 254, 12 2010-06-17 16:58 horus_vg0-opt
brw-rw---- 1 root disk 254, 14 2010-06-17 16:58 horus_vg0-srv
brw-rw---- 1 root disk 254, 13 2010-06-17 16:58 horus_vg0-tmp
brw-rw---- 1 root disk 254, 10 2010-06-17 16:58 horus_vg0-usr
brw-rw---- 1 root disk 254, 11 2010-06-17 16:58 horus_vg0-usr+local
brw-rw---- 1 root disk 254, 8 2010-06-17 16:58 horus_vg0-var
brw-rw---- 1 root disk 254, 9 2010-06-17 16:58 horus_vg0-var+log
brw-rw---- 1 root disk 254, 7 2010-06-17 16:58 horus_vg1-srv+backup
brw-rw---- 1 root disk 254, 6 2010-06-17 16:58 horus_vg1-srv+mysql
brw-rw---- 1 root disk 254, 0 2010-06-17 16:58 mpath0
brw-rw---- 1 root disk 254, 1 2010-06-17 16:58 mpath0-part1
brw-rw---- 1 root disk 254, 2 2010-06-17 16:58 mpath1
brw-rw---- 1 root disk 254, 4 2010-06-17 16:58 mpath1-part1
# cat /var/lib/multipath/bindings
# Multipath bindings, Version : 1.0
# NOTE: this file is automatically maintained by the multipath program.
# You should not need to edit this file in normal circumstances.
#
# Format:
# alias wwid
#
mpath0 222810001550c3cb3
mpath1 2220e0001558b5168
mpath2 222de000155468f10
# multipath -v 3 -l
ram0: device node name blacklisted
ram1: device node name blacklisted
ram2: device node name blacklisted
ram3: device node name blacklisted
ram4: device node name blacklisted
ram5: device node name blacklisted
ram6: device node name blacklisted
ram7: device node name blacklisted
ram8: device node name blacklisted
ram9: device node name blacklisted
ram10: device node name blacklisted
ram11: device node name blacklisted
ram12: device node name blacklisted
ram13: device node name blacklisted
ram14: device node name blacklisted
ram15: device node name blacklisted
sda: not found in pathvec
sda: mask = 0x1
sda: dev_t = 8:0
sda: size = 1465149168
sda: subsystem = scsi
sda: vendor = ATA
sda: product = Hitachi HUA72107
sda: rev = A74A
sda: h:b:t:l = 0:0:0:0
sdb: not found in pathvec
sdb: mask = 0x1
sdb: dev_t = 8:16
sdb: size = 1465149168
sdb: subsystem = scsi
sdb: vendor = ATA
sdb: product = Hitachi HUA72107
sdb: rev = A74A
sdb: h:b:t:l = 0:0:1:0
sdc: not found in pathvec
sdc: mask = 0x1
sdc: dev_t = 8:32
sdc: size = 1465149168
sdc: subsystem = scsi
sdc: vendor = ATA
sdc: product = Hitachi HUA72107
sdc: rev = A74A
sdc: h:b:t:l = 0:0:2:0
hda: device node name blacklisted
sdd: not found in pathvec
sdd: mask = 0x1
sdd: dev_t = 8:48
sdd: size = 1465149168
sdd: subsystem = scsi
sdd: vendor = ATA
sdd: product = Hitachi HUA72107
sdd: rev = A74A
sdd: h:b:t:l = 0:0:3:0
sde: not found in pathvec
sde: mask = 0x1
sde: dev_t = 8:64
sde: size = 27343744512
sde: subsystem = scsi
sde: vendor = Promise
sde: product = VTrak E610f
sde: rev = 1006
sde: h:b:t:l = 1:0:0:0
sde: tgt_node_name = 0x2501000155d2394
sdf: not found in pathvec
sdf: mask = 0x1
sdf: dev_t = 8:80
sdf: size = 27343744512
sdf: subsystem = scsi
sdf: vendor = Promise
sdf: product = VTrak E610f
sdf: rev = 1006
sdf: h:b:t:l = 1:0:0:1
sdf: tgt_node_name = 0x2501000155d2394
sdg: not found in pathvec
sdg: mask = 0x1
sdg: dev_t = 8:96
sdg: size = 27343744512
sdg: subsystem = scsi
sdg: vendor = Promise
sdg: product = VTrak E610f
sdg: rev = 1006
sdg: h:b:t:l = 1:0:0:2
sdg: tgt_node_name = 0x2501000155d2394
sdh: not found in pathvec
sdh: mask = 0x1
sdh: dev_t = 8:112
sdh: size = 27343744512
sdh: subsystem = scsi
sdh: vendor = Promise
sdh: product = VTrak E610f
sdh: rev = 1006
sdh: h:b:t:l = 1:0:1:0
sdh: tgt_node_name = 0x2501000155d2394
sdi: not found in pathvec
sdi: mask = 0x1
sdi: dev_t = 8:128
sdi: size = 27343744512
sdi: subsystem = scsi
sdi: vendor = Promise
sdi: product = VTrak E610f
sdi: rev = 1006
sdi: h:b:t:l = 1:0:1:1
sdi: tgt_node_name = 0x2501000155d2394
sdj: not found in pathvec
sdj: mask = 0x1
sdj: dev_t = 8:144
sdj: size = 27343744512
sdj: subsystem = scsi
sdj: vendor = Promise
sdj: product = VTrak E610f
sdj: rev = 1006
sdj: h:b:t:l = 1:0:1:2
sdj: tgt_node_name = 0x2501000155d2394
sdk: not found in pathvec
sdk: mask = 0x1
sdk: dev_t = 8:160
sdk: size = 27343744512
sdk: subsystem = scsi
sdk: vendor = Promise
sdk: product = VTrak E610f
sdk: rev = 1006
sdk: h:b:t:l = 1:0:2:0
sdk: tgt_node_name = 0x2500000155d2394
sdl: not found in pathvec
sdl: mask = 0x1
sdl: dev_t = 8:176
sdl: size = 27343744512
sdl: subsystem = scsi
sdl: vendor = Promise
sdl: product = VTrak E610f
sdl: rev = 1006
sdl: h:b:t:l = 1:0:2:1
sdl: tgt_node_name = 0x2500000155d2394
sdm: not found in pathvec
sdm: mask = 0x1
sdm: dev_t = 8:192
sdm: size = 27343744512
sdm: subsystem = scsi
sdm: vendor = Promise
sdm: product = VTrak E610f
sdm: rev = 1006
sdm: h:b:t:l = 1:0:2:2
sdm: tgt_node_name = 0x2500000155d2394
sdn: not found in pathvec
sdn: mask = 0x1
sdn: dev_t = 8:208
sdn: size = 27343744512
sdn: subsystem = scsi
sdn: vendor = Promise
sdn: product = VTrak E610f
sdn: rev = 1006
sdn: h:b:t:l = 1:0:3:0
sdn: tgt_node_name = 0x2500000155d2394
sdo: not found in pathvec
sdo: mask = 0x1
sdo: dev_t = 8:224
sdo: size = 27343744512
sdo: subsystem = scsi
sdo: vendor = Promise
sdo: product = VTrak E610f
sdo: rev = 1006
sdo: h:b:t:l = 1:0:3:1
sdo: tgt_node_name = 0x2500000155d2394
sdp: not found in pathvec
sdp: mask = 0x1
sdp: dev_t = 8:240
sdp: size = 27343744512
sdp: subsystem = scsi
sdp: vendor = Promise
sdp: product = VTrak E610f
sdp: rev = 1006
sdp: h:b:t:l = 1:0:3:2
sdp: tgt_node_name = 0x2500000155d2394
loop0: device node name blacklisted
loop1: device node name blacklisted
loop2: device node name blacklisted
loop3: device node name blacklisted
loop4: device node name blacklisted
loop5: device node name blacklisted
loop6: device node name blacklisted
loop7: device node name blacklisted
dm-0: device node name blacklisted
dm-1: device node name blacklisted
dm-2: device node name blacklisted
dm-4: device node name blacklisted
dm-6: device node name blacklisted
dm-7: device node name blacklisted
dm-8: device node name blacklisted
dm-9: device node name blacklisted
dm-10: device node name blacklisted
dm-11: device node name blacklisted
dm-12: device node name blacklisted
dm-13: device node name blacklisted
dm-14: device node name blacklisted
dm-15: device node name blacklisted
dm-16: device node name blacklisted
===== paths list =====
uuid hcil dev dev_t pri dm_st chk_st vend/prod/rev
0:0:0:0 sda 8:0 -1 [undef][undef] ATA ,Hitachi HUA72107
0:0:1:0 sdb 8:16 -1 [undef][undef] ATA ,Hitachi HUA72107
0:0:2:0 sdc 8:32 -1 [undef][undef] ATA ,Hitachi HUA72107
0:0:3:0 sdd 8:48 -1 [undef][undef] ATA ,Hitachi HUA72107
1:0:0:0 sde 8:64 -1 [undef][undef] Promise ,VTrak E610f
1:0:0:1 sdf 8:80 -1 [undef][undef] Promise ,VTrak E610f
1:0:0:2 sdg 8:96 -1 [undef][undef] Promise ,VTrak E610f
1:0:1:0 sdh 8:112 -1 [undef][undef] Promise ,VTrak E610f
1:0:1:1 sdi 8:128 -1 [undef][undef] Promise ,VTrak E610f
1:0:1:2 sdj 8:144 -1 [undef][undef] Promise ,VTrak E610f
1:0:2:0 sdk 8:160 -1 [undef][undef] Promise ,VTrak E610f
1:0:2:1 sdl 8:176 -1 [undef][undef] Promise ,VTrak E610f
1:0:2:2 sdm 8:192 -1 [undef][undef] Promise ,VTrak E610f
1:0:3:0 sdn 8:208 -1 [undef][undef] Promise ,VTrak E610f
1:0:3:1 sdo 8:224 -1 [undef][undef] Promise ,VTrak E610f
1:0:3:2 sdp 8:240 -1 [undef][undef] Promise ,VTrak E610f
params = 1 queue_if_no_path 0 1 1 round-robin 0 4 1 8:80 100 8:128 100
8:176 100 8:224 100
status = 2 0 0 0 1 1 A 0 4 0 8:80 A 0 8:128 A 0 8:176 A 0 8:224 A 0
mpath1 (2220e0001558b5168) dm-2 Promise ,VTrak E610f
[size=13T][features=1 queue_if_no_path][hwhandler=0]
\_ round-robin 0 [prio=0][active]
\_ 1:0:0:1 sdf 8:80 [active][undef]
\_ 1:0:1:1 sdi 8:128 [active][undef]
\_ 1:0:2:1 sdl 8:176 [active][undef]
\_ 1:0:3:1 sdo 8:224 [active][undef]
params = 1 queue_if_no_path 0 1 1 round-robin 0 4 1 8:64 100 8:112 100
8:160 100 8:208 100
status = 2 0 0 0 1 1 A 0 4 0 8:64 A 0 8:112 A 0 8:160 A 0 8:208 A 0
mpath0 (222810001550c3cb3) dm-0 Promise ,VTrak E610f
[size=13T][features=1 queue_if_no_path][hwhandler=0]
\_ round-robin 0 [prio=0][active]
\_ 1:0:0:0 sde 8:64 [active][undef]
\_ 1:0:1:0 sdh 8:112 [active][undef]
\_ 1:0:2:0 sdk 8:160 [active][undef]
\_ 1:0:3:0 sdn 8:208 [active][undef]
and restarting multipath-tools causes mpath2 to appear.
# /etc/init.d/multipath-tools stop
(wait a few sec)
# /etc/init.d/multipath-tools start
# ls -l /dev/mapper
total 0
crw-rw---- 1 root root 10, 59 2010-06-17 16:58 control
brw-rw---- 1 root disk 254, 16 2010-06-17 16:58 horus_vg0-data
brw-rw---- 1 root disk 254, 15 2010-06-17 16:58 horus_vg0-local
brw-rw---- 1 root disk 254, 12 2010-06-17 16:58 horus_vg0-opt
brw-rw---- 1 root disk 254, 14 2010-06-17 16:58 horus_vg0-srv
brw-rw---- 1 root disk 254, 13 2010-06-17 16:58 horus_vg0-tmp
brw-rw---- 1 root disk 254, 10 2010-06-17 16:58 horus_vg0-usr
brw-rw---- 1 root disk 254, 11 2010-06-17 16:58 horus_vg0-usr+local
brw-rw---- 1 root disk 254, 8 2010-06-17 16:58 horus_vg0-var
brw-rw---- 1 root disk 254, 9 2010-06-17 16:58 horus_vg0-var+log
brw-rw---- 1 root disk 254, 7 2010-06-17 16:58 horus_vg1-srv+backup
brw-rw---- 1 root disk 254, 6 2010-06-17 16:58 horus_vg1-srv+mysql
brw-rw---- 1 root disk 254, 0 2010-06-17 16:58 mpath0
brw-rw---- 1 root disk 254, 1 2010-06-17 16:58 mpath0-part1
brw-rw---- 1 root disk 254, 2 2010-06-17 16:58 mpath1
brw-rw---- 1 root disk 254, 4 2010-06-17 16:58 mpath1-part1
brw-rw---- 1 root disk 254, 3 2010-06-17 17:00 mpath2
brw-rw---- 1 root disk 254, 5 2010-06-17 17:00 mpath2-part1
syslog shows less this time:
# grep -e multipath -e udev -e kpartx /var/log/syslog
Jun 17 16:55:40 install kernel: [22479.819716] multipathd[7291]: segfault at a ip 7f40f7fdd9fb sp 7fffffffd830 error 4 in
libc-2.7.so[7f40f7f6a000+14a000]
Jun 17 16:58:34 install kernel: [ 58.060260] udevd version 125 started
Jun 17 16:58:34 install kernel: [ 66.523478] device-mapper: multipath: version 1.0.5 loaded
Jun 17 16:58:34 install kernel: [ 66.695709] device-mapper: multipath round-robin: version 1.0.0 loaded
Note the segfault at shutdown, see bug #580312
This has now occurred on two systems, running lenny/amd64 kernels
and the same version of multipath-tools.
Please advise what we can do to help diagnose this bug and resolve it.
Kind regards,
Vince
-- Package-specific info:
Contents of /etc/multipath.conf:
defaults {
user_friendly_names yes
}
blacklist {
devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
devnode "^hd[a-z][[0-9]*]"
devnode "^cciss!c[0-9]d[0-9]*[p[0-9]*]"
device {
vendor MegaRAID
}
device {
vendor APPLE
}
device {
vendor ATA
}
device {
vendor DELL
}
}
devices {
device {
vendor "Promise"
product "VTrak"
path_grouping_policy multibus
getuid_callout "/lib/udev/scsi_id -g -u -s /block/%n"
path_checker readsector0
path_selector "round-robin 0"
hardware_handler "0"
failback immediate
rr_weight uniform
rr_min_io 100
no_path_retry 20
features "1 queue_if_no_path"
product_blacklist "VTrak V-LUN"
}
}
(lspci; lspci -n) | sort|grep -i fibre
0f:00.0 0c04: 1000:0646 (rev 02)
0f:00.0 Fibre Channel: LSI Logic / Symbios Logic FC949ES Fibre Channel Adapter (rev 02)
0f:00.1 0c04: 1000:0646 (rev 02)
0f:00.1 Fibre Channel: LSI Logic / Symbios Logic FC949ES Fibre Channel Adapter (rev 02)
10:00.0 0c04: 1000:0646 (rev 02)
10:00.0 Fibre Channel: LSI Logic / Symbios Logic FC949ES Fibre Channel Adapter (rev 02)
10:00.1 0c04: 1000:0646 (rev 02)
10:00.1 Fibre Channel: LSI Logic / Symbios Logic FC949ES Fibre Channel Adapter (rev 02)
-- System Information:
Debian Release: 5.0.4
APT prefers stable
APT policy: (500, 'stable')
Architecture: amd64 (x86_64)
Kernel: Linux 2.6.26-2-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_AU.UTF-8, LC_CTYPE=en_AU.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash
Versions of packages multipath-tools depends on:
ii initscripts 2.86.ds1-61 Scripts for initializing and shutt
ii kpartx 0.4.8-14+lenny2 create device mappings for partiti
ii libaio1 0.3.107-3 Linux kernel AIO access library -
ii libc6 2.7-18lenny2 GNU C Library: Shared libraries
ii libdevmapper1.02.1 2:1.02.27-4 The Linux Kernel Device Mapper use
ii libncurses5 5.7+20081213-1 shared libraries for terminal hand
ii libreadline5 5.2-3.1 GNU readline and history libraries
ii lsb-base 3.2-20 Linux Standard Base 3.2 init scrip
ii udev 0.125-7+lenny3 /dev/ and hotplug management daemo
multipath-tools recommends no packages.
Versions of packages multipath-tools suggests:
pn multipath-tools-boot <none> (no description available)
-- no debconf information
More information about the pkg-lvm-maintainers
mailing list