Bug#592149: Fwd: mdadm
Florian Heigl
florian.heigl at gmail.com
Sat Aug 7 20:01:50 UTC 2010
I didn't manage to correcly use the --body=/var/tmp/mdadm.bug option
on my last try thus losing the actual report content. please find it
below.
---------- Forwarded message ----------
From: root <root at waxh0002>
Date: 2010/8/7
Subject: mdadm
To: florian.heigl at gmail.com
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
From: root <root at waxh0002>
To: Debian Bug Tracking System <submit at bugs.debian.org>
Subject: mdadm: wishlist item: please ionice the array check
Message-ID: <20100807192238.22025.20584.reportbug at waxh0002>
X-Mailer: reportbug 3.48
Date: Sat, 07 Aug 2010 21:22:38 +0200
Package: mdadm
Version: 2.6.7.2-3
Severity: important
mdadm came with a cronjob that does a periodic check of the configured arrays.
this check doesn't limit it's bandwidth or priority and basically it is a DOS
of the system for the runtime. ping will still work, but even ssh login might
be delayed beyond the standard timeouts.
the situation is worse with vm's running on top of the affected (host) system.
once i tracked down the issue i was able to completely resolve it by changing
the severity class of the scan to "idle". it continued without very much of a
slowdown, but the system was perfectly snappy.
this is much more the behavious that other mechanisms in larger arrays use for
their background verifies, and would actually enable a weekly check w/o any
negative impact.
We're not talking cpu issue here, the system was a largely idle core i7.
note this report might relate to:
#556610
#508123
i had written down the commands used at:
http://deranfangvomende.wordpress.com/2010/07/04/using-ionice-to-stop-mdresync-from-killing-the-system/
unfortunately i can't link to the cpu / io rate graphs anymore due to a
password protection.
it should be possible to call the verify/resync using ionice to save the hassle
of tracking the pid.
Best regards,
Flo
-- Package-specific info:
--- mount output
/dev/mapper/vg00-lvol01 on / type ext3 (rw)
tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
procbususb on /proc/bus/usb type usbfs (rw)
udev on /dev type tmpfs (rw,mode=0755)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=620)
/dev/md0 on /boot type ext2 (rw)
/dev/mapper/vg00-lvol03 on /usr type ext3 (rw)
/dev/mapper/vg00-lvol04 on /var type ext3 (rw)
/dev/mapper/vg00-lvol05 on /var/log type ext3 (rw)
/dev/mapper/vg00-lvol06 on /tmp type ext3 (rw)
/dev/mapper/vg00-lvol07 on /home type ext3 (rw)
/dev/mapper/vgxen-lvol01 on /xen type ext3 (rw)
/dev/mapper/vgxen-lvol02 on /xen/vbd/01 type ext3 (rw)
/dev/mapper/vgxen-lvol03 on /xen/dists type xfs (rw)
--- mdadm.conf
DEVICES /dev/sda* /dev/sdb*
ARRAY /dev/md0 level=raid1 num-devices=2 metadata=0.90
UUID=8338c8ba:565b0a83:776c2c25:004bd7b2
ARRAY /dev/md1 level=raid1 num-devices=2 metadata=0.90
UUID=1534f5ef:dfed0a34:776c2c25:004bd7b2
ARRAY /dev/md2 level=raid1 num-devices=2 metadata=0.90
UUID=41008bdc:74a94d83:776c2c25:004bd7b2
ARRAY /dev/md3 level=raid1 num-devices=2
UUID=d3e0bd33:45396281:ac3f7ccc:bdf0e125
--- /proc/mdstat:
Personalities : [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md3 : active raid1 sda4[0] sdb4[1]
560331008 blocks [2/2] [UU]
md2 : active raid1 sda3[0] sdb3[1]
104864192 blocks [2/2] [UU]
md1 : active raid1 sda2[0] sdb2[1]
67111424 blocks [2/2] [UU]
md0 : active raid1 sda1[0] sdb1[1]
264960 blocks [2/2] [UU]
unused devices: <none>
--- /proc/partitions:
major minor #blocks name
8 0 732574584 sda
8 1 265072 sda1
8 2 67111537 sda2
8 3 104864287 sda3
8 4 560331135 sda4
8 16 732574584 sdb
8 17 265072 sdb1
8 18 67111537 sdb2
8 19 104864287 sdb3
8 20 560331135 sdb4
9 0 264960 md0
9 1 67111424 md1
9 2 104864192 md2
9 3 560331008 md3
253 0 1048576 dm-0
253 1 1048576 dm-1
253 2 8388608 dm-2
253 3 2097152 dm-3
253 4 1048576 dm-4
253 5 524288 dm-5
253 6 524288 dm-6
7 0 5242880 loop0
7 1 5242880 loop1
7 10 10485760 loop10
7 11 524288 loop11
7 12 5219328 loop12
253 7 1048576 dm-7
253 8 103813120 dm-8
253 9 10485760 dm-9
253 10 102400 dm-10
253 11 41943040 dm-11
--- initrd.img-2.6.26-2-xen-amd64:
44127 blocks
e1e2d0e985196fecaf41fb42e9968af2 ./scripts/local-top/mdadm
5ceceb31d008fee1d7b0373ec19628c4
./lib/modules/2.6.26-2-xen-amd64/kernel/drivers/md/linear.ko
cc91d062297b3f326cc2f2a24e000925
./lib/modules/2.6.26-2-xen-amd64/kernel/drivers/md/dm-snapshot.ko
f54348c105b8b0435e9e4886fc8336e1
./lib/modules/2.6.26-2-xen-amd64/kernel/drivers/md/raid1.ko
105424ca699590f455ccb663afde5a0b
./lib/modules/2.6.26-2-xen-amd64/kernel/drivers/md/md-mod.ko
01e72701b32954d21ec44586d6ea8377
./lib/modules/2.6.26-2-xen-amd64/kernel/drivers/md/dm-log.ko
5945159b9f6ceffc20c9ad2d16b065e1
./lib/modules/2.6.26-2-xen-amd64/kernel/drivers/md/raid0.ko
9263fed31bc961e5af858568bb0328f1
./lib/modules/2.6.26-2-xen-amd64/kernel/drivers/md/raid456.ko
1b4e6fc0839047d1ebf53f2abad7c49c
./lib/modules/2.6.26-2-xen-amd64/kernel/drivers/md/raid10.ko
508edaca7ef5aa3efba277a028f67d58
./lib/modules/2.6.26-2-xen-amd64/kernel/drivers/md/dm-mod.ko
b023002710eab722487d38cb9de562e7
./lib/modules/2.6.26-2-xen-amd64/kernel/drivers/md/multipath.ko
91f661d59a8973e915241d1954213674
./lib/modules/2.6.26-2-xen-amd64/kernel/drivers/md/dm-mirror.ko
1e3828eef5ebb2b42ba71d08fc4247d4 ./sbin/mdadm
49652b302b29a8d420952c75f589aae9 ./etc/mdadm/mdadm.conf
--- /proc/modules:
dm_mirror 21120 0 - Live 0xffffffffa0169000
dm_log 14212 1 dm_mirror, Live 0xffffffffa0164000
dm_snapshot 19400 0 - Live 0xffffffffa015e000
dm_mod 59248 28 dm_mirror,dm_log,dm_snapshot, Live 0xffffffffa014e000
raid10 24192 0 - Live 0xffffffffa0101000
raid456 127520 0 - Live 0xffffffffa00e0000
async_xor 8448 1 raid456, Live 0xffffffffa00dc000
async_memcpy 6912 1 raid456, Live 0xffffffffa00d9000
async_tx 11764 3 raid456,async_xor,async_memcpy, Live 0xffffffffa00d5000
xor 10384 2 raid456,async_xor, Live 0xffffffffa00d1000
raid1 24832 4 - Live 0xffffffffa00c9000
raid0 10624 0 - Live 0xffffffffa00c5000
md_mod 81700 8 raid10,raid456,raid1,raid0, Live 0xffffffffa00b0000
--- /var/log/syslog:
--- volume detail:
/dev/sda is not recognised by mdadm.
/dev/sda1:
Magic : a92b4efc
Version : 00.90.00
UUID : 8338c8ba:565b0a83:776c2c25:004bd7b2
Creation Time : Wed Oct 7 01:54:31 2009
Raid Level : raid1
Used Dev Size : 264960 (258.79 MiB 271.32 MB)
Array Size : 264960 (258.79 MiB 271.32 MB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 0
Update Time : Sat Aug 7 06:26:13 2010
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Checksum : 91a3eb71 - correct
Events : 80
Number Major Minor RaidDevice State
this 0 8 1 0 active sync /dev/sda1
0 0 8 1 0 active sync /dev/sda1
1 1 8 17 1 active sync /dev/sdb1
--
/dev/sda2:
Magic : a92b4efc
Version : 00.90.00
UUID : 1534f5ef:dfed0a34:776c2c25:004bd7b2
Creation Time : Wed Oct 7 01:54:31 2009
Raid Level : raid1
Used Dev Size : 67111424 (64.00 GiB 68.72 GB)
Array Size : 67111424 (64.00 GiB 68.72 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 1
Update Time : Sat Aug 7 21:13:51 2010
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Checksum : b12ee761 - correct
Events : 78
Number Major Minor RaidDevice State
this 0 8 2 0 active sync /dev/sda2
0 0 8 2 0 active sync /dev/sda2
1 1 8 18 1 active sync /dev/sdb2
--
/dev/sda3:
Magic : a92b4efc
Version : 00.90.00
UUID : 41008bdc:74a94d83:776c2c25:004bd7b2
Creation Time : Wed Oct 7 01:54:31 2009
Raid Level : raid1
Used Dev Size : 104864192 (100.01 GiB 107.38 GB)
Array Size : 104864192 (100.01 GiB 107.38 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 2
Update Time : Sat Aug 7 21:13:50 2010
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Checksum : 73f6d060 - correct
Events : 78
Number Major Minor RaidDevice State
this 0 8 3 0 active sync /dev/sda3
0 0 8 3 0 active sync /dev/sda3
1 1 8 19 1 active sync /dev/sdb3
--
/dev/sda4:
Magic : a92b4efc
Version : 00.90.00
UUID : d3e0bd33:45396281:ac3f7ccc:bdf0e125
Creation Time : Sun Nov 1 01:21:45 2009
Raid Level : raid1
Used Dev Size : 560331008 (534.37 GiB 573.78 GB)
Array Size : 560331008 (534.37 GiB 573.78 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 3
Update Time : Sat Aug 7 20:17:40 2010
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Checksum : e5263e84 - correct
Events : 42
Number Major Minor RaidDevice State
this 0 8 4 0 active sync /dev/sda4
0 0 8 4 0 active sync /dev/sda4
1 1 8 20 1 active sync /dev/sdb4
--
/dev/sdb is not recognised by mdadm.
/dev/sdb1:
Magic : a92b4efc
Version : 00.90.00
UUID : 8338c8ba:565b0a83:776c2c25:004bd7b2
Creation Time : Wed Oct 7 01:54:31 2009
Raid Level : raid1
Used Dev Size : 264960 (258.79 MiB 271.32 MB)
Array Size : 264960 (258.79 MiB 271.32 MB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 0
Update Time : Sat Aug 7 06:26:13 2010
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Checksum : 91a3eb83 - correct
Events : 80
Number Major Minor RaidDevice State
this 1 8 17 1 active sync /dev/sdb1
0 0 8 1 0 active sync /dev/sda1
1 1 8 17 1 active sync /dev/sdb1
--
/dev/sdb2:
Magic : a92b4efc
Version : 00.90.00
UUID : 1534f5ef:dfed0a34:776c2c25:004bd7b2
Creation Time : Wed Oct 7 01:54:31 2009
Raid Level : raid1
Used Dev Size : 67111424 (64.00 GiB 68.72 GB)
Array Size : 67111424 (64.00 GiB 68.72 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 1
Update Time : Sat Aug 7 21:13:51 2010
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Checksum : b12ee773 - correct
Events : 78
Number Major Minor RaidDevice State
this 1 8 18 1 active sync /dev/sdb2
0 0 8 2 0 active sync /dev/sda2
1 1 8 18 1 active sync /dev/sdb2
--
/dev/sdb3:
Magic : a92b4efc
Version : 00.90.00
UUID : 41008bdc:74a94d83:776c2c25:004bd7b2
Creation Time : Wed Oct 7 01:54:31 2009
Raid Level : raid1
Used Dev Size : 104864192 (100.01 GiB 107.38 GB)
Array Size : 104864192 (100.01 GiB 107.38 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 2
Update Time : Sat Aug 7 21:13:50 2010
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Checksum : 73f6d072 - correct
Events : 78
Number Major Minor RaidDevice State
this 1 8 19 1 active sync /dev/sdb3
0 0 8 3 0 active sync /dev/sda3
1 1 8 19 1 active sync /dev/sdb3
--
/dev/sdb4:
Magic : a92b4efc
Version : 00.90.00
UUID : d3e0bd33:45396281:ac3f7ccc:bdf0e125
Creation Time : Sun Nov 1 01:21:45 2009
Raid Level : raid1
Used Dev Size : 560331008 (534.37 GiB 573.78 GB)
Array Size : 560331008 (534.37 GiB 573.78 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 3
Update Time : Sat Aug 7 20:17:40 2010
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Checksum : e5263e96 - correct
Events : 42
Number Major Minor RaidDevice State
this 1 8 20 1 active sync /dev/sdb4
0 0 8 4 0 active sync /dev/sda4
1 1 8 20 1 active sync /dev/sdb4
--
--- /proc/cmdline
root=/dev/mapper/vg00-lvol01 ro console=tty0
--- grub legacy:
module /vmlinuz-2.6.26-2-xen-amd64
root=/dev/mapper/vg00-lvol01 ro console=tty0
kernel /vmlinuz-2.6.26-2-xen-amd64 root=/dev/mapper/vg00-lvol01 ro
kernel /vmlinuz-2.6.26-2-xen-amd64
root=/dev/mapper/vg00-lvol01 ro single
kernel /vmlinuz-2.6.26-2-amd64 root=/dev/mapper/vg00-lvol01 ro
kernel /vmlinuz-2.6.26-2-amd64 root=/dev/mapper/vg00-lvol01 ro single
--- udev:
ii udev 0.125-7+lenny3 /dev/ and hotplug management daemon
cd6f5683974ea65603f04ec699b3cff2 /etc/udev/rules.d/65_mdadm.vol_id.rules
--- /dev:
brw-rw---- 1 root disk 9, 0 2010-03-02 21:50 /dev/md0
brw-rw---- 1 root disk 9, 1 2010-03-02 21:50 /dev/md1
brw-rw---- 1 root disk 9, 2 2010-03-02 21:50 /dev/md2
brw-rw---- 1 root disk 9, 3 2010-03-02 21:50 /dev/md3
/dev/disk/by-id:
total 0
lrwxrwxrwx 1 root root 9 2010-03-02 21:50
ata-SAMSUNG_HD753LJ_S13UJ90S506353 -> ../../sda
lrwxrwxrwx 1 root root 10 2010-03-02 21:50
ata-SAMSUNG_HD753LJ_S13UJ90S506353-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 2010-03-02 21:50
ata-SAMSUNG_HD753LJ_S13UJ90S506353-part2 -> ../../sda2
lrwxrwxrwx 1 root root 10 2010-03-02 21:50
ata-SAMSUNG_HD753LJ_S13UJ90S506353-part3 -> ../../sda3
lrwxrwxrwx 1 root root 10 2010-03-02 21:50
ata-SAMSUNG_HD753LJ_S13UJ90S506353-part4 -> ../../sda4
lrwxrwxrwx 1 root root 9 2010-03-02 21:50
ata-SAMSUNG_HD753LJ_S13UJ90S506356 -> ../../sdb
lrwxrwxrwx 1 root root 10 2010-03-02 21:50
ata-SAMSUNG_HD753LJ_S13UJ90S506356-part1 -> ../../sdb1
lrwxrwxrwx 1 root root 10 2010-03-02 21:50
ata-SAMSUNG_HD753LJ_S13UJ90S506356-part2 -> ../../sdb2
lrwxrwxrwx 1 root root 10 2010-03-02 21:50
ata-SAMSUNG_HD753LJ_S13UJ90S506356-part3 -> ../../sdb3
lrwxrwxrwx 1 root root 10 2010-03-02 21:50
ata-SAMSUNG_HD753LJ_S13UJ90S506356-part4 -> ../../sdb4
lrwxrwxrwx 1 root root 9 2010-03-02 21:50
md-uuid-1534f5ef:dfed0a34:776c2c25:004bd7b2 -> ../../md1
lrwxrwxrwx 1 root root 9 2010-03-02 21:50
md-uuid-41008bdc:74a94d83:776c2c25:004bd7b2 -> ../../md2
lrwxrwxrwx 1 root root 9 2010-03-02 21:50
md-uuid-8338c8ba:565b0a83:776c2c25:004bd7b2 -> ../../md0
lrwxrwxrwx 1 root root 9 2010-03-02 21:50
md-uuid-d3e0bd33:45396281:ac3f7ccc:bdf0e125 -> ../../md3
lrwxrwxrwx 1 root root 9 2010-03-02 21:50
scsi-SATA_SAMSUNG_HD753LJS13UJ90S506353 -> ../../sda
lrwxrwxrwx 1 root root 10 2010-03-02 21:50
scsi-SATA_SAMSUNG_HD753LJS13UJ90S506353-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 2010-03-02 21:50
scsi-SATA_SAMSUNG_HD753LJS13UJ90S506353-part2 -> ../../sda2
lrwxrwxrwx 1 root root 10 2010-03-02 21:50
scsi-SATA_SAMSUNG_HD753LJS13UJ90S506353-part3 -> ../../sda3
lrwxrwxrwx 1 root root 10 2010-03-02 21:50
scsi-SATA_SAMSUNG_HD753LJS13UJ90S506353-part4 -> ../../sda4
lrwxrwxrwx 1 root root 9 2010-03-02 21:50
scsi-SATA_SAMSUNG_HD753LJS13UJ90S506356 -> ../../sdb
lrwxrwxrwx 1 root root 10 2010-03-02 21:50
scsi-SATA_SAMSUNG_HD753LJS13UJ90S506356-part1 -> ../../sdb1
lrwxrwxrwx 1 root root 10 2010-03-02 21:50
scsi-SATA_SAMSUNG_HD753LJS13UJ90S506356-part2 -> ../../sdb2
lrwxrwxrwx 1 root root 10 2010-03-02 21:50
scsi-SATA_SAMSUNG_HD753LJS13UJ90S506356-part3 -> ../../sdb3
lrwxrwxrwx 1 root root 10 2010-03-02 21:50
scsi-SATA_SAMSUNG_HD753LJS13UJ90S506356-part4 -> ../../sdb4
/dev/disk/by-path:
total 0
lrwxrwxrwx 1 root root 9 2010-03-02 21:50
pci-0000:00:1f.2-scsi-0:0:0:0 -> ../../sda
lrwxrwxrwx 1 root root 10 2010-03-02 21:50
pci-0000:00:1f.2-scsi-0:0:0:0-part1 -> ../../sda1
lrwxrwxrwx 1 root root 10 2010-03-02 21:50
pci-0000:00:1f.2-scsi-0:0:0:0-part2 -> ../../sda2
lrwxrwxrwx 1 root root 10 2010-03-02 21:50
pci-0000:00:1f.2-scsi-0:0:0:0-part3 -> ../../sda3
lrwxrwxrwx 1 root root 10 2010-03-02 21:50
pci-0000:00:1f.2-scsi-0:0:0:0-part4 -> ../../sda4
lrwxrwxrwx 1 root root 9 2010-03-02 21:50
pci-0000:00:1f.2-scsi-1:0:0:0 -> ../../sdb
lrwxrwxrwx 1 root root 10 2010-03-02 21:50
pci-0000:00:1f.2-scsi-1:0:0:0-part1 -> ../../sdb1
lrwxrwxrwx 1 root root 10 2010-03-02 21:50
pci-0000:00:1f.2-scsi-1:0:0:0-part2 -> ../../sdb2
lrwxrwxrwx 1 root root 10 2010-03-02 21:50
pci-0000:00:1f.2-scsi-1:0:0:0-part3 -> ../../sdb3
lrwxrwxrwx 1 root root 10 2010-03-02 21:50
pci-0000:00:1f.2-scsi-1:0:0:0-part4 -> ../../sdb4
/dev/disk/by-uuid:
total 0
lrwxrwxrwx 1 root root 9 2010-03-02 21:50
0063cac1-5c5a-461d-9956-58d88bdbd6d6 -> ../../md0
/dev/md:
total 0
-- System Information:
Debian Release: 5.0.4
APT prefers stable
APT policy: (500, 'stable')
Architecture: amd64 (x86_64)
Kernel: Linux 2.6.26-2-xen-amd64 (SMP w/8 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash
Versions of packages mdadm depends on:
ii debconf 1.5.24 Debian configuration management sy
ii libc6 2.7-18lenny2 GNU C Library: Shared libraries
ii lsb-base 3.2-20 Linux Standard Base 3.2 init scrip
ii makedev 2.3.1-88 creates device files in /dev
ii udev 0.125-7+lenny3 /dev/ and hotplug management daemo
Versions of packages mdadm recommends:
pn exim4 | mail-transport-agent <none> (no description available)
ii module-init-tools 3.4-1 tools for managing Linux kernel mo
mdadm suggests no packages.
-- debconf information:
mdadm/autostart: true
mdadm/mail_to: root
mdadm/initrdstart_msg_errmd:
* mdadm/initrdstart: all
mdadm/initrdstart_msg_errconf:
mdadm/initrdstart_notinconf: false
mdadm/initrdstart_msg_errexist:
mdadm/initrdstart_msg_intro:
mdadm/autocheck: false
mdadm/initrdstart_msg_errblock:
mdadm/start_daemon: true
--
'Sie brauchen sich um Ihre Zukunft keine Gedanken zu machen'
More information about the pkg-mdadm-devel
mailing list