Bug#515815: mdadm: With 3ware 9000With 3ware 9500S-12, initramfs assembles before devices settled

Anthony DeRobertis anthony at derobert.net
Tue Feb 17 19:27:00 UTC 2009


Package: mdadm
Version: 2.6.7.2-1
Severity: important

With a 3ware 9500S-12, with all the disks configured as single-disk
arrays (e.g., configured to use software RAID instead of the hardware
RAID), an unfortunate sequence of events occurs:

1. 3ware kernel module loaded
2. /dev/sd* start being created by udev
3. mdadm tries to assemble the arrays; fails.
4. /dev/sd* finish being created
5. boot fails (because the arrays were not assembled)

Someones --- its a race condition after all --- the array will be
assembled, but degraded.

udevsettle is not enough to make sure all devices needed are present. At
least not with 3ware hardware. (And, I'd guess, with USB and Firewire as
well.)

I solved this on my machine with a
/etc/initramfs-tools/scripts/init-premount/local_3ware that loads the
3ware module then sleeps 10s. (Attached)

I'm not sure what the real solution is; but for now maybe something
like:

1. Attempt to assemble arrays.
2. If all arrays (needed for boot) have all members present, run them.
   Continue with boot.
3. Otherwise, wait 10s and attempt assembly again.
4. If all arrays (needed for boot) are runnable --- even if degraded ---
   run them.
5. Otherwise, go back to (3).


-- Package-specific info:
--- mount output
/dev/mapper/Kero-root on / type ext3 (rw,noatime,user_xattr,errors=remount-ro,commit=300)
tmpfs on /lib/init/rw type tmpfs (rw,nosuid,mode=0755)
proc on /proc type proc (rw,noexec,nosuid,nodev)
sysfs on /sys type sysfs (rw,noexec,nosuid,nodev)
procbususb on /proc/bus/usb type usbfs (rw)
udev on /dev type tmpfs (rw,mode=0755)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=620)
/dev/mapper/Kero-srv on /srv type ext4 (rw,noatime,user_xattr,errors=remount-ro,commit=300,stripe=576,barrier=0,commit=300)
/dev/md0 on /boot type ext2 (rw,sync,noatime)
/dev/mapper/Kero-var on /var type ext3 (rw,noatime,user_xattr,errors=remount-ro,commit=300,commit=300)
nfsd on /proc/fs/nfsd type nfsd (rw)

--- mdadm.conf
# mdadm.conf
#
# Please refer to mdadm.conf(5) for information about this file.
#

# by default, scan all partitions (/proc/partitions) for MD superblocks.
# alternatively, specify devices to scan, using wildcards if desired.
DEVICE partitions

# auto-create devices with Debian standard permissions
CREATE owner=root group=disk mode=0660 auto=yes

# automatically tag new arrays as belonging to the local system
HOMEHOST <system>

# instruct the monitoring daemon where to send mail alerts
MAILADDR root

# definitions of existing MD arrays
ARRAY /dev/md0 level=raid1 num-devices=11 UUID=77b40b96:8b6c84dc:a60ecb55:bfd75636
   spares=1
ARRAY /dev/md1 level=raid10 num-devices=11 UUID=5dbce27d:fa98a252:4bb2c254:bf75a219
   spares=1
ARRAY /dev/md2 level=raid6 metadata=1.0 num-devices=11 UUID=0ee820db:5b2662a2:79cae9a9:cd057e88 name=2

# This file was auto-generated on Sat, 07 Feb 2009 17:04:55 -0500
# by mkconf $Id$

--- /proc/mdstat:
Personalities : [raid6] [raid5] [raid4] [raid1] [raid10] 
md1 : active raid10 sda2[0] sdl2[11](S) sdk2[10] sdj2[9] sdi2[8] sdh2[7] sdg2[6] sdf2[5] sde2[4] sdd2[3] sdc2[2] sdb2[1]
      2738560 blocks 64K chunks 2 near-copies [11/11] [UUUUUUUUUUU]
      bitmap: 0/168 pages [0KB], 8KB chunk

md0 : active raid1 sdj1[0] sdl1[11](S) sdk1[10] sdc1[9] sdi1[8] sdh1[7] sdg1[6] sdf1[5] sde1[4] sdd1[3] sdb1[2] sda1[1]
      128384 blocks [11/11] [UUUUUUUUUUU]
      bitmap: 0/16 pages [0KB], 4KB chunk

md2 : active raid6 sda3[0] sdl3[11](S) sdk3[10] sdj3[9] sdi3[8] sdh3[7] sdg3[6] sdf3[5] sde3[4] sdd3[3] sdc3[2] sdb3[1]
      8783320320 blocks super 1.0 level 6, 256k chunk, algorithm 2 [11/11] [UUUUUUUUUUU]
      bitmap: 0/466 pages [0KB], 1024KB chunk

unused devices: <none>

--- /proc/partitions:
major minor  #blocks  name

   8        0  976551936 sda
   8        1     128488 sda1
   8        2     498015 sda2
   8        3  975924652 sda3
   8       16  976551936 sdb
   8       17     128488 sdb1
   8       18     498015 sdb2
   8       19  975924652 sdb3
   8       32  976551936 sdc
   8       33     128488 sdc1
   8       34     498015 sdc2
   8       35  975924652 sdc3
   8       48  976551936 sdd
   8       49     128488 sdd1
   8       50     498015 sdd2
   8       51  975924652 sdd3
   8       64  976551936 sde
   8       65     128488 sde1
   8       66     498015 sde2
   8       67  975924652 sde3
   8       80  976551936 sdf
   8       81     128488 sdf1
   8       82     498015 sdf2
   8       83  975924652 sdf3
   8       96  976551936 sdg
   8       97     128488 sdg1
   8       98     498015 sdg2
   8       99  975924652 sdg3
   8      112  976551936 sdh
   8      113     128488 sdh1
   8      114     498015 sdh2
   8      115  975924652 sdh3
   8      128  976551936 sdi
   8      129     128488 sdi1
   8      130     498015 sdi2
   8      131  975924652 sdi3
   8      144  976551936 sdj
   8      145     128488 sdj1
   8      146     498015 sdj2
   8      147  975924652 sdj3
   8      160  976551936 sdk
   8      161     128488 sdk1
   8      162     498015 sdk2
   8      163  975924652 sdk3
   8      176  976551936 sdl
   8      177     128488 sdl1
   8      178     498015 sdl2
   8      179  975924652 sdl3
   9        2 8783320320 md2
 253        0    9764864 dm-0
 253        1    3903488 dm-1
 253        2 8697311232 dm-2
   9        0     128384 md0
   9        1    2738560 md1

--- initrd.img-2.6.28.4:
45540 blocks
etc/mdadm
etc/mdadm/mdadm.conf
lib/modules/2.6.28.4/kernel/drivers/md/dm-region-hash.ko
lib/modules/2.6.28.4/kernel/drivers/md/multipath.ko
lib/modules/2.6.28.4/kernel/drivers/md/raid1.ko
lib/modules/2.6.28.4/kernel/drivers/md/raid456.ko
lib/modules/2.6.28.4/kernel/drivers/md/dm-snapshot.ko
lib/modules/2.6.28.4/kernel/drivers/md/dm-log.ko
lib/modules/2.6.28.4/kernel/drivers/md/raid10.ko
lib/modules/2.6.28.4/kernel/drivers/md/dm-mod.ko
lib/modules/2.6.28.4/kernel/drivers/md/md-mod.ko
lib/modules/2.6.28.4/kernel/drivers/md/dm-mirror.ko
lib/modules/2.6.28.4/kernel/drivers/md/linear.ko
lib/modules/2.6.28.4/kernel/drivers/md/raid0.ko
scripts/local-top/mdadm
sbin/mdadm

--- /proc/modules:
raid10 24320 1 - Live 0xffffffffa02cb000
raid1 24960 1 - Live 0xffffffffa02bf000
dm_mirror 18048 0 - Live 0xffffffffa0199000
dm_region_hash 16768 1 dm_mirror, Live 0xffffffffa0192000
dm_log 14084 2 dm_mirror,dm_region_hash, Live 0xffffffffa0189000
dm_snapshot 20552 0 - Live 0xffffffffa017e000
dm_mod 59744 10 dm_mirror,dm_log,dm_snapshot, Live 0xffffffffa016a000
raid456 126496 1 - Live 0xffffffffa0146000
md_mod 84388 6 raid10,raid1,raid456, Live 0xffffffffa012c000
async_xor 8064 1 raid456, Live 0xffffffffa0125000
async_memcpy 6528 1 raid456, Live 0xffffffffa011e000
async_tx 11884 3 raid456,async_xor,async_memcpy, Live 0xffffffffa0116000
xor 9616 2 raid456,async_xor, Live 0xffffffffa010e000

--- /var/log/syslog:

--- volume detail:
/dev/sda1:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : 77b40b96:8b6c84dc:a60ecb55:bfd75636
  Creation Time : Sat Feb  7 15:52:41 2009
     Raid Level : raid1
  Used Dev Size : 128384 (125.40 MiB 131.47 MB)
     Array Size : 128384 (125.40 MiB 131.47 MB)
   Raid Devices : 11
  Total Devices : 12
Preferred Minor : 0

    Update Time : Thu Feb 12 19:24:02 2009
          State : clean
Internal Bitmap : present
 Active Devices : 11
Working Devices : 12
 Failed Devices : 0
  Spare Devices : 1
       Checksum : a556b0b9 - correct
         Events : 138


      Number   Major   Minor   RaidDevice State
this     1       8        1        1      active sync   /dev/sda1

   0     0       8      145        0      active sync   /dev/sdj1
   1     1       8        1        1      active sync   /dev/sda1
   2     2       8       17        2      active sync   /dev/sdb1
   3     3       8       49        3      active sync   /dev/sdd1
   4     4       8       65        4      active sync   /dev/sde1
   5     5       8       81        5      active sync   /dev/sdf1
   6     6       8       97        6      active sync   /dev/sdg1
   7     7       8      113        7      active sync   /dev/sdh1
   8     8       8      129        8      active sync   /dev/sdi1
   9     9       8       33        9      active sync   /dev/sdc1
  10    10       8      161       10      active sync   /dev/sdk1
  11    11       8      177       11      spare   /dev/sdl1
--
/dev/sda2:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : 5dbce27d:fa98a252:4bb2c254:bf75a219
  Creation Time : Sat Feb  7 15:53:36 2009
     Raid Level : raid10
  Used Dev Size : 497920 (486.33 MiB 509.87 MB)
     Array Size : 2489600 (2.37 GiB 2.55 GB)
   Raid Devices : 11
  Total Devices : 12
Preferred Minor : 1

    Update Time : Thu Feb 12 15:54:19 2009
          State : clean
Internal Bitmap : present
 Active Devices : 11
Working Devices : 12
 Failed Devices : 0
  Spare Devices : 1
       Checksum : 9fd45aa3 - correct
         Events : 12

         Layout : near=2, far=1
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     0       8        2        0      active sync   /dev/sda2

   0     0       8        2        0      active sync   /dev/sda2
   1     1       8       18        1      active sync   /dev/sdb2
   2     2       8       34        2      active sync   /dev/sdc2
   3     3       8       50        3      active sync   /dev/sdd2
   4     4       8       66        4      active sync   /dev/sde2
   5     5       8       82        5      active sync   /dev/sdf2
   6     6       8       98        6      active sync   /dev/sdg2
   7     7       8      114        7      active sync   /dev/sdh2
   8     8       8      130        8      active sync   /dev/sdi2
   9     9       8      146        9      active sync   /dev/sdj2
  10    10       8      162       10      active sync   /dev/sdk2
  11    11       8      178       11      spare   /dev/sdl2
--
/dev/sda3:
          Magic : a92b4efc
        Version : 1.0
    Feature Map : 0x1
     Array UUID : 0ee820db:5b2662a2:79cae9a9:cd057e88
           Name : 2
  Creation Time : Sat Feb  7 16:15:49 2009
     Raid Level : raid6
   Raid Devices : 11

 Avail Dev Size : 1951849032 (930.71 GiB 999.35 GB)
     Array Size : 17566640640 (8376.43 GiB 8994.12 GB)
  Used Dev Size : 1951848960 (930.71 GiB 999.35 GB)
   Super Offset : 1951849288 sectors
          State : clean
    Device UUID : a050580d:67591b04:fa1a315e:97d47686

Internal Bitmap : -234 sectors from superblock
    Update Time : Tue Feb 17 14:11:32 2009
       Checksum : 3f775fc5 - correct
         Events : 6

     Chunk Size : 256K

    Array Slot : 0 (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
   Array State : Uuuuuuuuuuu
--
/dev/sdb1:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : 77b40b96:8b6c84dc:a60ecb55:bfd75636
  Creation Time : Sat Feb  7 15:52:41 2009
     Raid Level : raid1
  Used Dev Size : 128384 (125.40 MiB 131.47 MB)
     Array Size : 128384 (125.40 MiB 131.47 MB)
   Raid Devices : 11
  Total Devices : 12
Preferred Minor : 0

    Update Time : Thu Feb 12 19:24:02 2009
          State : clean
Internal Bitmap : present
 Active Devices : 11
Working Devices : 12
 Failed Devices : 0
  Spare Devices : 1
       Checksum : a556b0cb - correct
         Events : 138


      Number   Major   Minor   RaidDevice State
this     2       8       17        2      active sync   /dev/sdb1

   0     0       8      145        0      active sync   /dev/sdj1
   1     1       8        1        1      active sync   /dev/sda1
   2     2       8       17        2      active sync   /dev/sdb1
   3     3       8       49        3      active sync   /dev/sdd1
   4     4       8       65        4      active sync   /dev/sde1
   5     5       8       81        5      active sync   /dev/sdf1
   6     6       8       97        6      active sync   /dev/sdg1
   7     7       8      113        7      active sync   /dev/sdh1
   8     8       8      129        8      active sync   /dev/sdi1
   9     9       8       33        9      active sync   /dev/sdc1
  10    10       8      161       10      active sync   /dev/sdk1
  11    11       8      177       11      spare   /dev/sdl1
--
/dev/sdb2:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : 5dbce27d:fa98a252:4bb2c254:bf75a219
  Creation Time : Sat Feb  7 15:53:36 2009
     Raid Level : raid10
  Used Dev Size : 497920 (486.33 MiB 509.87 MB)
     Array Size : 2489600 (2.37 GiB 2.55 GB)
   Raid Devices : 11
  Total Devices : 12
Preferred Minor : 1

    Update Time : Thu Feb 12 15:54:19 2009
          State : clean
Internal Bitmap : present
 Active Devices : 11
Working Devices : 12
 Failed Devices : 0
  Spare Devices : 1
       Checksum : 9fd45ab5 - correct
         Events : 12

         Layout : near=2, far=1
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     1       8       18        1      active sync   /dev/sdb2

   0     0       8        2        0      active sync   /dev/sda2
   1     1       8       18        1      active sync   /dev/sdb2
   2     2       8       34        2      active sync   /dev/sdc2
   3     3       8       50        3      active sync   /dev/sdd2
   4     4       8       66        4      active sync   /dev/sde2
   5     5       8       82        5      active sync   /dev/sdf2
   6     6       8       98        6      active sync   /dev/sdg2
   7     7       8      114        7      active sync   /dev/sdh2
   8     8       8      130        8      active sync   /dev/sdi2
   9     9       8      146        9      active sync   /dev/sdj2
  10    10       8      162       10      active sync   /dev/sdk2
  11    11       8      178       11      spare   /dev/sdl2
--
/dev/sdb3:
          Magic : a92b4efc
        Version : 1.0
    Feature Map : 0x1
     Array UUID : 0ee820db:5b2662a2:79cae9a9:cd057e88
           Name : 2
  Creation Time : Sat Feb  7 16:15:49 2009
     Raid Level : raid6
   Raid Devices : 11

 Avail Dev Size : 1951849032 (930.71 GiB 999.35 GB)
     Array Size : 17566640640 (8376.43 GiB 8994.12 GB)
  Used Dev Size : 1951848960 (930.71 GiB 999.35 GB)
   Super Offset : 1951849288 sectors
          State : clean
    Device UUID : 56fd3db6:23282774:938b921c:a9268c1e

Internal Bitmap : -234 sectors from superblock
    Update Time : Tue Feb 17 14:11:32 2009
       Checksum : aedf9de3 - correct
         Events : 6

     Chunk Size : 256K

    Array Slot : 1 (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
   Array State : uUuuuuuuuuu
--
/dev/sdc1:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : 77b40b96:8b6c84dc:a60ecb55:bfd75636
  Creation Time : Sat Feb  7 15:52:41 2009
     Raid Level : raid1
  Used Dev Size : 128384 (125.40 MiB 131.47 MB)
     Array Size : 128384 (125.40 MiB 131.47 MB)
   Raid Devices : 11
  Total Devices : 12
Preferred Minor : 0

    Update Time : Thu Feb 12 19:24:02 2009
          State : clean
Internal Bitmap : present
 Active Devices : 11
Working Devices : 12
 Failed Devices : 0
  Spare Devices : 1
       Checksum : a556b0e9 - correct
         Events : 138


      Number   Major   Minor   RaidDevice State
this     9       8       33        9      active sync   /dev/sdc1

   0     0       8      145        0      active sync   /dev/sdj1
   1     1       8        1        1      active sync   /dev/sda1
   2     2       8       17        2      active sync   /dev/sdb1
   3     3       8       49        3      active sync   /dev/sdd1
   4     4       8       65        4      active sync   /dev/sde1
   5     5       8       81        5      active sync   /dev/sdf1
   6     6       8       97        6      active sync   /dev/sdg1
   7     7       8      113        7      active sync   /dev/sdh1
   8     8       8      129        8      active sync   /dev/sdi1
   9     9       8       33        9      active sync   /dev/sdc1
  10    10       8      161       10      active sync   /dev/sdk1
  11    11       8      177       11      spare   /dev/sdl1
--
/dev/sdc2:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : 5dbce27d:fa98a252:4bb2c254:bf75a219
  Creation Time : Sat Feb  7 15:53:36 2009
     Raid Level : raid10
  Used Dev Size : 497920 (486.33 MiB 509.87 MB)
     Array Size : 2489600 (2.37 GiB 2.55 GB)
   Raid Devices : 11
  Total Devices : 12
Preferred Minor : 1

    Update Time : Thu Feb 12 15:54:19 2009
          State : clean
Internal Bitmap : present
 Active Devices : 11
Working Devices : 12
 Failed Devices : 0
  Spare Devices : 1
       Checksum : 9fd45ac7 - correct
         Events : 12

         Layout : near=2, far=1
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     2       8       34        2      active sync   /dev/sdc2

   0     0       8        2        0      active sync   /dev/sda2
   1     1       8       18        1      active sync   /dev/sdb2
   2     2       8       34        2      active sync   /dev/sdc2
   3     3       8       50        3      active sync   /dev/sdd2
   4     4       8       66        4      active sync   /dev/sde2
   5     5       8       82        5      active sync   /dev/sdf2
   6     6       8       98        6      active sync   /dev/sdg2
   7     7       8      114        7      active sync   /dev/sdh2
   8     8       8      130        8      active sync   /dev/sdi2
   9     9       8      146        9      active sync   /dev/sdj2
  10    10       8      162       10      active sync   /dev/sdk2
  11    11       8      178       11      spare   /dev/sdl2
--
/dev/sdc3:
          Magic : a92b4efc
        Version : 1.0
    Feature Map : 0x1
     Array UUID : 0ee820db:5b2662a2:79cae9a9:cd057e88
           Name : 2
  Creation Time : Sat Feb  7 16:15:49 2009
     Raid Level : raid6
   Raid Devices : 11

 Avail Dev Size : 1951849032 (930.71 GiB 999.35 GB)
     Array Size : 17566640640 (8376.43 GiB 8994.12 GB)
  Used Dev Size : 1951848960 (930.71 GiB 999.35 GB)
   Super Offset : 1951849288 sectors
          State : clean
    Device UUID : 334f9740:d916ba8d:8e2ba820:186db8bf

Internal Bitmap : -234 sectors from superblock
    Update Time : Tue Feb 17 14:11:32 2009
       Checksum : f80dc4e1 - correct
         Events : 6

     Chunk Size : 256K

    Array Slot : 2 (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
   Array State : uuUuuuuuuuu
--
/dev/sdd1:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : 77b40b96:8b6c84dc:a60ecb55:bfd75636
  Creation Time : Sat Feb  7 15:52:41 2009
     Raid Level : raid1
  Used Dev Size : 128384 (125.40 MiB 131.47 MB)
     Array Size : 128384 (125.40 MiB 131.47 MB)
   Raid Devices : 11
  Total Devices : 12
Preferred Minor : 0

    Update Time : Thu Feb 12 19:24:02 2009
          State : clean
Internal Bitmap : present
 Active Devices : 11
Working Devices : 12
 Failed Devices : 0
  Spare Devices : 1
       Checksum : a556b0ed - correct
         Events : 138


      Number   Major   Minor   RaidDevice State
this     3       8       49        3      active sync   /dev/sdd1

   0     0       8      145        0      active sync   /dev/sdj1
   1     1       8        1        1      active sync   /dev/sda1
   2     2       8       17        2      active sync   /dev/sdb1
   3     3       8       49        3      active sync   /dev/sdd1
   4     4       8       65        4      active sync   /dev/sde1
   5     5       8       81        5      active sync   /dev/sdf1
   6     6       8       97        6      active sync   /dev/sdg1
   7     7       8      113        7      active sync   /dev/sdh1
   8     8       8      129        8      active sync   /dev/sdi1
   9     9       8       33        9      active sync   /dev/sdc1
  10    10       8      161       10      active sync   /dev/sdk1
  11    11       8      177       11      spare   /dev/sdl1
--
/dev/sdd2:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : 5dbce27d:fa98a252:4bb2c254:bf75a219
  Creation Time : Sat Feb  7 15:53:36 2009
     Raid Level : raid10
  Used Dev Size : 497920 (486.33 MiB 509.87 MB)
     Array Size : 2489600 (2.37 GiB 2.55 GB)
   Raid Devices : 11
  Total Devices : 12
Preferred Minor : 1

    Update Time : Thu Feb 12 15:54:19 2009
          State : clean
Internal Bitmap : present
 Active Devices : 11
Working Devices : 12
 Failed Devices : 0
  Spare Devices : 1
       Checksum : 9fd45ad9 - correct
         Events : 12

         Layout : near=2, far=1
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     3       8       50        3      active sync   /dev/sdd2

   0     0       8        2        0      active sync   /dev/sda2
   1     1       8       18        1      active sync   /dev/sdb2
   2     2       8       34        2      active sync   /dev/sdc2
   3     3       8       50        3      active sync   /dev/sdd2
   4     4       8       66        4      active sync   /dev/sde2
   5     5       8       82        5      active sync   /dev/sdf2
   6     6       8       98        6      active sync   /dev/sdg2
   7     7       8      114        7      active sync   /dev/sdh2
   8     8       8      130        8      active sync   /dev/sdi2
   9     9       8      146        9      active sync   /dev/sdj2
  10    10       8      162       10      active sync   /dev/sdk2
  11    11       8      178       11      spare   /dev/sdl2
--
/dev/sdd3:
          Magic : a92b4efc
        Version : 1.0
    Feature Map : 0x1
     Array UUID : 0ee820db:5b2662a2:79cae9a9:cd057e88
           Name : 2
  Creation Time : Sat Feb  7 16:15:49 2009
     Raid Level : raid6
   Raid Devices : 11

 Avail Dev Size : 1951849032 (930.71 GiB 999.35 GB)
     Array Size : 17566640640 (8376.43 GiB 8994.12 GB)
  Used Dev Size : 1951848960 (930.71 GiB 999.35 GB)
   Super Offset : 1951849288 sectors
          State : clean
    Device UUID : 36428a47:fa6b19a9:fa849642:8af5eee2

Internal Bitmap : -234 sectors from superblock
    Update Time : Tue Feb 17 14:11:32 2009
       Checksum : 5f84eee5 - correct
         Events : 6

     Chunk Size : 256K

    Array Slot : 3 (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
   Array State : uuuUuuuuuuu
--
/dev/sde1:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : 77b40b96:8b6c84dc:a60ecb55:bfd75636
  Creation Time : Sat Feb  7 15:52:41 2009
     Raid Level : raid1
  Used Dev Size : 128384 (125.40 MiB 131.47 MB)
     Array Size : 128384 (125.40 MiB 131.47 MB)
   Raid Devices : 11
  Total Devices : 12
Preferred Minor : 0

    Update Time : Thu Feb 12 19:24:02 2009
          State : clean
Internal Bitmap : present
 Active Devices : 11
Working Devices : 12
 Failed Devices : 0
  Spare Devices : 1
       Checksum : a556b0ff - correct
         Events : 138


      Number   Major   Minor   RaidDevice State
this     4       8       65        4      active sync   /dev/sde1

   0     0       8      145        0      active sync   /dev/sdj1
   1     1       8        1        1      active sync   /dev/sda1
   2     2       8       17        2      active sync   /dev/sdb1
   3     3       8       49        3      active sync   /dev/sdd1
   4     4       8       65        4      active sync   /dev/sde1
   5     5       8       81        5      active sync   /dev/sdf1
   6     6       8       97        6      active sync   /dev/sdg1
   7     7       8      113        7      active sync   /dev/sdh1
   8     8       8      129        8      active sync   /dev/sdi1
   9     9       8       33        9      active sync   /dev/sdc1
  10    10       8      161       10      active sync   /dev/sdk1
  11    11       8      177       11      spare   /dev/sdl1
--
/dev/sde2:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : 5dbce27d:fa98a252:4bb2c254:bf75a219
  Creation Time : Sat Feb  7 15:53:36 2009
     Raid Level : raid10
  Used Dev Size : 497920 (486.33 MiB 509.87 MB)
     Array Size : 2489600 (2.37 GiB 2.55 GB)
   Raid Devices : 11
  Total Devices : 12
Preferred Minor : 1

    Update Time : Thu Feb 12 15:54:19 2009
          State : clean
Internal Bitmap : present
 Active Devices : 11
Working Devices : 12
 Failed Devices : 0
  Spare Devices : 1
       Checksum : 9fd45aeb - correct
         Events : 12

         Layout : near=2, far=1
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     4       8       66        4      active sync   /dev/sde2

   0     0       8        2        0      active sync   /dev/sda2
   1     1       8       18        1      active sync   /dev/sdb2
   2     2       8       34        2      active sync   /dev/sdc2
   3     3       8       50        3      active sync   /dev/sdd2
   4     4       8       66        4      active sync   /dev/sde2
   5     5       8       82        5      active sync   /dev/sdf2
   6     6       8       98        6      active sync   /dev/sdg2
   7     7       8      114        7      active sync   /dev/sdh2
   8     8       8      130        8      active sync   /dev/sdi2
   9     9       8      146        9      active sync   /dev/sdj2
  10    10       8      162       10      active sync   /dev/sdk2
  11    11       8      178       11      spare   /dev/sdl2
--
/dev/sde3:
          Magic : a92b4efc
        Version : 1.0
    Feature Map : 0x1
     Array UUID : 0ee820db:5b2662a2:79cae9a9:cd057e88
           Name : 2
  Creation Time : Sat Feb  7 16:15:49 2009
     Raid Level : raid6
   Raid Devices : 11

 Avail Dev Size : 1951849032 (930.71 GiB 999.35 GB)
     Array Size : 17566640640 (8376.43 GiB 8994.12 GB)
  Used Dev Size : 1951848960 (930.71 GiB 999.35 GB)
   Super Offset : 1951849288 sectors
          State : clean
    Device UUID : a1f1324d:d5e5deb5:142c7c5e:ca50d4d4

Internal Bitmap : -234 sectors from superblock
    Update Time : Tue Feb 17 14:11:32 2009
       Checksum : 7fbe1a86 - correct
         Events : 6

     Chunk Size : 256K

    Array Slot : 4 (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
   Array State : uuuuUuuuuuu
--
/dev/sdf1:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : 77b40b96:8b6c84dc:a60ecb55:bfd75636
  Creation Time : Sat Feb  7 15:52:41 2009
     Raid Level : raid1
  Used Dev Size : 128384 (125.40 MiB 131.47 MB)
     Array Size : 128384 (125.40 MiB 131.47 MB)
   Raid Devices : 11
  Total Devices : 12
Preferred Minor : 0

    Update Time : Thu Feb 12 19:24:02 2009
          State : clean
Internal Bitmap : present
 Active Devices : 11
Working Devices : 12
 Failed Devices : 0
  Spare Devices : 1
       Checksum : a556b111 - correct
         Events : 138


      Number   Major   Minor   RaidDevice State
this     5       8       81        5      active sync   /dev/sdf1

   0     0       8      145        0      active sync   /dev/sdj1
   1     1       8        1        1      active sync   /dev/sda1
   2     2       8       17        2      active sync   /dev/sdb1
   3     3       8       49        3      active sync   /dev/sdd1
   4     4       8       65        4      active sync   /dev/sde1
   5     5       8       81        5      active sync   /dev/sdf1
   6     6       8       97        6      active sync   /dev/sdg1
   7     7       8      113        7      active sync   /dev/sdh1
   8     8       8      129        8      active sync   /dev/sdi1
   9     9       8       33        9      active sync   /dev/sdc1
  10    10       8      161       10      active sync   /dev/sdk1
  11    11       8      177       11      spare   /dev/sdl1
--
/dev/sdf2:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : 5dbce27d:fa98a252:4bb2c254:bf75a219
  Creation Time : Sat Feb  7 15:53:36 2009
     Raid Level : raid10
  Used Dev Size : 497920 (486.33 MiB 509.87 MB)
     Array Size : 2489600 (2.37 GiB 2.55 GB)
   Raid Devices : 11
  Total Devices : 12
Preferred Minor : 1

    Update Time : Thu Feb 12 15:54:19 2009
          State : clean
Internal Bitmap : present
 Active Devices : 11
Working Devices : 12
 Failed Devices : 0
  Spare Devices : 1
       Checksum : 9fd45afd - correct
         Events : 12

         Layout : near=2, far=1
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     5       8       82        5      active sync   /dev/sdf2

   0     0       8        2        0      active sync   /dev/sda2
   1     1       8       18        1      active sync   /dev/sdb2
   2     2       8       34        2      active sync   /dev/sdc2
   3     3       8       50        3      active sync   /dev/sdd2
   4     4       8       66        4      active sync   /dev/sde2
   5     5       8       82        5      active sync   /dev/sdf2
   6     6       8       98        6      active sync   /dev/sdg2
   7     7       8      114        7      active sync   /dev/sdh2
   8     8       8      130        8      active sync   /dev/sdi2
   9     9       8      146        9      active sync   /dev/sdj2
  10    10       8      162       10      active sync   /dev/sdk2
  11    11       8      178       11      spare   /dev/sdl2
--
/dev/sdf3:
          Magic : a92b4efc
        Version : 1.0
    Feature Map : 0x1
     Array UUID : 0ee820db:5b2662a2:79cae9a9:cd057e88
           Name : 2
  Creation Time : Sat Feb  7 16:15:49 2009
     Raid Level : raid6
   Raid Devices : 11

 Avail Dev Size : 1951849032 (930.71 GiB 999.35 GB)
     Array Size : 17566640640 (8376.43 GiB 8994.12 GB)
  Used Dev Size : 1951848960 (930.71 GiB 999.35 GB)
   Super Offset : 1951849288 sectors
          State : clean
    Device UUID : 5e7ae894:68afbd84:e45dc89f:a5abbaa3

Internal Bitmap : -234 sectors from superblock
    Update Time : Tue Feb 17 14:11:32 2009
       Checksum : a684f982 - correct
         Events : 6

     Chunk Size : 256K

    Array Slot : 5 (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
   Array State : uuuuuUuuuuu
--
/dev/sdg1:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : 77b40b96:8b6c84dc:a60ecb55:bfd75636
  Creation Time : Sat Feb  7 15:52:41 2009
     Raid Level : raid1
  Used Dev Size : 128384 (125.40 MiB 131.47 MB)
     Array Size : 128384 (125.40 MiB 131.47 MB)
   Raid Devices : 11
  Total Devices : 12
Preferred Minor : 0

    Update Time : Thu Feb 12 19:24:02 2009
          State : clean
Internal Bitmap : present
 Active Devices : 11
Working Devices : 12
 Failed Devices : 0
  Spare Devices : 1
       Checksum : a556b123 - correct
         Events : 138


      Number   Major   Minor   RaidDevice State
this     6       8       97        6      active sync   /dev/sdg1

   0     0       8      145        0      active sync   /dev/sdj1
   1     1       8        1        1      active sync   /dev/sda1
   2     2       8       17        2      active sync   /dev/sdb1
   3     3       8       49        3      active sync   /dev/sdd1
   4     4       8       65        4      active sync   /dev/sde1
   5     5       8       81        5      active sync   /dev/sdf1
   6     6       8       97        6      active sync   /dev/sdg1
   7     7       8      113        7      active sync   /dev/sdh1
   8     8       8      129        8      active sync   /dev/sdi1
   9     9       8       33        9      active sync   /dev/sdc1
  10    10       8      161       10      active sync   /dev/sdk1
  11    11       8      177       11      spare   /dev/sdl1
--
/dev/sdg2:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : 5dbce27d:fa98a252:4bb2c254:bf75a219
  Creation Time : Sat Feb  7 15:53:36 2009
     Raid Level : raid10
  Used Dev Size : 497920 (486.33 MiB 509.87 MB)
     Array Size : 2489600 (2.37 GiB 2.55 GB)
   Raid Devices : 11
  Total Devices : 12
Preferred Minor : 1

    Update Time : Thu Feb 12 15:54:19 2009
          State : clean
Internal Bitmap : present
 Active Devices : 11
Working Devices : 12
 Failed Devices : 0
  Spare Devices : 1
       Checksum : 9fd45b0f - correct
         Events : 12

         Layout : near=2, far=1
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     6       8       98        6      active sync   /dev/sdg2

   0     0       8        2        0      active sync   /dev/sda2
   1     1       8       18        1      active sync   /dev/sdb2
   2     2       8       34        2      active sync   /dev/sdc2
   3     3       8       50        3      active sync   /dev/sdd2
   4     4       8       66        4      active sync   /dev/sde2
   5     5       8       82        5      active sync   /dev/sdf2
   6     6       8       98        6      active sync   /dev/sdg2
   7     7       8      114        7      active sync   /dev/sdh2
   8     8       8      130        8      active sync   /dev/sdi2
   9     9       8      146        9      active sync   /dev/sdj2
  10    10       8      162       10      active sync   /dev/sdk2
  11    11       8      178       11      spare   /dev/sdl2
--
/dev/sdg3:
          Magic : a92b4efc
        Version : 1.0
    Feature Map : 0x1
     Array UUID : 0ee820db:5b2662a2:79cae9a9:cd057e88
           Name : 2
  Creation Time : Sat Feb  7 16:15:49 2009
     Raid Level : raid6
   Raid Devices : 11

 Avail Dev Size : 1951849032 (930.71 GiB 999.35 GB)
     Array Size : 17566640640 (8376.43 GiB 8994.12 GB)
  Used Dev Size : 1951848960 (930.71 GiB 999.35 GB)
   Super Offset : 1951849288 sectors
          State : clean
    Device UUID : c1f30943:7dcabc75:4a09a0f8:d0d06aeb

Internal Bitmap : -234 sectors from superblock
    Update Time : Tue Feb 17 14:11:32 2009
       Checksum : e62d5e8c - correct
         Events : 6

     Chunk Size : 256K

    Array Slot : 6 (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
   Array State : uuuuuuUuuuu
--
/dev/sdh1:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : 77b40b96:8b6c84dc:a60ecb55:bfd75636
  Creation Time : Sat Feb  7 15:52:41 2009
     Raid Level : raid1
  Used Dev Size : 128384 (125.40 MiB 131.47 MB)
     Array Size : 128384 (125.40 MiB 131.47 MB)
   Raid Devices : 11
  Total Devices : 12
Preferred Minor : 0

    Update Time : Thu Feb 12 19:24:02 2009
          State : clean
Internal Bitmap : present
 Active Devices : 11
Working Devices : 12
 Failed Devices : 0
  Spare Devices : 1
       Checksum : a556b135 - correct
         Events : 138


      Number   Major   Minor   RaidDevice State
this     7       8      113        7      active sync   /dev/sdh1

   0     0       8      145        0      active sync   /dev/sdj1
   1     1       8        1        1      active sync   /dev/sda1
   2     2       8       17        2      active sync   /dev/sdb1
   3     3       8       49        3      active sync   /dev/sdd1
   4     4       8       65        4      active sync   /dev/sde1
   5     5       8       81        5      active sync   /dev/sdf1
   6     6       8       97        6      active sync   /dev/sdg1
   7     7       8      113        7      active sync   /dev/sdh1
   8     8       8      129        8      active sync   /dev/sdi1
   9     9       8       33        9      active sync   /dev/sdc1
  10    10       8      161       10      active sync   /dev/sdk1
  11    11       8      177       11      spare   /dev/sdl1
--
/dev/sdh2:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : 5dbce27d:fa98a252:4bb2c254:bf75a219
  Creation Time : Sat Feb  7 15:53:36 2009
     Raid Level : raid10
  Used Dev Size : 497920 (486.33 MiB 509.87 MB)
     Array Size : 2489600 (2.37 GiB 2.55 GB)
   Raid Devices : 11
  Total Devices : 12
Preferred Minor : 1

    Update Time : Thu Feb 12 15:54:19 2009
          State : clean
Internal Bitmap : present
 Active Devices : 11
Working Devices : 12
 Failed Devices : 0
  Spare Devices : 1
       Checksum : 9fd45b21 - correct
         Events : 12

         Layout : near=2, far=1
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     7       8      114        7      active sync   /dev/sdh2

   0     0       8        2        0      active sync   /dev/sda2
   1     1       8       18        1      active sync   /dev/sdb2
   2     2       8       34        2      active sync   /dev/sdc2
   3     3       8       50        3      active sync   /dev/sdd2
   4     4       8       66        4      active sync   /dev/sde2
   5     5       8       82        5      active sync   /dev/sdf2
   6     6       8       98        6      active sync   /dev/sdg2
   7     7       8      114        7      active sync   /dev/sdh2
   8     8       8      130        8      active sync   /dev/sdi2
   9     9       8      146        9      active sync   /dev/sdj2
  10    10       8      162       10      active sync   /dev/sdk2
  11    11       8      178       11      spare   /dev/sdl2
--
/dev/sdh3:
          Magic : a92b4efc
        Version : 1.0
    Feature Map : 0x1
     Array UUID : 0ee820db:5b2662a2:79cae9a9:cd057e88
           Name : 2
  Creation Time : Sat Feb  7 16:15:49 2009
     Raid Level : raid6
   Raid Devices : 11

 Avail Dev Size : 1951849032 (930.71 GiB 999.35 GB)
     Array Size : 17566640640 (8376.43 GiB 8994.12 GB)
  Used Dev Size : 1951848960 (930.71 GiB 999.35 GB)
   Super Offset : 1951849288 sectors
          State : clean
    Device UUID : b372c462:d4375559:a812f44c:8963f818

Internal Bitmap : -234 sectors from superblock
    Update Time : Tue Feb 17 14:11:32 2009
       Checksum : 6b61e6ec - correct
         Events : 6

     Chunk Size : 256K

    Array Slot : 7 (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
   Array State : uuuuuuuUuuu
--
/dev/sdi1:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : 77b40b96:8b6c84dc:a60ecb55:bfd75636
  Creation Time : Sat Feb  7 15:52:41 2009
     Raid Level : raid1
  Used Dev Size : 128384 (125.40 MiB 131.47 MB)
     Array Size : 128384 (125.40 MiB 131.47 MB)
   Raid Devices : 11
  Total Devices : 12
Preferred Minor : 0

    Update Time : Thu Feb 12 19:24:02 2009
          State : clean
Internal Bitmap : present
 Active Devices : 11
Working Devices : 12
 Failed Devices : 0
  Spare Devices : 1
       Checksum : a556b147 - correct
         Events : 138


      Number   Major   Minor   RaidDevice State
this     8       8      129        8      active sync   /dev/sdi1

   0     0       8      145        0      active sync   /dev/sdj1
   1     1       8        1        1      active sync   /dev/sda1
   2     2       8       17        2      active sync   /dev/sdb1
   3     3       8       49        3      active sync   /dev/sdd1
   4     4       8       65        4      active sync   /dev/sde1
   5     5       8       81        5      active sync   /dev/sdf1
   6     6       8       97        6      active sync   /dev/sdg1
   7     7       8      113        7      active sync   /dev/sdh1
   8     8       8      129        8      active sync   /dev/sdi1
   9     9       8       33        9      active sync   /dev/sdc1
  10    10       8      161       10      active sync   /dev/sdk1
  11    11       8      177       11      spare   /dev/sdl1
--
/dev/sdi2:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : 5dbce27d:fa98a252:4bb2c254:bf75a219
  Creation Time : Sat Feb  7 15:53:36 2009
     Raid Level : raid10
  Used Dev Size : 497920 (486.33 MiB 509.87 MB)
     Array Size : 2489600 (2.37 GiB 2.55 GB)
   Raid Devices : 11
  Total Devices : 12
Preferred Minor : 1

    Update Time : Thu Feb 12 15:54:19 2009
          State : clean
Internal Bitmap : present
 Active Devices : 11
Working Devices : 12
 Failed Devices : 0
  Spare Devices : 1
       Checksum : 9fd45b33 - correct
         Events : 12

         Layout : near=2, far=1
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     8       8      130        8      active sync   /dev/sdi2

   0     0       8        2        0      active sync   /dev/sda2
   1     1       8       18        1      active sync   /dev/sdb2
   2     2       8       34        2      active sync   /dev/sdc2
   3     3       8       50        3      active sync   /dev/sdd2
   4     4       8       66        4      active sync   /dev/sde2
   5     5       8       82        5      active sync   /dev/sdf2
   6     6       8       98        6      active sync   /dev/sdg2
   7     7       8      114        7      active sync   /dev/sdh2
   8     8       8      130        8      active sync   /dev/sdi2
   9     9       8      146        9      active sync   /dev/sdj2
  10    10       8      162       10      active sync   /dev/sdk2
  11    11       8      178       11      spare   /dev/sdl2
--
/dev/sdi3:
          Magic : a92b4efc
        Version : 1.0
    Feature Map : 0x1
     Array UUID : 0ee820db:5b2662a2:79cae9a9:cd057e88
           Name : 2
  Creation Time : Sat Feb  7 16:15:49 2009
     Raid Level : raid6
   Raid Devices : 11

 Avail Dev Size : 1951849032 (930.71 GiB 999.35 GB)
     Array Size : 17566640640 (8376.43 GiB 8994.12 GB)
  Used Dev Size : 1951848960 (930.71 GiB 999.35 GB)
   Super Offset : 1951849288 sectors
          State : clean
    Device UUID : dd595515:f79b36b8:2f06c51f:de1b5c01

Internal Bitmap : -234 sectors from superblock
    Update Time : Tue Feb 17 14:11:32 2009
       Checksum : 3808de16 - correct
         Events : 6

     Chunk Size : 256K

    Array Slot : 8 (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
   Array State : uuuuuuuuUuu
--
/dev/sdj1:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : 77b40b96:8b6c84dc:a60ecb55:bfd75636
  Creation Time : Sat Feb  7 15:52:41 2009
     Raid Level : raid1
  Used Dev Size : 128384 (125.40 MiB 131.47 MB)
     Array Size : 128384 (125.40 MiB 131.47 MB)
   Raid Devices : 11
  Total Devices : 12
Preferred Minor : 0

    Update Time : Thu Feb 12 19:24:02 2009
          State : clean
Internal Bitmap : present
 Active Devices : 11
Working Devices : 12
 Failed Devices : 0
  Spare Devices : 1
       Checksum : a556b147 - correct
         Events : 138


      Number   Major   Minor   RaidDevice State
this     0       8      145        0      active sync   /dev/sdj1

   0     0       8      145        0      active sync   /dev/sdj1
   1     1       8        1        1      active sync   /dev/sda1
   2     2       8       17        2      active sync   /dev/sdb1
   3     3       8       49        3      active sync   /dev/sdd1
   4     4       8       65        4      active sync   /dev/sde1
   5     5       8       81        5      active sync   /dev/sdf1
   6     6       8       97        6      active sync   /dev/sdg1
   7     7       8      113        7      active sync   /dev/sdh1
   8     8       8      129        8      active sync   /dev/sdi1
   9     9       8       33        9      active sync   /dev/sdc1
  10    10       8      161       10      active sync   /dev/sdk1
  11    11       8      177       11      spare   /dev/sdl1
--
/dev/sdj2:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : 5dbce27d:fa98a252:4bb2c254:bf75a219
  Creation Time : Sat Feb  7 15:53:36 2009
     Raid Level : raid10
  Used Dev Size : 497920 (486.33 MiB 509.87 MB)
     Array Size : 2489600 (2.37 GiB 2.55 GB)
   Raid Devices : 11
  Total Devices : 12
Preferred Minor : 1

    Update Time : Thu Feb 12 15:54:19 2009
          State : clean
Internal Bitmap : present
 Active Devices : 11
Working Devices : 12
 Failed Devices : 0
  Spare Devices : 1
       Checksum : 9fd45b45 - correct
         Events : 12

         Layout : near=2, far=1
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     9       8      146        9      active sync   /dev/sdj2

   0     0       8        2        0      active sync   /dev/sda2
   1     1       8       18        1      active sync   /dev/sdb2
   2     2       8       34        2      active sync   /dev/sdc2
   3     3       8       50        3      active sync   /dev/sdd2
   4     4       8       66        4      active sync   /dev/sde2
   5     5       8       82        5      active sync   /dev/sdf2
   6     6       8       98        6      active sync   /dev/sdg2
   7     7       8      114        7      active sync   /dev/sdh2
   8     8       8      130        8      active sync   /dev/sdi2
   9     9       8      146        9      active sync   /dev/sdj2
  10    10       8      162       10      active sync   /dev/sdk2
  11    11       8      178       11      spare   /dev/sdl2
--
/dev/sdj3:
          Magic : a92b4efc
        Version : 1.0
    Feature Map : 0x1
     Array UUID : 0ee820db:5b2662a2:79cae9a9:cd057e88
           Name : 2
  Creation Time : Sat Feb  7 16:15:49 2009
     Raid Level : raid6
   Raid Devices : 11

 Avail Dev Size : 1951849032 (930.71 GiB 999.35 GB)
     Array Size : 17566640640 (8376.43 GiB 8994.12 GB)
  Used Dev Size : 1951848960 (930.71 GiB 999.35 GB)
   Super Offset : 1951849288 sectors
          State : clean
    Device UUID : 25ee411f:4f3c0bc1:fc77505b:a3168abe

Internal Bitmap : -234 sectors from superblock
    Update Time : Tue Feb 17 14:11:32 2009
       Checksum : 43837f4a - correct
         Events : 6

     Chunk Size : 256K

    Array Slot : 9 (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
   Array State : uuuuuuuuuUu
--
/dev/sdk1:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : 77b40b96:8b6c84dc:a60ecb55:bfd75636
  Creation Time : Sat Feb  7 15:52:41 2009
     Raid Level : raid1
  Used Dev Size : 128384 (125.40 MiB 131.47 MB)
     Array Size : 128384 (125.40 MiB 131.47 MB)
   Raid Devices : 11
  Total Devices : 12
Preferred Minor : 0

    Update Time : Thu Feb 12 19:24:02 2009
          State : clean
Internal Bitmap : present
 Active Devices : 11
Working Devices : 12
 Failed Devices : 0
  Spare Devices : 1
       Checksum : a556b16b - correct
         Events : 138


      Number   Major   Minor   RaidDevice State
this    10       8      161       10      active sync   /dev/sdk1

   0     0       8      145        0      active sync   /dev/sdj1
   1     1       8        1        1      active sync   /dev/sda1
   2     2       8       17        2      active sync   /dev/sdb1
   3     3       8       49        3      active sync   /dev/sdd1
   4     4       8       65        4      active sync   /dev/sde1
   5     5       8       81        5      active sync   /dev/sdf1
   6     6       8       97        6      active sync   /dev/sdg1
   7     7       8      113        7      active sync   /dev/sdh1
   8     8       8      129        8      active sync   /dev/sdi1
   9     9       8       33        9      active sync   /dev/sdc1
  10    10       8      161       10      active sync   /dev/sdk1
  11    11       8      177       11      spare   /dev/sdl1
--
/dev/sdk2:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : 5dbce27d:fa98a252:4bb2c254:bf75a219
  Creation Time : Sat Feb  7 15:53:36 2009
     Raid Level : raid10
  Used Dev Size : 497920 (486.33 MiB 509.87 MB)
     Array Size : 2489600 (2.37 GiB 2.55 GB)
   Raid Devices : 11
  Total Devices : 12
Preferred Minor : 1

    Update Time : Thu Feb 12 15:54:19 2009
          State : clean
Internal Bitmap : present
 Active Devices : 11
Working Devices : 12
 Failed Devices : 0
  Spare Devices : 1
       Checksum : 9fd45b57 - correct
         Events : 12

         Layout : near=2, far=1
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this    10       8      162       10      active sync   /dev/sdk2

   0     0       8        2        0      active sync   /dev/sda2
   1     1       8       18        1      active sync   /dev/sdb2
   2     2       8       34        2      active sync   /dev/sdc2
   3     3       8       50        3      active sync   /dev/sdd2
   4     4       8       66        4      active sync   /dev/sde2
   5     5       8       82        5      active sync   /dev/sdf2
   6     6       8       98        6      active sync   /dev/sdg2
   7     7       8      114        7      active sync   /dev/sdh2
   8     8       8      130        8      active sync   /dev/sdi2
   9     9       8      146        9      active sync   /dev/sdj2
  10    10       8      162       10      active sync   /dev/sdk2
  11    11       8      178       11      spare   /dev/sdl2
--
/dev/sdk3:
          Magic : a92b4efc
        Version : 1.0
    Feature Map : 0x1
     Array UUID : 0ee820db:5b2662a2:79cae9a9:cd057e88
           Name : 2
  Creation Time : Sat Feb  7 16:15:49 2009
     Raid Level : raid6
   Raid Devices : 11

 Avail Dev Size : 1951849032 (930.71 GiB 999.35 GB)
     Array Size : 17566640640 (8376.43 GiB 8994.12 GB)
  Used Dev Size : 1951848960 (930.71 GiB 999.35 GB)
   Super Offset : 1951849288 sectors
          State : clean
    Device UUID : c46ec57c:0bb9c0fa:44d96904:cb4ab3ea

Internal Bitmap : -234 sectors from superblock
    Update Time : Tue Feb 17 14:11:32 2009
       Checksum : afff1216 - correct
         Events : 6

     Chunk Size : 256K

    Array Slot : 10 (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
   Array State : uuuuuuuuuuU
--
/dev/sdl1:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : 77b40b96:8b6c84dc:a60ecb55:bfd75636
  Creation Time : Sat Feb  7 15:52:41 2009
     Raid Level : raid1
  Used Dev Size : 128384 (125.40 MiB 131.47 MB)
     Array Size : 128384 (125.40 MiB 131.47 MB)
   Raid Devices : 11
  Total Devices : 12
Preferred Minor : 0

    Update Time : Thu Feb 12 19:24:02 2009
          State : clean
Internal Bitmap : present
 Active Devices : 11
Working Devices : 12
 Failed Devices : 0
  Spare Devices : 1
       Checksum : a556b177 - correct
         Events : 138


      Number   Major   Minor   RaidDevice State
this    11       8      177       11      spare   /dev/sdl1

   0     0       8      145        0      active sync   /dev/sdj1
   1     1       8        1        1      active sync   /dev/sda1
   2     2       8       17        2      active sync   /dev/sdb1
   3     3       8       49        3      active sync   /dev/sdd1
   4     4       8       65        4      active sync   /dev/sde1
   5     5       8       81        5      active sync   /dev/sdf1
   6     6       8       97        6      active sync   /dev/sdg1
   7     7       8      113        7      active sync   /dev/sdh1
   8     8       8      129        8      active sync   /dev/sdi1
   9     9       8       33        9      active sync   /dev/sdc1
  10    10       8      161       10      active sync   /dev/sdk1
  11    11       8      177       11      spare   /dev/sdl1
--
/dev/sdl2:
          Magic : a92b4efc
        Version : 00.90.00
           UUID : 5dbce27d:fa98a252:4bb2c254:bf75a219
  Creation Time : Sat Feb  7 15:53:36 2009
     Raid Level : raid10
  Used Dev Size : 497920 (486.33 MiB 509.87 MB)
     Array Size : 2489600 (2.37 GiB 2.55 GB)
   Raid Devices : 11
  Total Devices : 12
Preferred Minor : 1

    Update Time : Thu Feb 12 15:54:19 2009
          State : clean
 Active Devices : 11
Working Devices : 12
 Failed Devices : 0
  Spare Devices : 1
       Checksum : 9fd45b63 - expected 9fd45a63
         Events : 12

         Layout : near=2, far=1
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this    11       8      178       11      spare   /dev/sdl2

   0     0       8        2        0      active sync   /dev/sda2
   1     1       8       18        1      active sync   /dev/sdb2
   2     2       8       34        2      active sync   /dev/sdc2
   3     3       8       50        3      active sync   /dev/sdd2
   4     4       8       66        4      active sync   /dev/sde2
   5     5       8       82        5      active sync   /dev/sdf2
   6     6       8       98        6      active sync   /dev/sdg2
   7     7       8      114        7      active sync   /dev/sdh2
   8     8       8      130        8      active sync   /dev/sdi2
   9     9       8      146        9      active sync   /dev/sdj2
  10    10       8      162       10      active sync   /dev/sdk2
  11    11       8      178       11      spare   /dev/sdl2
--
/dev/sdl3:
          Magic : a92b4efc
        Version : 1.0
    Feature Map : 0x1
     Array UUID : 0ee820db:5b2662a2:79cae9a9:cd057e88
           Name : 2
  Creation Time : Sat Feb  7 16:15:49 2009
     Raid Level : raid6
   Raid Devices : 11

 Avail Dev Size : 1951849032 (930.71 GiB 999.35 GB)
     Array Size : 17566640640 (8376.43 GiB 8994.12 GB)
  Used Dev Size : 1951848960 (930.71 GiB 999.35 GB)
   Super Offset : 1951849288 sectors
          State : clean
    Device UUID : 1bd49765:a600828e:a7853994:5e8e58e6

Internal Bitmap : -234 sectors from superblock
    Update Time : Thu Feb 12 19:23:51 2009
       Checksum : b80160b2 - correct
         Events : 6

     Chunk Size : 256K

    Array Slot : 11 (0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
   Array State : uuuuuuuuuuu
--

--- /proc/cmdline
root=/dev/mapper/Kero-root ro 

--- grub:
kernel		/vmlinuz-2.6.28.4 root=/dev/mapper/Kero-root ro 
kernel		/vmlinuz-2.6.28.4 root=/dev/mapper/Kero-root ro single
kernel		/vmlinuz-2.6.26-1-amd64 root=/dev/mapper/Kero-root ro 
kernel		/vmlinuz-2.6.26-1-amd64 root=/dev/mapper/Kero-root ro single


-- System Information:
Debian Release: 5.0
  APT prefers testing
  APT policy: (500, 'testing')
Architecture: amd64 (x86_64)

Kernel: Linux 2.6.28.4 (SMP w/2 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash

Versions of packages mdadm depends on:
ii  debconf                       1.5.24     Debian configuration management sy
ii  libc6                         2.7-18     GNU C Library: Shared libraries
ii  lsb-base                      3.2-20     Linux Standard Base 3.2 init scrip
ii  makedev                       2.3.1-88   creates device files in /dev
ii  udev                          0.125-7    /dev/ and hotplug management daemo

Versions of packages mdadm recommends:
ii  exim4-daemon-light [mail-tran 4.69-9     lightweight Exim MTA (v4) daemon
ii  module-init-tools             3.4-1      tools for managing Linux kernel mo

mdadm suggests no packages.

-- debconf information:
* mdadm/autostart: true
* mdadm/mail_to: root
  mdadm/initrdstart_msg_errmd:
* mdadm/initrdstart: /dev/md2
  mdadm/initrdstart_msg_errconf:
  mdadm/initrdstart_notinconf: false
  mdadm/initrdstart_msg_errexist:
  mdadm/initrdstart_msg_intro:
* mdadm/autocheck: true
  mdadm/initrdstart_msg_errblock:
* mdadm/start_daemon: true





More information about the pkg-mdadm-devel mailing list