RAID 5 broken ?

[Bruno TATU]

Hello everyone,

I would like to share a little stupid story and sorry for my bad english. :)


Yesterday i've read this topic : 
https://blog.stalkr.net/2009/10/reducing-number-of-devices-in-raid-5.html 
for move from 4 discs to 3 discs.

Indeed, on dics /dev/sdb1 have bad smart attribute so i remove it in 
raid /dev/md5 and mainwhile the sync a second discs are missing 
/dev/sdd1(?).

After a night i see my sync was stopped at 93%!!

So, how i can active the sdd1 (this is "_" ?) to continue the sync ?

Can I dump this superblock for a recovery ?

May be a reboot ? I'm not sure...


My setup (jessie) :

# mdadm -V
mdadm - v3.3.2 - 21st August 2014

# uname -a
Linux nas.domaine.com 3.16.0-4-amd64 #1 SMP Debian 3.16.43-2+deb8u2 
(2017-06-26) x86_64 GNU/Linux

# grep name /proc/cpuinfo | awk -F ':' '{print $2}'
  AMD Sempron(tm) Processor LE-1250

The state of my raid :

#  mdadm --detail /dev/md5

/dev/md5:
         Version : 1.2
   Creation Time : Sat Apr  8 16:55:58 2017
      Raid Level : raid5
      Array Size : 781158400 (744.97 GiB 799.91 GB)
   Used Dev Size : 390579200 (372.49 GiB 399.95 GB)
    Raid Devices : 3
   Total Devices : 4
     Persistence : Superblock is persistent

   Intent Bitmap : Internal

     Update Time : Mon Aug 14 23:18:13 2017
           State : clean, degraded, reshaping
  Active Devices : 3
Working Devices : 3
  Failed Devices : 1
   Spare Devices : 0

          Layout : left-symmetric
      Chunk Size : 512K

  Reshape Status : 93% complete
   Delta Devices : -1, (4->3)

            Name : nas.local.net:5
            UUID : d52c653d:da37e6b1:9e7c3426:c36089af
          Events : 8928

     Number   Major   Minor   RaidDevice State
        0       8        1        0      active sync   /dev/sda1
        2       0        0        2      removed
        2       8       33        2      active sync   /dev/sdc1

        1       8       17        -      faulty   /dev/sdb1
        4       8       49        3      active sync   /dev/sdd1

# cat /proc/mdstat

Personalities : [raid6] [raid5] [raid4]
md5 : active raid5 sdb1[1](F) sda1[0] sdd1[4] sdc1[2]
       781158400 blocks super 1.2 level 5, 512k chunk, algorithm 2 [3/2] 
[U_U]
       [==================>..]  reshape = 93.7% (366077828/390579200) 
finish=67708.2min speed=6K/sec
       bitmap: 3/3 pages [12KB], 65536KB chunk

unused devices: <none>


My commands :

   mdadm --manage /dev/md5 --set-faulty /dev/sdb1
   mdadm --manage /dev/md5 --remove /dev/sdb1

   resize2fs -M -p /dev/md5

[It's may be at this moment when a other disk missing]

   mdadm /dev/md5 --grow --array-size=781158400

   mdadm /dev/md5 --grow --raid-devices=3 --backup-file=/tmp/backup

[here is stopped at 93,7%...]

    mdadm /dev/md5 --add /dev/sdb1

    mdadm /dev/md5 --set-faulty /dev/sdb1

[here in /proc/mdstat we have sdb1[1](F)]


Tell me if you need more informations and if needed, i can mount a 
remote disk.

Thanks a lot!

Regards