Bug#854830: apt updates initramfs incorrectly, breaks RAID/mdadm

[Julian Andres Klode]

Control: reassign -1 
On Fri, Feb 10, 2017 at 01:38:45PM -0800, Brad Doster wrote:
> Package: apt
> Version: 1.0.9.8.4

Thanks for the laugh :) APT is your package manager, not your RAID
manager. Reassigning there.


> 
> During initial installation of Debian 8, I used mdadm to create 2 RAID
> arrays as follows:
> 
> /dev/md0 - RAID 1 - 4 drives - contains /boot only
> /dev/md1 - RAID 5 - 3 drives + 1 spare - contains /
> 
> Both are needed to successfully boot the system.
> 
> During installation I name the machine HOSTA.  A few days later, I decided
> to change the machine's host name to HOSTB.  After a few weeks of
> successful operation of HOSTB, I ran apt-update followed by apt-upgrade.
> Among the updates were linux-image updates, which caused initramfs to be
> updated.  After the updates, rebooting the system landed me here:
> 
> =====
> Gave up waiting for root device. Common problems:
>   — Boot args (cat /proc/cmdline)
>     — Check rootdelay= (did the system wait long enough?)
>     — Check root= (did the system wait for the right device?)
>   — Missing modules (cat /proc/modules; ls /dev)
> ALERT! /dev/disk/by-uuid/054a6dcc-d280-4aff-9977 does not exist.
> Dropping to a shell!
> 
> BusyBox v.1.22.1 (Ubuntu 1:1.22.0-9deb8u1) built-in shell (ash)
> Enter 'help' for list of built-in commands.
> 
> /bin/sh: can't access tty: job control turned off
> (initramfs)
> =====
> 
> In the initramfs shell, /etc/mdadm/mdamd.conf contained the new hostname,
> HOSTB instead of HOSTA:
> 
> =====
> # definitions of existing MD arrays
> ARRAY /dev/md/0  metadata=1.2 UUID=413a2cf3:ef0784ac:1e135c73:7a1723c6
> name=HOSTB:0
> ARRAY /dev/md/1  metadata=1.2 UUID=ae1ab583:e5e30fb9:e2c807a2:b5e95f42
> name=HOSTB:1
>    spares=1
> =====
> 
> Editing mdadm.conf to use HOSTA in place of HOSTB allowed me to assemble my
> arrays and continue booting the system:
> 
> =====
> (initramfs) mdadm --assemble --scan
> mdadm: /dev/md/0 has been started with 4 drives.
> mdadm: /dev/md/1 has been started with 3 drives and 1 spare.
> (initramfs) exit
> =====
> 
> After a successful boot, mdadm -D confirms that the original host name,
> HOSTA is part of the array configuraion:
> 
> =====
> mdadm -D /dev/md1
> /dev/md1:
>         Version : 1.2
>   Creation Time : Thu Dec 29 09:02:43 2016
>      Raid Level : raid5
>      Array Size : 2130618368 (2031.92 GiB 2181.75 GB)
>   Used Dev Size : 1065309184 (1015.96 GiB 1090.88 GB)
>    Raid Devices : 3
>   Total Devices : 4
>     Persistence : Superblock is persistent
> 
>   Intent Bitmap : Internal
> 
>     Update Time : Thu Feb  9 17:25:16 2017
>           State : clean
>  Active Devices : 3
> Working Devices : 4
>  Failed Devices : 0
>   Spare Devices : 1
> 
>          Layout : left-symmetric
>      Chunk Size : 512K
> 
>            Name : HOSTA:1       # <--- here is the original hostname given
> to the machine during installation
>            UUID : ae1ab583:e5e30fb9:e2c807a2:b5e95f42
>          Events : 4831
> 
>     Number   Major   Minor   RaidDevice State
>        0       8        3        0      active sync   /dev/sda3
>        1       8       19        1      active sync   /dev/sdb3
>        2       8       35        2      active sync   /dev/sdc3
> 
>        3       8       51        -      spare   /dev/sdd3
> =====
> 
> In this case, the updates to initramfs should NOT have changed the hostname
> in the mdadm.conf file.  Doing so prevented the arrays from being assembled
> and the system from booting.
> 
> To get around the issue until the next apt-upgrade installs something
> causing initramfs to be updated again, on the up and running system, edit
> /etc/mdadm/mdadm.conf to use the original hostname and then run
> update-initramfs to install the change.  Once done, the system reboots
> normally again.
> 
> Once the issue is known, it is also possible to run apt-upgrade to
> completion, then before rebooting, edit mdamd.conf and manually execute
> update-initramfs.  This will prevent an untimely visit to the BusyBox
> screen, and in the case of remote upgrades being done, will prevent the
> need to physically visit the server in order to complete the boot process.
> 
> While this may be considered a problem with mdadm - why is it using a
> potentially volatile parameter like hostname in its permanent
> configuration? - until that is addressed, apt-upgrade should not modify the
> hostname used in the mdadm.conf file.

-- 
Debian Developer - deb.li/jak | jak-linux.org - free software dev
                  |  Ubuntu Core Developer |
When replying, only quote what is necessary, and write each reply
directly below the part(s) it pertains to ('inline').  Thank you.