Bug#716820: lvm2: /dev/mapper/vg-lv symlink creation races with fsck, causing boot failure

[Jimmy]

Package: lvm2
Version: 2.02.98-3
Severity: important

Dear Maintainer,

After a recent upgrade of packages on my server, it sometimes fails to boot.
I cannot see any pattern as to whether it boots or not. It looks very much
like a race condition.

When the machine fails to boot, fsck fails to check one or more of the
filesystem(s) listed in fstab, which are on LVM2 LVs and are named like:
/dev/mapper/$vg-$lv

When this happens I see messages on the console of the form:
/dev/mapper/$vg-$lv: nonexistent device ("nofail" fstab option may be used to
skip this device)

The particular LV that cannot be fsckd changes each failed boot.
To reiterate, sometimes all LVs are fine, there is no problem at all, and boot
proceeds normally.

When fsck fails, I can get an sulogin shell; from there, I can fsck and mount
all LVM2-backed filesystems just fine.
There are no errors on any of the filesystems or block devices involved.

Editing /etc/fstab and adding the 'nofail' option to each LVM2-backed FS
causes these errors to be disregarded at bootup; bootup then proceeds
normally every time.

However I am concerned that this workaround would cause real filesystem errors
discovered at bootup to be ignored, which I regard as dangerous.
Therefore I backed out this change.

Editing /etc/fstab and changing the device names from /dev/mapper/$vg-$lv
to /dev/$vg-$lv seemed to make no difference whatsoever - the seemingly-random
failures occurred as previously.

Editing /etc/fstab and changing the device names from /dev/$vg-$lv to UUID=
form seems to work around the problem - I have had several successful bootups
in a row after this change.
However as the problem is intermittent, I don't 100% trust this workaround.

I have not attempted to use the real device names /dev/dm-$nn in fstab.

I believe that if UUID= works, I must have working LVs, in which case I am
just lacking the symlinks in /dev. So it seems as if udev is slow or bad?

I will add supplementary information shortly.

FYI the following existing bugs appear to be related, but distinct, problems:
#568838 (but is about the LABEL= case, which I am not using)
#715184 (but he seems to lack the LV, whereas I *think* I have an LV but lack symlinks)
#397509 (but he, unlike me, has /dev/mapper/$vg-$lv symlinks)

Thank you for taking the time to look at this report!

-- System Information:
Debian Release: jessie/sid
  APT prefers unstable
  APT policy: (500, 'unstable'), (500, 'testing'), (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 3.9.0 (SMP w/8 CPU cores)
Locale: LANG=en_AU.UTF-8, LC_CTYPE=en_AU.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages lvm2 depends on:
ii  dmsetup                   2:1.02.77-3
ii  initscripts               2.88dsf-41
ii  libc6                     2.17-7
ii  libdevmapper-event1.02.1  2:1.02.77-3
ii  libdevmapper1.02.1        2:1.02.77-3
ii  libreadline5              5.2+dfsg-2
ii  libudev0                  175-7.2
ii  lsb-base                  4.1+Debian12

lvm2 recommends no packages.

lvm2 suggests no packages.

-- no debconf information