Bug#774082: systemd: system with LVM and lvmetad does not bot after upgrade to jessie

[Peter Rajnoha]

On 04/23/2015 04:12 PM, Michael Biebl wrote:
> Am 23.04.2015 um 11:40 schrieb Peter Rajnoha:
>> On 04/23/2015 11:04 AM, Michael Biebl wrote:
>>> Looking at the udev rule shipped by Debian's lvm2 package, it has:
>>>
>>> 69-lvm-metad.rules:ACTION!="remove", ENV{LVM_PV_GONE}=="1",
>>> RUN+="/bin/systemd-run /sbin/lvm pvscan --cache $major:$minor",
>>> GOTO="lvm_end"
>>>
>>
>> That rule is to cover immediate removal of the PV by erasing
>> it's PV header directly (e.g. by running "dd"). So when the
>> device is still in the system, but it loses the PV abruptly.
> 
> Why does this use systemd-run and not SYSTEMD_WANTS?

Because lvm2-pvscan at major:minor.service is bound to device
existence - once the major:minor device appears in the system
and it is a PV, the lvm2-pvscan at major:minor.service is
instantiated through that SYSTEMD_WANTS.

And, what is more important, once you use SYSTEMD_WANTS
to instantiate a service, you can call SYSTEMD_WANTS
again as many times as you want, it won't run again
if the service exists.

However, if you erase the PV header directly and the device
itself still stays in the system (that's exactly the
ACTION!="remove", ENV{LVM_PV_GONE}=="1"), you need
to run lvm2-pvscan too for lvmetad to notice the
PV header got erased and hence the device is not
a PV anymore for all LVM tools to have the proper
information. This is not needed in case you erase
the PV header with proper pvremove call, but still
there may be situations in which people just call
"dd if=/dev/zero of=/dev/the_pv ..." and they
expect it all works.

Alternatively, instead of calling that systemd-run,
we could call "systemctl stop lvm2-pvscan at major:minor.service"
(the "ExecStop" action of that service also
notifies lvmetad about the new state) but I was
advised by systemd folk that this is dangerous
for various reasons and any systemctl shouldn't be
called from within udev rules.

Unfortunately, all calls from udev where we need to
be sure they're executed, need to be run as services.
So we can't simply call "pvscan --cache" directly from
udev rule as we may risk udev killing this process
if the system if heavy-loaded and event handling
timeouts (yes, we've had reports for these kind of
problems as well).

So this is a dark corner which is still not quite
resolved in a nice way as I'd like and it still
requires better approach. We even have upstream bug
for this: https://bugzilla.redhat.com/show_bug.cgi?id=1063813

So the systemd-run is actually a workaround for now
until we resolve this properly.

I tried to discuss this with systemd team and I asked
for something like "SYSTEMD_UNWANTS", but they were
strictly against this (because of some internal
problems it may bring). So while we can bind service
existence to the device existence, we can't bind it
to the "signature" existence on that device, unfortunately.

-- 
Peter