Bug#800845: autopkgtest: Add support for nested VMs

[Christian Seiler]

Hi Martin,

So I've spent some time again on this and have separated this into
three patches:

0001-adt-virt-qemu-Implement-support-for-nested-base-imag.patch

  This adds the image specified to adt-virt-qemu as an additional read
  only virtio block device (with serial BASEIMAGE) to the VM (in
  contrast to my first patch, this is the actual data, not a qcow2
  image).

  It sets the ADT_BASEIMAGE environment variable to that device.

  It also does two things with udev in setup-testbed:

    1) it creates an udev rule to create a symlink /dev/baseimage to
       the new image (ADT_BASEIMAGE remains the absolute path for now,
       though)

    2) it modifies 60-persistent-storage.rules to skip blkid processing
       of UUID etc. for the base image, since the base image is likely
       the same as the main image, so UUIDs would match. Otherwise the
       /dev/disk/by-uuid symlinks would point to the base image drive.

  The same as with my first patch: if multiple images are specified on
  the adt-virt-qemu command line, it is impossible to determine what
  an approrpriate base image is, hence my patch doesn't set the base
  image in that case, though you may specify one explicitly via the
  command line.

0002-adt-virt-qemu-Use-host-CPU-type-by-default.patch

  Passes -cpu host to QEMU by default _unless_ --qemu-command= is
  specified. (If qemu-command is specified, it might be the case that
  someone wants to run a fully emulated different architecture, and
  then -cpu host will fail spectacularly.)

  This can be overridden by the --qemu-cpu option for adt-virt-qemu,
  which then replaces the value of QEMU's -cpu parameter. If
  --qemu-cpu=qemu-default is specified, the -cpu option is dropped
  completely.

0003-Add-needs-baseimage-restriction.patch

  I split this out from the first patch, because this is what you
  didn't like the first time around. I still think a new restriction is
  a good idea, in order to be able to distinguish between skipped tests
  and tests that were actually successful.

  (Of course, without a base image, one could do vmdebootstrap within
  the outer VM, but that would be a huge waste of resources, without
  a base image the only sensible thing to do is to skip the tests. And
  I'd really prefer .)

All patches also update the documentation. Note that since autopkgtest
specification is now part of Debian policy, if you agree to my third
patch, an update to the policy package would also be required, which
should probably only follow after this has been in use for a while, so
that we could still change it if we determine some problems. I'd be
willing to take care of that part.

I've tested this with Debian sid and Ubuntu Xenial images (created via
vmdebootstrap and adt-buildvm-ubuntu-cloud, as per the manpage) and it
works on both.

CAVEAT though:

  The first patch that adds the base image as an additional read-only
  drive has a slight problem if old images are used: since UUIDs are
  the same for filesystems on the main virtio drive and the base image
  "drive" (because udev processing of the base image "drive" is not
  disabled), partitions on both drives will have the same UUID, and
  apparently (at least on sid) udev selects the second drive for the
  symlinks, so /dev/disk/by-uuid/* will point to /dev/vdb*.

  This is not a problem for the default images (even those that were
  created from older versions), because they only contain one partition
  and the initiramfs is not affected, but if someone has multiple
  partitions in their test image, this will cause problems.

  I don't think this is a huge deal, beacuse you have to regenerate VM
  base images for ADT regularly anyway, but this is something to keep
  in mind (and maybe requires a NEWS entry?).

  I don't have a good solution for that other than to disable the base
  image logic unless explicitly enabled - but I'd really like to see
  this on by default.

I'd really like to see this moving forward.

Thoughts?

Regards,
Christian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-adt-virt-qemu-Implement-support-for-nested-base-imag.patch
Type: text/x-diff
Size: 7090 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/autopkgtest-devel/attachments/20160226/74e1f74b/attachment.patch>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0002-adt-virt-qemu-Use-host-CPU-type-by-default.patch
Type: text/x-diff
Size: 3927 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/autopkgtest-devel/attachments/20160226/74e1f74b/attachment-0001.patch>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0003-Add-needs-baseimage-restriction.patch
Type: text/x-diff
Size: 4010 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/autopkgtest-devel/attachments/20160226/74e1f74b/attachment-0002.patch>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: OpenPGP digital signature
URL: <http://lists.alioth.debian.org/pipermail/autopkgtest-devel/attachments/20160226/74e1f74b/attachment.sig>