Bug#800845: autopkgtest: Add support for nested VMs

[Martin Pitt]

Hello Christian,

Christian Seiler [2016-03-03  2:01 +0100]:
> > Maybe this is all trying to be overly abstract, and what we really
> > want is just "Restrictions: virt-{qemu,lxc,lxd,schroot,...}"?
> 
> No, it isn't. Cloning this bug to track the SKIP stuff.

Ack, since this is now tracked in #816564, let's drop the thread in
this bug.

> >> 0002-adt-virt-qemu-Use-host-CPU-type-by-default.patch
> >>
> >>   Passes -cpu host to QEMU by default _unless_ --qemu-command= is
> >>   specified. (If qemu-command is specified, it might be the case that
> >>   someone wants to run a fully emulated different architecture, and
> >>   then -cpu host will fail spectacularly.)
> > 
> > As I already wrote before, I really don't like this as a default, as
> > it introduces unpredictability into testbeds. We've seen tests that
> > behave differently with different CPU models (X.org's LLVM
> > software render and mesa for example), and getting random test
> > passes/failures depending on which hw you run it on is really
> > non-obvious.
> 
> Isn't this then a bug of the testbed?

How so? Host hardware having different capabilities is not a bug :-)
But I'd like to abstract away from this as much as possible for
production runs, i. e. make the test environment predictable as much
as we can.

> Also note that if you run those tests (e.g. mesa) in e.g. LXC - or
> on a dedicated machine (doesn't Ubuntu have some bare metal machines
> where tests requiring machine isolation are run on?), you get the
> same issues.

That's correct, yes. But that doesn't mean that this is intended, just
way harder to avoid :-) We do run LXC tests for armhf and s390x where
we don't yet have QEMU support, and indeed we have much more jitter
there. But for i386/amd64/ppc64el we use Openstack instances (which
are effectively QEMU).

> > However, I'd be totally fine with changing the default to
> > "-cpu SandyBridge". This is what nova-compute-qemu uses by default
> > (apparently), so this both provides a reasonably rich CPU architecture
> > and increases compatibility to test runs on a cloud instance, and
> > keeps predictability. WDYT?
> 
> I don't have AMD hardware to test here, but if I try to do it the
> other way around, e.g. "-cpu Opteron_G5" on my Haswell box, nested
> KVM doesn't work. Problem is that AMD and Intel have completely
> different KVM support - AMD has something called 'svm', Intel calls
> it's technology 'vmx'. And if we start to emulate SandyBridge,
> nested KVM won't work on AMD at all. (CPU features are disabled if
> the host CPU doesn't support them. There are some warnings present.)
> That's why I went with -cpu host - it's very nice and simple. :-)

Ah, I didn't have that in mind, thanks for the heads-up.

> Of course, we could pick SandyBridge for Intel and something else
> for AMD (no idea what would be a good idea there, I haven't followed
> AMD for a while) and reduce the possible CPUs to just 2.

That sounds like a reasonable first start. So if the host has "vmx",
use SandyBridge, if it has "svm" use Opteron_G5, and if it has
neither, just continue to use whatever QEMU defaults to?

That said, at least on the machines I tested, nested kvm does work in
principle (maybe some subtle bugs) with the default -cpu too.

> This is a tangent, but wasn't somebody working on qemu testbeds for
> the Debian CI?

I'm not aware of that. However, it's more realistic to use the ssh
runner with the nova setup script, given that production debci already
runs in the cloud. (This won't be able to use this nesting trick,
though).

Thanks,

Martin

-- 
Martin Pitt                        | http://www.piware.de
Ubuntu Developer (www.ubuntu.com)  | Debian Developer  (www.debian.org)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/autopkgtest-devel/attachments/20160304/2e8acb63/attachment.sig>