Bug#875914: autopkgtest: support Conflicts: in debian/tests/control

[Martin Pitt]

Hello Daniel,

Daniel Kahn Gillmor [2018-03-21 10:12 +0000]:
> afaict, the autopkgtest spec doesn't say that all
> implementations/backends will start from a minimal environment.

This is by intent, as it should not conceptually be limited to the
standardized minimal testbeds, for several reasons:

 * It's reasonable to run an autotest (manually) on an existing machine, even a
   full desktop. Both the null and the ssh runner support this well.

 * It not all that easy to define what "minimal" precisely is. E. g. just
   "Priority: required" (more or less debootstrap) works for a schroot, but is
   usually not enough for a container and by far not enough for a VM.

 * Often really minimal testbeds are also impractical. E. g. cloud images (or
   Ubuntu's official LXC images even) often carry a lot of packages that are
   undesirable for an official CI system (as they are too big), but are
   nevertheless practical for developer usage as they are readily and easily
   available.

> does this help explain why an explicit conflicts would be useful in
> avoiding false negatives?

I do agree that this is legitimate in some corner cases. The actual need just
didn't occur very often so far, so it hasn't been a priority to actually
implement this. Furthermore this isn't actually that simple: If autopkgtest
detects that a conflicted package is installed, it certainly shouldn't just
fail or skip the test, as that would be rather pointless. It needs to actually
try and remove the packages; but this has significantly harder corner cases
than installing new packages; for example:

 - What if another package depends on the conflicted package, but has other
   alternatives? How do you pick a different one? Or should you remove that package as well?

 - What happens if you conflict to an essential/required package? (I think in
   this case the test should actually fail as "invalid test").

 - What happens if you conflict to a non-required package which is nevertheless
   required for your test environment? (e. g. your network driver or
   network-manager or openssh-server)

A naïve implementation coudl just leave all these questions to "apt purge -y",
and fail the test if that fails. Maybe this is even good enough for the use
cases that you have in mind?

For these reasons autopkgtest has never uninstalled packages so far, but
provides a new testbed if a subsequent test has fewer test dependencies than
its predecessor. Again, I'm not saying that it's impossible or undesirable,
just that there are quite a number of traps.

Thanks,

Martin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/autopkgtest-devel/attachments/20180322/d7a6ba5b/attachment.sig>