Bug#851556: autopkgtest: please add Restrictions for network access, or Features for lack of network access

[Martin Pitt]

Hello Simon,

Simon McVittie [2017-01-16  9:57 +0000]:
> Since this comes up repeatedly, would you be OK with having some text in
> the specification that explicitly says "it is not a bug for a test to
> make itself visible on the network"? That way it's at least unambiguous.

Yes, I think that does make sense. Proposed patch attached (language/legibility
updates greatly appreciated!)

> > Such a test should add
> > "Restrictions: breaks-testbed" if it's doing something dangerous
> 
> I would have expected that we would want to force the testbed to be
> reverted after messing with apt configuration (as this patch does,
> although I didn't make that clear enough in the commit message), even if
> the testbed is not actually "broken". But I can keep using breaks-testbed
> for that in nss-mdns if you prefer.

I think break-testbeds is a generic way of indicating that (1) autopkgtest
needs to revert after running this test and (2) it will be skipped on "unsafe"
backends which don't provide revert. As we don't have a way to partially revert
only certain aspects, it seems that providing partial versions of
breaks-testbeds is both hard to maintain in the autopkgtest implementation and
also hard to keep track of for a package maintainer over the history of
changing tests.

Martin

-- 
Martin Pitt                        | http://www.piware.de
Ubuntu Developer (www.ubuntu.com)  | Debian Developer  (www.debian.org)
-------------- next part --------------
>From d08c58f5e2efdf5c19a321cf0a011923478e49ce Mon Sep 17 00:00:00 2001
From: Martin Pitt <martin at piware.de>
Date: Mon, 16 Jan 2017 12:43:19 +0100
Subject: [PATCH] doc/README.package-tests.rst: Document network access

---
 doc/README.package-tests.rst | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/doc/README.package-tests.rst b/doc/README.package-tests.rst
index 4775a07..1e9cba5 100644
--- a/doc/README.package-tests.rst
+++ b/doc/README.package-tests.rst
@@ -314,4 +314,23 @@ Example:
         reboot
     fi
 
+Network access
+--------------
+autopkgtest needs access to the network at least for downloading test
+dependencies and possibly dist-upgrading testbeds. In environments with
+restricted internet access you need to set up an apt proxy and configure
+the testbed to use it. (Note that the standard tools like
+autopkgtest-build-lxc or mk-sbuild automatically use the apt proxy from
+the host system.)
+
+In general, tests are also allowed to access the internet. As this
+usually makes tests less reliable, this should be kept to a minimum; but
+for many packages their main purpose is to interact with remote web
+services and thus their testing should actually cover those too, to
+ensure that the distribution package keeps working with their
+corresponding web service.
+
+Debian's and Ubuntu's production CI infrastructure do allow unrestricted
+network access (possibly through a proxy).
+
 .. vim: ft=rst tw=72
-- 
2.9.3