Bug#851232: debci: does not understand space-separated tests

[Simon McVittie]

On Tue, 20 Mar 2018 at 16:22:15 +0100, Paul Gevers wrote:
> On Fri, 13 Jan 2017 07:12:36 +0000 Simon McVittie <smcv at debian.org> wrote:
> > The autopkgtest spec[1] says "Test names are separated by comma and/or
> > whitespace" (which I admit is weird, but it's what it says) and the
> > reference implementation implements that:
> > <https://anonscm.debian.org/git/autopkgtest/autopkgtest.git/tree/lib/testdesc.py?h=4.3#n413>
> > 
> > debian/tests/control in nss-mdns 0.10-8 has space-separated Tests (because
> > I misremembered what is allowed) and it looks as though ci.debian.net
> > responds to this by only running the first test per stanza:
> > https://ci.debian.net/data/packages/unstable/amd64/n/nss-mdns/20170113_063407.autopkgtest.log.gz
> > 
> > There is no practical impact right now for this particular package,
> > because I don't think nss-mdns' tests can be relied on to work in a
> > container anyway, so they're all skipped.
> 
> Do you believe the issue is still present? It seems you removed the test
> from nss-mdns and when I test this (albeit within the autopkgtest test
> suite) it seems currently to work as expected.

https://ci.debian.net/data/autopkgtest/unstable/amd64/n/nss-mdns/91/log.gz says:

autopkgtest [01:40:14]: testing package nss-mdns version 0.10-8
autopkgtest [01:40:14]: build not needed
after-resolve        SKIP Test requires machine-level isolation but testbed does not provide that
nss-behaviour        SKIP Test requires machine-level isolation but testbed does not provide that
autopkgtest [01:40:14]: @@@@@@@@@@@@@@@@@@@@ summary
after-resolve        SKIP Test requires machine-level isolation but testbed does not provide that
nss-behaviour        SKIP Test requires machine-level isolation but testbed does not provide that

but there are more tests than just those two:

Tests: after-resolve before-resolve jessie-after-resolve jessie-before-resolve multiarch-purge multiarch-remove remove-reinstall
Depends: avahi-daemon, avahi-utils, dbus
Restrictions: allow-stderr, breaks-testbed, isolation-machine, needs-root

Tests: nss-behaviour
Depends: dbus
Restrictions: allow-stderr, breaks-testbed, isolation-machine, needs-root

ci.debian.net can't currently run these tests anyway (they're marked as
needing isolation-machine) so they'd be skipped regardless. Perhaps this
is just misleading output, using after-resolve as shorthand for
"after-resolve and everything with the same dependencies"?

https://autopkgtest.ubuntu.com/packages/nss-mdns/bionic/amd64 (which I
think is also debci) supports isolation-machine, and that does run all
the tests, except for multiarch-* which are patched out (they don't
work on Ubuntu's infrastructure).

    smcv