Bug#788659: autopkgtest: Provide option to not implicitly run undeclared tests
dom at earth.li
Sat Jun 13 23:18:28 UTC 2015
To allow test execution environments to discover packages which provide
tests, their source packages should have a ``Testsuite:`` header
containing ``autopkgtest`` (which is currently the only defined value).
Multiple values get comma separated, as usual in control files.
My expectation having read this was that adt-run would only attempt
to test packages if the Testsuite header is present; however adt-run
actually appears to not look for this header at all; it always processes
tests for a package, including automagic test suites set up by
autodep8. This is probably fine in the case of invoking adt-run with
a single package, but not if you want to look for regressions using
adt-run; for example I ran it against all packages I'd been rebuilding
against perl 5.22, and found quite a few failures which didn't appear to
be related to perl 5.22 but were instead tests which just didn't work.
Please could adt-run have an option to exit without running any tests
if the source package in question doesn't define any? I think this
would be useful in other QA contexts too. I would say that this should
be the default, but that is probably too difficult to change now.
More information about the autopkgtest-devel