Bug#832751: autopkgtest: aborts the entire test run when one test has unsatisfiable dependencies

[Antonio Terceiro]

Package: autopkgtest
Version: 4.0.2
Severity: normal

Just found with redmine:
https://ci.debian.net/data/packages/unstable/arm64/r/redmine/20160728_031417.autopkgtest.log.gz

The third test block failed during the dependency installation phase, and then
the entire test run was aborted. I would that specific test to be marked as
failed, or whatever, and that the remaining tests would still run, and I would
have the PASS/FAIL summary at the end.

I can also reproduce this with the latest version in unstable, with a minimal
test case:

----------------8<----------------8<----------------8<-----------------
$ find
.
./debian
./debian/tests
./debian/tests/control
$ cat debian/tests/control
Test-Command: true
Depends: package-that-does-not-exist

Test-Command: true
$ autopkgtest -B . -- null
autopkgtest [11:00:56]: version 4.0.2
autopkgtest [11:00:56]: host homer; command line: /usr/bin/autopkgtest -B . -- null
autopkgtest [11:00:56]: testbed dpkg architecture: amd64
autopkgtest [11:00:56]: testbed running kernel: Linux 4.6.0-1-amd64 #1 SMP Debian 4.6.3-1 (2016-07-04)
autopkgtest [11:00:56]: @@@@@@@@@@@@@@@@@@@@ built-tree .
autopkgtest [11:00:56]: test command1: preparing testbed
E: Unable to locate package package-that-does-not-exist
blame: .
badpkg: Test dependencies are unsatisfiable. A common reason is that your testbed is out of date with respect to the archive, and you need to use a current testbed, or try "--setup-commands ro-apt-update".
autopkgtest [11:00:57]: ERROR: erroneous package: Test dependencies are unsatisfiable. A common reason is that your testbed is out of date with respect to the archive, and you need to use a current testbed, or try "--setup-commands ro-apt-update".
----------------8<----------------8<----------------8<-----------------

Note that while I am here forcing the issue with an unexisting package, the
result is the same as it was in the case of redmine, when some dependency
failed to configure, and caused the whole thing to abort.


-- System Information:
Debian Release: stretch/sid
  APT prefers unstable
  APT policy: (500, 'unstable'), (500, 'testing'), (1, 'experimental')
Architecture: amd64 (x86_64)

Kernel: Linux 4.6.0-1-amd64 (SMP w/4 CPU cores)
Locale: LANG=pt_BR.UTF-8, LC_CTYPE=pt_BR.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages autopkgtest depends on:
ii  apt-utils       1.3~pre2
ii  libdpkg-perl    1.18.9
ii  procps          2:3.3.12-2
ii  python3         3.5.1-4
ii  python3-debian  0.1.28

Versions of packages autopkgtest recommends:
ii  autodep8  0.6

Versions of packages autopkgtest suggests:
ii  lxc          1:2.0.3-1
pn  lxd-client   <none>
ii  qemu-system  1:2.6+dfsg-3
ii  qemu-utils   1:2.6+dfsg-3
ii  schroot      1.6.10-2

-- no debconf information
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/autopkgtest-devel/attachments/20160728/4c9b669f/attachment-0001.sig>