Bug#741322: autopkgtest: responds poorly to timeout, should display stdout/stderr and move on

[Simon McVittie]

Package: autopkgtest
Version: 2.9.1
Severity: normal

I recently added autopkg tests to dbus. They worked in a schroot on my laptop,
but failed with a timeout on ci.debian.net, presumably because the schroot
was not minimal (it was originally for sbuild, so it has build-essential)
or because there's some other environmental difference.

However, from the output that was logged, I can't tell what the difference
or failure was, because adt-run logged the extensive shell command that it
invoked, but not any of its output. It also aborted the entire test run
when the timeout was reached, rather than moving on to the next autopkgtest
(admittedly, dbus only has two, because I'm representing all the upstream
tests as one big autopkgtest at the moment, but it was the first one that
failed).

<http://ci.debian.net/data/unstable-amd64/packages/d/dbus/2014-02-27.log>:
> adt-run: version 2.7.2
> adt-run: $ vserver: adt-virt-schroot debci-unstable-amd64
> adt-run: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ builds ...
> adt-run: @@@@@@@@@@@@@@@@@@@@ tree tree0
> adt-run: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ builds done.
> adt-run: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ tests ...
> adt-run: @@@@@@@@@@@@@@@@@@@@ test tree tree0
> adt-run: @@@@@@@@@@ run_tests ...
> adt-run: & tree0t-installed-tests: preparing
> adt-run: & tree0t-installed-tests:  processing dependency dbus
...
> adt-run: dependencies: install-deps: running ['/usr/lib/pbuilder/pbuilder-satisfydepends-classic', '--binary-all', '--internal-chrootexec', '/tmp/tmpJx9O1I/satdep-auxverb', '-c', '/tmp/tmpJx9O1I/deps.dsc']
...
> Setting up dbus-1-dbg:amd64 (1.8.0-2) ...
> Processing triggers for libc-bin (2.18-3) ...
> -> Finished parsing the build-deps
> adt-run: & tree0t-installed-tests: [----------------------------------------
> adt-run: testbed failed: sent `execute su,-s,/bin/sh,debci,-c,/tmp/adt-run.7lKKGR/tree0-tree/debian/tests/installed-tests /dev/null /tmp/adt-run.7lKKGR/tree0t-installed-tests-stdout /tmp/adt-run.7lKKGR/tree0t-installed-tests-stderr /tmp/adt-run.7lKKGR/tree0-tree timeout=10000 env=TMPDIR=/tmp/adt-run.7lKKGR/tree0t-installed-tests-testtmp/tmpdir env=ADTTMP=/tmp/adt-run.7lKKGR/tree0t-installed-tests-testtmp/adttmp env=ADT_ARTIFACTS=/tmp/adt-run.7lKKGR/tree0t-installed-tests-testtmp/artifacts env=LANG=C', got `timeout', expected `ok...'

As it happens, all the current dbus installed-tests are meant to be
fast (< 1 minute), so for the next upload I'm going to wrap each test in
coreutils timeout(1) and see what happens; but it seems wrong to be
second-guessing adt-run's own timeout logic?

Regards,
    S