Bug#817079: autopkgtest: *3* copies for running a test?!

[Rene Engelhard]

Package: autopkgtest
Version: 3.19.3
Severity: important

Hi,

I have the following test:

--- snip ---
Tests: junit-subsequentcheck
Depends: libreoffice, libreoffice-subsequentcheckbase, libreoffice-sdbc-hsqldb, [...]
--- snip --

Then I run

$ adt-run -B -s -d --unbuilt-tree . --timeout-copy=10000 --- null

from the *source package*.

(there's no build needed, as junit-subsequentcheck already does built what it
needs. copy timeout choosen as "randomly big", the default for sure is not
enough for LO. Note this:

(jessie)rene at frodo ..Office/libreoffice/libreoffice-5.1.1.3 % du -hs
3.1G	.
)

So let's see.

adt-run: DBG: Parsed options: Namespace(apt_pocket=[], auto_control=True, copy=[], env=[], gainroot=None, logfile=None, output_dir=None, set_lang=None, setup_commands=[], shell=False, shell_fail=True, summary=None, timeout_build=None, timeout_copy=10000, timeout_factor=1.0, timeout_install=None, timeout_short=None, timeout_test=None, user=None, verbosity=2)
adt-run: DBG: Remaining arguments: ['-B', '--unbuilt-tree', '.']
adt-run: DBG: Interpreted actions: ['--no-built-binaries', '--unbuilt-tree', '.']
adt-run: DBG: Virt runner arguments: ['null']
adt-run: DBG: testbed init
adt-run [00:35:16]: version 3.19.3
adt-run [00:35:16]: host frodo; command line: /usr/bin/adt-run -B -s -d --unbuilt-tree . --timeout-copy=10000 --- null
adt-run: DBG: got reply from testbed: ok
adt-run: DBG: testbed open, scratch=None
adt-run: DBG: sending command to testbed: open
adt-run: DBG: got reply from testbed: ok /tmp/adt-run.c6BSZ8
adt-run: DBG: sending command to testbed: print-execute-command
adt-run: DBG: got reply from testbed: ok env
adt-run: DBG: sending command to testbed: capabilities
adt-run: DBG: got reply from testbed: ok isolation-machine downtmp-host=/tmp/adt-run.c6BSZ8
adt-run: DBG: testbed capabilities: ['isolation-machine', 'downtmp-host=/tmp/adt-run.c6BSZ8']
adt-run: DBG: testbed command ['dpkg', '--print-architecture'], kind short, sout pipe, serr pipe, env []
adt-run: DBG: testbed command exited with code 0
adt-run [00:35:17]: testbed dpkg architecture: amd64
adt-run: DBG: testbed command ['which', 'eatmydata'], kind short, sout pipe, serr pipe, env []
adt-run: DBG: testbed command exited with code 1
adt-run: DBG: testbed command ['which', 'dpkg-query'], kind short, sout pipe, serr pipe, env []
adt-run: DBG: testbed command exited with code 0
adt-run: DBG: testbed command ['sh', '-ec', "dpkg-query --show -f '${Package}\\t${Version}\\n' > /tmp/adt-run.c6BSZ8/testbed-packages"], kind short, sout raw, serr pipe, env []
adt-run: DBG: testbed command exited with code 0
adt-run: DBG: sending command to testbed: copyup /tmp/adt-run.c6BSZ8/testbed-packages /tmp/adt-run.output.4ufg40yp/testbed-packages
adt-run: DBG: got reply from testbed: ok
adt-run: DBG: testbed command ['uname', '-srv'], kind short, sout pipe, serr pipe, env []
adt-run: DBG: testbed command exited with code 0
adt-run [00:35:17]: testbed running kernel: Linux 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt20-1+deb8u4 (2016-02-29)
adt-run: DBG: Binaries: initialising
adt-run [00:35:17]: @@@@@@@@@@@@@@@@@@@@ unbuilt-tree .
adt-run: DBG: blame += .
adt-run: DBG: testbed reset: modified=False, deps_installed=[](r: False), deps_new=[](r: False)
adt-run: DBG: testbed command ['mkdir', '-p', '/tmp/adt-run.c6BSZ8'], kind short, sout raw, serr pipe, env []
adt-run: DBG: testbed command exited with code 0
adt-run: DBG: sending command to testbed: copydown ./ /tmp/adt-run.c6BSZ8/ubtree-./


Here it copies the source tree to /tmp/. OK. I assume ubtree means
"unbuilt tree". #1

adt-run: DBG: got reply from testbed: ok
adt-run: DBG: testbed command ['sh', '-ec', 'exec 3>&1 >&2; set -x; cd /; builddir=$(mktemp -d /tmp/adt-run.c6BSZ8/build.XXX); cd $builddir; cp -rd --preserve=timestamps -- "/tmp/adt-run.c6BSZ8/ubtree-." real-tree; chmod -R a+rX .; cd [a-z0-9]*/.; pwd >&3; sed -n "1 {s/).*//; s/ (/\\n/; p}" debian/changelog >&3; set +e; grep -q "^Restrictions:.*\\bbuild-needed\\b" debian/tests/control 2>/dev/null; echo $? >&3'], kind build, sout pipe, serr raw, env []
+ cd /
+ mktemp -d /tmp/adt-run.c6BSZ8/build.XXX
+ builddir=/tmp/adt-run.c6BSZ8/build.Pdm
+ cd /tmp/adt-run.c6BSZ8/build.Pdm
+ cp -rd --preserve=timestamps -- /tmp/adt-run.c6BSZ8/ubtree-. real-tree

Here it copies the source tree from #1 to _an other_ "real" tree. Why? This is
cp #2.

+ chmod -R a+rX .
+ cd real-tree/.
+ pwd
+ sed -n 1 {s/).*//; s/ (/\n/; p} debian/changelog
+ set +e
+ grep -q ^Restrictions:.*\bbuild-needed\b debian/tests/control
+ echo 1
adt-run: DBG: testbed command exited with code 0
adt-run [00:52:46]: testing package libreoffice version 1:5.1.1~rc3-2
adt-run [00:52:46]: build not needed
adt-run: DBG: sending command to testbed: copyup /tmp/adt-run.c6BSZ8/build.Pdm/real-tree/ /tmp/adt-run.output.4ufg40yp/tests-tree/

What on earth? Another copy? From "real-tree" to "tests-tree"?

adt-run [00:52:46]: testing package libreoffice version 1:5.1.1~rc3-2
adt-run [00:52:46]: build not needed
adt-run: DBG: sending command to testbed: copyup /tmp/adt-run.c6BSZ8/build.Pdm/real-tree/ /tmp/adt-run.output.4ufg40yp/tests-tree/
adt-run: DBG: got reply from testbed: ok
adt-run: DBG: processing dependency libreoffice
[...]

AH, FINALLY.

Copying 3.1GB three times? Especially as there's no build-needed etc.
At least "real-tree" to "test-tree" can be avoided, if you already have a
"ubtree"? And if you have "ubtree" and don't need a build, why copy it over
again? It's not as you run into the dir you started it from.

1 copy to /tmp should be enough here. Takes already enough time and disk.

Or maybe

01:01 < terceiro> if should be probably mounted inside the testbed instead of 
                  copied

Regards,

Rene

-- System Information:
Debian Release: stretch/sid
  APT prefers unstable-debug
  APT policy: (500, 'unstable-debug'), (500, 'buildd-unstable'), (500, 'unstable'), (1, 'experimental-debug'), (1, 'buildd-experimental'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 3.16.0-4-amd64 (SMP w/4 CPU cores)
Locale: LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages autopkgtest depends on:
ii  apt-utils       1.2.5
ii  libdpkg-perl    1.18.4
ii  procps          2:3.3.11-3
ii  python3         3.5.1-2
ii  python3-debian  0.1.27

Versions of packages autopkgtest recommends:
ii  autodep8  0.3

Versions of packages autopkgtest suggests:
pn  lxc          <none>
pn  lxd-client   <none>
ii  qemu-system  1:2.5+dfsg-5
ii  qemu-utils   1:2.5+dfsg-5
pn  schroot      <none>

-- no debconf information