Bug#741330: autopkgtest: please add ability to wrap a script/runner/adverb around existing tests

[Martin Pitt]

Hey Simon,

this has been stuck for a while, so let's see how we can get at least
some of this working.

Simon McVittie [2014-04-04 20:42 +0100]:
> Not quite: it can be a shell command line (with multiple words). The
> idea was that you can do
> 
>     Interpreter: dbus-run-session -- env XDG_RUNTIME_DIR=$ADTTMP --
> 
> to wrap the test in both dbus-run-session and env. Perhaps this is
> over-engineering and I should be using debian/tests/environment.sh
> (containing that command-line + "$@") as my Interpreter, but I think

I recently refactored the test metadata parsing to be more easily
extensible (as I added support for Ubuntu's click packages). During
that course, the internal "test" representation can now hold either a
path to an executable (as it always has) or a shell command. This
isn't exposed yet for debian/tests/control, but it would be fairly
simple now to write the above as

  Test-Command: <piece of shell>

so you can put any adverbs or other wrappers directly into the control
file. We could support multiple Test-Command: lines in each paragraph
so that you can still have a common Restrictions:/Depends etc, and of
course being a shell command you can use globbing, `find`, etc.

This has an obvious semantics, and with a shell command you should be
able to do pretty much anything. Does that suit your needs?

> > * Read tests from a file:
> > 
> > Tests: < debian/tests/testlist
> 
> If it can be a file in the filesystem, I think that would be fine -
> packagers of GNOME stuff could arrange for it to be generated during
> the build and installed alongside the *.test files. If not, the only
> mechanism I can see for arranging for it to be generated would be to
> repeat the entire package build, which seems a shame when the GNOME
> installed-tests are specifically designed to not need that.

If we actually need this, it's not too hard to implement now, but I
wonder if there's a real-world use case for this already?

> > * Globbing:
> > 
> > Tests: glib*.test
> 
> In GNOME installed-tests, the actual files of interest are
> /usr/share/installed-tests/$PACKAGE/*.test and
> /usr/share/installed-tests/$PACKAGE/**/*.test (using the rsync/zsh
> notation where ** matches any number of levels of directories), so
> there would need to be a way to say "cut off this prefix when naming
> the tests".

Likewise, it's now rather easy to implement, and perhaps even better
than "Tests: <" from above?

Thanks,

Martin
-- 
Martin Pitt                        | http://www.piware.de
Ubuntu Developer (www.ubuntu.com)  | Debian Developer  (www.debian.org)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.alioth.debian.org/pipermail/autopkgtest-devel/attachments/20140705/409b6733/attachment.sig>