Bug#720458: autopkgtest: allow tests to depend on Build-Depends installed

[Martin Pitt]

Hey Stefano,

Stefano Zacchiroli [2013-08-30 15:10 +0200]:
> And I wonder if we couldn't have better support for it at a different
> level. All in all, the need of the suggested feature seems to be
> justified by the desire to avoid duplication of information which is
> already written elsewhere (namely: in a *subset* of
> Build-Dependencies). Can't we, say, build something more general on top
> of, say, substvars to better support this need?

General substvars only exist during package build, but I don't mind
adding a specific ${builddepends} or something to adt.

> Of course there is nothing worry with running build-time test suites
> *again* via autopkgtest, but it does smell of bad design.

Why? If test suites are designed to run both against the built source
tree as well as the installed program, that's perfect? Then one
doesn't need to maintain two separate test suites. It's certainly not
usually necessary to test all the fine details again in an autopkgtest
(we usually only need a smoketest), but we already found curious bugs
by running detailled tests in autopkgtest.

> Or are we hinting here at the fact that it would be better (in an
> ideal world we're only starting to scratch :)) to run *all* test
> suites via autopkgtest, therefore relieving the buildds of the need
> of running them?

I don't think that either replaces the other. Tests during build have
the advantage that they run on all architectures, as you already said,
and moreover that they don't even cause debs to be built from the
faulty source. As long as Debian doesn't have a "staging area" for
builds and tests before .debs get released to unstable users (like
ubuntu is using with devel-proposed), that's a good way to have bugs
detected as early as possible IMHO.

> An obvious problem with that idea is that we will probably never
> have autopkgtest runners for all the architectures for which we have
> buildds...

Running autopkgtest on all arches is extremely difficult due to the
lack of proper virtualization. And if we had them for all arches, they
wouldn't be any machine usage relief any more, to the contrary
(preparing a VM for testing is much more expensive than a "make check"
during pacakge build where everything is already installed and the
test doesn't need root).

Thanks,

Martin

-- 
Martin Pitt                        | http://www.piware.de
Ubuntu Developer (www.ubuntu.com)  | Debian Developer  (www.debian.org)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.alioth.debian.org/pipermail/autopkgtest-devel/attachments/20130830/c1a495fa/attachment.sig>