DEP-8 extension proposal: Add source package header

[Stefano Zacchiroli]

[ Adding autopkgtest-devel to recipients, where I've also just bounced
  your mail. If you're interested in this topic, please consider
  subscribing to that, very low traffic, list. ]

On Wed, Jun 13, 2012 at 09:44:46AM +0200, Martin Pitt wrote:
> in Ubuntu we have started adding some autopkgtests to packages, using
> the DEP-8 debian/tests/control standard [1]. We now also have a test
> execution environment [2] which automatically triggers autopkgtest
> runs for a package and all its reverse dependencies on each upload.

Oh, that's nice, thanks! Is the test execution environment something
that can easily deployed elsewhere? While autopkgtest itself (the
package) has received attention in Debian, we haven't yet managed to
restart test executions. I'm happy to consider alternatives if they're
more readily available for deployment than existing alternatives.

I'm also curious about the implementation: do you actually use
autopkgtest as low level test runner for Jenkins integration or...?

> The main problem that we are facing with this is that it is not easily
> discoverable which source packages actually ship tests. Right now this
> is a hardcoded list, which does not scale at all. Thus I would like to
> propose to add a source package header, so that test execution
> environments can find out packages with tests by merely scanning
> Sources.gz (or iterating over the package dict with python-apt and the
> like).
> 
> What do you think about something like this? (Diff against [3])

As a temporary alternative to a hardcoded list, Stuart's proposal to use
Contents-source.gz is clearly better.

But as a long term solution I like source metadata, as you propose,
more. It is more independent from the source package layout and it makes
it trivial to access the metadata via APT, as you point out. More
profoundly, I'm inclined to consider testsuite information as first
class source metadata, no less than other similar metadata like the
Vcs-* fields. The space bloat problem seems negligible in this case,
especially considering we're talking about Sources rather than Packages.

As long as it stays as a XS-* header (which, for the history, it's also
how Vcs-* fields came into existence), no tool change is needed. We
should just agree upon a name. But if we want to have hopes to promote
it to something more official, I think it should rather be independent
from autopkgtest. How about something like "XS-Testsuite: runtime",
where the key is actually a space separate list of values values.  Right
now only "runtime" [1] makes sense, but others might appear in the
future.

Cheers.


[1] or maybe "autopkgtest" if we want to be specific about the
    testrunner, but that doesn't seem to be a god idea
-- 
Stefano Zacchiroli zack@{upsilon.cc,pps.jussieu.fr,debian.org} . o .
Maître de conférences ......  http://upsilon.cc/zack ......  . . o
Debian Project Leader .......  @zack on identi.ca .......  o o o « the
first rule of tautology club is the first rule of tautology club »
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 828 bytes
Desc: Digital signature
URL: <http://lists.alioth.debian.org/pipermail/autopkgtest-devel/attachments/20120613/d8c733bb/attachment-0001.pgp>