MBF: packages with autopkgtest tests but no XS-Testsuite

Sun Nov 4 18:01:12 UTC 2012

It looks like we have already 68 packages in the Debian archive shipping
autopkgtest [1] tests, which is great!  But unfortunately only 3 of them
declare the presence of the testsuite adding the recommended [2]
"XS-Testsuite: autopkgtest" header to their source stanza in
debian/control.

The dd-list of packages shipping autopkgtest tests but lacking the
header is attached. Please let me know if you notice any false positive.

I'd like to go ahead with a "Severity: wishlist" mass bug filing to
kindly ask maintainers to add the missing header.  The proposed mail
template for mass-bug is attached.

To avoid new instances of the problem in the future, I've also requested
a new lintian check [3]. In the meantime it'd be useful to ask
maintainers to add XS-Testsuite where needed.

Many thanks to Holger Levsen for setting up http://jenkins.debian.net/
and initiating this discussion on -qa [4], and to Jakub Wilk for making
me realize how many autopkgtest-enabled packages we already have in the
archive.

Cheers.

[1]: http://packages.qa.debian.org/a/autopkgtest.html
[2]: http://anonscm.debian.org/gitweb/?p=autopkgtest/autopkgtest.git;a=blob_plain;f=doc/README.package-tests;hb=HEAD
[3]: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=692282
[4]: https://lists.debian.org/debian-qa/2012/11/msg00009.html
-- 
Stefano Zacchiroli  . . . . . . .  zack at upsilon.cc . . . . o . . . o . o
Maître de conférences . . . . . http://upsilon.cc/zack . . . o . . . o o
Debian Project Leader . . . . . . @zack on identi.ca . . o o o . . . o .
« the first rule of tautology club is the first rule of tautology club »
-------------- next part --------------
Adam Schmalhofer <Adam.Schmalhofer at gmx.de>
   apipkg
   execnet
   pytest-xdist

Andreas Tille <tille at debian.org>
   mafft (U)

Antonio Terceiro <terceiro at debian.org>
   pstreams
   rake (U)
   ruby-switch (U)

Arnaud Fontaine <arnau at debian.org>
   python-mechanize (U)
   zc.buildout (U)
   zope.testbrowser (U)

Brian Sutherland <brian at vanguardistas.net>
   bobo (U)
   python-chameleon (U)
   python-mechanize (U)
   sourcecodegen (U)
   transaction (U)
   zc.buildout (U)
   zc.lockfile (U)
   zconfig (U)
   zdaemon (U)
   zodb (U)
   zope.authentication (U)
   zope.browser (U)
   zope.cachedescriptors (U)
   zope.component (U)
   zope.configuration (U)
   zope.contenttype (U)
   zope.copy (U)
   zope.dottedname (U)
   zope.event (U)
   zope.exceptions (U)
   zope.hookable (U)
   zope.i18n (U)
   zope.i18nmessageid (U)
   zope.interface (U)
   zope.location (U)
   zope.proxy (U)
   zope.publisher (U)
   zope.schema (U)
   zope.security (U)
   zope.sendmail (U)
   zope.sqlalchemy (U)
   zope.testbrowser (U)
   zope.testing (U)
   zope.testrunner (U)
   zope.traversing (U)

Charles Plessy <plessy at debian.org>
   mafft (U)

Christian Perrier <bubulle at debian.org>
   samba4 (U)

Debian Bazaar Maintainers <pkg-bazaar-maint at lists.alioth.debian.org>
   bzr
   bzr-git
   bzr-rewrite
   bzr-stats
   bzr-svn
   bzr-upload

Debian Med Packaging Team <debian-med-packaging at lists.alioth.debian.org>
   mafft

Debian Python Modules Team <python-modules-team at lists.alioth.debian.org>
   gamera (U)
   python-byteplay (U)
   sphinx (U)

Debian QA Group <packages at qa.debian.org>
   bzr-fastimport
   bzr-loom

Debian Ruby Extras Maintainers <pkg-ruby-extras-maintainers at lists.alioth.debian.org>
   rake
   ruby-switch

Debian/Ubuntu Zope Team <pkg-zope-developers at lists.alioth.debian.org>
   bobo
   python-chameleon
   python-mechanize
   sourcecodegen
   transaction
   zc.buildout
   zc.lockfile
   zconfig
   zdaemon
   zodb
   zope.authentication
   zope.browser
   zope.cachedescriptors
   zope.component
   zope.configuration
   zope.contenttype
   zope.copy
   zope.deprecation
   zope.dottedname
   zope.event
   zope.exceptions
   zope.hookable
   zope.i18n
   zope.i18nmessageid
   zope.interface
   zope.location
   zope.proxy
   zope.publisher
   zope.schema
   zope.security
   zope.sendmail
   zope.sqlalchemy
   zope.testbrowser
   zope.testing
   zope.testrunner
   zope.traversing

Fabio Tranchitella <kobold at debian.org>
   bobo (U)
   python-chameleon (U)
   python-mechanize (U)
   sourcecodegen (U)
   transaction (U)
   zc.buildout (U)
   zc.lockfile (U)
   zconfig (U)
   zdaemon (U)
   zodb (U)
   zope.authentication (U)
   zope.browser (U)
   zope.cachedescriptors (U)
   zope.component (U)
   zope.configuration (U)
   zope.contenttype (U)
   zope.copy (U)
   zope.dottedname (U)
   zope.event (U)
   zope.exceptions (U)
   zope.hookable (U)
   zope.i18n (U)
   zope.i18nmessageid (U)
   zope.interface (U)
   zope.location (U)
   zope.proxy (U)
   zope.publisher (U)
   zope.schema (U)
   zope.security (U)
   zope.sqlalchemy (U)
   zope.testbrowser (U)
   zope.testing (U)
   zope.traversing (U)

Jakub Wilk <jwilk at debian.org>
   gamera
   ocrad
   python-byteplay
   sphinx

Jeff Bailey <jbailey at raspberryginger.com>
   bzr (U)

Jelmer Vernooij <jelmer at debian.org>
   bzr (U)
   bzr-git (U)
   bzr-rewrite (U)
   bzr-svn (U)
   python-fastimport
   samba4 (U)

Jérémy Bobbio <lunar at debian.org>
   python-mechanize (U)

Koichi Akabe <vbkaisetsu at gmail.com>
   bzr-stats (U)
   bzr-upload (U)

KURASHIKI Satoru <lurdan at gmail.com>
   bzr-email

Ludovic Brenta <lbrenta at debian.org>
   libgtkada

Maintainer: Debian QA Group <packages at qa.debian.org>
   bzr-upload

martin f. krafft <madduck at debian.org>
   pstreams (U)

Martin Pitt <mpitt at debian.org>
   udisks (U)

Matthias Klose <doko at ubuntu.com>
   python-mechanize (U)
   zope.dottedname (U)
   zope.testing (U)

Michael Biebl <biebl at debian.org>
   udisks (U)

Nicolas Boulenguez <nicolas.boulenguez at free.fr>
   libgmpada
   libgtkada (U)
   libncursesada
   libtexttools

Noèl Köthe <noel at debian.org>
   samba4 (U)

Ond?ej Surý <ondrej at debian.org>
   rake (U)

Paul van Tilburg <paulvt at debian.org>
   rake (U)

Piotr Lewandowski <piotr.lewandowski at gmail.com>
   sinntp

Python Applications Packaging Team <python-apps-team at lists.alioth.debian.org>
   sinntp (U)

Reinhard Tartler <siretart at tauware.de>
   bzr (U)

Robert Collins <robertc at robertcollins.net>
   bzr-svn (U)

Samba Debian Maintainers <pkg-samba-maint at lists.alioth.debian.org>
   samba4

Steinar H. Gunderson <sesse at debian.org>
   samba4 (U)

Steve Langasek <vorlon at debian.org>
   mawk
   samba4 (U)

TANIGUCHI Takaki <takaki at debian.org>
   zope.deprecation (U)

Utopia Maintenance Team <pkg-utopia-maintainers at lists.alioth.debian.org>
   udisks

Wouter van Heyst <larstiq at larstiq.dyndns.org>
   bzr (U)

-------------- next part --------------
[ This is an automated bug report, submitted as part of the mass bug
  filing discussed at TODO-ADD-URL-TO-DEVEL-THREAD-HERE ]

According to the Contents file, #PACKAGE# ships a test suite for the
autopkgtest testing framework [1]. Thanks a bunch for that, those test
suites are a great way to improve the overall quality of the Debian
archive!

However, the presence of test suites in packages should be easily
discoverable, to allow test runners to pick suitable packages and run
their test suites.  According to the autopkgtest specification [2], if
you ship an autopkgtest test suite you should add the following header
to the *source stanza* of your debian/control file:

  XS-Testsuite: autopkgtest

It seems that #PACKAGE# lacks such a header.
Can you please add it?

Thanks for considering,
Cheers.

[1]: http://packages.qa.debian.org/a/autopkgtest.html
[2]: http://anonscm.debian.org/gitweb/?p=autopkgtest/autopkgtest.git;a=blob_plain;f=doc/README.package-tests;hb=HEAD
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 828 bytes
Desc: Digital signature
URL: <http://lists.alioth.debian.org/pipermail/autopkgtest-devel/attachments/20121104/a4672c72/attachment.pgp>