Contributing autopkgtest tests to important packages

[Antonio Terceiro]

Hello Christian,

On Sun, Mar 15, 2015 at 04:06:33PM +0100, Christian Kastner wrote:
> Hi,
> 
> I'm thinking of contributing autopkgtest tests to various important [1]
> packages that don't have any yet. Since I will be applying the same
> pattern over and over again, I'd like to ask for feedback first.

That is a very worthwhile initiative!

> The tests I want to add are just of a very basic nature; I merely want
> to increase the _breadth_ of testing. Increasing the _depth_ will be up
> to the individual maintainers as they have the experience with the
> package necessary to do this.

If you end up looking for a place to consolidate shared code that will
be used in multiple packages, we can think about adding it to autodep8.

> I plan to add the following tests:
> 
>   Test: upstream-testsuite{-flavor}
>         If the upstream source has a testsuite that can be invoked with
>         `make test` or some such, use it. The -flavor suffix is
>         optional, and useful for eg: python2/python3 distinctions.
> 
>   Motive:  no explanation necessary
>   Example: [2]

Note that `make test` and friends will usually run against the local
source tree, what somehow defeats the purpose of testing the installed
package (of course, it is still better than nothing!).

In the case of Ruby packages, for example, we provide a test runner that
will run the upstream test suite against the installed package when under
autopkgtest:

http://sources.debian.net/src/gem2deb/0.12/lib/gem2deb/test_runner.rb/#L77

The Perl team also provides a test runner that will do something
similar.

Another issue is that for compiled languages, a build may be necessary,
what brings two problems:

- the build may take a lot of time, slowing the whole CI process down.
- we are, again, not testing the code that is actually in the archive.

-- 
Antonio Terceiro <terceiro at debian.org>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.alioth.debian.org/pipermail/autopkgtest-devel/attachments/20150316/074ebdd4/attachment.sig>