autopkgtest'ing against multiple Python versions
Jakub Wilk
jwilk at debian.org
Sat Apr 21 20:33:24 UTC 2012
I started adding DEP-8 tests to my packages. It worked reasonably well
so far (after I fixed #648148 localy...). However, I have doubts about
which strategy to choose when it comes to testing with multiple Python
versions. The following schemes come to my mind:
1) Test only with default version:
Tests: test-default
Depends: python-pet-module, python
This is very straight-forward to implement. Also, it looks like that
soon we'll have only one supported Python 2.X version anyway, so not
testing against non-default versions wouldn't be a big deal.
2) Provide one test that is run against all supported versions
(pyversions -r):
Tests: test-all
Depends: python-pet-module, python-all
This is also very easy to implement. The disadvantage is that in some
cases a (non-root) user won't be able to run any test even though he has
some (or maybe even all) Python interpreters installed.
3) Provide one test that is run against all installed versions
(pyversions -i):
Tests: test-installed
Depends: python-pet-module
This is a variant of 2 which is more friendly to users who can't or
don't want to install extra packages. It's also less useful for users
who would install test dependencies automatically anyway (since then
you do want to install python-all). This issue could be fixed if
autopkgtest supported weak dependencies, e.g.:
Tests: test-installed
Depends: python-pet-module
Recommends: python-all
4) Provide two tests:
- for testing with default version;
- for testing with all versions.
Tests: test-default
Depends: python-pet-module, python
Tests: test-all
Depends: python-pet-module, python-all
It his case one could always run at least the first test. But on the
other hand the second one is wasteful...
5) Provide two tests:
- for testing with default version;
- for testing with non-default versions.
Tests: test-default
Depends: python-pet-module, python
Tests: test-nondefault
Depends: python-pet-module, python-all
It's a less wasteful variant of 4. However, it's also more difficult to
implement.
What do you think? I'm currently leaning towards 3.
--
Jakub Wilk
More information about the autopkgtest-devel
mailing list