autopkgtest: Generic Test Harness / Network Client Service testing

[Christian Seiler]

Dear autopkgtest Maintainers,

I'm unsure if this is the correct list for this - if not, please
redirect me to the proper place for that. Also, please Cc me on
replies, as I'm not subscribed.

I have two questions regarding autopkgtest that I hope you might be
able to help me with:

1. I'm currently working on the list of bugs in the open-iscsi package
and there's a wishlist bug there to add autopkgtests for that package.
It also comes with a patch.[1]

The patch itself contains a file debian/tests/testlib.py, which is
basically a python library providing all sorts of useful helpers for
testing daemons etc. But it's also >1000 lines long and there are a
couple of things in there that I'd like to improve. A quick codesearch
in Debian revealed that this library has already been copied to 3 other
Debian packages:
http://sources.debian.net/src/postfix/2.11.3-1/debian/tests/testlib.py/
http://sources.debian.net/src/nut/2.7.2-4/debian/tests/testlib.py/
http://sources.debian.net/src/mailman/1:2.1.20-1/debian/tests/testlib.py/

Personally, I think having an embedded copy of the whole thing is
probably a bad idea in the long run - especially if I start improving
it for the convenience of the package I'm currently working on,
effectively creating divergent versions of the same piece of software.

I think it would make sense to package that - and autopkgtest seems to
be the best place for that (maybe an additional binary package?). What
do you think?


2. In order to properly test open-iscsi, one needs to be able to log in
to an iSCSI target. The patch in bug[1] doesn't really do this by
default, it only tests logging in to the target if one modifies the
test script and manually hacks in a host name - which will definitely
not be generic enough.

Of course, I could simply install targetcli (from LIO) on the same host
and log in to localhost, but then it would be really hard to expand the
automatic tests to situations like whether iSCSI login at boot works
properly or even having root on iSCSI or the such. I think a couple of
other services would have similar problems (think NFS etc.) if
autopktest were to be thoroughly implemented for them.

Can one tell autopkgtest to provision two VMs? One which is configured
to provide some generic services - and then a second VM that contains
the package itself (both can see each other and IP addresses of the own
and other VMs are available for scripts) in order to test whether it
works properly against the service that's set up.

If that's not possible, do you think this idea has some merit?



Thanks in advance!

Best regards,
Christian

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=710261