DEP8 tests using the built package source

[Jakub Wilk]

* Antonio Terceiro <terceiro at debian.org>, 2014-03-17, 09:59:
>>>>>You have to make sure that the any tests will run against the code 
>>>>>that is _installed_ and not against the code that was just built.

What I've been doing to my packages, is to copy all the bits necessary 
to run tests from the package directory to $ADTTMP, then chdir to 
$ADTTMP, and run tests from there. This greatly mitigates the risk of 
accidentally testing against not-installed code.

>>What bothers me is that the current DEP8 spec says that packages can 
>>rely on having their source tree in the built state by stating 
>>"Restrictions: build-needed", but effectively that imposes too much of 
>>a burden on the testing infrastructure. (That's not a complaint, I 
>>don't think we should require another buildd network to run tests, at 
>>least not until we've got as much test code as binary-targeted source 
>>code.) It's the kind of expectation that makes sense in a 
>>"traditional" CI setting (e.g. Jenkins with Maven for Java projects, 
>>where the project is built and its tests run in the same environment), 
>>but with DEP8's aim of testing the installed binaries it seems less 
>>useful to me. Wouldn't it make sense to change DEP8 to encourage 
>>building whatever is strictly required for the tests, and perhaps drop 
>>"build-needed" altogether?

The problem with with "build-needed" is that it causes the full package 
build, which is often way more than the tests actually need.

But the alternative of building the required bits inside the tests 
itself is not appealing either, because to do that you would typically 
need to add stuff to Depends.

It would be great if there was a way to tell the test runner to run a 
dedicated debian/rules target (say, "debian/rules adt-prepare"), instead 
of doing full-blown build.

-- 
Jakub Wilk