DEP8 tests using the built package source

[Antonio Terceiro]

Adding the autopkgtest-devel list to involve the current maintainers.

On Sun, Mar 16, 2014 at 11:27:06PM +0100, Stephen Kitt wrote:
> (Hi Ian, I'm adding you to the discussion since I'm talking about changes to
> DEP8. I've snipped the exchanges but I hope you can get the gist of it
> without needing to spend time looking anything else up!)
> 
> On Sun, 16 Mar 2014 17:39:20 -0300, Antonio Terceiro <terceiro at debian.org>
> wrote:
> > On Sun, Mar 16, 2014 at 08:39:38PM +0100, Stephen Kitt wrote:
> > > On Sun, 16 Mar 2014 15:41:15 -0300, Antonio Terceiro <terceiro at debian.org>
> > > wrote:
> > > > ... this is an important point. You have to make sure that the any
> > > > tests will run against the code that is _installed_ and not against
> > > > the code that was just built. Also, it would be really nice on the
> > > > test infrastructure if you could build strictly the bits you need to
> > > > the tests instead of building everything. e.g. the ideal would be
> > > > build _only_ the unit test files (assuming they need to be built)
> > > > and not all the other code (since you are supposed to run the tests
> > > > against the installed version of the package)
> > > 
> > > Indeed, thanks to Jakub for pointing that out. I've reworked the
> > > upstream tests to build using the installed package.
> > > 
> > > Your point concerning building only the required bits effectively
> > > means that we shouldn't use "build-needed" in the tests/control
> > > "Restrictions" field, but manually control the build only for the
> > > purposes of running the tests, am I right?
> > 
> > most probably, yes.
> > 
> > It's always a compromise.  Sometimes it's reasonably easy to patch the
> > upstream test suite to make it not expect locally-built binaries, not
> > use local includes etc. Sometimes it may be a lot harder, so YMMV.
> 
> Right, and in this particular instance it's not particularly difficult.
> 
> What bothers me is that the current DEP8 spec says that packages can rely on
> having their source tree in the built state by stating "Restrictions:
> build-needed", but effectively that imposes too much of a burden on the
> testing infrastructure. (That's not a complaint, I don't think we should
> require another buildd network to run tests, at least not until we've got as
> much test code as binary-targeted source code.) It's the kind of expectation
> that makes sense in a "traditional" CI setting (e.g. Jenkins with Maven for
> Java projects, where the project is built and its tests run in the same
> environment), but with DEP8's aim of testing the installed binaries it seems
> less useful to me. Wouldn't it make sense to change DEP8 to encourage
> building whatever is strictly required for the tests, and perhaps drop
> "build-needed" altogether?
> 
> Regards,
> 
> Stephen



-- 
Antonio Terceiro <terceiro at debian.org>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.alioth.debian.org/pipermail/autopkgtest-devel/attachments/20140317/fa979618/attachment.sig>