DEP8 tests using the built package source

Stephen Kitt skitt at debian.org
Mon Mar 24 23:45:35 UTC 2014


On Thu, 20 Mar 2014 08:24:15 +0100, Martin Pitt <mpitt at debian.org> wrote:
> Stephen Kitt [2014-03-19 23:54 +0100]:
> Well, it depends what you want to do. In CI we *always* want to test
> the packages from the actual archive of course. But as a package
> maintainer you may want to (and should) run your autopkgtest for an
> updated package *before* you upload it to the archive, and you want to
> test them with locally built binaries instead.

Of course, and ideally with the same setup as is used on the CI platforms in
Debian and Ubuntu.

> > Likewise for me. I find it particularly complicated to imagine what
> > variants of adt-run should be used to make sure a particular test-suite
> > is suitable for the CI platforms (ci.debian.net and Ubuntu's), which
> > basically means having to look up the source code of the platforms to
> > figure out what's going to happen, or uploading a package and waiting for
> > the results...
> 
> Recent versions of autopkgtest now ship the new
> /usr/share/doc/autopkgtest/README.running-tests.gz which I hope
> explains the use cases and how to run them.

It does explain the various use cases, but I found auto-package-testing more
helpful in that it contains the actual launcher scripts (prepare-testbed and
run-adt-test) used on the CI platforms.

> > To me, autopkgtest test-suites are nice for two things:
> > * tests which can't be run within a build (I started looking into all this
> >   for libevdev, which includes a test-suite that must be run as root);
> > * integration tests which combine multiple binary packages (e.g. Martin's
> >   example with the toolchain; I'm also working on a toolchain test-suite
> > for mingw-w64, which would combine the mingw-w64 toolchain and wine to
> > make sure the toolchain is working correctly).
> 
> Yes, for these. But also, you can do reverse dependencies testing and
> thus ensure that package A still works if one of its dependencies (say
> libfoo) gets updated, to prevent libfoo from landing if it breaks API.
> Uploading libfoo won't usually cause an upload/build of A otherwise,
> so running tests during build won't catch this.

Yup, that came under "combine multiple binary packages" from my point of
view :-).

> Also, autopkgtests test *packages* as they will appear on an actual
> system, unlike tests during build. The latter won't tell you if you
> forgot to install some files or put them into the wrong place, or your
> package has a missing Depends.

Ahah, that's an excellent point.

> > I'm wondering how all this meshes with Ubuntu's "fast migration" approach:
> > I'm guessing it doesn't see build-time tests which are run as part of the
> > build... 
> 
> It kind of does by rejecting packages which don't build on all
> platforms, i. e. if tests during build fail on a particular arch.

Right, I was thinking of the positive side; as I understood it there is some
sort of bonus in the "proposed" migration in Ubuntu for packages with working
autopkgtests. If that is correct then it would be better to use tests during
the build and also as autopkgtests, wouldn't it?

> > Would it be worth running them as autopkgtest tests *as well* for
> > this use case?
> 
> If you can, sure. Otherwise a quick smoke test is usually enough. Let
> the upstream tests during package build cover the fine details, and
> just let autopkgtest check that your package works in general (i. e.
> you can start a program, import a Python module, build a simple
> 5-liner against libfoo-dev, and so on).

That's what I liked about DEP8 at first — it helps avoid brown-bag errors ;-).

Regards,

Stephen
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/autopkgtest-devel/attachments/20140325/7e273e57/attachment.sig>


More information about the autopkgtest-devel mailing list