mass autopkgtest Perl run [2014-09-20]

Niko Tyni ntyni at debian.org
Sun Sep 21 20:10:50 UTC 2014


On Sat, Sep 20, 2014 at 02:15:13PM +0200, Martin Pitt wrote:
> Niko Tyni [2014-09-18 23:56 +0300]:
> > Once #762013 (autopkgtest: detect test.pl in perl packages) is fixed, I
> > think one more full rebuild with that and pkg-perl-autopkgtest 0.16 would
> > be good as running test.pl will probably pick up a few new failures too.
> 
> I ran the whole batch again yesterday. As usual the logs are on
> 
>   https://people.debian.org/~mpitt/tmp/logs-20140920.tar.xz
>   https://people.debian.org/~mpitt/tmp/perl-failed-tests.txt
>   https://people.debian.org/~mpitt/tmp/perl-failures/

Thanks once again.

> With the new pkg-perl-autopkgtest 0.16 

The logs look like you only had pkg-perl-autopkgtest 0.15 available.
The main change is that t/boilerplate.t tests are ignored with 0.16.
My tests indicate this made 31 new packages pass.

> there are 37 new successes, but
> also 32 regressions, i. e. these passed earlier and now fail:

> I think these new failures deserve a closer look. 

Right. I went through these, and found indeed a couple of bugs in
pkg-perl-autopkgtest. I've just uploaded 0.17 which fixes 21 of those 32
regressions. The rest are either real bugs in the packages or tradeoffs:

- including dummy MANIFEST+MANIFEST.SKIP files in the test directory
  makes more new passing packages than new failed ones (I think).

- setting AUTOMATED_TESTING and NONINTERACTIVE_TESTING is the right thing
  to do even though it activates author tests needing extra dependencies
  in some buggy packages

>   https://people.debian.org/~mpitt/tmp/perl-failures/libapp-termcast-perl/auto-build-deps-stdout

FWIW this particular one is due to a new upstream release that now needs
files outside t/, so it requires a bit of manual tuning.

I see libgnome2-vfs-perl used to skip tests with "You have no ~/.gnome"
but didn't in the last go, so presumably your test setup has changed a bit.
(The package is probably buggy if it looks in $HOME, though; bug filed.)

libextutils-autoinstall-perl got killed due to a timeout, but builds
fine for me. Is that reproducible for you?

> > Unless I've broken something, I think the passing packages from that
> > could go live on the ci.debian.net whitelist and we'll start fixing the
> > remaining ones and adding the explicit Testsuite headers.
> 
> Yes, I agree. So far I kept perl.txt up to date with new
> pkg-perl-autopkgtest versions, so at no point there should be any new
> failures. (I'm sure there will be a handful due to flaky tests, timing
> issues, etc.)

Yeah, at least libtest-cgi-multipart-perl is undeterministic (and RC-buggy
because of that.)

So would you like me to update the whitelist with the 32+21 new passing
packages, or do we do one more test run with pkg-perl-autopkgtest 0.17?
(In the latter case, waiting for the binNMUs in #760123 would be prudent
as we're currently in a small Perl transition.)

PS: I've just documented all this a bit at
 http://pkg-perl.alioth.debian.org/autopkgtest.html
Comments welcome.
-- 
Niko Tyni   ntyni at debian.org



More information about the autopkgtest-devel mailing list