Results for mass autopkgtest run for all ruby packages [2014-09-17]

[Martin Pitt]

Hello Ruby maintainers,

Antonio recently worked on getting centralized support for Ruby
package tests into autopkgtest [1].

Similarly to the recent mass-run of Perl packages, I now ran all the
ruby packages [2] through this.

To avoid the problem of having to kill hanging tests (most often just
in their cleanup, the actual tests usually work fine) I ran them with
--timeout-test=600 (i. e. 10 minutes). This killed 2 tests (grep for
"testbed failed: timed out"). Apologies if this killed a legitimately
running test, I'm happy to re-run those without a short timeout (or
you just do it yourself).

FYI, the whole run took ~ 3.5 hours; that's with a filled apt-cacher-ng,
directory (i. e. not tarball) schroot, and putting overlays in tmpfs,
i. e. pretty much as fast as you can get this.

Results:

  Total:    491       [2]
  Passing:  372 (76%)
  Failing:  119 (24%) <https://people.debian.org/~mpitt/tmp/ruby-failed-tests.txt>

The failed logs are in
https://people.debian.org/~mpitt/tmp/ruby-failures/

The complete (i. e. passed and failed) collection of logs is in
https://people.debian.org/~mpitt/tmp/ruby-20140917.tar.xz (450 kB)
I prefixed the directories with "fail-" and "pass-" in the tarball.

The command to run this:

for p in $(< ~/debian/debian-ci-config/ruby.txt); do if env -u DISPLAY adt-run -o $p --timeout-test=600 $p --- schroot sid; then mv $p pass-$p; else mv $p fail-$p; fi; done; alert
(autopkgtest 3.5.1)

Like with Perl, I suspect that there's a few common types of failures
which can be fixed centrally in gem2deb-test-runner or autopkgtest.

Once we want to actually mass-run those on ci.d.n. I recommend to
blacklist the known failures, to avoid introducing a lot of red.

Martin

[1] http://lists.alioth.debian.org/pipermail/autopkgtest-devel/2014-September/001066.html
[2] http://anonscm.debian.org/cgit/collab-maint/debian-ci-config.git/tree/ruby.txt
-- 
Martin Pitt                        | http://www.piware.de
Ubuntu Developer (www.ubuntu.com)  | Debian Developer  (www.debian.org)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.alioth.debian.org/pipermail/autopkgtest-devel/attachments/20140917/bacf1a2d/attachment.sig>