[Pkg-octave-devel] more detail on test failures in buildd logs

[Mike Miller]

Hi,

As you know, Octave's build succeeds even when some unit tests fail in
dh_auto_test. I wonder if the DOG thinks it would be desirable, and if
so, possible, to see the full test outputs for failing tests and have
them available either as part of the package or on a publicly available
web site. I think it would be useful to see which tests fail and why on
a particular architecture.

First, do you know if it is possible to get at the contents of the
buildd packaging directory that resulted in a given binary package (or
failure) on the buildd network? I assume that it is discarded once the
build is finished.

One possibility of seeing the test results would be to install the full
fntests.log file as part of one of the binary packages. I don't favor
this option because the file serves no purpose to end users and because
it's mostly redundant, all we really want is the failures, not a full
list of passing tests.

I discussed with upstream whether it would be possible to parse the
fntests.log file to extract only the failures, which could be written to
a file or to stdout. I now have an awk script that does this to some
extent, still needs some improvement.

Any other clever ideas?

If not I will likely work on refining this awk script and see if I can
get it merged upstream. The goal there would be for a 'make check' with
--disable-silent-rules to automatically dump test failures to stdout.

Thanks,

-- 
mike