[Pkg-octave-devel] First try on autopkgtest for OF packages

[Sébastien Villemot]

On Sun, Aug 27, 2017 at 05:04:34PM +0200, Rafael Laboissière wrote:
> * Sébastien Villemot <sebastien at debian.org> [2017-08-26 22:01]:
> 
> > On Fri, Aug 25, 2017 at 09:14:44PM +0200, Rafael Laboissière wrote:
> > > I have designed a very simple way of adding autopkgtest functionality to the
> > > OF packages.  You can try it with the following commands:
> > > 
> > > [snip]
> > 
> > I did not yet give a try to it, so I may have missed something, but are
> > you sure that it catches test failures? When the unit tests are run at
> > build time, failures are never fatal (we have to look at the build log
> > to catch them), and my understanding is that the autopkgtest relies on
> > the same codebase. If we want the autopkgtest to be useful, it should
> > actually catch test failures through the appropriate exit code.
> 
> Yes, I confirm that it does not catch test failures.  In order to do that,
> we must change the check-pkg script, such that it would be able to collect
> the failures and optionally exit with an error code if any of the tests
> fails.

Ok, sounds good. Note that we will need to whitelist some test failures in some
packages, which can probably be achieved by patching the .m file and use the
expected failure functionality of Octave.

I am also wondering if we should not make test failures fatal at build time.
This would probably cause a few FTBFS in the short term, but in the longer run
this would improve the quality of the packages.

> My goal was just to make a proof of concept in which it is possible to use
> the same script in both build and aufopkgtest contexts.

Thanks for that.

-- 
⢀⣴⠾⠻⢶⣦⠀  Sébastien Villemot
⣾⠁⢠⠒⠀⣿⡁  Debian Developer
⢿⡄⠘⠷⠚⠋⠀  http://sebastien.villemot.name
⠈⠳⣄⠀⠀⠀⠀  http://www.debian.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/pkg-octave-devel/attachments/20170829/2fb9b91d/attachment.sig>