Bug#823979: autodep8: Support for ELPA testsuites

[Antonio Terceiro]

Control: tag -1 + moreinfo

On Tue, May 10, 2016 at 02:53:07PM -0700, Sean Whitton wrote:
> Package: autodep8
> Version: 0.5.1
> Severity: wishlist
> Tags: patch
> 
> Dear maintainers,
> 
> Please consider applying the following patch which adds support for the
> testsuites of Emacs Lisp addon packages.

Hi, thanks for your patch. Comments below.

> diff --git a/support/elpa/detect b/support/elpa/detect
> new file mode 100755
> index 0000000..7fd6287
> --- /dev/null
> +++ b/support/elpa/detect
> @@ -0,0 +1,18 @@
> +#!/bin/sh
> +
> +# Maintainer: Sean Whitton <spwhitton at spwhitton.name>
> +
> +# We check that: this package uses dh_elpa, debhelper compat level is
> +# at least 10 (see dh_elpa_test(1)), and the user hasn't disabled
> +# dh_elpa_test, and that there is actually a testsuite to run
> +
> +set -e
> +
> +# check that the file exists first, so that if it doesn't cat doesn't
> +# produce stderr output which disrupts other test/*_test.sh
> +test -e debian/compat
> +test "$(cat debian/compat)" -ge 10
> +(! grep -q ".*DH_ELPA_TEST_DISABLE.*" debian/rules)
> +grep-dctrl -q -F Build-Depends dh-elpa debian/control
> +( grep -q -r ".*(ert-deftest.*" . || \
> +        grep-dctrl -q -F Build-Depends elpa-buttercup debian/control )
> diff --git a/support/elpa/generate b/support/elpa/generate
> new file mode 100755
> index 0000000..3d89bc7
> --- /dev/null
> +++ b/support/elpa/generate
> @@ -0,0 +1,19 @@
> +#!/bin/sh
> +
> +# Maintainer: Sean Whitton <spwhitton at spwhitton.name>
> +
> +# Remarks:
> +
> +# 1. Many ELPA test suites write to the build tree, so we need that
> +#    restriction in the general case.
> +
> +# 2. We invoke `debian/rules build' rather than simply `dh_elpa_test'
> +#    because that ensures that any DH_ELPA_TEST_* environment
> +#    variables the user has specifies in debian/rules are exported
> +#    such that dh_elpa_test can see them.  See dh_elpa_test(1).
> +
> +cat <<EOF
> +Test-Command: debian/rules build
> +Depends: @, @builddeps@
> +Restrictions: rw-build-tree
> +EOF

Even without having a clue about ELPA, or emacs in general, I can guess
with a certain degree of confidence that those tests will be testing the
source tree, not the installed packages. The purpose of
autopkgtest/DEP-8 is testing the installed binary packages, not the
source tree.

Is it possible to run the tests against the installed packages, making sure
that code from the source tree is not used (except of course the test code
itself)?

If it's possible, it would be useful if dh_elpa_test could get input
from outside of debian/rules so that tests  can be run be used
independent of a build (e.g. dh_elpa_test --autopkgtest). For an example
on how that can be done, you can look at the "FILES" section of
dh_ruby(1) (package gem2deb), in special the description of
debian/ruby-test-files.yaml, debian/ruby-tests.rake,
debian/ruby-tests.rb, and gem2deb-test-runner(1)

you could for example make dh_elpa_test read environment variables form
debian/dh_test_elpa.conf or something similar, and make it also have a
--autopkgtest switch that makes sure the tests will not load code from
the source tree, then use that switch here in autodep8:

Test-Command: dh_test_elpa --autopkgtest
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/autopkgtest-devel/attachments/20160511/bf4c1a95/attachment.sig>