[debhelper-devel] dh_auto_test - Integration with CTest/CDash

[Niels Thykier]

Kyle Edwards:
> Hello all,
> 

Hi :)

> I'm currently working on a Debian package for the upcoming VTK 9, and as
> part of this endeavor, I'm setting up nightly .deb builds and nightly
> testing on Debian Sid. One thing that I'd like to be able to do is
> combine these into one step: the same process that builds the nightly
> .debs also submits the dh_auto_test results to our CDash server. (CDash
> is a companion product of CTest - it is a dashboard server which hosts
> the results of nightly tests from test machines which choose to submit
> their results.)
> > [... More context ...]
> 

Thanks for the context. :)

> What I would like to have is the option to submit the results of
> dh_auto_test to CDash, though certainly not a requirement to do so.

How would you normally do this without the Debian packaging stack?  Is
it something like:

  cmake ...
  make/ninja build
  make test/ninja test
  cdash ... ?

Note that cmake is not a very strong suit for me so rather assume I know
less rather than more.

> Here are some of the challenges I see:
> 
> 1. Deciding how much of this process to leave to the CTest dashboard
> script and how much to leave to the Debian packaging system.

Indeed.  As far as I can tell the "Debian packaging system" covers both
debian/rules and debhelper.  See below (on splitting the build steps)

> 2. The dashboard script also has site-local settings, which could
> potentially be challenging to pass to the builder, especially if the
> builder is sbuild.
> 

If it does not contain passwords or sensitive material, then it could be
copied into the debian dir before the source package was built and have
the build use the package from debian/.  But given the nature of what it
is doing, I have a feeling that you will shoot down that assumption.

> I'm still a bit of a beginner on Debian packaging. Is it possible (and
> practical) to separate out the build steps? For now I've just been
> running "gbp buildpackage", which does everything at once, but if the
> build steps can be run separately, I may be able to write a dashboard
> script which executes these steps.
> 

gbp buildpackage effectively just calls debian/rules (via some layers of
indirection and some magic to optimize git workflows).  Without having
seen your rules file, I assume it is something like:

"""
#!/usr/bin/make -f

%:
	dh $@ --buildsystem cmake+ninja
"""

If so, then you can leverage debhelper's override targets to override
the test target.  E.g. via:


"""
[...]

override_dh_auto_test:
	dh_auto_test [... options for dh_auto_test ...]
	[... other commands to run here ...]
"""

See also dh(1) (Examples).  Please keep in mind that most dh_auto_*
helpers will pass anything after a "--" to to the underlying build
system.  E.g. if you need to pass an extra flag to cmake, then you can
use the following example:



"""
[...]

override_dh_auto_configure:
	dh_auto_configure --buildsystem cmake+ninja -- \
 		[... options for cmake ...]
"""


> Any other ideas would be appreciated.
> 
> Kyle
> 
> [...]

I hope this will be enough to get you further. :)

Thanks,
~Niels