[pkg-bioc] Problem building CRAN packages

[Johannes Ranke]

Hi Dirk,

* Dirk Eddelbuettel <edd at debian.org> [060620 15:10]:
> 
> Hi Johannes,
> 
> And a CC to Andrew who may be joining the fun shortly...

Looking forward to that!
 
> On 20 June 2006 at 10:27, Johannes Ranke wrote:
> | Dear all,
> | 
> | After fixing a small typo in cran2deb.pl (committed), and adding a
> | couple of build-dependencies to the 
> | 
> | 	  ## List of packages that build once their dependencies are
> | 	  ## available in the build environment.
> | 
> | (not committed), I have successfully built 485 packages so far.
> 
> Pretty good start!  Congrats! 
> 
> I only got 461 when I played with some modifications a few weeks ago.... Some
> of my changed are still not checked in, and I am not sure if it makes sense
> now. 

I did not change any of the logic, since I only partly understand it. I
just added a couple of packages to the "cannotbuild" array, with
comments like "needs r-cran-thisandthat" or "fails with ...".

I just want to make sure, that I am doing the right thing, before I
commit those.
 
> | 1) Why are these dependencies not handled in the array that seems to
> | be made for build dependencies:
> | 
> | 	my %buildDeps=(
> | 	           arules    => ["r-cran-matrix"], ##
> | 				...
> | 				...

Well, in @cannotbuild there is the comment

	## List of packages that build once their dependencies are
    ## available in the build environment.
    ## To be installed, for example, by something like:
    ## dpkg -i builds/r-cran-...
	## The number in parenthesis is the number of run of this scripts
	## found necessary before commenting out the non-build of the package

And then, there is lines like

	   "distr",         # r-cran-setrng.

This looks like a build dependency to me.

Later, in the associative array (or however this is called in perl)
named %buildDeps, build dependencies are specified by lines like

	   distr     => ["r-cran-setrng"],

Only now I noticed that these are even (partially?) duplicated.  I
understand that the entries in the first array disable the build attempt
until they are commented out. 

The later entries are used 
	a) to determine if the package requires packages from contrib
	b) is added to $packages{$package}{'debianbuilddependencies'} 
	   which is in turn later printed as $bdps, if not empty.

Now that I analyzed this, I can tentatively answer the question myself:

The first array is useful to make the skript proceed, while the second
array tells the user what he should install in order to make the skript
proceed.

Please correct me, if I didn't get it right. If I did, my conclusion
is that it would be preferable to put all build dependencies in one
place (probably %buildDeps), check for them, build an array of
unsatisfied build dependencies, print it for the user, and skip packages
with unsatisfied build-dependencies, instead of aborting.


> Not sure I understand the question.  Are you asking why particular
> dependencies are missing?  That could be simply because the script has not
> been updated in a while.

>  
> | 2) At the moment, I repeatedly encounter failures like
> | 
> | building package builds/r-cran-pcurve_0.6.2-0.r2d.1_amd64.deb ...
> | debuild: fatal error at line 1224:
> | fakeroot debian/rules binary failed
> | fakeroot debian/rules clean
> | dpkg-source -b pcurve-0.6.2
> | dpkg-source: cannot represent change to src/Getlam.o: binary file
> | contents changed
> | dpkg-source: warning: executable mode 0755 of `src/pcurve.so' will not
> | be represented in diff
> | dpkg-source: cannot represent change to src/pcurve.so: binary file
> | contents changed
> | dpkg-source: unrepresentable changes to source
> 
> Standard building errors indicating that the directory wasn't cleaned which
> should happen from 'R CMD INSTALL' as well as from the script which removes
> directories.   There must be a reason that failed here..

You mean a reason why the directory wasn't cleaned? I can't think of
anything it would have be special to this package and some others that
gave me similar errors.
 
> | dpkg-buildpackage: source package is pcurve
> | dpkg-buildpackage: source version is 0.6.2-0.r2d.1
> | dpkg-buildpackage: source changed by CRAN/BioC packagers
> | <pkg-bioc-devel at lists.alioth.debian.org>
> | dpkg-buildpackage: host architecture amd64
> | dpkg-buildpackage: source version without epoch 0.6.2-0.r2d.1
> | test -x debian/rules
> | test "`id -u`" = 0
> | dh_clean
> | dpkg-source: building pcurve using existing pcurve_0.6.2.orig.tar.gz
> | dpkg-source: building pcurve in pcurve_0.6.2-0.r2d.1.diff.gz
> | dpkg-source: building pcurve in pcurve_0.6.2-0.r2d.1.dsc
> | build failed for builds/r-cran-pcurve_0.6.2-0.r2d.1_amd64.deb built in
> | builds/pcurve-0.6.2
> | 
> | What is the problem? The format of the source version string?
> 
> No, I think the build errors above drive that...
> 
> | 3) Did anybody ever create a condensed version of cran2deb.pl for
> | building just an exactly defined subset of packages? I'd be interested.
> 
> We need to redo the script, at least in parts. The "big loop" around the core
> is hundreds of lines.

Yes, the first step would be to split the script.
 
> If and when it has been refactored, it should be possible to call for just
> one file.  In fact we could even rearchitecture such that each call is
> treated like just one (re-reading the meta information may be costly that
> way, though).

As I said in my last mail to Gregor Gorjanc, I am thinking of an array
of package names (maybe already including build-dependencies), as input
to the script. In this way, things that only need to be done once, would
only be done once (ideally).
> 
> Hope this helps, Dirk

It sure did!

Johannes