[buildd-tools-devel] Making sbuild releases

[Roger Leigh]

On Mon, Sep 29, 2014 at 05:46:20PM +0100, Wookey wrote:
> +++ Wookey [2014-09-29 12:41 +0100]:
> > +++ Roger Leigh [2014-09-28 09:45 +0100]:
> > > On Thu, Sep 25, 2014 at 10:10:30PM +0100, Wookey wrote:
> > > > +++ Benjamin Drung [2014-09-25 16:50 +0200]:
> > > > > Hi,
> > > > > 
> > > > > due to some bugs (#700522, #714883, #756265), I like to offer my help to
> > > > > maintain sbuild. Do you have any rules for committing to the git
> > > > > repository and for doing releases?
> > > > 
> > > > I'd like some advice on procedure too.
> > > 
> > > I'll add notes on how to make a release to git, i.e. what to tag,
> > > test and run to make the release tarball.  It's not too involved,
> > > but it probably needs writing down.
> 
> I found the HACKING file which actually gives some useful details.
> 
> However it seys to use 'make check' which is good advice but as a
> makefile is not built directly in the source tree you need to run it
> elsewhere. It seems to build in debian/build but running make check in
> there does not work due to 'wrong number of ../' 
> 
> Running make -f debian/rules/Makefile check works OK

It may be the case that the test scripts don't work out-of-tree.  You
certainly can run

  ./configure --enable-chroot-checks

and then "make check".  The option here enables checks which require
a working schroot/sbuild setup on the host system.

> (Is there a reason that we don't run make check during the normal
> package build? That is generally deemed good practice?)

The checks are for the most part reliant on a local schroot setup to
test against (unstable chroot).  This isn't present in a typical
build environment.  This is also why the tests probably only work
in-tree since it's only made sense to test locally before making a
commit or release from git.

> It also tells you to use ./configure --enable-maintainer-mode which now gives:
> configure: WARNING: unrecognized options: --enable-maintainer-mode.
> but make distcheck does now run.  I guess enable-maintainer-mode is obsolete now?

I think so, and this should probably be removed.

> make distcheck fails near the end with recipie for target 'distcleancheck' failed:
> EROR: files left in build directory after distclean 
> <list of manpages>

Hmm, that could be a bug in the Makefile.am.

See the bottom of configure.ac and man/Makefile.am.  The manpages are
preprocessed by configure (config.status) to add the release version and
date etc.  And symlinks are created on install/uninstall.  Are the listed
manpages "real" or symlinks?  Or both?

> It does make an sbuild-ver dir and tarball, which is good, but
> does it _inside_ the sbuild git dir, which seems wrong. Should I be
> invoking it some other way? or is this just because it didn't complete?

This is typical behaviour for automake "make dist".  It goes into the
build directory which, in this case, is the source (git) directory.

> make check at this point works OK, which I guess is what the HACKING
> file intended people to do, despite listing the actions with make
> check first?

make check should work straight after running configure. You might need
to run "make all" first, but I dont't think that's strictly needed.

> Also ./bootstrap currently runs autoreconf, but the normal package
> build doesn't. In line with the rest of debian we should probably be
> re-autoconfing during build. Anybody have philosophical obhjections to
> that?

Not me.  The bootstrap is needed to set the version number from NEWS,
update the ChangeLog etc. when building from git.  But for the debian
package build itself, should be absolutely fine to autoreconf as needed.

> So all this more-or less works, but it seems that the HACKING file is
> slightly out of date, and I may not be issuing quite the right runes as
> there is some breakage. 

I think you've made good progress :)

The only other missing bits to add to HACKING if not already present are:

  ./scripts/git-tag-release to GPG-sign a release
  ./scripts/git-tag-debian to GPG-sign a Debian release

(run both for making a new release; if you have a back-branch for e.g.
backports you can just reuse the old tarball and tag the debian changes
and just have them in the .diff.gz)


> I'll prod about some more. Is there a place where our 'upstream'
> releases go? I note that
> https://alioth.debian.org/projects/buildd-tools/ only has v0.55 (may
> 2007) so I guess that's not the place :-)
> Or do we just immediately use the orig.tar.gz created to make a debian
> -1 release?

I used to copy all our sbuild (and schroot) releases onto alioth.  But
then there were disc space restrictions and there wasn't space to
continue with all the history and add new releases, so I stopped.  It
would be nice to have that, but I now just rename it to use as the
.orig.tar and have snapshots/archive.d.o store it, and use this as the
-1 release.

> git clone git://git.debian.org/git/buildd-tools/sbuild
> !patch (attached) (patch -p2 < 0.63-release.patch) to update NEWS,  UNRELEASED -> unstable!
> in that dir do
> scripts/git-tag-release
> ./bootstrap
> ./configure
> make distcheck (make distcleancheck fails due to leftover manpages)
> make check (passes, optional)
> mv sbuild-0.64.3.tar.gz ../sbuild-0.64.3.orig.tar.gz
> sudo rm -r sbuild-0.64.3  (tidying up due to failed distcleancheck?)

I think so.  If you run plain "make dist" you shouldn't have a
directory left to remove.

> make distclean (more tidying?)

> dpkg-buildpackage

I'd run dpkg-buildpackage in the unpacked tarball from the above steps.
(Or "sbuild -s -A -d $dist .").  Definitely don't do it in the git
repo, it needs to be in the unpacked release.

> That process generates a sensibly-sized orig.tar.gz of 570K (i.e not
> full of build artifacts), and plausible-looking packages with the
> right versions, so apart from the man-page distcleancheck issue may be
> somewhere near the expected procedure.
> 
> Am I doing anything wrong to have the distcleancheck failure and orig
> tarball that needs copying up a level, or does it just need fixing?

I think as above, copy the tarball to an empty build directory, unpack
and build that.

For distclean/distcheck, I think these are problems with the automake
files, but they shouldn't stop you making a release.

> Should I be using the generated-then-unpacked sbuild.orig.tar.gz to do
> the debian dpkg-buildpackage step (as opposed to the git checkout) -
> or does it not actually matter?

Yes you should, and yes it does matter ;-)

> Should I update the HACKING file to clarify the stuff I've stumbled over above?

That would be greatly appreciated.

> I realise these are a lot of dumb questions, but without having done
> this before it seems easy to make a wrong assumption about exactly how this is
> intended to be used.

No problem, none of the questions are dumb, and I'd certainly prefer that
questions are asked rather than mistakes being made!  I've generally been
the only one making full releases, doing the tagging/testing/tarball
creation for a good few years, so documenting this and sharing the
expertise is definitely a good thing.  And if anything is clearly daft,
please don't hesitate to point it out so we can see if it could be
improved!


Thanks,
Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux    http://people.debian.org/~rleigh/
 `. `'   schroot and sbuild  http://alioth.debian.org/projects/buildd-tools
   `-    GPG Public Key      F33D 281D 470A B443 6756 147C 07B3 C8BC 4083 E800