[Buildd-tools-devel] buildd-tools sbuild post-etch

[Roger Leigh]

Hi all,

This mail is just to outline a few thoughts I have about where to take
sbuild after the release of Etch.  It's basically a wish-list of
things I'd like to propose for Lenny.  Comments and criticism are
welcome, as are any additional ideas for changes.

I've CC'd debian-devel for wider comments, since most sbuild users are
not subscribed to the -devel list.

For reference, the current sbuild is available here:
  svn://svn.debian.org/svn/buildd-tools/trunk/sbuild


1. Source Dependencies
----------------------

source-dependencies files have been deprecated since the advent of
package build dependencies, and the andrea output on buildd.debian.org
was turned off sometime last year.  This was unnoticed until kmuto
reported update-sourcedeps was broken (#409642).

As a result, I have removed update-sourcedeps in SVN.  Hopefully this
will make Etch.

Support for reading and using source-dependencies files is still in
place in sbuild, and will remain for compatibility for Etch.  I'd like
to completely strip this out of sbuild for Lenny since it is obsolete,
unused and rather complex.


2. Security (and Extensibility)
-------------------------------

I have been concerned for some time with the security of sbuild, and
the potential for abuse.  Currently, sbuild needs:

a) sudu access to both the host and chroot system, typically
   unrestricted and passwordless.  This is a major flaw, wide open for
   abuse if someone accesses your account.  I also find it dangerous
   for the user themselves with full sudo access--there's a great
   potential for accidents.
b) schroot access, with the individual user granted passwordless root
   access to the chroot environment.  While not as easy to abuse, it's
   still trivial to gain complete root access to the entire system
   from within the chroot with only a little thought (this applies to
   having root inside any chroot).

I'd like to propose a rethink of how we run sbuild.

Currently the chroots are located in $build_dir/chroot-$dist (sudo) or
called sbuild-$dist (chroots managed by schroot).  What I'd like to do
is split schroot into a client and server, which communicate over a
socket.  The server can run as root, and use schroot to access the
chroot.  The client simply submits and retrieves build jobs to the
server.  This eliminates all the need for the user to have *any* root
privileges to build, which I think will be a big improvement in
security.

With this setup, the chroot will be under the control of sbuild,
rather than the user, but it could still be configured to use the old
locations if desired.

While this sounds like a big change, in reality it will be a small set
of changes.  We can write a simple client and server in C++, and the
server will manage the chroot and run a slightly modified form of the
existing sbuild, as the user who requested the build.

This would be in effect a mini "build daemon" that will allow jobs to
be queued.  This has the potential to farm out jobs to multiple
chroots, and (in the very far future) allow remote job submission if
we want to let it listen on a network socket to allow distributed
building.

However, I'd like to keep things very small and simple to begin with.

One thing I'd like to do is merge it with schroot (or rather, split
out libsbuild from schroot, which can then be shared by both).  This
will allow the server to manage all the chrooted building quite
simply.  The code has in fact already been written for this purpose
for over a year, and is now well tested and debugged.  Support is also
already integrated in sbuild for over a year.  All we will be doing is
rearranging how the pieces fit together.


3. Perl
-------

sbuild is currently written in Perl, and is mostly an uncommented,
unmanageable and unmaintainable tangle which is they way it is due to
having "evolved" into a spaghetti mess over the course of many years.
Over the last year, I have attempted to gradually split it into more
maintainable chunks.  However, this is complicated due to the
requirements of [Part 4].

With the above changes in mind, I'd like to gradually rewrite pieces
of it in (clear and commented) C++ and move them into the server.
Hopefully this will allow for the removal of most of the accumulated
cruft, as well as making it understandable again.

"apt-get source schroot" for an example of the type of coding I
prefer.  This comes with a complete API reference, with everything
extensively commented.


4. Upstream
-----------

Upstream is http://svn.cyberhqz.com/svn/wanna-build

While I am continuing to track upstream, I have never yet received a
single reply to any of the patchsets I sent.  So far, one trivial
single line patch has been applied upstream.  I don't think that there
is much interest upstream to merge any of our changes, most of which
are in fact bugfixes and usability fixes.  As a result, by tying
ourselves to upstream for compatibility, we can't make much progress
unless patches get merged both ways.

Is it better to make a complete break?


5. buildd
---------

Currently, we only provide sbuild in the Debian archive.  For a few
months, a complete buildd setup has existed in our SVN repository.

This is available here:
  svn://svn.debian.org/svn/buildd-tools/trunk/buildd

This, like sbuild, is tracking upstream.  However, it needs some work
before it will be fit for the Debian archive, due to these
considerations (amongst others):
- LSB compliance
- A good, working default configuration
- Integration with buildd-tools sbuild, in particular the sbuild perl
  modules and schroot support, and the configuration files.
- Up-to-date packaging

Getting wanna-build to (optionally) use a proper relational database
such as PostgreSQL is also something I would like, while continuing to
support the current format.

If anyone is interested in working on it, please drop a mail to the
buildd-tools-devel list.


Regards,
Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux             http://people.debian.org/~rleigh/
 `. `'   Printing on GNU/Linux?       http://gutenprint.sourceforge.net/
   `-    GPG Public Key: 0x25BFB848   Please GPG sign your mail.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 188 bytes
Desc: not available
Url : http://lists.alioth.debian.org/pipermail/buildd-tools-devel/attachments/20070210/d979bdd7/attachment.pgp