[Buildd-tools-devel] schroot: union filesystem based schroots

[Roger Leigh]

Jan-Marek Glogowski <glogow at fbihome.de> writes:

> The attached patch implements session chroots using a union filesystem
> like aufs or unionfs.

WOW!, thanks, this is a really exciting addition to schroot!  I really
like the idea of using union filesystems.

(I played with aufs for the first time last week at work, where I
helped set up a diskless netbooting compute cluster with Debian Live.)

> It's a first implementation based on the code for LVM snapshots.
> It's not well tested yet (just Etch and unionfs 1.4), but I want to get
> some feedback, especially how to solve the following problem:
>
> Old unionfs crashs, if you change any of it's branches not throught
> unionfs while unionfs is overlaying them. Therefore I need some lock when
> someone wants to enter the parent changeroot, while sessions are running.
> This should block new sessions from startup, and should block the parent
> until all sessions are closed.

It should be possible to create some form of lockfile to do this.  We
already have lock facilities (sbuild/sbuild-lock.h), but these are for
either files or block devices, rather than mounts.  As a result, I
don't think they would be appropriate.  One mechanism would be to look
at existing sessions, where we can lock the session files, but it
would still be slightly racy I think.

Overall, I really like the patch, especially with the care taken to
put everything in the manual pages.  There are a few minor syntactical
issues, but they can be taken care of later.

One thing I'd just like to suggest for consideration: Currently you
are deriving sbuild::chroot_directory, which makes sense.  But, why
should a union mount be restricted to directories?  Consider also that
block-device can be mounted and that mount location overlaid with a
union.  It doesn't make sense for "file" and "lvm-snapshot", but for
all other types (including "plain"), there is a use for this.

Additionally, consider that what you get is conceptually very similar
to "cloned" chroots such as "lvm-snapshot" and "file":

block-device    -> snapshot
[source: block-device]

tarfile/zipfile -> unpacked and not repacked
[source: tarfile/zipfile -> unpacked then repacked]

Is the "union" a chroot type, or something "extra" you could
potentially do with /any/ chroot?  In all the chroot types so far, the
"type" is referring to the /source/ of the data, be it a directory,
file, device etc., they all result in a directory we can chroot into.
A union filesystem, OTOH, is more of a specialisation, like a source
chroot.  It might even be considered as a specialisation of a source
chroot (which you have done in your code).

Basically, what I'm suggesting is that maybe it would, conceptually,
fit better /inside/, or derived from chroot_source as opposed to being
a totally separate type, then you could use it with all chroot types.
Because it doesn't really apply to file or lvm-snapshot, maybe the
other chroot types should inherit from a class derived from
chroot_source.  Or, maybe we should have directory-union, plain-union,
and block-device-union types which derive from e.g. sbuild_directory
and sbuild_source_union?  This would give a new -union type for each
type we want to add support for.

Oh, one other thought.  For the directory type, we bind mount it by
default.  If union mounts are reliable and generally desirable,
there's no reason why we couldn't also set up union mounts by default.

Note: I'm not actually suggesting you do any of this yet!--these are
just some ideas I thought of, and I would be very interested to hear
your thoughts (or anyone else reading the list).


Other minor suggestions/questions:

Should mount not be called with -p ?

Is it our place to modprobe kernel modules?  Or, should be admin load
the modules specifically?

Why does gettextize need adding to bootstrap?  We already call
autopoint to set it up.  Also, why does the gettext version change in
configure.ac?--through I have no objection to using a later version.

debian/control: We can't downgrade the boost version to build in
unstable, so I can't apply that part.

overlay_base: While a temporary directory is OK, we also need to
consider that the sessions should (by default) get preserved over a
reboot.  As a result, we might want to create /var/lib/schroot/overlay
for the purpose.  If having it on a tmpfs is important, we can allow
the user to customise this in schroot.conf.


This is what I came up with with a brief review--I'll have a more
detailed look later on.  Again, many thanks for taking the time to do
this!


Kind regards,
Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux             http://people.debian.org/~rleigh/
 `. `'   Printing on GNU/Linux?       http://gutenprint.sourceforge.net/
   `-    GPG Public Key: 0x25BFB848   Please GPG sign your mail.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 188 bytes
Desc: not available
Url : http://lists.alioth.debian.org/pipermail/buildd-tools-devel/attachments/20080122/168668ba/attachment.pgp