[cut-team] Rethinking CUT

[Colin Watson]

On Thu, Sep 09, 2010 at 10:13:36PM -0400, Michael Gilbert wrote:
>     1.  The provided testing installation media is often broken, missing
>         functionality, and at times unable to install testing itself.
[...]
> Issue #1 has various vectors for solutions that don't necessarily
> invoke the need for the complexity of the current CUT proposals.  A few
> approaches that I can think of are:
> 
>     a.  Use the "official" stable media (relabeled as testing) to
>         install testing. This is how I install testing on my systems
>         since I can't really tell the state of the devel
>         debian-installer until I try it and run into a problem.
>         Obviously this approach could be problematic since only hardware
>         that's compatible with the previous installer will be supported.
>         Perhaps a known incompatible hardware list could be downloaded
>         and displayed to the user?  Also, bugs that break the testing
>         installation functionality (such as [0]) would have to be
>         treated with the utmost urgency.  This would be a stop-gap
>         solution for most of testing's lifetime until new d-i reaches a
>         release candidate state.

There's a vicious circle here.  We need to be able to work on d-i, if
nothing else so that we can have something good for the next stable
release; continuing to use the old one hinders that goal, and only makes
it *less* likely that at some point we'll be able to switch over to a
new version because hardly anyone will have been trying to use it.  Plus
there are a lot of contortions required to make the old d-i work with a
newer distribution.  In general we're happy to go through some
contortions to make new d-i work with older Debian, but the other way
round tends to require a time machine.

Concrete example: on 2 July, I uploaded a new version of grub2 that
fixed a clutch of RC bugs and involved some changes to its debconf
interaction.  I uploaded a matching version of grub-installer a mere
five and a half hours later.  Yet to this day we get reports from users
of squeeze d-i alpha 1 that they get asked a spurious question during
installation.  When you think about it, this means that testing is
actually *worse* than unstable, and using even older stable media is
just going to make that problem worse.  (The code in grub2 is already
horrifically complicated and as a matter of good software engineering I
would prefer that we not require it to be complicated further because
our process can't cope with keeping grub2 and grub-installer at vaguely
matching versions.)

> [0] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=529475

That bug was fixed the day after it was reported.  I don't know whether
that's breaking for alpha 1 users, but it seems quite plausible that it
might be.  We can only fix this kind of thing by way of a process that
encourages using a reasonably current installer rather than old media.

>     b.  Provide a live cd copier like the one used on Ubuntu's live
>         installation media or as Stefano pointed out Linux Mint's new
>         live testing installer.

Based on my experience in Ubuntu, I would say that this doesn't meet all
the requirements people have of d-i.  Copying a live filesystem is OK
for fairly standardised desktops but it's very difficult to make it
flexible.

In the Ubuntu case, the way we get the live installer working on a new
release (after merging lots of stuff from Debian) involves getting d-i
working first, so I don't see that as an alternative anyway - and Debian
Live's installer is a d-i component.

>     c.  Keep the testing installer in a "constantly" supported/working
>         state.  I'm not sure how feasible that is due to the flux that
>         testing goes through during transitions, etc., but it would be
>         the ideal solution.  Perhaps, and admittedly I'm not familiar
>         with d-i's current development process, all installer
>         development should happen in experimental instead of
>         testing/unstable. This would free up testing to house only
>         (mostly) stable d-i packages; thus providing a constantly
>         working installer.

If there's a problem with d-i's process, it's more the other way round:
getting new installer code into testing takes too long, and problems
remain unfixed in testing for too long.  I don't think we have a serious
problem with major installer regressions remaining unfixed in unstable,
particularly not given that d-i development is relatively slow at the
moment, and so I don't think that moving d-i development to experimental
would help.  In general I'd rather that d-i be coupled more tightly to
the rest of Debian, not less.

See my grub2/grub-installer example above, which is also relevant here.

>         In order to preserve this state, all non-bugfix/non-planned
>         uploads of packages with udebs to unstable would need to be
>         blocked to preserve the stability of the installer due to the
>         automatic unstable->testing transition process.

This is already the case: packages with udebs are all blocked
automatically, and require acks from debian-boot before they get
promoted, to try to avoid situations where they break the last published
alpha of the installer.  (But of course there's no way to express
dependencies or breakages between debs and udebs in a programmatic way,
and it isn't necessarily sane to e.g. block a new version of grub2 from
testing for months while we wait for a new d-i release ...)

The problem with d-i development at the moment is more that we're very
slow at producing new d-i *releases*.  One of the demotivating factors
there is, as I understand it, that when you try to prepare those
releases you're constantly fighting to get the rest of the distribution
sorted out, since if the set of packages you're working with
fundamentally can't be installed to give you a usable system then it's
pretty hard to test certain parts of the installer.  Your choices right
now are to work with stable (too old), testing (would be nice except for
the way sometimes it breaks and then it tends to take a week to fix
anything), unstable (breaks all the time).

Fundamentally, the fact that people want any given d-i release to keep
working with a target that's going to keep on moving after the d-i
release is a very difficult engineering problem to solve.  Snapshotting
the thing we're installing every so often, and thereby coupling a given
version of our not-yet-released installer to a given version of our
not-yet-released distribution, would make things so much easier for our
users.  This in turn would take some of the pressure off us so that we
can deal with the business of producing new software rather than dealing
with fiddly compatibility issues between slightly different versions of
our existing software.


Things are better than in the bad old days of boot-floppies; Anthony
doesn't have to send repeated mails to debian-devel-announce pleading
for someone, anyone to do any work on the installer at all or else we
can't release.  But still, observationally, d-i release managers tend to
find other things more interesting or else burn out.  In order to scale,
we need to make this less painful.

I realise perhaps as much as anybody that Ubuntu's development model is
very different, but what we do there is: every few weeks, we say "OK,
we're rolling a new milestone release now", we get the archive
installable, sync up kernel versions with d-i if necessary, fix anything
that's broken in d-i (usually not too much), do a few test runs, and
mark it as a milestone release (though media only, we don't snapshot the
archive).  I've been handling d-i for nearly every Ubuntu milestone
release for six years and I haven't burnt out.  Obviously the fact that
I draw a paycheque for it helps, but it's really just not that painful.
So, why is it so hard in Debian, with basically the same codebase?

That's a bit of a rhetorical question, and of course I can come up with
lots of possible answers (part-time development model, developers not
necessarily having release production as their primary goal, running
install tests is time-consuming, trade-off between automatic checks on
testing and easy promotion of fixes, etc.).  Ubuntu's model certainly
has its own drawbacks and for the avoidance of doubt I'm not saying that
Debian should just work like Ubuntu.  But the contrast between our
respective abilities to produce new sets of installation media, even the
basic kind as opposed to full CDs, is very marked, and as a d-i
developer I observe that contrast and would like that part of the
process to be easier.  The thing I do in my free time really ought to be
at least as much fun as the thing I'm paid to do!

-- 
Colin Watson                                       [cjwatson at debian.org]