[cut-team] Rethinking CUT

[Michael Gilbert]

On Fri, 10 Sep 2010 10:13:19 +0100, Colin Watson wrote:
> On Thu, Sep 09, 2010 at 10:13:36PM -0400, Michael Gilbert wrote:
> >     1.  The provided testing installation media is often broken, missing
> >         functionality, and at times unable to install testing itself.
> [...]
> > Issue #1 has various vectors for solutions that don't necessarily
> > invoke the need for the complexity of the current CUT proposals.  A few
> > approaches that I can think of are:
> > 
> >     a.  Use the "official" stable media (relabeled as testing) to
> >         install testing. This is how I install testing on my systems
> >         since I can't really tell the state of the devel
> >         debian-installer until I try it and run into a problem.
> >         Obviously this approach could be problematic since only hardware
> >         that's compatible with the previous installer will be supported.
> >         Perhaps a known incompatible hardware list could be downloaded
> >         and displayed to the user?  Also, bugs that break the testing
> >         installation functionality (such as [0]) would have to be
> >         treated with the utmost urgency.  This would be a stop-gap
> >         solution for most of testing's lifetime until new d-i reaches a
> >         release candidate state.
> 
> There's a vicious circle here.  We need to be able to work on d-i, if
> nothing else so that we can have something good for the next stable
> release; continuing to use the old one hinders that goal, and only makes
> it *less* likely that at some point we'll be able to switch over to a
> new version because hardly anyone will have been trying to use it.  

So, the goal here is to provide something to user who just want
something that works.  These are the type of people that probably
wouldn't submit a bug report anyway if they ran into a problem.  They
would just get frustrated and give up.  

I think as long as you have enough people working on the new d-i, and
call for testers at times, you should have enough of a base to find
most of the bugs in that new version. But in the meantime, you also
have something else constantly working for the timid.

> Plus
> there are a lot of contortions required to make the old d-i work with a
> newer distribution.  In general we're happy to go through some
> contortions to make new d-i work with older Debian, but the other way
> round tends to require a time machine.

My idea there was that behind the scenes the installer would install a
stable base system install, then do a dist-upgrade to testing.  That
also has its problems since stable->testing upgrades are not
necessarily guaranteed while testing is under development.  Perhaps
that could be solved by automatically running piparts upgrade tests on
all unstable uploads and preventing failures there from entering
testing.

> >     b.  Provide a live cd copier like the one used on Ubuntu's live
> >         installation media or as Stefano pointed out Linux Mint's new
> >         live testing installer.
> 
> Based on my experience in Ubuntu, I would say that this doesn't meet all
> the requirements people have of d-i.  Copying a live filesystem is OK
> for fairly standardised desktops but it's very difficult to make it
> flexible.
> 
> In the Ubuntu case, the way we get the live installer working on a new
> release (after merging lots of stuff from Debian) involves getting d-i
> working first, so I don't see that as an alternative anyway - and Debian
> Live's installer is a d-i component.
> 
> >     c.  Keep the testing installer in a "constantly" supported/working
> >         state.  I'm not sure how feasible that is due to the flux that
> >         testing goes through during transitions, etc., but it would be
> >         the ideal solution.  Perhaps, and admittedly I'm not familiar
> >         with d-i's current development process, all installer
> >         development should happen in experimental instead of
> >         testing/unstable. This would free up testing to house only
> >         (mostly) stable d-i packages; thus providing a constantly
> >         working installer.
> 
> If there's a problem with d-i's process, it's more the other way round:
> getting new installer code into testing takes too long, and problems
> remain unfixed in testing for too long.  I don't think we have a serious
> problem with major installer regressions remaining unfixed in unstable,
> particularly not given that d-i development is relatively slow at the
> moment, and so I don't think that moving d-i development to experimental
> would help.  In general I'd rather that d-i be coupled more tightly to
> the rest of Debian, not less.
> 
> See my grub2/grub-installer example above, which is also relevant here.
> 
> >         In order to preserve this state, all non-bugfix/non-planned
> >         uploads of packages with udebs to unstable would need to be
> >         blocked to preserve the stability of the installer due to the
> >         automatic unstable->testing transition process.
> 
> This is already the case: packages with udebs are all blocked
> automatically, and require acks from debian-boot before they get
> promoted, to try to avoid situations where they break the last published
> alpha of the installer.  (But of course there's no way to express
> dependencies or breakages between debs and udebs in a programmatic way,
> and it isn't necessarily sane to e.g. block a new version of grub2 from
> testing for months while we wait for a new d-i release ...)
> 
> The problem with d-i development at the moment is more that we're very
> slow at producing new d-i *releases*.  One of the demotivating factors
> there is, as I understand it, that when you try to prepare those
> releases you're constantly fighting to get the rest of the distribution
> sorted out, since if the set of packages you're working with
> fundamentally can't be installed to give you a usable system then it's
> pretty hard to test certain parts of the installer.  Your choices right
> now are to work with stable (too old), testing (would be nice except for
> the way sometimes it breaks and then it tends to take a week to fix
> anything), unstable (breaks all the time).
> 
> Fundamentally, the fact that people want any given d-i release to keep
> working with a target that's going to keep on moving after the d-i
> release is a very difficult engineering problem to solve.  Snapshotting
> the thing we're installing every so often, and thereby coupling a given
> version of our not-yet-released installer to a given version of our
> not-yet-released distribution, would make things so much easier for our
> users.  This in turn would take some of the pressure off us so that we
> can deal with the business of producing new software rather than dealing
> with fiddly compatibility issues between slightly different versions of
> our existing software.
> 
> 
> Things are better than in the bad old days of boot-floppies; Anthony
> doesn't have to send repeated mails to debian-devel-announce pleading
> for someone, anyone to do any work on the installer at all or else we
> can't release.  But still, observationally, d-i release managers tend to
> find other things more interesting or else burn out.  In order to scale,
> we need to make this less painful.
> 
> I realise perhaps as much as anybody that Ubuntu's development model is
> very different, but what we do there is: every few weeks, we say "OK,
> we're rolling a new milestone release now", we get the archive
> installable, sync up kernel versions with d-i if necessary, fix anything
> that's broken in d-i (usually not too much), do a few test runs, and
> mark it as a milestone release (though media only, we don't snapshot the
> archive).  I've been handling d-i for nearly every Ubuntu milestone
> release for six years and I haven't burnt out.  Obviously the fact that
> I draw a paycheque for it helps, but it's really just not that painful.
> So, why is it so hard in Debian, with basically the same codebase?
> 
> That's a bit of a rhetorical question, and of course I can come up with
> lots of possible answers (part-time development model, developers not
> necessarily having release production as their primary goal, running
> install tests is time-consuming, trade-off between automatic checks on
> testing and easy promotion of fixes, etc.).  Ubuntu's model certainly
> has its own drawbacks and for the avoidance of doubt I'm not saying that
> Debian should just work like Ubuntu.  But the contrast between our
> respective abilities to produce new sets of installation media, even the
> basic kind as opposed to full CDs, is very marked, and as a d-i
> developer I observe that contrast and would like that part of the
> process to be easier.  The thing I do in my free time really ought to be
> at least as much fun as the thing I'm paid to do!

Thanks for the very interesting discussion.  This serves as a strong
foundation for further CIT discussion. It sounds like it would be very
useful to adapt some of Ubuntu's processes to stabilize d-i development.

Best wishes,
Mike