[cut-team] Thoughts on Snapshots

[Michael Gilbert]

Hi,

So, with debconf going on, I thought it may be prescient to think about
the current state of CUT, or at least the snapshot subset that I've
been working on for a while now.

At this point, I've put out a release every month since March (except
for July), without much fanfare, and absolutely no sense of trouble.  I
have received a couple of nice comments and some thanks from a few
people, and rather surprisingly zero bug reports. So either what I'm
doing is working successfully for everyone that's tried it and they're
completely happy (unlikely), or perhaps there just aren't a whole lot
of people interested in using a snapshotted distro (and thus corner
cases/problems aren't being uncovered), or I just haven't done a good
job of selling the thing (and in fact I've been trying to avoid exactly
that; putting big disclaimers in announcements and such; not writing
articles for lwn, etc).

So, as to the second one (possibly little interest in a snapshot
solution), I can certainly see where people may be coming from.  I
personally don't want to be running a static OS that gets no (security
or otherwise) updates for a month or however long period, and maybe
that's sufficient justification enough to abandon the idea outright.
Something quicker is much more desirable.  Looking at my nightly builds,
I have only had two real problems leading to downtime over this whole
period (and never at the time of the monthly release since I worked
hard to get the problems solved before then). Those were a change to
the archive (shasums vs. md5sums) and a hardcoded reliance on squeeze's
2.6.32 modules in d-i. 

The first issue was fixed in base-installer 1.119, and in the meantime I
created an ugly workaround in my build scripts to keep things going.
Note that even though this issue was fixed incredibly fast in unstable,
it took over two months to finally transition to testing. That problem
itself could have been more appropriately fixed by a TPU, but I don't
have upload rights, and thought it would be more trouble than it was
worth to find a sufficiently interested sponsor; although if something
like that happens again, that is the road I'm going to try first.

The second issue is that the kernel version is hardcoded to squeeze in
d-i's config files. I just worked on creating a solution for that (bug
#617943), but it's come up in a couple different ways now that I solved
with various ugly workarounds in my build scripts.

In the meantime (when there weren't problems breaking the builds),
installations have tested flawlessly (spot-checked since I can't test
every one), and even when I went back after solving those, the failed
builds when redone tested flawless (again spot-checked). Anyway, these
really good results lead me to think that a better solution may be
providing the nightly builds themselves (along with the possibility to
stick to an old known-good version if something is broken).  This will
solve the ugly out-of-dateness issue plaguing the current approach.

I know this sounds a lot like the existing d-i daily builds that are
already officially available as part of the project. However, there are
a couple subtle differences.  One is that software versions are exactly
the same on the snapshot media and on the remote to be downloaded, so
there is less risk of incompatibility (vice testing installation media
differing from packages on the remote archive). The second is that once
a snapshot is known to work, it will always work (since the remote data
remains the same).  Whereas, the current d-i daily builds can work just
fine one day and fail the next due to changes/transitions/removals
in testing.

However, in order to make such a system really robust, I think this
will also require nightly build testing (a theme that's come up
elsewhere recently with respect to improving general package quality in
the archive). In other words, each nightly build will be run through a
few common installation cases before its presented as "good" for
download. I'm planning to look at the possibility of doing that with
preseeding when I find some actual free time, but that's not likely for
a few months.

As for not selling the thing (as mentioned way back in the second
paragraph), I could try to do that more, but I don't want to step on
toes as seemed to be a (mis)perception of what I was doing a while back.
And not to mention I'm not really convinced myself that the current
monthly (or longer) snapshots are the right solution anyway.

So, anyway, my plan is to continue the monthly thing for now, and once I
get some time away from real life, I'll look at the new nightly idea.
Feel free to discuss, but please no tomatoes this time :)

Best wishes,
Mike