[cut-team] Contantly Usable Testing, and Debian "rolling"

Tue Aug 10 09:20:07 UTC 2010

On 09/08/10 at 15:12 -0400, Joey Hess wrote:
> > Lucas and Asheesh want clarity as to which of the proposals you guys
> > are interested in working on. Snapshotting the whole archive? Or
> > just getting more users onto testing or something similar (e.g.,
> > what we've called "rolling").
> > 
> > Once you've read this email, reply saying which approach you like!
> 
> I am interested in both. In particular, I want there to be snapshots
> because they are needed to ensure installation always works and provide
> a firm base, and I want users to be able to easily upgrade to testing
> or similar, and have that also work well.
> 
> I don't think these ideas are in opposition, indeed doing each makes
> doing the other easier. Just, I think that snapshotting is the best way to
> get started, because it's an incremental step up from the current d-i
> releases, and wih testing frozen, this is the easiest point in the
> release cycle to make one or more snapshot releases.

I'm fine with doing both:
- snapshotting of testing
- making testing more usable, possibly with a new very-close-to-testing
  suite

The question about snapshotting of testing is how much manpower we want
to put on this. Do we want to do security support of those snapshots?
For all issues, or just the very-critical ones? If those snapshots are
only used to install a system that users have to upgrade as soon as the
installation is finished, we don't need to put as much effort to support
them security-wise as if the snapshots were targetted for general use.
I'd prefer that those snapshots would only be used for installation, and
are explicitely not security-supported.

> > Implementation problems for Debian rolling
> > ------------------------------------------
> > 
> > If we snapshot just enough of testing to do an install, then we
> > could offer installation media. But testing isn't perfect. It's used
> > to prepare stable releases, and not intended for general use. ("RT"
> > below refers to the release team.) Some problems:
> > 
> > * RT removes packages not suitable for stable release. but maybe
> > they are still suitable for general use (e.g clamav)
> > * RT sometimes need to break testing (transitions)
> > * RT sometimes FREEZES testing!
> > 
> > One action we can take to determine how real these problems are:
> > 
> >     ACT: Find real cases of testing breakages. Talk to the release
> > team about the secnarios and possible compromises. Perhaps through
> > that discussion we'll learn that we don't need a separate suite at
> > all.
> > 
> > So we propose that a separate suite, "rolling," be created. rolling
> > is in sync with testing except when it's a good idea to diverge.
> > (For example, rolling would not freeze; rolling could keep packages
> > unsuitable for a stable release.)
> 
> The release team necessarily has to yank packages sometimes for transitions.

After discussing that with the release team, it is not clear if such
removals are as frequent as you seem to think. Also, when asked during
the BOF, nobody was able to come up with a good case of an important
package being removed for a transition.

> A suite without such oversight would tend to always lag what is available
> in testing. Especially desktop environments would tend to fall behind
> due to their complex dependency graphs. That is one of the problems I
> have with the idea of adding another testing-like suite, rather than 
> finding ways to improve testing.

Can you elaborate on that? I'm not sure I understand why this "rolling"
suite would lag much behind testing.

On the other hand, while the "testing is broken by release team for
transition purposes" problem is not so important IMHO (as we lack proof
that it's really a problem), there are other problems with using testing
as the suite directly used by users:
- testing is used to prepare the next stable release. The release team
  removes packages that are not suitable for a stable release, but could
  perfectly be suitable for a rolling release. Good examples are packages
  that rely on an online service to achieve functionality (anti-virus,
  get-video-from-youtube stuff, etc).
- testing is sometimes frozen. During the freeze, it would be great if
  we could find a way for users of the rolling release to continue to
  receive updated packages (new upstream releases), which wouldn't be
  the case if they used testing.
That's why I think that it would be better to have another suite,
"rolling", which would allow us to make different compromises than those
of the release team. This is not a way to say that the release team is
sometimes wrong: they just make compromises that are perfectly
reasonable with their goals.
That suite would try to be the same as testing most of the time, with
the following exceptions:
- packages that are suitable for a rolling release, but not for the next
  stable release, would not be removed
- during freeze, rolling would continue to get migrations from unstable

> > Another problem with telling users to use testing (or anything
> > similar, like rolling): The security situation is unclear.
> > 
> > ACT: Talk to security team to figure out what's the current level of
> > security support for testing, and how it could be improved if
> > needed. (Find out if the Testing Security Team is still active.)
> > 
> > Maybe we simply will tell people that the only way to get security
> > updates is through a dist-upgrade.
> > 
> > ACT: Decide if that's enough. (It's fine with Lucas and Asheesh.)
> 
> I covered this in my talk. 
> 
> * Testing still tends to have fewer security holes than stable,
>   and more than unstable. http://security-tracker.debian.org/tracker/
>   has some great lists for comparing the security holes in the 3 suites.
> * Stable always has a relatively large backlog of security holes. It is
>   not unusual for many of them to be high impact either.
> * Stable gets fixes for very serious security holes faster than testing.
> * Transitions can block security fixes from reaching testing sometimes.
> * The Testing Security Team has only used the testing-security queue 5
>   times since the last stable release, but all the infrastructure is there,
>   and could be used more by the CUT team to backport fixes from unstable
>   if a transition was blocking them from reaching testing.
> * No accouncements are done for fixes that propigate from unstable, 
>   leaving users needing to dist-upgrade. This is something the CUT team
>   should address.
> * There is still a perception problem around testing's security that
>   does not match well with reality, and from a publicity POV, that is
>   one of the problems CUT can address.

All of those problems seem easily fixable by a small shift of
priorities. If we decide (as a project) that rolling should be a usable
(for users) and security-supported suite, then we could prioritize
getting updates in more.

> > Installation:
> >     possible solutions:
> >         - only provide snapshots of the installation media itself
> > (CD images)
> >         PB: cannot support netboot / business card CD
> >         PB: we cannot update the installation media
> > 
> >     ACT: figure out when/if the installer updates all packages to
> > current testing
> 
> I think that d-i and CD team members will all agree that making d-i
> media that targets a snapshot of testing will result in it all working
> much better for much longer than the current status of targeting testing.

Can't we target installing a snapshot, and upgrading to rolling at the
end of the installation process? Wouldn't that solve the problems with
trying to get an installer that can be tested?

> >     ACT: decide what about security support for the installation
> > media/snapshot?
> >         what are the scenarios that need to be addressed, since we
> > are upgrading at the end of installation anyway?
> 
> Mostly not a problem. d-i upgrades, so there are no security issues from
> using old media, assuming d-i itself, and apt are secure.

So d-i would upgrade to rolling?

> Case in point: All stable release media is full of packages with security
> holes; many vendors don't bother to update to stable point releases
> until they run out of existing stock, and this does not cause problems.
> 
> > Social/communication issues
> > ---------------------------
> > 
> > Naming:
> > 
> > 'cut' is opaque if we don't do snapshots.
> > 
> > 'testing' is sub-optimal. Possible options: 'next', ...
> > 
> > For implementing a snapshot-based system, "cut" is okay. Other
> > choices: fresh, current.
> > 
> > For a new suite that updates similarly to how testing updates, Lucas
> > and Asheesh like the name "rolling". Do you?
> 
> I don't dislike it at all, but "cut" is supposed to be at least usable as
> a way to describe both a snapshot and as an acronym to describe a rolling,
> constantly usable testing release.

I think that "cut" is a great acronym to describe the snapshots, but
that "rolling" is much better to describe the rolling release itself.
We should aim for users saying: "I installed Debian using CUT-3, and am
now running Debian rolling."
-- 
| Lucas Nussbaum
| lucas at lucas-nussbaum.net   http://www.lucas-nussbaum.net/ |
| jabber: lucas at nussbaum.fr             GPG: 1024D/023B3F4F |