Bug#369882: cyrus-doc-2.2 upgrade issues

[Ross Boylan]

On Thu, 2006-06-15 at 23:41 -0300, Henrique de Moraes Holschuh wrote:
> On Thu, 15 Jun 2006, Ross Boylan wrote:
> > Does this mean that Debian departed from upstream for 2.1?  Because the
> 
> Debian 2.1 is far closer to upstream 2.2 than you would believe.  Look at
> the size of the Debian diff for 2.1 if you doubt it.  It is the most
> advanced Cyrus 2.1 on earth :-p   So, basically, yes, we departed from
> upstream for 2.1 (and upstream took my patches for 2.2 :-) ).
> 
Thank you for clarifying that.

> > upstream notes seem to imply that the databases were not skiplist at
> > 2.1.
> 
> They were not.  But I knew better, and switched them (AND added logic to
switched all of them, or some of them (remarks below seem to imply the
latter)?
> detect database type mismatches before cyrus was started when I noticed I
> might need to do such a switch) as soon as it was clear they were stable
> enough.  Cyrus 2.2 and newer can do database type switch at runtime, so the
> old database type mismatch detection stuff can actually be used to
> auto-detect what the initial 2.2 config for database types should be.
It sounds as if it could auto-detect the type of db in use and spare us
all some trouble.  Though I guess even then one would need to specify
the format when the db's were first created.
> 
> > Also, the news that it's all skiplist is suprising in view of the
> > exchange in 342660.  As I noted earlier, it seems to say that Debian
> > uses bdb, although it also refers to that as being consistent with
> > upstream (which it's not, if upstream is skiplist).
> 
> Upstream uses a mix of skiplist and bdb in 2.2, and would have done that too
> eventually for 2.1.  Debian uses a mix of skiplist and bdb 3.2 in cyrus 2.1
> (rock stable).  I think we use skiplist + bdb 4.x (faster, not nearly as
> rock-stable) for 2.2.
If 2.1 and 2.2 on Debian use different bdb formats wouldn't that require
conversion of all bdb databases on upgrade?  Since the 2 reported
successful upgrades (earlier messages in this bugreport) did not mention
doing such a thing, that suggests the bdb format is the same for cyrus
2.1 and 2.2 on Debian.  Or am I missing something?
> 
> > I assume those instructions apply only to sieve scripts stored on the
> > imap server, not those in regular user home directories (another point
> 
> It applies to *all* scripts AFAIK. That means scripts stored on regular user
> homes that don't get compiled, don't run.  And it means that when the time
> comes to require a full recompile because of bytecode changes (this HAS
> happened at least once), they must be recompiled as well.
I can't see anything in the docs about how a user would byte-compile a
script in their homedir. (Well, uploading it to the sealed server
compiles it, but then the script is in the server, not the home
directory).

>From the next sentence on I have a feeling I may have lost what you are
getting at.
> 
> Cyrus is a black box,  Debian never partake of any misguided attempts to
> twart that by placing files on user homes (but we didn't remove the ability
> to shoot one's self on the foot either.  Maybe that was a big mistake).
Are you referring to any files other than .sieve?
If .sieve in home directories is not compiled there is a performance
penalty, and possibly a late discovery of syntax errors penalty, but I
don't see how this would lead to anything terrible.

> 
> I don't think Cyrus even tries to protect itself against someone messing
> with its stuff behind its back other than the casual sanity checking of
> values it gets from files in the spool, so it will probably not notice
> someone messed with the sieve scripts and bytecode (to recompile it by
> itself).
This sounds as if it involves messing with stuff in directories that
cyrus manages.  I think that's what one is doing if following the
upstream upgrade advice:
"To upgrade existing scripts (outside of home directories), you can run
the tools/masssievec perl script included with the distribution. It
requires a path to your sievec binary. For example: 
masssievec /usr/src/cyrus/sieve/sievec"


Note that file:///usr/share/doc/cyrus-doc-2.2/html/man.html does not
have an entry for masssievec or sievec.  So I'm speculating.

The phrase "(outside of home directories)" in the upstream advice is
obscure to me, but I think it means that the operation works on the
scripts in the server, not in the home directories.

> 
> > it would be useful to clarify).  (I mean the IMAP server conceptually;
> > user home directories might be on the same box as the IMAP server, but
> > they aren't under the control of the server).
> 
> Cyrus has control of *everything* pertaining its spool, or rather it was
> designed to, and *must* have control of *everything* pertaining to its
> spool.  I don't think this changed even for 2.3 (beta), let alone for 2.2.
> 
> My strong suggestion is for everyone to follow the black-box rule, because
> otherwise they will have to deal with the pain that *will* come sooner or
> later.  It is a pity upstream doesn't stress this strongly enough.
Does this mean you recommend against enabling sieveusehomedir, against
doing the command line starting with masssievec above, or something
else?

By the way,
http://cyrusimap.web.cmu.edu/twiki/bin/view/Cyrus/WhatDatabaseBackend
discusses the different Cyrus databases and their recommended formats,
and file:///usr/share/doc/cyrus-doc-2.2/html/man/imapd.conf.5.html lists
all the databases that can be configured (I count 8).  I suppose each
database listed in imapd.conf may correspond to 1 or more physical files
because their is one database per user or per mailbox, or because the
concept is implemented using several different databases, or because the
database itself uses several different files.

As an afterthought, python byte compiles python programs.  If it detects
there is no compiled program, or that the compilation was done with an
earlier version, it just recompiles.  If Cyrus did something like that
there'd be no need for manual (re)compilation of scripts on upgrade.