Any advice on big upgrade lenny->jessie?

Sun May 14 05:27:20 UTC 2017

The new installation is not running.  There are lots of database errors, including some that appear to be from bdb.  It looks as if there still are bdb files despite my attempt to convert them all to skiplist.  cyrus-db-types.active is absent and the .txt version is not exactly what I copied in, suggesting some of my files were overwritten with the standard ones from the package.

Any ideas?

I told the package not to purge the spool and lib directories.

In more detail:

There is no cyrus-db-types.active, though there is a txt version:
ANNOTATION skiplist
DBENGINE BerkeleyDB5.3
DUPLICATE skiplist
MBOX skiplist
PTS skiplist
QUOTA quotalegacy
SEEN skiplist
STATUSCACHE skiplist
SUBS flat
TLS skiplist
USERDENY flat

See further down for a comparison with the cyrus-db-types.txt file I copied in before the installation.

The huge log file begins
May 13 16:29:36 ross-mail cyrus/master[44503]: process started
May 13 16:29:36 ross-mail cyrus/master[44508]: about to exec /usr/sbin/cyrus
May 13 16:29:36 ross-mail cyrus/ctl_cyrusdb[44508]: DBERROR db5: BDB1538 Program version 5.3 doesn't match environment version 0.2063
May 13 16:29:36 ross-mail cyrus/ctl_cyrusdb[44508]: DBERROR db5: BDB1521 Recovery function for LSN 127 8428253 failed
May 13 16:29:36 ross-mail cyrus/ctl_cyrusdb[44508]: DBERROR db5: BDB0061 PANIC: Invalid argument
May 13 16:29:36 ross-mail cyrus/ctl_cyrusdb[44508]: DBERROR: critical database situation
May 13 16:29:36 ross-mail cyrus/master[44503]: process 44508 exited, status 75
May 13 16:29:36 ross-mail cyrus/master[44520]: about to exec /usr/sbin/cyrus
May 13 16:29:36 ross-mail cyrus/cyr_expire[44520]: DBERROR db5: BDB0060 PANIC: fatal region error detected; run recovery
May 13 16:29:36 ross-mail cyrus/cyr_expire[44520]: DBERROR: critical database situation
May 13 16:29:36 ross-mail cyrus/master[44503]: process 44520 exited, status 75
May 13 16:29:36 ross-mail cyrus/master[44523]: about to exec /usr/sbin/cyrus
May 13 16:29:36 ross-mail cyrus/tls_prune[44523]: DBERROR db5: BDB0060 PANIC: fatal region error detected; run recovery
May 13 16:29:36 ross-mail cyrus/tls_prune[44523]: DBERROR: critical database situation
May 13 16:29:36 ross-mail cyrus/master[44503]: process 44523 exited, status 75
May 13 16:29:36 ross-mail cyrus/master[44503]: WARNING: cannot find executable for service 'pop3' -- ignored
May 13 16:29:36 ross-mail cyrus/master[44503]: WARNING: cannot find executable for service 'nntp' -- ignored
May 13 16:29:36 ross-mail cyrus/master[44503]: WARNING: cannot find executable for service 'http' -- ignored
May 13 16:29:36 ross-mail cyrus/master[44503]: unable to setsocketopt(IP_TOS): Operation not supported
May 13 16:29:36 ross-mail cyrus/master[44503]: unable to setsocketopt(IP_TOS): Operation not supported
May 13 16:29:36 ross-mail cyrus/master[44503]: ready for work

which is weird because I converted everything to skiplist but it looks as if it's trying bdb.
/var/lib/cryus/db (and there are also db.backupN directories) does look as if it's still bdb:
root at ross-mail:/var/lib/cyrus/db# file __db.001
__db.001: data
root at ross-mail:/var/lib/cyrus/db# file log.0000000127
log.0000000127: Berkeley DB (Log, version 8, native byte-order)
root at ross-mail:/var/lib/cyrus/db# ls -l
total 16036
-rw------- 1 cyrus mail 14770175 May 13 21:34 __db.001
-rw------- 1 cyrus mail    49151 May 13 21:34 __db.002
-rw------- 1 cyrus mail    49151 May 13 21:34 __db.003
-rw------- 1 cyrus mail  8428497 May 11 16:52 log.0000000127
-rw------- 1 cyrus mail        4 Sep 30  2014 skipstamp

After I upgraded inside the chroot for the old system I copied
usr/lib/cyrus var/lib/cyrus var/backup/cyrus-imapd
onto the target system, which did not have cyrus installed at that point.
I also copied /var/spool/cyrus, though I did that before the database conversions.

I executed the lines from cyrus-common.postinst that created the cyrus user to be sure the ids were right.  I previously took steps to ensure the numeric ids matched between systems.

The I installed sasl and cyrus.  There were no visible complaints from aptitude; obviously lots of complaints in the log.

I have yet to create any users in sasl.

The post-installation cyrus-db-types.txt is not identical to the version I copied. The post-installation file's timestamp is Nov 7, 2016 (vs May 11, 2017 for the one I copied).  So maybe the package's version got copied over the one I put in place?  Here's what I had:
ANNOTATION skiplist
DBENGINE BerkeleyDB4.2
DUPLICATE skiplist
MBOX skiplist
PTS skiplist
QUOTA quotalegacy
SEEN skiplist
SUBS flat
TLS skiplist
Which is the same as the one currently in /usr/lib/cyrus except the latter has a line for USERDENY and a newer DBENGINE
________________________________________
From: Henrique de Moraes Holschuh <hmh at debian.org>
Sent: Friday, May 12, 2017 10:26:56 AM
To: Boylan, Ross
Cc: Ondřej Surý; Pkg-Cyrus-imapd-Debian-devel at lists.alioth.debian.org
Subject: Re: Any advice on big upgrade lenny->jessie?

On Fri, 12 May 2017, Boylan, Ross wrote:
> I tried to build db4.2-util from source on jessie, but couldn't even

Try installing the old binary packages.  That often works on stuff that
doesn't have nasty reverse-dependencies.

> get configure to run:
> https://lists.debian.org/debian-devel/2017/05/msg00069.html.  Maybe I
> could have just installed the deb from wheezy, but that didn't seem
> trustworthy.

ldd the resulting binaries if the install goes through.  If it is just
glibc and berkeley DB libs, it will work just fine.

> Instead I ran the wheezy system in a chroot, copied the current
> cyrus-upgrade-db script into it, edited upgrade cyrus-db-types.txt to
> change all the berkeley db's to skiplist (except I left the DBENGINE
> line intact) and ran the upgrade script in the chroot.  I had to alter
> the path to cvt_cyrusdb so the script could find it.
>
> I copied the resulting files to the new system and deleted
> cyrus-db-types.active.

Yeah, that will do it :)

> My next step is to install sasl, and then I'll try installing cyrus.

Good luck...

--
  Henrique Holschuh