Bug#706862: Database destroyed during upgrade from squeeze

[Ben Hutchings]

On Mon, 2013-05-06 at 07:23 +0200, Ondřej Surý wrote:
> I still miss the answer for:
> 
> >> Could you check the permissions on /var/lib/cyrus and it's contents?

This is from the full system backup that ran just before the upgrade:

# ls -al /restore/shadbolt-var/20130505/tree/lib/cyrus
total 158828
drwxr-x--- 12 cyrus mail      4096 May  5 03:01 .
drwxr-xr-x 72 root  root      4096 Apr  4 23:03 ..
-rw------- 15 cyrus mail       144 Apr  1 06:00 annotations.db
drwx------  2 cyrus mail      4096 Apr 14 18:30 db
drwx------  2 cyrus mail      4096 May  5 03:01 db.backup1
drwx------  2 cyrus mail      4096 May  5 02:31 db.backup2
-rw-------  1 cyrus mail 162291712 May  5 03:01 deliver.db
drwx------  2 cyrus mail      4096 Aug 17  2004 log
-rw------- 13 cyrus mail     62668 Apr 15 13:27 mailboxes.db
drwx------  2 cyrus mail      4096 Sep  7  2009 msg
drwx------  2 cyrus mail     36864 May  5 03:08 proc
drwx------ 28 cyrus mail      4096 Aug 17  2004 quota
drwxr-xr-x 28 cyrus mail      4096 Sep  7  2009 sieve
drwxr-x---  2 cyrus mail      4096 Feb 15  2011 socket
-rw-------  1 cyrus mail     28672 May  5 02:31 tls_sessions.db
drwx------ 28 cyrus mail      4096 Aug 17  2004 user
# ls -al /restore/shadbolt-var/20130505/tree/lib/cyrus/db
total 41448
drwx------  2 cyrus mail     4096 Apr 14 18:30 .
drwxr-x--- 12 cyrus mail     4096 May  5 03:01 ..
-rw-------  1 cyrus mail    24576 May  5 03:08 __db.001
-rw-------  1 cyrus mail  5464064 May  5 03:08 __db.002
-rw-------  1 cyrus mail   663552 May  5 03:08 __db.003
-rw-------  1 cyrus mail    98304 May  5 03:08 __db.004
-rw-------  1 cyrus mail 25575424 May  5 03:08 __db.005
-rw-------  1 cyrus mail    49152 May  5 03:08 __db.006
-rw-------  1 cyrus mail 10485760 May  5 03:01 log.0000000124
-rw------- 15 cyrus mail        4 Apr  1 06:00 skipstamp

The current state (after some vain attempts to recover) is:

# ls -al /var/lib/cyrus
total 208
drwxr-x--- 14 cyrus mail  4096 May  7 03:17 .
drwxr-xr-x 76 root  root  4096 May  7 05:46 ..
-rw-------  1 cyrus mail   144 May  7 03:17 annotations.db
drwx------  2 cyrus mail  4096 May  7 03:17 db
drwx------  2 cyrus mail  4096 May  7 03:17 db.backup1
drwx------  2 cyrus mail  4096 May  7 02:58 db.backup2
drwx------  2 cyrus mail  4096 Apr 14 18:30 db.broken
-rw-------  1 cyrus mail 51196 May  7 03:17 deliver.db
drwx------ 29 cyrus mail  4096 May  5 06:25 lock
drwx------  2 cyrus mail  4096 Aug 17  2004 log
-rw-------  1 cyrus mail 23684 May  7 03:17 mailboxes.db
drwx------  2 cyrus mail  4096 Sep  7  2009 msg
drwx------  2 cyrus mail 69632 May  5 14:51 proc
drwx------ 28 cyrus mail  4096 Aug 17  2004 quota
drwxr-xr-x 28 cyrus mail  4096 Sep  7  2009 sieve
drwxr-x---  2 cyrus mail  4096 Feb 15  2011 socket
drwx------ 28 cyrus mail  4096 Aug 17  2004 user
-rw-------  1 cyrus mail     0 May  5 05:02 user_deny.db
# ls -al /var/lib/cyrus/db
total 34124
drwx------  2 cyrus mail     4096 May  7 03:17 .
drwxr-x--- 14 cyrus mail     4096 May  7 03:17 ..
-rw-------  1 cyrus mail      144 May  5 04:01 annotations.db
-rw-------  1 cyrus mail    24576 May  8 06:25 __db.001
-rw-------  1 cyrus mail  6479872 May  8 06:26 __db.002
-rw-------  1 cyrus mail   663552 May  8 06:25 __db.003
-rw-------  1 cyrus mail   163840 May  7 03:17 __db.004
-rw-------  1 cyrus mail 29696000 May  7 03:17 __db.005
-rw-------  1 cyrus mail    40960 May  7 03:17 __db.006
-rw-------  1 cyrus mail 10485760 May  7 03:17 log.0000000124
-rw-------  1 cyrus mail    62668 May  5 04:01 mailboxes.db
-rw-------  1 cyrus mail        4 May  7 03:17 skipstamp

> And could it be that you ran out of free space in /tmp?

It's a tmpfs which appears to to have a capacity of 2G (there is 4G of
swap, barely used).  I don't know how much free space it had during the
upgrade, of course.

> I have tested the migration script throughly, but there still might be
> some corner cases unhandled.

Well, note that migration was triggered by running the init script
'status' action (which is itself a bug - only 'start' should do that)
while the upgraded package was in the unpacked state.  Are you sure that
doesn't make a difference?

> Well, it would help me to understand what has happened if I could test
> with real data. So, yes, it would be nice to lay my hands on full
> backup.
[...]

I'm afraid these databases seem to include private information that I am
not prepared to disclose.

How about I try restoring the system backup on some other machine and
re-run the upgrade with 'set -x' added to upgrade-db?

Ben.

-- 
Ben Hutchings
For every action, there is an equal and opposite criticism. - Harrison
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 828 bytes
Desc: This is a digitally signed message part
URL: <http://lists.alioth.debian.org/pipermail/pkg-cyrus-imapd-debian-devel/attachments/20130509/cff00fc1/attachment-0002.pgp>