[Popcon-developers] Bug#440493: popularity-contest: Fails to report using http method

[Frans Pop]

Package: popularity-contest
Version: 1.41
Severity: normal

I've been getting errors for a long time during weekly popcon runs on one
of my boxes.

>From /var/log/messages:
Aug 26 06:47:48 elrond popularity-contest: unable to submit report to http://popcon.debian.org/cgi-bin/popcon.cgi.
Aug 19 06:47:50 elrond popularity-contest: unable to submit report to http://popcon.debian.org/cgi-bin/popcon.cgi.
Aug 12 06:47:47 elrond popularity-contest: unable to submit report to http://popcon.debian.org/cgi-bin/popcon.cgi.
Aug  5 06:48:10 elrond popularity-contest: unable to submit report to http://popcon.debian.org/cgi-bin/popcon.cgi.

The fallback to mail works correctly, but it should not need to fall back to
that. I have no idea why it fails, especially since it works correctly when
I run the same script manually as root.

Could it be that the server is just getting DoSed because too many reports
are being submitted at the same time (as they are all cronned at the same time,
at least in my TZ)? Or is there something subtle that fails because it is being
run from cron?


Output from a cron run with set -x in the cron script and a -d added in the
call to popcon-upload is shown below.

+ '[' '!' -f /usr/sbin/popularity-contest ']'
+ unset MAILFROM
+ unset MAILTO
+ unset MY_HOSTID
+ unset PARTICIPATE
+ unset SUBMITURLS
+ unset USEHTTP
+ . /usr/share/popularity-contest/default.conf
++ PARTICIPATE=no
++ MAILTO=survey at popcon.debian.org
++ SUBMITURLS=http://popcon.debian.org/cgi-bin/popcon.cgi
++ USEHTTP=yes
+ . /etc/popularity-contest.conf
++ MY_HOSTID=3eb1575275db421caca3e0859d9727eb
++ PARTICIPATE=yes
++ USEHTTP=yes
+ '[' -z survey at popcon.debian.org ']'
+ '[' yes = no ']'
+ '[' -z yes ']'
+ cd /var/log
+ umask 022
+ savelog -c 7 popularity-contest
+ POPCON=/var/log/popularity-contest
+ run_popcon
+ su -c 'sh -c /usr/sbin/popularity-contest' nobody
+ SUBMITTED=no
+ '[' http://popcon.debian.org/cgi-bin/popcon.cgi ']'
+ '[' yes = yes ']'
+ for URL in '$SUBMITURLS'
+ RET=0
+ setsid /usr/share/popularity-contest/popcon-upload -d -u http://popcon.debian.org/cgi-bin/popcon.cgi -f /var/log/popularity-contest
timeout in popcon-upload
+ RET=4
+ '[' 4 = 0 ']'
+ logger -t popularity-contest 'unable to submit report to http://popcon.debian.org/cgi-bin/popcon.cgi (4).'
+ '[' yes '!=' no ']'
+ '[' survey at popcon.debian.org ']'
++ which sendmail
+ '[' -x /usr/sbin/sendmail ']'
+ '[' -n '' ']'
+ echo 'To: survey at popcon.debian.org'
+ echo 'Subject: popularity-contest submission'
+ echo 'MIME-Version: 1.0'
+ echo 'Content-Type: text/plain'
+ echo
+ cat /var/log/popularity-contest
+ do_sendmail
+ '[' -n '' ']'
+ sendmail -oi survey at popcon.debian.org
+ SUBMITTED=yes
+ '[' yes '!=' yes ']'


Same output when run manually from a shell (as root).

# /etc/cron.weekly/popularity-contest
+ '[' '!' -f /usr/sbin/popularity-contest ']'
+ unset MAILFROM
+ unset MAILTO
+ unset MY_HOSTID
+ unset PARTICIPATE
+ unset SUBMITURLS
+ unset USEHTTP
+ . /usr/share/popularity-contest/default.conf
++ PARTICIPATE=no
++ MAILTO=survey at popcon.debian.org
++ SUBMITURLS=http://popcon.debian.org/cgi-bin/popcon.cgi
++ USEHTTP=yes
+ . /etc/popularity-contest.conf
++ MY_HOSTID=3eb1575275db421caca3e0859d9727eb
++ PARTICIPATE=yes
++ USEHTTP=yes
+ '[' -z survey at popcon.debian.org ']'
+ '[' yes = no ']'
+ '[' -z yes ']'
+ cd /var/log
+ umask 022
+ savelog -c 7 popularity-contest
+ POPCON=/var/log/popularity-contest
+ run_popcon
+ su -c 'sh -c /usr/sbin/popularity-contest' nobody
+ SUBMITTED=no
+ '[' http://popcon.debian.org/cgi-bin/popcon.cgi ']'
+ '[' yes = yes ']'
+ for URL in '$SUBMITURLS'
+ RET=0
+ setsid /usr/share/popularity-contest/popcon-upload -d -u http://popcon.debian.org/cgi-bin/popcon.cgi -f /var/log/popularity-contest
+ '[' 0 = 0 ']'
+ SUBMITTED=yes
+ '[' yes '!=' yes ']'
+ '[' yes '!=' yes ']'

-- System Information:
Debian Release: 4.0
  APT prefers stable
  APT policy: (500, 'stable')
Architecture: i386 (i686)
Shell:  /bin/sh linked to /bin/bash
Kernel: Linux 2.6.18-5-686
Locale: LANG=en_US, LC_CTYPE=en_US (charmap=ISO-8859-1)

Versions of packages popularity-contest depends on:
ii  debconf [debconf-2.0]         1.5.11     Debian configuration management sy
ii  dpkg                          1.13.25    package maintenance system for Deb

Versions of packages popularity-contest recommends:
ii  cron                          3.0pl1-100 management of regular background p
ii  exim4                         4.63-17    metapackage to ease exim MTA (v4) 
ii  exim4-daemon-light [mail-tran 4.63-17    lightweight exim MTA (v4) daemon
pn  mime-construct                <none>     (no description available)

-- debconf information:
  popularity-contest/submiturls:
  popularity-contest/hostid-failed:
* popularity-contest/participate: true