[Pkg-awstats-devel] Trying to get /usr/share/awstats/tools/update.sh working for multiple sites

[Ken Neighbors]

I have created a patch to fix this problem I reported earlier (it is 
patch number 0006 below).  I have also included a couple of patches I 
posted to bugs.debian.org, and I included some patches for additional 
features and some suggestions.  Please feel free to take any of the 
patches or give me some feedback if you'd like something done 
differently, or if you'd like me to post these to bugs.debian.org.

I hope this is helpful, and I really appreciate the work your team has 
put into making this Debian package.

You can access the patchset here:

   http://www.nsds.com/software/dist/awstats/awstats-patches.mbox.txt

Or see them individually here:

   http://www.nsds.com/software/dist/awstats/

Here is a summary of my proposed changes:

-------------------------------------------------------------------------------
0001 Add patch to report error when config file exists but cannot be read.

   /usr/lib/cgi-bin/awstats.pl

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=572353

-------------------------------------------------------------------------------
0002 Add patch to fix truncated nested included config files.

   /usr/lib/cgi-bin/awstats.pl

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=575545

-------------------------------------------------------------------------------
0003 Show error messages from cron jobs (update.sh and buildstatic.sh)

   /usr/share/awstats/tools/buildstatic.sh
   /usr/share/awstats/tools/update.sh

TMPFILE is used to record the output of 
awstats/awstats_buildstaticpages.pl (from both stdout and stderr).  If 
the return status of awstats/awstats_buildstaticpages.pl is nonzero, 
then an error occurred and the content of TMPFILE is printed to stderr 
to show up in a cron email report.  Note that a failure of 
awstats.pl/awstats_buildstaticpages.pl due to a bad config file will now 
no longer stop it from processing additional config files; 
update.sh/buildstatic.sh will continue to run 
awstats.pl/awstats_buildstaticpages.pl on other config files.  This 
feature has both good and bad consequences: an error in a single config 
file will not interrupt processing of the remaining config files (good); 
but an error in awstats.local.conf (which is included in all config 
files) will generate a repeated error message for multiple config files 
(bad).  I think the good outweighs the bad, but if you think that 
update.sh/buildstatic.sh should stop after the first error, then you can 
add "exit 1" after "cat $TMPFILE >&2" in both scripts.

-------------------------------------------------------------------------------
0004 Add option to disable nightly generation of static html reports.

   /etc/default/awstats
   /usr/share/awstats/tools/buildstatic.sh
   /usr/share/docs/awstats/README.Debian

With inspiration from /etc/default/rsync and /etc/init.d/rsync, I set up 
an option in /etc/default/awstat called AWSTATS_ENABLE_BUILDSTATICPAGES. 
  If set to "yes" then buildstatic.sh will run every night to build 
static pages.  If set to "no" (which I think should be the default, but 
feel free to make up your own mind), then buildstatic.sh will not build 
static pages.  I placed a short note in README.Debian about it.

-------------------------------------------------------------------------------
0005 Use "nice" to lower the priority of the AWStats processes when 
updating databases and generating static html reports from cron.

   /etc/default/awstats
   /usr/share/awstats/tools/buildstatic.sh
   /usr/share/awstats/tools/update.sh

Again with inspiration from /etc/default/rsync and /etc/init.d/rsync, I 
set up an option in /etc/default/awstat called AWSTATS_NICE.  It sets 
the niceness level for awstats.pl and awstats_buildstaticpages.pl run 
from update.sh and buildstatic.sh.  Default value is 10.

Note that unlike /etc/init.d/rsync, I did not include error checking for 
this value.  Here is some code from /etc/init.d/rsync you might consider 
(or just let me know if you'd like me to create a patch that includes 
this type of error-checking on AWSTATS_NICE):

     case "x$RSYNC_NICE" in
         x[0-9])         RSYNC_NICE_PARM="--nicelevel $RSYNC_NICE";;
         x[1-9][0-9])    RSYNC_NICE_PARM="--nicelevel $RSYNC_NICE";;
         x)              ;;
         *)              log_warning_msg "Value of RSYNC_NICE in 
$RSYNC_DEFAULTS_FILE must be a value between 0 and 19 (inclusive);"
                         log_warning_msg "ignoring RSYNC_NICE now."
                         ;;
     esac

-------------------------------------------------------------------------------
0006 Do not use implied default config file /etc/awstats/awstats.conf 
but instead require sysadmin to install one or more explicit config 
file(s) so that update.sh and buildstatic.sh will work for either 
single-site or multi-site installations.  This fixes the problem where 
awstats.pl complains about undefined SiteDomain when using separate 
config files as described in README.Debian for multiple stats.

   /usr/share/awstats/tools/buildstatic.sh
   /usr/share/awstats/tools/update.sh
   /usr/share/docs/awstats/README.Debian
   /usr/share/docs/awstats/examples/awstats.default.conf
   debian/rules

I removed "/etc/conf/awstats.conf" from the list of config files to 
process because it always generates an error message when trying to set 
up awstats for multiple sites.  I think it is better to treat the 
single-site case as a special case of multiple sites where there is only 
one site, served by a single config file "awstats.default.conf" which 
contains one line: Include "awstats.conf".

-------------------------------------------------------------------------------
0007 Improve documentation of logfile permissions and suggest 
EnableLockForUpdate=1.

   /usr/share/docs/awstats/README.Debian

-------------------------------------------------------------------------------
0008 Cosmetic improvements to cron scripts (update.sh and buildstatic.sh).

   /usr/share/awstats/tools/buildstatic.sh
   /usr/share/awstats/tools/update.sh

Feel free to take or leave any of these changes.

I used a different method to initialize the values read from 
/etc/default/awstats (again with inspiration from /etc/init.d/rsync and 
other init files).  Specifically, instead of:
   [ -r "$DEFAULT" ] && . "$DEFAULT"
   -NICE=${AWSTATS_NICE:-10}
I did this:
   AWSTATS_NICE=10
   [ -r "$DEFAULT" ] && . "$DEFAULT"
This method reduces the need for another environment variable ("NICE") 
because it can use the existing environment variable ("AWSTATS_NICE") 
defined in /etc/default/awstats (similarly for "AWSTATS_LANG" and 
"AWSTATS_ENABLE_BUILDSTATICPAGES").

I moved AWSTATS_LANG, Y, and m (the date calls) out of the 
buildstatic.sh loop since they should remain constant for all config files.

I used an alternate method to extract the config file name using posix 
shell parameter expansions (without calling the external programs ls and 
sed).

-------------------------------------------------------------------------------

Thanks again,
Ken