[Pkg-clamav-commits] [SCM] packaging for clamav-unoffical-sigs branch, master, updated. debian/3.3-2-28-g88dc755
Paul Wise
pabs at debian.org
Sat Sep 26 08:49:29 UTC 2009
The following commit has been merged in the master branch:
commit 7f86fc1e31719c072197beaa6167e029ec9f0566
Author: Paul Wise <pabs at debian.org>
Date: Sat Sep 26 15:44:57 2009 +0800
Imported Upstream version 3.6
diff --git a/CHANGELOG b/CHANGELOG
index 3630b01..4342790 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -3,6 +3,31 @@ written by Bill Landry (bill at inetmsg.com). The script provides a simple
way to download, test and run the third-party ClamAV signature databases
provided by Sanesecurity, MSRBL, SecuriteInfo, MalwarePatrol, and OITC.
+Version 3.6 (updated 2009-08-23)
+ - Added "tr" to remove Windows CRLF from signatures in local.ign
+ monitoring section.
+ - Updated signature database monitoring section to better handle
+ rearrangement of signature database file name placement in the
+ configuration file.
+ - Removed several of the config file reload options in favor of
+ simplicity and most reliable options.
+ - Changed rsync mirror lookup from 'host' to 'dig' with the hope
+ that 'dig' is more universally consistent between OS platforms.
+ Issue reported by Al Sterman.
+ - Added the '-u' (timestamp check) flag to the rsync downloads so
+ that signature databases will not be downloaded from mirrors that
+ are out of sync and hosting old files. Requested by Wolfgang Breyha
+ - Added a configuration variable that will provide the ability to
+ scan a HAM (non-spam) directory with new signature databases and
+ automatically remove signatures that trigger from the database
+ before implementing. Requested by Mike Cardwell.
+ - Added the '-t' flag to the script to output third-party signatures
+ that trigger during the HAM directory scan, but only if the 'ham_dir'
+ variable is enabled in the configuration file and hit were found.
+ - Updated required utilities section of the config file. Requested
+ by Micah Anderson.
+ - Updated Manual page, README, and INSTALL files.
+
Version 3.5.4 (updated 2009-06-25)
- Removed an unnecessary early database reload when a change was
detected by the script in the local.ign signature bypass file.
diff --git a/INSTALL b/INSTALL
index 0d0072c..591a9c8 100644
--- a/INSTALL
+++ b/INSTALL
@@ -50,8 +50,8 @@ Configure:
Edit the clamav-unofficial-sigs.conf file
IMPORTANT CONFIGURATION CONSIDERATIONS:
- Make sure that the PATH statement correctly defines the location of your binary files. These
- include: find, xargs, sed, awk, cut, grep, tail, chown, chmod, cmp, diff, gzip, ls, cp, mv,
- test, gpg, host, sleep, urandom, chksum, rsync, curl, socat, etc. It has been reported that
+ include: find, sed, awk, cut, grep, tail, chown, chmod, cmp, diff, dig, host, gzip, ls, cp, mv,
+ test, gpg, xargs, sleep, urandom, chksum, rsync, curl, socat, etc. It has been reported that
on Sun systems, the GNU utilities should be used rather than the default Sun versions.
- System shell setting: Based on user feedback, it has been reported that "sh" works best for
BSD variants, "ksh" for Sun Solaris, and "bash" for Linux variants. If you experience problems
@@ -119,5 +119,9 @@ To completely remove the script and all of its associated files, databases and w
To clamscan integrity test a specific database file:
/path/to/clamav-unofficial-sigs.sh -s filename (e.g., -s junk.ndb)
+To output third-party signature names that triggered during local HAM directory scanning,
+ if enabled in the configuration file:
+ /path/to/clamav-unofficial-sigs.sh -t
+
To check version:
/path/to/clamav-unofficial-sigs.sh -v
diff --git a/README b/README
index 3a66dab..7eb6970 100644
--- a/README
+++ b/README
@@ -60,6 +60,8 @@ Script (clamav-unofficial-sigs.sh) features & capabilities:
have to redo the configuration with each new script update.
- The script can hexadecimal encode (for usage) and decode (for viewing) virus signatures.
- Ability to create a hexadecimal signature database file from a clear text ascii file.
+- Ability to enable scanning of a local HAM (non-spam) directory for false-positive hits from
+ third-party signatures and removal of errant signatures from databases before implementing.
- Script logging can be enabled/disabled in the configuration file.
- Includes cron, manual, and logrotate files.
diff --git a/clamav-unofficial-sigs.8 b/clamav-unofficial-sigs.8
index 5105c74..49997c2 100644
--- a/clamav-unofficial-sigs.8
+++ b/clamav-unofficial-sigs.8
@@ -1,5 +1,5 @@
.\" Manual page for clamav-unofficial-sigs.sh
-.TH clamav-unofficial-sigs 8 "June 25, 2009" "Version 3.5.4" "SCRIPT COMMANDS"
+.TH clamav-unofficial-sigs 8 "August 23, 2009" "Version 3.6" "SCRIPT COMMANDS"
.SH NAME
clamav-unofficial-sigs \- Download, test, and install third-party ClamAV signature databases.
.SH SYNOPSIS
@@ -56,6 +56,10 @@ Test the integrity of a third-party signature database with clamscan.
Only specify the filename as listed in the configuration file
and the script will search for it in the work directory.
.TP
+.B \-t
+If HAM directory scanning is enabled in the script's configuration file, then output
+the names of any third-party signatures that triggered during the HAM directory scan.
+.TP
.B \-v
Print the script version and date information.
.SH SEE ALSO
diff --git a/clamav-unofficial-sigs.conf b/clamav-unofficial-sigs.conf
index 810ca35..ee43b9d 100644
--- a/clamav-unofficial-sigs.conf
+++ b/clamav-unofficial-sigs.conf
@@ -25,11 +25,11 @@
# script, please try editing the top line of the script file and changing
# "sh" to either "ksh" or "bash" before reporting a problem.
-# Set and export the appropriate program paths for your OS platform. These
-# include: find, xargs, sed, awk, cut, grep, tail, chown, chmod, cmp, diff,
-# gzip, ls, cp, mv, test, gpg, host, sleep, urandom, cksum, rsync, curl,
-# socat, etc. It has been reported that on Sun systems, the GNU utilities
-# should be used rather than the default Sun versions of these utilities.
+# Set and export the appropriate program paths for your OS platform. Required
+# utilities include: find, xargs, sed, awk, cut, dig, grep, tail, chown, chmod,
+# cmp, diff, gzip, ls, cp, mv, test, gpg, host, sleep, cksum, rsync, curl, perl,
+# and optionally socat. It's been reported that on Sun systems, the GNU utilities
+# should be used rather than the default Sun OS versions of these utilities.
PATH="/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin"
export PATH
@@ -47,6 +47,11 @@ clam_dbs="/var/lib/clamav"
# Set path to clamd.pid file (see clamd.conf for path location).
clamd_pid="/var/run/clamd.pid"
+# To enable "ham" (non-spam) directory scanning and removal of
+# signatures that trigger on ham messages, uncomment the following
+# variable and set it to the appropriate ham message directory.
+#ham_dir="/path/to/ham-test/directory"
+
# If you would like to reload the clamd databases after an update,
# change the following variable to "yes".
reload_dbs="no"
@@ -58,15 +63,7 @@ reload_dbs="no"
reload_opt="clamdscan --reload" # Default
# - The next variable signals clamd's Process ID (PID) to reload databases
#reload_opt="kill -USR2 `cat $clamd_pid`"
-# - The next variable uses perl to signal clamd's local socket to reload databases
-#reload_opt="perl -MIO::Socket::UNIX -we '$s = IO::Socket::UNIX->new(shift); $s->print("RELOAD"); print $s->getline; $s->close' $clamd_socket"
-# - The next variable uses socat to signal clamd's local socket to reload databases
-#reload_opt="echo RELOAD | socat - $clamd_socket"
-# - The next variable uses perl to signal clamd's tcp/ip address:port to reload databases (check address:port at EOL)
-#reload_opt="perl -MIO::Socket::INET -we '$s = IO::Socket::INET->new(shift); $s->print("RELOAD"); print $s->getline; $s->close' 127.0.0.1:3310"
-# - The next variable uses socat to signal clamd's tcp/ip address:port to reload databases
-#reload_opt="echo RELOAD | socat - tcp4-connect:127.0.0.1:3310"
-# - The next variable signals clamd to do a full service stop/start
+# - The next variable signals linux based systems to do a full clamd service stop/start
#reload_opt="service clamd restart"
# - Use the next variable to set a custom or system specific reload/restart option
#reload_opt=""
diff --git a/clamav-unofficial-sigs.sh b/clamav-unofficial-sigs.sh
index 8288e94..3aac700 100755
--- a/clamav-unofficial-sigs.sh
+++ b/clamav-unofficial-sigs.sh
@@ -21,7 +21,7 @@
default_config="/etc/clamav-unofficial-sigs.conf"
-version="v3.5.4 (updated 2009-06-25)"
+version="v3.6 (updated 2009-08-23)"
output_ver="
`basename $0` $version
"
@@ -38,7 +38,7 @@ ClamAV Unofficial Signature Databases Update Script - $version
either modified or removed from the third-party database.
-c Direct script to use a specific configuration file
- e.g.: '-c /path/to/`basename "$default_config"`'
+ e.g.: '-c /path/to/`basename "$default_config"`'.
-d Decode a third-party signature either by signature name
(e.g: Sanesecurity.Junk.15248) or hexadecimal string.
@@ -53,12 +53,12 @@ ClamAV Unofficial Signature Databases Update Script - $version
be used in any '*.ndb' signature database file.
-g GPG verify a specific Sanesecurity database file
- e.g.: '-g filename.ext' (do not include file path)
+ e.g.: '-g filename.ext' (do not include file path).
- -h Display this script's help and usage information
+ -h Display this script's help and usage information.
-i Output system and configuration information for
- viewing or possible debugging purposes
+ viewing or possible debugging purposes.
-m Make a signature database from an ascii file containing
data strings, with one data string per line. Additional
@@ -68,9 +68,13 @@ ClamAV Unofficial Signature Databases Update Script - $version
its associated files and databases from the system.
-s Clamscan integrity test a specific database file
- e.g.: '-s filename.ext' (do not include file path)
+ e.g.: '-s filename.ext' (do not include file path).
- -v Output script version and date information
+ -t If HAM directory scanning is enabled in the script's
+ configuration file, then output names of any third-party
+ signatures that triggered during the HAM directory scan.
+
+ -v Output script version and date information.
Alternative to using '-c': Place config file in /etc ($default_config)
"
@@ -97,7 +101,7 @@ perms () {
}
# Take input from the commandline and process.
-while getopts 'bc:defg:himrs:v' option ; do
+while getopts 'bc:defg:himrs:tv' option ; do
case $option in
b) no_default_config
echo "Input a third-party signature name that you wish to bypass due to false-positives"
@@ -118,7 +122,7 @@ while getopts 'bc:defg:himrs:v' option ; do
cp -f local.ign "$config_dir" 2>/dev/null
echo "$sig_ign" | tr -d "\r" >> "$config_dir/local.ign"
echo "$file_sig" | tr -d "\r" >> "$config_dir/monitor-ign.txt"
- if clamscan --quiet -d $config_dir/local.ign $config_dir/scan-test.txt
+ if clamscan --quiet -d "$config_dir/local.ign" "$config_dir/scan-test.txt"
then
if rsync -cqt $config_dir/local.ign $clam_dbs
then
@@ -364,7 +368,7 @@ while getopts 'bc:defg:himrs:v' option ; do
echo ""
echo ""
echo "Signature database file created at: $path_file"
- if clamscan --quiet -d $path_file $config_dir/scan-test.txt 2>/dev/null
+ if clamscan --quiet -d "$path_file" "$config_dir/scan-test.txt" 2>/dev/null
then
echo ""
echo "Clamscan reports database integrity tested good."
@@ -457,7 +461,7 @@ while getopts 'bc:defg:himrs:v' option ; do
then
echo "Clamscan integrity testing: $db_file"
echo ""
- if clamscan --quiet -d $db_file $config_dir/scan-test.txt ; then
+ if clamscan --quiet -d "$db_file" "$config_dir/scan-test.txt" ; then
echo "Clamscan reports that '$input' database integrity tested GOOD"
fi
else
@@ -470,6 +474,23 @@ while getopts 'bc:defg:himrs:v' option ; do
echo ""
exit
;;
+ t) no_default_config
+ if [ -n "$ham_dir" ]
+ then
+ if [ -s "$config_dir/whitelist.hex" ]
+ then
+ echo "The following third-party signatures triggered hits during the HAM Directory scan:"
+ echo ""
+ grep -h -f "$config_dir/whitelist.hex" "$work_dir"/*/*.ndb | cut -d ":" -f1
+ else
+ echo "No third-party signatures have triggered hits during the HAM Directory scan."
+ fi
+ else
+ echo "Ham directory scanning is not currently enabled in the script's configuration file."
+ fi
+ echo ""
+ exit
+ ;;
v) echo "$output_ver"
exit
;;
@@ -538,9 +559,48 @@ if [ "$user_configuration_complete" != "yes" ]
exit 1
fi
+# If "ham_dir" variable is set, then create initial whitelist files (skipped if first-time script run).
+test_dir="$work_dir/test"
+if [ -n "$ham_dir" -a -d "$work_dir" -a ! -d "$test_dir" ] ; then
+ if [ -d "$ham_dir" ]
+ then
+ mkdir -p "$test_dir"
+ cp -f "$work_dir"/*/*.ndb "$test_dir"
+ clamscan --infected --no-summary -d "$test_dir" "$ham_dir"/* | \
+ sed 's/\.UNOFFICIAL FOUND//' | awk '{print $NF}' >> "$config_dir/whitelist.txt"
+ grep -h -f "$config_dir/whitelist.txt" "$test_dir"/* | \
+ cut -d "*" -f2 | sort | uniq > "$config_dir/whitelist.hex"
+ cd "$test_dir"
+ for db_file in `ls`; do
+ grep -h -v -f "$config_dir/whitelist.hex" "$db_file" > "$db_file-tmp"
+ mv -f "$db_file-tmp" "$db_file"
+ if clamscan --quiet -d "$db_file" "$config_dir/scan-test.txt" 2>/dev/null ; then
+ if rsync -cqt $db_file $clam_dbs ; then
+ perms chown $clam_user:$clam_group $clam_dbs/$db_file
+ do_clamd_reload=1
+ fi
+ fi
+ done
+ if [ -s "$config_dir/whitelist.hex" ]
+ then
+ echo "*** Initial HAM directory scan whitelist file created in $config_dir ***"
+ echo ""
+ log "INFO - Initial HAM directory scan whitelist file created in $config_dir"
+ else
+ echo "No false-positives detected in initial HAM directory scan"
+ log "No false-positives detected in initial HAM directory scan"
+ fi
+ else
+ echo "Cannot locate HAM directory: $ham_dir"
+ echo "Skipping initial whitelist file creation. Fix 'ham_dir' path in config file"
+ log "WARNING - Cannot locate HAM directory: $ham_dir"
+ log "WARNING - Skipping initial whitelist file creation. Fix 'ham_dir' path in config file"
+ fi
+fi
+
# Check to see if the working directories have been created.
# If not, create them. Otherwise, ignore and proceed with script.
-mkdir -p "$ss_dir" "$msrbl_dir" "$si_dir" "$mbl_dir" "$config_dir" "$gpg_dir" "$add_dir"
+mkdir -p "$work_dir" "$ss_dir" "$msrbl_dir" "$si_dir" "$mbl_dir" "$config_dir" "$gpg_dir" "$add_dir"
# Set secured access permissions to the GPG directory
chmod 0700 "$gpg_dir"
@@ -654,53 +714,60 @@ fi
# Create files containing lists of current and previously active 3rd-party databases
# so that databases and/or backup files that are no longer being used can be removed.
+current_tmp="$config_dir/current-dbs.tmp"
current_dbs="$config_dir/current-dbs.txt"
previous_dbs="$config_dir/previous-dbs.txt"
-mv -f "$current_dbs" "$previous_dbs" 2>/dev/null
+sort "$current_dbs" > "$previous_dbs" 2>/dev/null
+rm -f "$current_dbs"
clamav_files () {
- echo "$clam_dbs/$db" >> "$current_dbs"
+ echo "$clam_dbs/$db" >> "$current_tmp"
if [ "$keep_db_backup" = "yes" ] ; then
- echo "$clam_dbs/$db-bak" >> "$current_dbs"
+ echo "$clam_dbs/$db-bak" >> "$current_tmp"
fi
}
if [ -n "$ss_dbs" ] ; then
for db in $ss_dbs ; do
- echo "$ss_dir/$db" >> "$current_dbs"
- echo "$ss_dir/$db.sig" >> "$current_dbs"
+ echo "$ss_dir/$db" >> "$current_tmp"
+ echo "$ss_dir/$db.sig" >> "$current_tmp"
clamav_files
done
fi
if [ -n "$msrbl_dbs" ] ; then
for db in $msrbl_dbs ; do
- echo "$msrbl_dir/$db" >> "$current_dbs"
+ echo "$msrbl_dir/$db" >> "$current_tmp"
clamav_files
done
fi
if [ -n "$si_dbs" ] ; then
for db in $si_dbs ; do
- echo "$si_dir/$db" >> "$current_dbs"
- echo "$si_dir/$db.gz" >> "$current_dbs"
+ echo "$si_dir/$db" >> "$current_tmp"
+ echo "$si_dir/$db.gz" >> "$current_tmp"
clamav_files
done
fi
if [ -n "$mbl_dbs" ] ; then
for db in $mbl_dbs ; do
- echo "$mbl_dir/$db" >> "$current_dbs"
+ echo "$mbl_dir/$db" >> "$current_tmp"
clamav_files
done
fi
if [ -n "$add_dbs" ] ; then
for db in $add_dbs ; do
- echo "$add_dir/$db" >> "$current_dbs"
+ echo "$add_dir/$db" >> "$current_tmp"
clamav_files
done
fi
# Remove 3rd-party databases and/or backup files that are no longer being used.
+sort "$current_tmp" > "$current_dbs" 2>/dev/null
+rm -f "$current_tmp"
db_changes="$config_dir/db-changes.txt"
+if [ ! -s "$previous_dbs" ] ; then
+ cp -f "$current_dbs" "$previous_dbs" 2>/dev/null
+fi
diff "$current_dbs" "$previous_dbs" 2>/dev/null | grep '>' | awk '{print $2}' > "$db_changes"
if [ -s "$db_changes" ] ; then
- if grep -vq "bak" $db_changes 2> /dev/null ; then
+ if grep -vq "bak" $db_changes 2>/dev/null ; then
do_clamd_reload=2
fi
comment ""
@@ -724,6 +791,7 @@ echo "$config_dir/msrbl-include-dbs.txt" >> "$purge"
echo "$config_dir/previous-dbs.txt" >> "$purge"
echo "$config_dir/scan-test.txt" >> "$purge"
echo "$config_dir/ss-include-dbs.txt" >> "$purge"
+echo "$config_dir/whitelist.hex" >> "$purge"
echo "$gpg_dir/publickey.gpg" >> "$purge"
echo "$gpg_dir/secring.gpg" >> "$purge"
echo "$gpg_dir/ss-keyring.gpg*" >> "$purge"
@@ -895,14 +963,14 @@ if [ -n "$ss_dbs" ] ; then
comment "======================================================================"
comment "Sanesecurity Database & GPG Signature File Updates"
comment "======================================================================"
- ss_mirror_ips=`host -T $ss_url | awk '{print $NF}'`
+ ss_mirror_ips=`dig +tcp +short $ss_url`
for ss_mirror_ip in $ss_mirror_ips ; do
ss_mirror_name=`host $ss_mirror_ip | awk '{print $NF}' | cut -d "(" -f2 | sed 's/\(.*\)./\1/'`
ss_mirror_site_info="$ss_mirror_name $ss_mirror_ip"
comment ""
comment "Sanesecurity mirror site used: $ss_mirror_site_info"
log "INFO - Sanesecurity mirror site used: $ss_mirror_site_info"
- if rsync $rsync_output_level $no_motd --files-from=$ss_include_dbs -ctz $contimeout \
+ if rsync $rsync_output_level $no_motd --files-from=$ss_include_dbs -ctuz $contimeout \
--timeout=30 --stats rsync://$ss_mirror_ip/sanesecurity $ss_dir 2>/dev/null
then
ss_rsync_success="1"
@@ -926,25 +994,57 @@ if [ -n "$ss_dbs" ] ; then
log "WARNING - Sanesecurity GPG Signature test FAILED on $db_file database - SKIPPING" ; false
fi
if [ "$?" = "0" ] ; then
- if clamscan --quiet -d $ss_dir/$db_file $config_dir/scan-test.txt 2>/dev/null
+ db_ext=`echo $db_file | cut -d "." -f2`
+ if [ -z "$ham_dir" -o "$db_ext" != "ndb" ]
then
- comment "Clamscan reports Sanesecurity $db_file database integrity tested good"
- log "INFO - Clamscan reports Sanesecurity $db_file database integrity tested good" ; true
+ if clamscan --quiet -d "$ss_dir/$db_file" "$config_dir/scan-test.txt" 2>/dev/null
+ then
+ comment "Clamscan reports Sanesecurity $db_file database integrity tested good"
+ log "INFO - Clamscan reports Sanesecurity $db_file database integrity tested good" ; true
+ else
+ echo "Clamscan reports Sanesecurity $db_file database integrity tested BAD - SKIPPING"
+ log "WARNING - Clamscan reports Sanesecurity $db_file database integrity tested BAD - SKIPPING" ; false
+ fi && \
+ (test "$keep_db_backup" = "yes" && cp -f $clam_dbs/$db_file $clam_dbs/$db_file-bak 2>/dev/null ; true) && \
+ if rsync -cqt $ss_dir/$db_file $clam_dbs
+ then
+ perms chown $clam_user:$clam_group $clam_dbs/$db_file
+ comment "Successfully updated Sanesecurity production database file: $db_file"
+ log "INFO - Successfully updated Sanesecurity production database file: $db_file"
+ ss_update=1
+ do_clamd_reload=1
+ else
+ echo "Failed to successfully update Sanesecurity production database file: $db_file - SKIPPING"
+ log "WARNING - Failed to successfully update Sanesecurity production database file: $db_file - SKIPPING" ; false
+ fi
else
- echo "Clamscan reports Sanesecurity $db_file database integrity tested BAD - SKIPPING"
- log "WARNING - Clamscan reports Sanesecurity $db_file database integrity tested BAD - SKIPPING" ; false
- fi && \
- (test "$keep_db_backup" = "yes" && cp -f $clam_dbs/$db_file $clam_dbs/$db_file-bak 2>/dev/null ; true) && \
- if rsync -cqt $ss_dir/$db_file $clam_dbs
- then
- perms chown $clam_user:$clam_group $clam_dbs/$db_file
- comment "Successfully updated Sanesecurity production database file: $db_file"
- log "INFO - Successfully updated Sanesecurity production database file: $db_file"
- ss_update=1
- do_clamd_reload=1
- else
- echo "Failed to successfully update Sanesecurity production database file: $db_file - SKIPPING"
- log "WARNING - Failed to successfully update Sanesecurity production database file: $db_file - SKIPPING"
+ grep -h -v -f "$config_dir/whitelist.hex" "$ss_dir/$db_file" > "$test_dir/$db_file"
+ clamscan --infected --no-summary -d "$test_dir/$db_file" "$ham_dir"/* | \
+ sed 's/\.UNOFFICIAL FOUND//' | awk '{print $NF}' > "$config_dir/whitelist.txt"
+ grep -h -f "$config_dir/whitelist.txt" "$test_dir/$db_file" | \
+ cut -d "*" -f2 | sort | uniq >> "$config_dir/whitelist.hex"
+ grep -h -v -f "$config_dir/whitelist.hex" "$test_dir/$db_file" > "$test_dir/$db_file-tmp"
+ mv -f "$test_dir/$db_file-tmp" "$test_dir/$db_file"
+ if clamscan --quiet -d "$test_dir/$db_file" "$config_dir/scan-test.txt" 2>/dev/null
+ then
+ comment "Clamscan reports Sanesecurity $db_file database integrity tested good"
+ log "INFO - Clamscan reports Sanesecurity $db_file database integrity tested good" ; true
+ else
+ echo "Clamscan reports Sanesecurity $db_file database integrity tested BAD - SKIPPING"
+ log "WARNING - Clamscan reports Sanesecurity $db_file database integrity tested BAD - SKIPPING" ; false
+ fi && \
+ (test "$keep_db_backup" = "yes" && cp -f $clam_dbs/$db_file $clam_dbs/$db_file-bak 2>/dev/null ; true) && \
+ if rsync -cqt $test_dir/$db_file $clam_dbs
+ then
+ perms chown $clam_user:$clam_group $clam_dbs/$db_file
+ comment "Successfully updated Sanesecurity production database file: $db_file"
+ log "INFO - Successfully updated Sanesecurity production database file: $db_file"
+ ss_update=1
+ do_clamd_reload=1
+ else
+ echo "Failed to successfully update Sanesecurity production database file: $db_file - SKIPPING"
+ log "WARNING - Failed to successfully update Sanesecurity production database file: $db_file - SKIPPING"
+ fi
fi
fi
fi
@@ -981,14 +1081,14 @@ if [ -n "$msrbl_dbs" ] ; then
comment "======================================================================"
comment "MSRBL Database File Updates"
comment "======================================================================"
- msrbl_mirror_ips=`host -T $msrbl_url | awk '{print $NF}'`
+ msrbl_mirror_ips=`dig +tcp +short $msrbl_url`
for msrbl_mirror_ip in $msrbl_mirror_ips ; do
msrbl_mirror_name=`host $msrbl_mirror_ip | awk '{print $NF}' | cut -d "(" -f2 | sed 's/\(.*\)./\1/'`
msrbl_mirror_site_info="$msrbl_mirror_name $msrbl_mirror_ip"
comment ""
comment "MSRBL mirror site used: $msrbl_mirror_site_info"
log "INFO - MSRBL mirror site used: $msrbl_mirror_site_info"
- if rsync $rsync_output_level $no_motd --files-from=$msrbl_include_dbs -ctz --stats \
+ if rsync $rsync_output_level $no_motd --files-from=$msrbl_include_dbs -ctuz --stats \
$contimeout --timeout=30 rsync://$msrbl_mirror_ip/msrbl $msrbl_dir 2>/dev/null
then
msrbl_rsync_success="1"
@@ -997,25 +1097,59 @@ if [ -n "$msrbl_dbs" ] ; then
comment ""
comment "Testing updated MSRBL database file: $db_file"
log "INFO - Testing updated MSRBL database file: $db_file"
- if clamscan --quiet -d $msrbl_dir/$db_file $config_dir/scan-test.txt 2>/dev/null
- then
- comment "Clamscan reports MSRBL $db_file database integrity tested good"
- log "INFO - Clamscan reports MSRBL $db_file database integrity tested good" ; true
- else
- echo "Clamscan reports MSRBL $db_file database integrity tested BAD - SKIPPING"
- log "WARNING - Clamscan reports MSRBL $db_file database integrity tested BAD - SKIPPING" ; false
- fi && \
- (test "$keep_db_backup" = "yes" && cp -f $clam_dbs/$db_file $clam_dbs/$db_file-bak 2>/dev/null ; true) && \
- if rsync -cqt $msrbl_dir/$db_file $clam_dbs
- then
- perms chown $clam_user:$clam_group $clam_dbs/$db_file
- comment "Successfully updated MSRBL production database file: $db_file"
- log "INFO - Successfully updated MSRBL production database file: $db_file"
- msrbl_update=1
- do_clamd_reload=1
- else
- echo "Failed to successfully update MSRBL production database file: $db_file - SKIPPING"
- log "WARNING - Failed to successfully update MSRBL production database file: $db_file - SKIPPING"
+ if [ "$?" = "0" ] ; then
+ db_ext=`echo $db_file | cut -d "." -f2`
+ if [ -z "$ham_dir" -o "$db_ext" != "ndb" ]
+ then
+ if clamscan --quiet -d "$msrbl_dir/$db_file" "$config_dir/scan-test.txt" 2>/dev/null
+ then
+ comment "Clamscan reports Sanesecurity $db_file database integrity tested good"
+ log "INFO - Clamscan reports Sanesecurity $db_file database integrity tested good" ; true
+ else
+ echo "Clamscan reports Sanesecurity $db_file database integrity tested BAD - SKIPPING"
+ log "WARNING - Clamscan reports Sanesecurity $db_file database integrity tested BAD - SKIPPING" ; false
+ fi && \
+ (test "$keep_db_backup" = "yes" && cp -f $clam_dbs/$db_file $clam_dbs/$db_file-bak 2>/dev/null ; true) && \
+ if rsync -cqt $msrbl_dir/$db_file $clam_dbs
+ then
+ perms chown $clam_user:$clam_group $clam_dbs/$db_file
+ comment "Successfully updated MSRBL production database file: $db_file"
+ log "INFO - Successfully updated MSRBL production database file: $db_file"
+ msrbl_update=1
+ do_clamd_reload=1
+ else
+ echo "Failed to successfully update MSRBL production database file: $db_file - SKIPPING"
+ log "WARNING - Failed to successfully update MSRBL production database file: $db_file - SKIPPING"
+ fi
+ else
+ grep -h -v -f "$config_dir/whitelist.hex" "$msrbl_dir/$db_file" > "$test_dir/$db_file"
+ clamscan --infected --no-summary -d "$test_dir/$db_file" "$ham_dir"/* | \
+ sed 's/\.UNOFFICIAL FOUND//' | awk '{print $NF}' > "$config_dir/whitelist.txt"
+ grep -h -f "$config_dir/whitelist.txt" "$test_dir/$db_file" | \
+ cut -d "*" -f2 | sort | uniq >> "$config_dir/whitelist.hex"
+ grep -h -v -f "$config_dir/whitelist.hex" "$test_dir/$db_file" > "$test_dir/$db_file-tmp"
+ mv -f "$test_dir/$db_file-tmp" "$test_dir/$db_file"
+ if clamscan --quiet -d "$test_dir/$db_file" "$config_dir/scan-test.txt" 2>/dev/null
+ then
+ comment "Clamscan reports Sanesecurity $db_file database integrity tested good"
+ log "INFO - Clamscan reports Sanesecurity $db_file database integrity tested good" ; true
+ else
+ echo "Clamscan reports Sanesecurity $db_file database integrity tested BAD - SKIPPING"
+ log "WARNING - Clamscan reports Sanesecurity $db_file database integrity tested BAD - SKIPPING" ; false
+ fi && \
+ (test "$keep_db_backup" = "yes" && cp -f $clam_dbs/$db_file $clam_dbs/$db_file-bak 2>/dev/null ; true) && \
+ if rsync -cqt $test_dir/$db_file $clam_dbs
+ then
+ perms chown $clam_user:$clam_group $clam_dbs/$db_file
+ comment "Successfully updated MSRBL production database file: $db_file"
+ log "INFO - Successfully updated MSRBL production database file: $db_file"
+ msrbl_update=1
+ do_clamd_reload=1
+ else
+ echo "Failed to successfully update MSRBL production database file: $db_file - SKIPPING"
+ log "WARNING - Failed to successfully update MSRBL production database file: $db_file - SKIPPING"
+ fi
+ fi
fi
fi
done
@@ -1088,26 +1222,61 @@ if [ -n "$si_dbs" ] ; then
comment ""
comment "Testing updated SecuriteInfo database file: $db_file"
log "INFO - Testing updated SecuriteInfo database file: $db_file"
- if clamscan --quiet -d $si_dir/$db_file $config_dir/scan-test.txt 2>/dev/null
- then
- comment "Clamscan reports SecuriteInfo $db_file database integrity tested good"
- log "INFO - Clamscan reports SecuriteInfo $db_file database integrity tested good" ; true
- else
- echo "Clamscan reports SecuriteInfo $db_file database integrity tested BAD - SKIPPING"
- log "WARNING - Clamscan reports SecuriteInfo $db_file database integrity tested BAD - SKIPPING" ; false
- fi && \
- (test "$keep_db_backup" = "yes" && cp -f $clam_dbs/$db_file $clam_dbs/$db_file-bak 2>/dev/null ; true) && \
- if rsync -cqt $si_dir/$db_file $clam_dbs
- then
- perms chown $clam_user:$clam_group $clam_dbs/$db_file
- comment "Successfully updated SecuriteInfo production database file: $db_file"
- log "INFO - Successfully updated SecuriteInfo production database file: $db_file"
- si_updates=1
- si_db_update=1
- do_clamd_reload=1
- else
- echo "Failed to successfully update SecuriteInfo production database file: $db_file - SKIPPING"
- log "WARNING - Failed to successfully update SecuriteInfo production database file: $db_file - SKIPPING"
+ if [ "$?" = "0" ] ; then
+ db_ext=`echo $db_file | cut -d "." -f2`
+ if [ -z "$ham_dir" -o "$db_ext" != "ndb" ]
+ then
+ if clamscan --quiet -d "$si_dir/$db_file" "$config_dir/scan-test.txt" 2>/dev/null
+ then
+ comment "Clamscan reports Sanesecurity $db_file database integrity tested good"
+ log "INFO - Clamscan reports Sanesecurity $db_file database integrity tested good" ; true
+ else
+ echo "Clamscan reports Sanesecurity $db_file database integrity tested BAD - SKIPPING"
+ log "WARNING - Clamscan reports Sanesecurity $db_file database integrity tested BAD - SKIPPING" ; false
+ fi && \
+ (test "$keep_db_backup" = "yes" && cp -f $clam_dbs/$db_file $clam_dbs/$db_file-bak 2>/dev/null ; true) && \
+ if rsync -cqt $si_dir/$db_file $clam_dbs
+ then
+ perms chown $clam_user:$clam_group $clam_dbs/$db_file
+ comment "Successfully updated SecuriteInfo production database file: $db_file"
+ log "INFO - Successfully updated SecuriteInfo production database file: $db_file"
+ si_updates=1
+ si_db_update=1
+ do_clamd_reload=1
+ else
+ echo "Failed to successfully update SecuriteInfo production database file: $db_file - SKIPPING"
+ log "WARNING - Failed to successfully update SecuriteInfo production database file: $db_file - SKIPPING"
+ fi
+ else
+ grep -h -v -f "$config_dir/whitelist.hex" "$si_dir/$db_file" > "$test_dir/$db_file"
+ clamscan --infected --no-summary -d "$test_dir/$db_file" "$ham_dir"/* | \
+ sed 's/\.UNOFFICIAL FOUND//' | awk '{print $NF}' > "$config_dir/whitelist.txt"
+ grep -h -f "$config_dir/whitelist.txt" "$test_dir/$db_file" | \
+ cut -d "*" -f2 | sort | uniq >> "$config_dir/whitelist.hex"
+ grep -h -v -f "$config_dir/whitelist.hex" "$test_dir/$db_file" > "$test_dir/$db_file-tmp"
+ mv -f "$test_dir/$db_file-tmp" "$test_dir/$db_file"
+ if clamscan --quiet -d "$test_dir/$db_file" "$config_dir/scan-test.txt" 2>/dev/null
+ then
+ comment "Clamscan reports Sanesecurity $db_file database integrity tested good"
+ log "INFO - Clamscan reports Sanesecurity $db_file database integrity tested good" ; true
+ else
+ echo "Clamscan reports Sanesecurity $db_file database integrity tested BAD - SKIPPING"
+ log "WARNING - Clamscan reports Sanesecurity $db_file database integrity tested BAD - SKIPPING" ; false
+ fi && \
+ (test "$keep_db_backup" = "yes" && cp -f $clam_dbs/$db_file $clam_dbs/$db_file-bak 2>/dev/null ; true) && \
+ if rsync -cqt $test_dir/$db_file $clam_dbs
+ then
+ perms chown $clam_user:$clam_group $clam_dbs/$db_file
+ comment "Successfully updated SecuriteInfo production database file: $db_file"
+ log "INFO - Successfully updated SecuriteInfo production database file: $db_file"
+ si_updates=1
+ si_db_update=1
+ do_clamd_reload=1
+ else
+ echo "Failed to successfully update SecuriteInfo production database file: $db_file - SKIPPING"
+ log "WARNING - Failed to successfully update SecuriteInfo production database file: $db_file - SKIPPING"
+ fi
+ fi
fi
fi
else
@@ -1174,25 +1343,59 @@ if [ -n "$mbl_dbs" ] ; then
comment ""
comment "Testing updated MalwarePatrol database file: $db_file"
log "INFO - Testing updated database file: $db_file"
- if clamscan --quiet -d $mbl_dir/$db_file $config_dir/scan-test.txt 2>/dev/null
- then
- comment "Clamscan reports MalwarePatrol $db_file database integrity tested good"
- log "INFO - Clamscan reports MalwarePatrol $db_file database integrity tested good" ; true
- else
- echo "Clamscan reports MalwarePatrol $db_file database integrity tested BAD - SKIPPING"
- log "WARNING - Clamscan reports MalwarePatrol $db_file database integrity tested BAD - SKIPPING" ; false
- fi && \
- (test "$keep_db_backup" = "yes" && cp -f $clam_dbs/$db_file $clam_dbs/$db_file-bak 2>/dev/null ; true) && \
- if rsync -cqt $mbl_dir/$db_file $clam_dbs
- then
- perms chown $clam_user:$clam_group $clam_dbs/$db_file
- comment "Successfully updated MalwarePatrol production database file: $db_file"
- log "INFO - Successfully updated MalwarePatrol production database file: $db_file"
- mbl_update=1
- do_clamd_reload=1
- else
- echo "Failed to successfully update MalwarePatrol production database file: $db_file - SKIPPING"
- log "WARNING - Failed to successfully update MalwarePatrol production database file: $db_file - SKIPPING"
+ if [ "$?" = "0" ] ; then
+ db_ext=`echo $db_file | cut -d "." -f2`
+ if [ -z "$ham_dir" -o "$db_ext" != "ndb" ]
+ then
+ if clamscan --quiet -d "$mbl_dir/$db_file" "$config_dir/scan-test.txt" 2>/dev/null
+ then
+ comment "Clamscan reports Sanesecurity $db_file database integrity tested good"
+ log "INFO - Clamscan reports Sanesecurity $db_file database integrity tested good" ; true
+ else
+ echo "Clamscan reports Sanesecurity $db_file database integrity tested BAD - SKIPPING"
+ log "WARNING - Clamscan reports Sanesecurity $db_file database integrity tested BAD - SKIPPING" ; false
+ fi && \
+ (test "$keep_db_backup" = "yes" && cp -f $clam_dbs/$db_file $clam_dbs/$db_file-bak 2>/dev/null ; true) && \
+ if rsync -cqt $mbl_dir/$db_file $clam_dbs
+ then
+ perms chown $clam_user:$clam_group $clam_dbs/$db_file
+ comment "Successfully updated MalwarePatrol production database file: $db_file"
+ log "INFO - Successfully updated MalwarePatrol production database file: $db_file"
+ mbl_update=1
+ do_clamd_reload=1
+ else
+ echo "Failed to successfully update MalwarePatrol production database file: $db_file - SKIPPING"
+ log "WARNING - Failed to successfully update MalwarePatrol production database file: $db_file - SKIPPING"
+ fi
+ else
+ grep -h -v -f "$config_dir/whitelist.hex" "$mbl_dir/$db_file" > "$test_dir/$db_file"
+ clamscan --infected --no-summary -d "$test_dir/$db_file" "$ham_dir"/* | \
+ sed 's/\.UNOFFICIAL FOUND//' | awk '{print $NF}' > "$config_dir/whitelist.txt"
+ grep -h -f "$config_dir/whitelist.txt" "$test_dir/$db_file" | \
+ cut -d "*" -f2 | sort | uniq >> "$config_dir/whitelist.hex"
+ grep -h -v -f "$config_dir/whitelist.hex" "$test_dir/$db_file" > "$test_dir/$db_file-tmp"
+ mv -f "$test_dir/$db_file-tmp" "$test_dir/$db_file"
+ if clamscan --quiet -d "$test_dir/$db_file" "$config_dir/scan-test.txt" 2>/dev/null
+ then
+ comment "Clamscan reports Sanesecurity $db_file database integrity tested good"
+ log "INFO - Clamscan reports Sanesecurity $db_file database integrity tested good" ; true
+ else
+ echo "Clamscan reports Sanesecurity $db_file database integrity tested BAD - SKIPPING"
+ log "WARNING - Clamscan reports Sanesecurity $db_file database integrity tested BAD - SKIPPING" ; false
+ fi && \
+ (test "$keep_db_backup" = "yes" && cp -f $clam_dbs/$db_file $clam_dbs/$db_file-bak 2>/dev/null ; true) && \
+ if rsync -cqt $test_dir/$db_file $clam_dbs
+ then
+ perms chown $clam_user:$clam_group $clam_dbs/$db_file
+ comment "Successfully updated MalwarePatrol production database file: $db_file"
+ log "INFO - Successfully updated MalwarePatrol production database file: $db_file"
+ mbl_update=1
+ do_clamd_reload=1
+ else
+ echo "Failed to successfully update MalwarePatrol production database file: $db_file - SKIPPING"
+ log "WARNING - Failed to successfully update MalwarePatrol production database file: $db_file - SKIPPING"
+ fi
+ fi
fi
else
comment ""
@@ -1235,7 +1438,7 @@ if [ -n "$add_dbs" ] ; then
db_file=`basename $db_url`
if [ "`echo $db_url | cut -d ":" -f1`" = "rsync" ]
then
- if ! rsync $rsync_output_level $no_motd $contimeout --timeout=30 -ctz --exclude=*.txt \
+ if ! rsync $rsync_output_level $no_motd $contimeout --timeout=30 -ctuz --exclude=*.txt \
--stats --exclude=*.sha256 --exclude=*.sig --exclude=*.gz $db_url $add_dir ; then
echo "Failed rsync connection to $base_url - SKIPPED $db_file update"
log "WARNING - Failed rsync connection to $base_url - SKIPPED $db_file update"
@@ -1256,7 +1459,7 @@ if [ -n "$add_dbs" ] ; then
if ! cmp -s $add_dir/$db_file $clam_dbs/$db_file ; then
comment ""
comment "Testing updated database file: $db_file"
- clamscan --quiet -d $add_dir/$db_file $config_dir/scan-test.txt 2>/dev/null
+ clamscan --quiet -d "$add_dir/$db_file" "$config_dir/scan-test.txt" 2>/dev/null
if [ "$?" = "0" ]
then
comment "Clamscan reports $db_file database integrity tested good"
@@ -1296,12 +1499,12 @@ if [ -s "$clam_dbs/local.ign" -a -s "$config_dir/monitor-ign.txt" ] ; then
cp -f local.ign "$config_dir/local.ign"
for entry in `cat "$config_dir/monitor-ign.txt" 2>/dev/null` ; do
sig_file=`echo "$entry" | tr -d "\r" | awk -F ":" '{print $1}'`
- sig_hex=`echo "$entry" | awk -F ":" '{print $NF}'`
+ sig_hex=`echo "$entry" | tr -d "\r" | awk -F ":" '{print $NF}'`
sig_name_old=`echo "$entry" | tr -d "\r" | awk -F ":" '{print $3}'`
sig_ign_old=`grep "$sig_name_old" "$config_dir/local.ign"`
sig_old=`echo "$entry" | tr -d "\r" | cut -d ":" -f3-`
sig_new=`grep -h "$sig_hex" "$sig_file" | tr -d "\r" 2>/dev/null`
- sig_mon_new=`grep -H -n "$sig_hex" "$sig_file"`
+ sig_mon_new=`grep -H -n "$sig_hex" "$sig_file" | tr -d "\r"`
if [ -n "$sig_new" ]
then
if [ "$sig_old" != "$sig_new" -o "$entry" != "$sig_mon_new" ] ; then
@@ -1326,7 +1529,7 @@ if [ -s "$clam_dbs/local.ign" -a -s "$config_dir/monitor-ign.txt" ] ; then
fi
done
if [ "$ign_updated" = "1" ] ; then
- if clamscan --quiet -d $config_dir/local.ign $config_dir/scan-test.txt
+ if clamscan --quiet -d "$config_dir/local.ign" "$config_dir/scan-test.txt"
then
if rsync -cqt $config_dir/local.ign $clam_dbs
then
@@ -1344,6 +1547,28 @@ if [ -s "$clam_dbs/local.ign" -a -s "$config_dir/monitor-ign.txt" ] ; then
fi
fi
+# Check for non-matching whitelist.hex signatures and remove them from the whitelist file (signature modified or removed).
+if [ -n "$ham_dir" ] ; then
+ if [ -s "$config_dir/whitelist.hex" ]
+ then
+ grep -h -f "$config_dir/whitelist.hex" "$work_dir"/*/*.ndb | cut -d "*" -f2 | tr -d "\r" | sort | uniq > "$config_dir/whitelist.tmp"
+ mv -f "$config_dir/whitelist.tmp" "$config_dir/whitelist.hex"
+ rm -f "$config_dir/whitelist.txt"
+ rm -f "$test_dir"/*.*
+ echo ""
+ echo "***********************************************************************"
+ echo "* Signature(s) triggered on HAM directory scan - signature(s) removed *"
+ echo "***********************************************************************"
+ log "WARNING - Signature(s) triggered on HAM directory scan - signature(s) removed"
+ else
+ comment ""
+ comment "================================================="
+ comment "= No signatures triggered on HAM directory scan ="
+ comment "================================================="
+ log "INFO - No signatures triggered on HAM directory scan"
+ fi
+fi
+
# Set appropriate directory and file access permissions
perms chown -R $clam_user:$clam_group "$clam_dbs"
if ! find "$clam_dbs" "$work_dir" -type f -exec chmod 0644 {} + 2>/dev/null ; then
@@ -1357,7 +1582,8 @@ fi
# Reload all clamd databases if updates detected and $reload_dbs" is
# set to "yes", and neither $reload_opt nor $do_clamd_reload are null.
-if [ "$reload_dbs" = "yes" -a -z "$reload_opt" ] ; then
+if [ "$reload_dbs" = "yes" -a -z "$reload_opt" ]
+ then
echo ""
echo "********************************************************************************************"
echo "* Check the script's configuration file, 'reload_dbs' enabled but no 'reload_opt' selected *"
--
packaging for clamav-unoffical-sigs
More information about the Pkg-clamav-commits
mailing list