[Amavisd-new-commits] [pkg-amavisd-new] 02/08: Imported Upstream version 2.9.0
Alexander Wirt
formorer at debian.org
Sat May 10 18:30:24 UTC 2014
This is an automated email from the git hooks/post-receive script.
formorer pushed a commit to branch master
in repository pkg-amavisd-new.
commit 44bb9073f0bbe3cf01a0b1ecb267709dea9bca3b
Author: Alexander Wirt <formorer at debian.org>
Date: Sat May 10 09:42:21 2014 +0200
Imported Upstream version 2.9.0
---
JpegTester.pm | 4 +-
LDAP.ldif | 5 +
LDAP.schema | 11 +
LICENSE | 29 +
MANIFEST | 66 +-
README_FILES/README.customize | 7 +
README_FILES/README.sql-mysql | 5 +-
RELEASE_NOTES | 691 +++++++++-
TinyRedis.pm | 435 +++++++
amavis-mc | 57 +-
amavis-mc_init.sh | 12 +-
amavis-services | 57 +-
amavisd | 2823 ++++++++++++++++++++++++++++++-----------
amavisd-agent | 55 +-
amavisd-nanny | 55 +-
amavisd-new-courier.patch | 57 +-
amavisd-new-qmqpqq.patch | 28 +-
amavisd-release | 78 +-
amavisd-signer | 75 +-
amavisd-snmp-subagent | 55 +-
amavisd-snmp-subagent-zmq | 55 +-
amavisd-snmp-subagent_init.sh | 24 +
amavisd-status | 59 +-
amavisd-submit | 72 +-
amavisd.conf | 3 +
amavisd.conf-default | 23 +-
p0f-analyzer.pl | 53 +-
p0f-analyzer.pl-old | 55 +-
28 files changed, 3783 insertions(+), 1166 deletions(-)
diff --git a/JpegTester.pm b/JpegTester.pm
index 0ac8233..f93fa40 100644
--- a/JpegTester.pm
+++ b/JpegTester.pm
@@ -1,6 +1,6 @@
package JpegTester;
-# Author: Mark Martinec <mark.martinec at ijs.si>, 2004-10;
-# The (new)BSD license applies to this package JpegTester;
+# Author: Mark Martinec <Mark.Martinec at ijs.si>, 2004-10;
+# The 2-clause BSD license applies to this package JpegTester.
use strict;
use re 'taint';
diff --git a/LDAP.ldif b/LDAP.ldif
index 9c389cf..a0b89a4 100644
--- a/LDAP.ldif
+++ b/LDAP.ldif
@@ -29,6 +29,7 @@
# Attribute Types
#-----------------
#
+# DO NOT EDIT!! Use ldapmodify.
dn: cn=amavisd,cn=schema,cn=config
objectClass: olcSchemaConfig
cn: amavisd
@@ -183,6 +184,10 @@ olcAttributeTypes: {45}( 1.3.6.1.4.1.15312.2.2.1.46 NAME 'amavisSaUserName' DE
SC 'SpamAssassin username (for Bayes and AWL lookups)' EQUALITY caseExactIA5M
atch SUBSTR caseExactSubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26{256
} SINGLE-VALUE )
+olcAttributeTypes: {46}( 1.3.6.1.4.1.15312.2.2.1.47 NAME 'amavisDisclaimerOpti
+ ons' DESC 'Altermime disclaimer map data' EQUALITY caseIgnoreIA5Match SUBSTR
+ caseIgnoreIA5SubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.26{256} SINGLE
+ -VALUE )
olcObjectClasses: {0}( 1.3.6.1.4.1.15312.2.2.2.1 NAME 'amavisAccount' DESC 'Am
avisd Account' SUP top AUXILIARY MAY ( amavisVirusLover $ amavisBypassVirusCh
ecks $ amavisSpamLover $ amavisBypassSpamChecks $ amavisBannedFilesLover $ am
diff --git a/LDAP.schema b/LDAP.schema
index 4d97d3c..a0f0f91 100644
--- a/LDAP.schema
+++ b/LDAP.schema
@@ -520,6 +520,17 @@ attributetype ( 1.3.6.1.4.1.15312.2.2.1.46
SYNTAX 1.3.6.1.4.1.1466.115.121.1.26{256}
SINGLE-VALUE )
+#dn: cn=schema
+#changetype: modify
+#add: attributetypes
+attributetype ( 1.3.6.1.4.1.15312.2.2.1.47
+ NAME 'amavisDisclaimerOptions'
+ DESC 'Altermime disclaimer map data'
+ EQUALITY caseIgnoreIA5Match
+ SUBSTR caseIgnoreIA5SubstringsMatch
+ SYNTAX 1.3.6.1.4.1.1466.115.121.1.26{256}
+ SINGLE-VALUE )
+
# Classes
#---------
diff --git a/LICENSE b/LICENSE
index 5b6e7c6..564db83 100644
--- a/LICENSE
+++ b/LICENSE
@@ -1,3 +1,32 @@
+=========
+2-Clause BSD License
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions
+are met:
+1. Redistributions of source code must retain the above copyright notice,
+ this list of conditions and the following disclaimer.
+2. Redistributions in binary form must reproduce the above copyright notice,
+ this list of conditions and the following disclaimer in the documentation
+ and/or other materials provided with the distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS
+BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+POSSIBILITY OF SUCH DAMAGE.
+
+The views and conclusions contained in the software and documentation are
+those of the authors and should not be interpreted as representing official
+policies, either expressed or implied, of the Jozef Stefan Institute.
+
+=========
GNU GENERAL PUBLIC LICENSE
Version 2, June 1991
diff --git a/MANIFEST b/MANIFEST
index 493af93..1fd9313 100644
--- a/MANIFEST
+++ b/MANIFEST
@@ -20,25 +20,36 @@ amavisd-release a program to request releasing a message from a quarantine
amavisd-submit a simple program to pass an email message to amavisd daemon
and to adjust its exit status according to a response received
-amavisd-agent a demo program to access and display SNMP-like counters
- being updated and made available as a Berkeley DB by amavisd
- (there currently is no equivalent to this utility when
- ZMQ is used instead of a Berkeley DB, use a SNMP AgentX
- and snmpbulkwalk for similar functionality)
+amavisd-signer A DKIM signing service daemon for amavisd. It uses an AM.PDP
+ protocol lookalike to receive a request from amavisd and
+ provides two services: choosing a signing key, and signing
+ a message digest with a chosen DKIM private key.
+ Amavisd will use this signing service configured through
+ a $dkim_signing_service setting if it is nonempty, otherwise
+ an internal signing service is used.
-amavisd-nanny a program to show status and keep an eye on health
- of child processes in amavisd-new, using Berkeley DB
+AMAVIS-MIB.txt The MIB module (SNMP Management information base)
+ describing amavisd-new statistics and health information.
+ Useful to a SNMP client program such as snmpwalk or Cacti;
+
+p0f-analyzer.pl a program to interface amavisd with a p0f v3 or v2 utility
+
+p0f-analyzer.pl-old a program to interface amavisd with a p0f v2 utility
+
+JpegTester.pm a Perl module needed if 'check-jpeg' AV checker entry
+ is enabled; to be placed in Perl include paths if needed;
+
+test-messages/ contains sample/test mail messages
+
+TODO missing features, wish list, ...
+
+
+Newer 0MQ-based utilities and auxilliary services:
amavisd-status equivalent to amavisd-nanny, but uses ZMQ as a communication
protocol instead of a Berkeley DB database;
is faster for high-traffic sites;
-amavisd-snmp-subagent a SNMP AgentX program, exporting the amavisd
- statistical counters and gauges database (stored as a
- Berkeley DB database) as well as a process health database
- to a snmpd daemon supporting AgentX protocol (RFC 2741),
- such a NET-SNMP;
-
amavisd-snmp-subagent-zmq equivalent to amavisd-snmp-subagent, but uses
ZMQ as a communications protocol instead of Berkeley DB;
is faster for high-traffic sites;
@@ -60,26 +71,25 @@ amavis-mc a master control program (to be started by a startup script,
amavis-mc_init.sh a sample FreeBSD rc.d shell script for starting
and stopping amavis-mc process
-AMAVIS-MIB.txt The MIB module (SNMP Management information base)
- describing amavisd-new statistics and health information.
- Useful to a SNMP client program such as snmpwalk or Cacti;
-amavisd-signer A DKIM signing service daemon for amavisd. It uses an AM.PDP
- protocol lookalike to receive a request from amavisd and
- provides two services: choosing a signing key, and signing
- a message digest with a chosen DKIM private key.
- Amavisd uses this signing service configured through
- a $dkim_signing_service setting if it is nonempty;
-p0f-analyzer.pl a program to interface amavisd with a p0f v2 utility
- (currently p0f v3 is not yet supported)
+Older utilities based on Berkeley DB (now superseded by 0MQ-based utilities):
-JpegTester.pm a Perl module needed if 'check-jpeg' AV checker entry
- is enabled; to be placed in Perl include paths if needed;
+amavisd-agent a demo program to access and display SNMP-like counters
+ being updated and made available as a Berkeley DB by amavisd
+ (there currently is no equivalent to this utility when
+ ZMQ is used instead of a Berkeley DB, use a SNMP AgentX
+ and snmpbulkwalk for similar functionality)
-test-messages/ contains sample/test mail messages
+amavisd-nanny a program to show status and keep an eye on health
+ of child processes in amavisd-new, using Berkeley DB
+
+amavisd-snmp-subagent a SNMP AgentX program, exporting the amavisd
+ statistical counters and gauges database (stored as a
+ Berkeley DB database) as well as a process health database
+ to a snmpd daemon supporting AgentX protocol (RFC 2741),
+ such a NET-SNMP;
-TODO missing features, wish list, ...
CONTRIBUTED WORK:
diff --git a/README_FILES/README.customize b/README_FILES/README.customize
index 26437f5..17b218d 100644
--- a/README_FILES/README.customize
+++ b/README_FILES/README.customize
@@ -302,6 +302,8 @@ The substitution text for the following simple macros is built-in:
k any recipient declared the message be killed ?
T list of triggered SA tests (only in $log_templ and $log_recip_templ)
+ report_json expands to a JSON representation of a structured log event
+
wrap with arguments: width, prefix, indent, string
will wrap a string to a multiline string of the specified width;
for details see comments before the 'sub wrap_string' in file amavisd;
@@ -335,6 +337,11 @@ The substitution text for the following simple macros is built-in:
join (just like a Perl function join): the first argument is a separator
string, remaining arguments are strings to be concatenated, with a
separator string inserted at every concatenation point;
+ rot13 replaces a string in its argument with an obfuscated string
+ where letters are shifted by 13 positions of an English alphabet
+ (a popular variant of a Caesar cipher to conceal spoilers);
+ this may serve to (poorly) hide strings such as mail Subject or
+ an e-mail address from casual browsing of a log;
limit takes two arguments: a string size limit and some string,
returning the string from the second argument unchanged if its size is
below the limit, or cropped to the size limit, with '[...]' appended;
diff --git a/README_FILES/README.sql-mysql b/README_FILES/README.sql-mysql
index f269a29..dddfa95 100644
--- a/README_FILES/README.sql-mysql
+++ b/README_FILES/README.sql-mysql
@@ -239,7 +239,10 @@ CREATE INDEX msgs_idx_sid ON msgs (sid);
CREATE INDEX msgs_idx_mess_id ON msgs (message_id); -- useful with pen pals
CREATE INDEX msgs_idx_time_num ON msgs (time_num);
-- alternatively when purging based on time_iso (instead of msgs_idx_time_num):
--- CREATE INDEX msgs_idx_time_iso ON msgs (time_iso);
+-- CREATE INDEX msgs_idx_time_iso ON msgs (time_iso);
+-- When using FOREIGN KEY contraints, InnoDB requires index on a field
+-- (an the field must be the first field in the index). Hence create it:
+-- CREATE INDEX msgs_idx_mail_id ON msgs (mail_id);
-- per-recipient information related to each processed message;
-- NOTE: records in msgrcpt without corresponding msgs.mail_id record are
diff --git a/RELEASE_NOTES b/RELEASE_NOTES
index 47ce98b..800a59b 100644
--- a/RELEASE_NOTES
+++ b/RELEASE_NOTES
@@ -1,4 +1,632 @@
---------------------------------------------------------------------------
+ May 9, 2014
+amavisd-new-2.9.0 release notes
+
+Contents:
+ COMPATIBILITY
+ NEW FEATURES SUMMARY
+ RELAXED LICENSE
+ BUG FIXES
+ NEW FEATURES
+ OTHER
+ WHY REDIS?
+
+
+COMPATIBILITY
+
+This version drops dependency on a Perl module Redis, and makes
+dependencies on modules Convert::TNEF and Convert::UUlib truly optional.
+
+The following change may affect third-party log parsers:
+
+To facilitate forensic log analysis and troubleshooting, log entries
+'FWD from' and 'SEND from' at level 1 now carry one additional
+prefixed information field which is the unique internal mail_id of
+the message, possibly followed by a parent_mail_id in parenthesis,
+e.g.:
+ (00525-02) XE9xnQYjrWyd FWD from <...> -> <...>, ...
+ (00495-02) v1pyIOMQkUYD(CIcqao-vCDO9) SEND from <...> -> <...>, ...
+
+No other incompatibilities with a previous version 2.8.1 are expected.
+
+
+NEW FEATURES SUMMARY
+
+- structured log/reporting to a Redis server in JSON format;
+
+- IP address reputation (uses a Redis server);
+
+- added two minor content categories to the major ccat CC_UNCHECKED
+ (encrypted (=1) and over-limits/mail-bomb (=2) );
+
+- introduced a by-recipient setting %final_destiny_maps_by_ccat.
+
+
+RELAXED LICENSE
+
+Some utility / auxiliary programs that were previously released under a
+3-clause BSD license, are now available under a more relaxed 2-clause BSD
+license (also known as a "Simplified BSD License" or a "FreeBSD License").
+
+Affected programs are: amavis-mc, amavis-services, amavisd-status,
+amavisd-snmp-subagent-zmq, amavisd-release, amavisd-submit, p0f-analyzer.pl,
+amavisd-nanny, amavisd-agent, amavisd-snmp-subagent, amavisd-signer,
+JpegTester.pm, and TinyRedis.pm.
+
+Note that TinyRedis.pm is provided in the package as a separate file
+and includes a documentation section. Its copy is also included in
+the file amavisd, so that the separate file is not needed for Amavis
+operation. The separate copy is provided under a 2-clause BSD license
+so that it may be useful for third parties if desired. Eventually it
+could be moved to CPAN as an independent module.
+
+A license of the main program 'amavisd' remains unchanged GPLv2.
+
+
+BUG FIXES
+
+- fixed "Insecure dependency in sprintf" in Sophos SAVI av-scanner,
+ reported by Maciej Uhlig;
+
+- fixed the interface code to virus scanners Sophie, Trophie and fpscand,
+ where a time-out on a long-running virus scan would leave a connection
+ to the virus scanner open and a late response from a scanner to a
+ previous request could be interpreted as a result of the current scan;
+ reported by David Schweikert;
+
+- fixed a bug in transforming an IPv6 alternative form IP address into
+ a preferred form. One effect of this bug was declaring an IPv4-mapped
+ IPv6 address as syntactically incorrect; reported by Patrick Domack;
+
+- if SQL logging was disabled a pen pals feature was non-functional even
+ when a Redis storage back-end was available and collecting data; now
+ pen pals is fully functional with a Redis database back-end and no SQL;
+
+- provided our own Redis client code, avoiding Redis CPAN module bugs,
+ its slowness and non-support for IPv6.
+ The noteworthy Redis CPAN module bug is the #38 (failing to re-select
+ a non-zero-index database after an automatic re-connect to a server).
+ See: https://github.com/melo/perl-redis/issues/38
+ https://github.com/melo/perl-redis/issues/28
+
+- fixed a regexp in parsing wildcarded signing domain in a DKIM key
+ declaration and in a wildcarded sender pattern of signing options
+ (this feature is rarely used, exists for compatibility with dkim_milter);
+
+- dropped hard-coded dependency on modules Convert::TNEF and Convert::UUlib.
+ The Convert::TNEF was made optional in amavisd-new-2.8.0, but the
+ program still failed if the module could not be loaded at startup.
+
+ Both of these modules are now loaded at run time when first used, if
+ specified in the @decoders setting. The use of module Convert::UUlib
+ (the do_ascii entry) is disabled in a default setting of @decoders,
+ and the module Convert::TNEF (the do_tnef entry) is not used
+ if an external TNEF decoder (the do_tnef_ext entry) is available,
+ or if disabled in the @decoders list;
+
+- import a missing do_log_safe() in Amavis::LDAP::Connection to avoid
+ a warning: _WARN: \t(in cleanup)
+ Undefined subroutine &Amavis::LDAP::Connection::do_log_safe
+ called at (eval 101) line 76 during global destruction;
+ a patch by Quanah Gibson-Mount;
+
+- at startup amavis may try to find a decoder for 7z and zip extensions
+ twice; a fix by Quanah Gibson-Mount;
+
+- fixed the amavisd-new-courier.patch which resulted in two instances
+ of sub post_bind_hook(). Only tested for syntax. Thanks to Eray Aslan.
+
+
+NEW FEATURES
+
+- Structured logging/reporting in JSON format is now available through
+ a redis server.
+
+ Each processed mail message and each generated mail message (e.g.
+ a delivery status notification) generates a structured data object
+ (internally a perl associative array). Its fields carry information
+ on most attributes of a mail message and its processing, similar
+ to what is available for logging via macros. Unlike a plain text
+ log which can be difficult to parse and inconsistent due to user
+ configurability of the log template, the data object contains
+ information in a structured form as key/value pairs, where each
+ value can be a scalar or a list or an associative array.
+
+ This internal data object is then serialized to a JSON format and
+ sent to a redis server, where it is appended to a list under a key
+ (arbitrary string) configured by $redis_logging_key setting. This
+ list serves as a queue of log events, which may be pulled from the
+ queue by some third party application, e.g. by a logstash utility
+ or by some home-grown program. Redis server is quite handy for this
+ purpose as it offers blocking requests for pulling events from a
+ queue, which makes it easy to interface with an event processing
+ program. The queue also allows for independent and asynchronous
+ operation between amavisd child processes filling the queue, and
+ a log analyzer pulling entries from the queue.
+
+ The structured logging to redis is enabled when @storage_redis_dsn
+ is configured (see below at the 'IP address reputation' section)
+ and the setting $redis_logging_key is set to some nonempty and
+ nonzero string, and the $redis_logging_queue_size_limit is set
+ to some positive integer value (corresponding to a maximal number
+ of entries allowed in a queue).
+
+ Both the $redis_logging_key and $redis_logging_queue_size_limit are
+ undefined by default, so structured logging to redis is disabled
+ by default even if @storage_redis_dsn is configured.
+
+ The string in $redis_logging_key determines the key in a redis
+ database where the event queue (a redis list) will be maintained.
+ Semantically it is a name of the queue. This setting is a component
+ of policy banks, so log entries can be fed into different redis
+ queues depending on a policy bank loaded for each mail message.
+
+ To prevent a queue in the redis server from growing out of bounds,
+ e.g. when an event-pulling program is temporarily nonfunctional or
+ its processing is falling behind, the $redis_logging_queue_size_limit
+ setting imposes a maximal number of events that amavisd may push into
+ the queue, i.e. the maximal queue size. If the queue size limit is
+ reached, new log events from amavisd are discarded as long as the
+ queue size is at the limit. As a redis database is kept in memory, it
+ makes sense to choose the value of $redis_logging_queue_size_limit low
+ enough so that it does not use too much memory if the log processing
+ program goes down, but also high enough so that short outages of
+ the log processing program do not lose any log events. The setting
+ $redis_logging_queue_size_limit is global (not a component of policy
+ banks).
+
+ And example setting:
+
+ @storage_redis_dsn = ( { server => '[::1]:6379', db_id => 1 } );
+
+ $redis_logging_queue_size_limit = 300000;
+ # takes about 250 MB of redis memory per 100000 log entries
+
+ $redis_logging_key = 'amavis-log';
+
+ $policy_bank{'MYNETS'} = {
+ originating => 1,
+ redis_logging_key => 'amavis-log-myusers', # overrides global setting
+ }
+
+ The oldest event may be pulled from listed queues by the redis command:
+ BLPOP amavis-log amavis-log-myusers 0
+
+ so from a command line this may look like:
+ $ redis-cli -h ::1 -p 6379 -n 1
+ BLPOP amavis-log 0
+
+ The BLPOP redis command blocks if the queue is empty and only returns
+ when the queue becomes nonempty, which makes it easy to use. For high
+ event rates it may be more efficient to batch one LLEN and multiple
+ BLPOP calls in a Lua script executed on a redis server and return events
+ in chunks.
+
+ An example of a logstash plugin configuration for pulling amavis log
+ events from a redis server and feeding them to Elasticsearch:
+
+ input {
+ redis {
+ type => "amavis"
+ host => "::1"
+ db => 1
+ data_type => "list"
+ key => "amavis-log"
+ codec => json {}
+ }
+ }
+ filter {
+ date { match => [ "time_unix", "UNIX" ] }
+ }
+ output {
+ # stdout { codec => rubydebug }
+ elasticsearch_http {
+ host => "127.0.0.1"
+ port => 9200
+ index_type => "%{type}"
+ document_id => "%{mail_id}"
+ codec => json {}
+ }
+ }
+
+ As an alternative for sending log events to a redis server, it is
+ possible to use a macro [:report_json] in a log template, which will
+ expand to a full JSON representation of a log event. As these strings
+ are fairly long (typically 2 kB to 3 kB), this is not a good solution
+ when logging to syslog. It may be usable when logging to a file, but
+ is not an efficient solution and has not been tested in production.
+
+ Here is a (fake) example of a structured log report entry in JSON
+ format, fields are loosely ordered by their semantics in this example.
+ Not all fields are always present. When a boolean fields is missing
+ it should be interpreted as a false.
+
+ {
+ "@timestamp" => "2014-05-06T09:29:47.048Z",
+ "time_unix" => 1399368587.048,
+ "time_iso_week_date" => "2014-W19-2",
+ "partition" => "19",
+
+ "type" => "amavis",
+ "host" => "mailer.example.net",
+ "src_ip" => "::1",
+ "dst_ip" => "::1",
+ "dst_port" => 10024,
+
+ "log_id" => "82329-04",
+ "mail_id" => "Jnk7NzYB8pvl",
+ "mail_id_related" => ["men7HTERZaOF"],
+
+ "client_port" => 41831,
+ "client_ip" => "2001:db8::143:1",
+ "ip_trace" => ["2001:db8::143:1", "192.0.2.242"],
+ "os_fp" => "Windows XP; dist: 6; raw_mtu: 1340; ...",
+
+ "originating" => true,
+ "policy_banks" => ["PROXY-ORIGINATING", "MYNETS"],
+
+ "size" => 302694,
+ "digest_body" => "a4a7db6307c140b12f57feaf076663f8",
+
+ "mail_from" => "mailing-list-1 at example.com",
+ "rcpt_to" => ["recip2 at example.org", "recip1 at example.net"],
+ "rcpt_num" => 2,
+
+ "message_id" => "<003701cf690d$b671b3f0$23551bd0 at example.com>",
+ "author" => ["sending-user at example.com"],
+ "to_addr" => ["recip1 at example.net"],
+ "cc_addr" => ["recip2 at example.org"],
+ "subject" => "Fw: An example 123 - test",
+ "subject_rot13" => "Sj: Na rknzcyr 123 - grfg",
+ "user_agent" => "Microsoft Office Outlook 12.0",
+ "is_bulk" => true,
+ "is_mlist" => true,
+
+ "action" => ["PASS"],
+ "actions_performed" => "RelayedInternal RelayedOutbound",
+ "checks_performed" => "V S H B F P",
+ "content_type" => "Clean",
+ "dkim_new_sig" => ["example.com"],
+ "dsn_sent" => false,
+ "elapsed" => { "Receiving" => 0.009,
+ "Decoding" => 0.053,
+ "VirusCheck" => 0.326
+ "SpamCheck" => 2.116,
+ "Sending" => 0.118,
+ "Amavis" => 0.215,
+ "Total" => 2.672,
+ },
+ "message" =>
+ "82329-04 PASS Clean <mailing-list-1 at example.com>
+ -> <recip2 at example.org>,<recip1 at example.net>",
+ "queued_as" => ["3gNFyR4Mfjzc3", "3gNFyR4n6Lzc4"],
+ "recipients" => [
+ { "action" => "PASS",
+ "ccat_main" => "Clean",
+ "queued_as" => "3gNFyR4Mfjzc3",
+ "rcpt_is_local" => false,
+ "rcpt_to" => "recip2 at example.org",
+ "smtp_code" => "250",
+ "smtp_response" => "250 2.0.0 from MTA(smtp:[::1]:10013): 250 2.0.0 Ok: queued as 3gNFyR4Mfjzc3",
+ "spam_score" => -2.0
+ },
+ { "action" => "PASS",
+ "ccat_main" => "Clean",
+ "mail_id_related" => "men7HTERZaOF",
+ "penpals_age" => 1114599,
+ "queued_as" => "3gNFyR4n6Lzc4",
+ "rcpt_is_local" => true,
+ "rcpt_to" => "recip1 at example.net",
+ "smtp_code" => "250",
+ "smtp_response" => "250 2.0.0 from MTA(smtp:[::1]:10013): 250 2.0.0 Ok: queued as 3gNFyR4n6Lzc4",
+ "spam_score" => -5.272
+ }
+ ],
+ "smtp_code" => ["250"],
+ "spam_score" => -2.0,
+ "tests" => ["ALL_TRUSTED", "AM.PENPAL", "BAYES_00",
+ "MSGID_MULTIPLE_AT", "RP_MATCHES_RCVD"],
+ "tests_ham" => ["AM.PENPAL","BAYES_00","ALL_TRUSTED","RP_MATCHES_RCVD"],
+ "tests_spam" => ["MSGID_MULTIPLE_AT"],
+ }
+
+
+- IP address reputation
+
+ When a Redis storage back-end is enabled, besides the existing pen pals
+ functionality, it now also offers information updating and retrieval
+ on IP address reputation. This function is enabled by default when
+ @storage_redis_dsn is nonempty, but can be disabled by setting
+ $enable_ip_repu to false (to 0 or undef), per policy bank if necessary.
+
+ For each mail message a list of public IP addresses (IPv4 or IPv6) is
+ collected from its 'Received' trace header fields in a mail header
+ section. A redis server maintains a database of each IP address
+ encountered. For each IP address an entry carries a set of counters
+ corresponding to the number of mail messages encountered in the past
+ having this IP address in a trace header. These counters show: a number
+ of spam messages, a number of ham messages, a number of banned or
+ infected messages, and a total number of messages. Also a timestamp of
+ the first and last encounter is kept. Each entry (a set of counters)
+ is subject to automatic expiry, so that infrequently encountered IP
+ addresses are eventually automatically purged from a database by a
+ redis server itself.
+
+ As a sending IP address may change its role (e.g. some machine was
+ infected (sending spam) but now has been cleaned, or a NAT-ted address
+ is reassigned to someone else), currently a crude way of data aging is
+ implemented by discarding entries older than three days since created.
+ This may be refined in the future.
+
+ When a new mail message is being processed, a lookup on all its public
+ IP addresses from a trace is done. For each IP address found in a
+ database a spam score is computed based on a ratio of ham versus
+ all messages, and based on a total number of messages. The largest
+ calculated spam score of all encountered IP addresses is then
+ contributed to a total spam score of a message.
+
+ A formula for computing spam score of each IP address is currently
+ hard-coded, is non-linear and takes into account the total number of
+ encounters of an IP address, diluted by the ratio of ham messages
+ versus all messages seen with this IP address. The computed score
+ cannot be negative, i.e. the IP reputation can only contribute to
+ spamminess of a message and cannot serve as a 'whitelisting' negative
+ score. For the exact formula in use see query_and_update_ip_reputation()
+ in file amavisd.
+
+ A time-to-live of each IP entry is assigned dynamically: frequently
+ encountered IP addresses are given longer expiration times (days),
+ infrequent IP addresses are short-lived and eventually expire,
+ typically in few hours.
+
+ It is possible to exclude certain IP addresses or networks from
+ contributing spam score by listing them in an @ip_repu_ignore_networks
+ list, e.g.:
+
+ @ip_repu_ignore_networks =
+ qw( 192.0.2.44 192.0.2.45 198.51.100.0/24 2001:db8::1:25 );
+
+ This does not preclude a redis lookup on an IP addresses matching
+ the list, but just takes a zero as its score and does not update
+ counters on such address. The mechanism is appropriate for excluding
+ site's own mailers (MSA and MX), or local (e.g. departmental) mailers,
+ which may on occasion emit a spammy message, but should never receive
+ a score penalty. There is no need to include private IP address networks
+ in the list, as these are already exempt from IP reputation database.
+
+ An associated list of lookup tables @ip_repu_ignore_maps (whose only
+ default entry is the \@ip_repu_ignore_networks) offers more flexibility
+ if needed, and is a member of policy banks.
+
+ Like other self-learning mechanisms (e.g. SpamAssassin's auto-learn,
+ AWL, TxRep), the quality of a result depends on a quality of other
+ spam-gauging rules - the better spam/ham classification works
+ (SpamAssassin), the more useful IP reputation becomes. For the purpose
+ of IP reputation's spam and ham counts, a mail is considered spam if
+ its score is at or above 5, and is considered ham when its final score
+ is below 0.5. This is currently hard-coded (see sub save_info_final).
+ Intermediate scores are considered unclassified.
+
+ A nice feature of the mechanism is that it reacts fairly quickly
+ to a new rush-in of unwanted messages from some IP address, either
+ foreign, or local.
+
+ For insight on the IP address reputation behaviour, search the log
+ for ' redis: IP '. At log level 2 only spammy hits are logged, at
+ log level 3 also the clean hits are shown. The log entry shows
+ spam, ham, banned+infected and unclassified counts for an IP address,
+ a percentage of unwanted (spam+banned+infected) messages out of the
+ total count, and the associated score.
+
+ Apart from starting a redis server on a loopback interface (except for
+ changing its 'bind' setting in redis.conf, no other configuration changes
+ are necessary, a database need not be initialized), here is an example
+ configuration in amavisd.conf:
+
+ @storage_redis_dsn = (
+ { server => '[::1]:6379', db_id => 1 },
+ { server => '127.0.0.1:6379', db_id => 1 },
+ );
+
+ # list your MX and MSA mailer IP addresses or networks here:
+ @ip_repu_ignore_networks = qw( 192.0.2.44 2001:db8::/64 );
+
+ A redis server needs to support Lua scripting, which is available
+ since version 2.6. Support for IPv6 is available since version 2.8.0
+ of the redis server.
+
+
+- Added support for decompressing LZ4 streams in mail attachments when
+ an external utility lz4c is available and the 'file' utility recognizes
+ such streams (probably since version file-5.17). Default settings
+ of @decoders and $map_full_type_to_short_type_re now recognize LZ4;
+ if these settings are replaced by a configuration file, the config
+ file needs to be updated to include the new entry.
+
+
+- Added two minor content categories to the major ccat CC_UNCHECKED
+ to allow distinguishing between reasons of decoders failure.
+ * a minor ccat 1 now indicates that at least one mail part was
+ encrypted or otherwise scrambled (e.g. password protected archive);
+ * a minor ccat 2 now indicates that some of the limits for protection
+ against mail bombs was exceeded (e.g. $MAXLEVELS, $MAXFILES,
+ $MAX_EXPANSION_QUOTA, $MAX_EXPANSION_FACTOR).
+ Based on a suggestion and a patch by Carsten Wolff.
+
+ The additional information can be used in any of the *_maps_by_ccat
+ settings, e.g.:
+ $subject_tag_maps_by_ccat{CC_UNCHECKED.',1'} =
+ [ '***UNCHECKED(Encrypted)*** ' ];
+ $subject_tag_maps_by_ccat{CC_UNCHECKED.',2'} =
+ [ '***UNCHECKED(OverLimit)*** ' ];
+ or:
+ $defang_by_ccat{CC_UNCHECKED.',2'} = 1;
+
+
+- introduced a setting %final_destiny_maps_by_ccat, which makes it
+ possible to specify by-recipient final destiny for each contents
+ category, e.g. use D_REJECT on spam to some users, and D_BOUNCE or
+ D_DISCARD or D_PASS for others. Introduced mostly for completeness.
+
+ As a backward compatibility measure the existing %final_destiny_by_ccat
+ is now an alias for the new %final_destiny_maps_by_ccat;
+
+
+- added a setting $outbound_disclaimers_only. When set to true and
+ disclaimers are enabled, it will only allow adding disclaimers
+ to non-local recipients. For backward compatibility the default
+ value is false (undef). Based on a patch by Quanah Gibson-Mount;
+
+
+- the $recipient_delimiter setting can now hold a multi-character string,
+ specifying all characters that can delimit an address extension from
+ a base e-mail address. Previously this setting was restricted to a
+ single character (typically a '+' or a '-').
+
+ When parsing existing e-mail address any of the characters in
+ $recipient_delimiter can delimit an address extension. When adding an
+ address extension (through %addr_extension_maps_by_ccat), the first
+ character in the $recipient_delimiter string is used as a delimiter.
+
+ The change is now in line with a postfix 2.11 that added support
+ for multi recipient-delimiters, and a similar feature in Dovecot.
+
+ A patch contributed by Patrick Domack.
+
+
+- added macros report_json and rot13 (to be used in a log template):
+
+ * the macro 'report_json' expands to a JSON representation of a
+ structured log event;
+
+ * the macro 'rot13' replaces a string in its argument with an obfuscated
+ string where letters are shifted by 13 positions of an English
+ alphabet (a popular variant of a Caesar cipher to conceal spoilers);
+ this may serve to (poorly) hide strings such as mail Subject or
+ an e-mail address from casual browsing of a log;
+
+
+OTHER
+
+- dropped dependency on a CPAN module Redis, implementing our own
+ client-side redis protocol implementation (Amavis::TinyRedis).
+ It is faster and smaller, and supports opening sessions with a
+ redis server over IPv6 (or over IPv4 or over a Unix socket).
+ The redis server supports IPv6 starting with version 2.8.0.
+
+ Currently supported options in @storage_redis_dsn are:
+ server, db_id, password, and ttl.
+
+ The 'server' specifies an INET or INET6 socket (a host IP address
+ or name and a port number) or an absolute path to a Unix socket.
+ An IPv6 address must be enclosed in square brackets. The default
+ value is '127.0.0.1:6379'. Match this with your redis configuration.
+
+ Option 'db_id' specifies a redis database index (given to a "SELECT"
+ redis command). Its value is a (small) integer, defaults to 0.
+ This allows for independent databases to co-exist on the same redis
+ server, e.g. an amavis database and a SpamAssassin Bayes database.
+
+ The 'ttl' option can override a global setting $storage_redis_ttl
+ on a per-server basis. Its value is an integer, representing a number
+ of seconds for expiration time of pen pals records. It defaults to
+ $storage_redis_ttl, which in turn defaults to 16 days (in seconds).
+ This setting does not affect IP reputation records, whose expiration
+ time is computed dynamically.
+
+ Example:
+ $storage_redis_ttl = 22*24*3600; # 22 days for pen pals records
+ @storage_redis_dsn = ( # alternative servers, use the first which works
+ { server => '[::1]:6379', db_id => 1 },
+ { server => '127.0.0.1:6379', db_id => 1, password => 'abc...' },
+ { server => '/tmp/redis.sock', db_id => 1, ttl => 8*24*3600 },
+ );
+
+ Btw, make sure to keep the setting $database_sessions_persistent
+ at its default value (1, i.e. enabled), otherwise Redis performance
+ will suffer somewhat.
+
+
+- store only essential information for pen pals operation to a Redis
+ storage back-end to save memory on a database server; information on
+ inbound messages is no longer stored there, i.e. only information on
+ originating messages is kept;
+
+- more informative logging of pen pals query results when using a Redis
+ storage back-end. The redis support code (Lua and protocol handling)
+ was largely rewritten for efficiency since amavisd-new 2.8.1.
+
+- added LDAP attribute amavisDisclaimerOptions 1.3.6.1.4.1.15312.2.2.1.47
+ to LDAP.schema; contributed by Quanah Gibson-Mount;
+
+- reduced EDNS payload size from 1240 bytes to a conservative default
+ of 1220 bytes when calling Mail::DKIM verifier;
+
+- optimization: filter for public IP addresses from a Received trace
+ only once;
+
+- added one digit of precision in the TIMING log report to reported small
+ elapsed times (below 5 ms);
+
+- in a milter setup (AM.PDP) the log-id wasn't unique; adding a request
+ sequence number to it; a patch by Andreas Schulze;
+
+- avoid writing a notification to stdout about a warm reload for the benefit
+ of a cron job; a patch by Andreas Schulze;
+
+- reduced log level on some of the less useful log messages in a milter
+ setup; a patch by Andreas Schulze;
+
+- documentation README.sql-mysql: added "CREATE INDEX msgs_idx_mail_id..."
+ with a note on an InnoDB requirement for a foreign key; by Jernej Porenta;
+
+
+WHY REDIS?
+
+A redis database was chosen initially because SpamAssassin 3.4.0 supports
+keeping its Bayes database in a redis server, which makes it very fast,
+so this makes a redis database readily available to amavisd too.
+
+Redis has some features that make it suitable for use as a pen pals
+database, for Bayes storage, and now for IP reputation and structured
+logging:
+
+- automatic expiration of entries based on key's individual time-to-live
+ setting makes explicit database maintenance unnecessary;
+
+- accessible over INET (or Unix sockets) allows several amavisd hosts
+ to use a common redis server, possibly running on a dedicated host;
+
+- supports Lua scripting, which makes it possible to perform multiple
+ basic operations in one go as a single application's functional
+ operation. It reduces multiple network round-trip times to a single
+ network transaction, reducing network packet rate and latency;
+
+- compared to SQL storage for pen pals (and for Bayes database),
+ the redis read speed is somewhat faster, and the write speed is
+ MUCH faster;
+
+- as an in-memory database with optional periodic disk persistence
+ it makes it suitable for use as a pen pals, as IP reputation and
+ as Bayes storage: it is fast, and a potential redis server restart
+ reloads data from the last snapshot, thus only losing the last
+ minute or two of updates when trouble strikes, which is acceptable
+ for these three databases.
+
+- makes it possible to eliminate SQL r/w storage if its only purpose
+ was to provide pen pals functionality (and SpamAssassin's Bayes);
+
+Caveat:
+
+Redis server does not offer access controls or strong authentication
+mechanisms. For running a server on the same host as amavisd is running
+the solution is straightforward: just bind the redis server to a
+loopback interface or use a Unix socket. If a network access is desired,
+consider protecting the redis server by a firewall (host-local, or
+on a dedicated subnet).
+
+
+---------------------------------------------------------------------------
June 28, 2013
amavisd-new-2.8.1 release notes
@@ -86,20 +714,21 @@ NEW FEATURES
in memory (with optional periodic persistence on disk), which might be
of concern for busy sites with a long time-to-live setting. Potential
drawbacks of a Redis server are also its lack of sophisticated access
- controls, and lack of IPv6 support in a current version.
+ controls.
A redis database may be shared between hosts running amavisd. It can
be accessed either locally over a Unix socket, or using an INET socket
- (IPv4 only) over a loopback interface (better security) or over a local
- network. Currently (version 2.6.14) a Redis server does not offer
- access over IPv6, which is planned (but not promised) for version 2.8.
+ (IPv4) over a loopback interface (better security) or over a local
+ network. Version 2.6.14 of a Redis server does not offer access over
+ IPv6, but version 2.8.0 and later does.
Required dependencies when Redis support is enabled are a perl module
- "Redis" ( http://search.cpan.org/dist/Redis/ ) version 1.954 or later,
- and a redis server ( http://redis.io/ ) with support for Lua scripting
- (i.e. version 2.6 or later). Most pen pals application-level details on
- queries and storage management is delegated to Lua scripts running on a
- Redis server.
+ "Redis" ( http://search.cpan.org/dist/Redis/ ) version 1.954 (or 1.956)
+ or later and a redis server ( http://redis.io/ ) with support for Lua
+ scripting (i.e. version 2.6 or later). Most pen pals application-level
+ details on queries and storage management is delegated to Lua scripts
+ running on a Redis server. Redis module currently requires a patch to
+ support communication with a redis server over IPv6.
Expiration time of items stored in a redis database is controlled by
a setting $storage_redis_ttl, which is a time-to-live time in seconds
@@ -466,7 +1095,8 @@ NEW FEATURES SUMMARY
busy host (with monitoring disabled, so as not to skew a measurement).
- Use a module IO::Socket::IP if available, instead of dealing directly
- with low-level modules IO::Socket::INET and IO::Socket::INET6;
+ with low-level modules IO::Socket::INET and IO::Socket::INET6.
+ The IO::Socket::IP is a Perl core module since Perl version 5.19.8;
- choose more appropriate defaults if running on an IPv6-only host
(like connecting to ::1 instead of 127.0.0.1 which may not exist);
@@ -540,33 +1170,33 @@ NEW FEATURES - 0MQ
Unix socket, and at least some of these services need visibility
of amavisd processes through signals (kill). At least the forwarding
service must be running when amavisd is operational with $enable_zmq
- at true, otherwise amavisd processing might eventually stall when
+ at true, otherwise amavisd child processes might eventually stall when
their message queue fills up. Preferably amavis-service processes
should be started before amavisd is started, although things would
eventually catch up even if started late or restarted during operation.
- amavisd-status is a user utility program, similar to amavisd-nanny,
which connects to amavis-service 0MQ socket and displays a status
- of running amavisd child processes. This program communicates
- with amavis-service processes through an inet socket and can
- in principle run on a different host (in which case sockets must
- not be bound to a loopback interface). The program can be started
- and stopped at any time, can run under any UID as long as it has
- access to a 0MQ socket $outer_sock_specs, and may run in multiple
- instances if necessary.
+ of running amavisd child processes. This program communicates with
+ amavis-service processes through an inet socket and can in principle
+ run on a different host (in which case sockets must be bound to a
+ publicly reachable interface, not loopback). The program can be
+ started and stopped at any time, can run under any UID as long as
+ it has access to a 0MQ socket $outer_sock_specs, and may run in
+ multiple instances if necessary.
- amavisd-snmp-subagent-zmq is a SNMP AgentX program, functionally
equivalent to amavisd-snmp-subagent. It collects information from
amavis-service processes and passes it as a MIB to an SNMP daemon.
This process communicates with amavis-service processes through an
inet socket and can in principle run on a different host (in which
- case sockets must not be bound to a loopback interface). If access
- to the amavisMta MIB (1.3.6.1.4.1.15312.2.1.3) is desired, the
- amavisd-snmp-subagent-zmq must run on the same host as Postfix
- in order to have access to its queue directories. In principle
- there could be more than one instance of amavisd-snmp-subagent-zmq
- running at the same time, although this hardly serves any practical
- purpose.
+ case sockets must be bound to a publicly reachable interface, not
+ loopback). If access to the amavisMta MIB (1.3.6.1.4.1.15312.2.1.3)
+ subtree is desired, the amavisd-snmp-subagent-zmq must run on
+ the same host as Postfix in order to have access to its queue
+ directories. In principle there could be more than one instance of
+ the amavisd-snmp-subagent-zmq running at the same time, although
+ this hardly serves any practical purpose.
The old amavisd-agent utility does not currently have a 0MQ equivalent;
use snmpbulkwalk with net-snmp and amavisd-snmp-subagent-zmq for similar
@@ -592,11 +1222,10 @@ NEW FEATURES - 0MQ
Required Perl modules are either:
ZeroMQ, which interfaces with a version 2 of a libzmq library
(in case of FreeBSD that would be ports net/p5-ZeroMQ and devel/zmq),
- or with a Crossroads I/O library libxs, which itself is similar to a
- version 3 of libzmq, but provides a zmq 2.1 compatibility interface;
+ or with a Crossroads I/O library libxs (now defunct);
or
ZMQ::LibZMQ2 and ZMQ::Constants modules
- with a version 2 of a libzmq library or with a Crossroads I/O library;
+ with a version 2 of a libzmq library (or with a Crossroads I/O library);
or
ZMQ::LibZMQ3 and ZMQ::Constants
with a version 3 of a libzmq library (FreeBSD ports: devel/zmq-devel).
@@ -624,9 +1253,9 @@ NEW FEATURES - 0MQ
PERFORMANCE with 0MQ
- When scanning messages for spam is enabled (using SpamAssassin), a
- spam scan takes most of the processing time and resources, so replacing
- a BerkeleyDB-based monitoring with a 0MQ-based monitoring brings some
+ When scanning messages for spam (using SpamAssassin), a spam scan
+ takes most of the processing time and resources, so replacing a
+ BerkeleyDB-based monitoring with a 0MQ-based monitoring brings some
speedup on a busy server, but the change is not dramatic.
But as an extreme counter-example: when DKIM signing passed messages,
diff --git a/TinyRedis.pm b/TinyRedis.pm
new file mode 100755
index 0000000..82a6d00
--- /dev/null
+++ b/TinyRedis.pm
@@ -0,0 +1,435 @@
+#
+# Copyright (c) 2013-2014 Mark Martinec
+# All rights reserved.
+#
+# See LICENSE AND COPYRIGHT section in POD text below for usage
+# and distribution rights.
+#
+
+package Redis::TinyRedis;
+
+use strict;
+use re 'taint';
+use warnings;
+
+use Errno qw(EINTR EAGAIN EPIPE ENOTCONN ECONNRESET ECONNABORTED);
+use IO::Socket::UNIX;
+use Time::HiRes ();
+
+use vars qw($VERSION $io_socket_module_name);
+BEGIN {
+ $VERSION = '1.000';
+ if (eval { require IO::Socket::IP }) {
+ $io_socket_module_name = 'IO::Socket::IP';
+ } elsif (eval { require IO::Socket::INET6 }) {
+ $io_socket_module_name = 'IO::Socket::INET6';
+ } elsif (eval { require IO::Socket::INET }) {
+ $io_socket_module_name = 'IO::Socket::INET';
+ }
+}
+
+sub new {
+ my($class, %args) = @_;
+ my $self = bless { args => {%args} }, $class;
+ my $outbuf = ''; $self->{outbuf} = \$outbuf;
+ $self->{batch_size} = 0;
+ $self->{server} = $args{server} || $args{sock} || '127.0.0.1:6379';
+ $self->{on_connect} = $args{on_connect};
+ return if !$self->connect;
+ $self;
+}
+
+sub DESTROY {
+ my $self = $_[0];
+ local($@, $!, $_);
+ undef $self->{sock};
+}
+
+sub disconnect {
+ my $self = $_[0];
+ local($@, $!);
+ undef $self->{sock};
+}
+
+sub connect {
+ my $self = $_[0];
+
+ $self->disconnect;
+ my $sock;
+ my $server = $self->{server};
+ if ($server =~ m{^/}) {
+ $sock = IO::Socket::UNIX->new(
+ Peer => $server, Type => SOCK_STREAM);
+ } else {
+ $sock = $io_socket_module_name->new(
+ PeerAddr => $server, Proto => 'tcp');
+ }
+ if ($sock) {
+ $self->{sock} = $sock;
+
+ $self->{sock_fd} = $sock->fileno; $self->{fd_mask} = '';
+ vec($self->{fd_mask}, $self->{sock_fd}, 1) = 1;
+
+ # an on_connect() callback must not use batched calls!
+ $self->{on_connect}->($self) if $self->{on_connect};
+ }
+ $sock;
+}
+
+# Receive, parse and return $cnt consecutive redis replies as a list.
+#
+sub _response {
+ my($self, $cnt) = @_;
+
+ my $sock = $self->{sock};
+ if (!$sock) {
+ $self->connect or die "Connect failed: $!";
+ $sock = $self->{sock};
+ };
+
+ my @list;
+
+ for (1 .. $cnt) {
+
+ my $result = <$sock>;
+ if (!defined $result) {
+ $self->disconnect;
+ die "Error reading from Redis server: $!";
+ }
+ chomp $result;
+ my $resp_type = substr($result, 0, 1, '');
+
+ if ($resp_type eq '$') { # bulk reply
+ if ($result < 0) {
+ push(@list, undef); # null bulk reply
+ } else {
+ my $data = ''; my $ofs = 0; my $len = $result + 2;
+ while ($len > 0) {
+ my $nbytes = read($sock, $data, $len, $ofs);
+ if (!$nbytes) {
+ $self->disconnect;
+ defined $nbytes or die "Error reading from Redis server: $!";
+ die "Redis server closed connection";
+ }
+ $ofs += $nbytes; $len -= $nbytes;
+ }
+ chomp $data;
+ push(@list, $data);
+ }
+
+ } elsif ($resp_type eq ':') { # integer reply
+ push(@list, 0+$result);
+
+ } elsif ($resp_type eq '+') { # status reply
+ push(@list, $result);
+
+ } elsif ($resp_type eq '*') { # multi-bulk reply
+ push(@list, $result < 0 ? undef : $self->_response(0+$result) );
+
+ } elsif ($resp_type eq '-') { # error reply
+ die "$result\n";
+
+ } else {
+ die "Unknown Redis reply: $resp_type ($result)";
+ }
+ }
+ \@list;
+}
+
+sub _write_buff {
+ my($self, $bufref) = @_;
+
+ if (!$self->{sock}) { $self->connect or die "Connect failed: $!" };
+ my $nwrite;
+ for (my $ofs = 0; $ofs < length($$bufref); $ofs += $nwrite) {
+ # to reliably detect a disconnect we need to check for an input event
+ # using a select; checking status of syswrite is not sufficient
+ my($rout, $wout, $inbuff); my $fd_mask = $self->{fd_mask};
+ my $nfound = select($rout=$fd_mask, $wout=$fd_mask, undef, undef);
+ defined $nfound && $nfound >= 0 or die "Select failed: $!";
+ if (vec($rout, $self->{sock_fd}, 1) &&
+ !sysread($self->{sock}, $inbuff, 1024)) {
+ # eof, try reconnecting
+ $self->connect or die "Connect failed: $!";
+ }
+ local $SIG{PIPE} = 'IGNORE'; # don't signal on a write to a widowed pipe
+ $nwrite = syswrite($self->{sock}, $$bufref, length($$bufref)-$ofs, $ofs);
+ next if defined $nwrite;
+ $nwrite = 0;
+ if ($! == EINTR || $! == EAGAIN) { # no big deal, try again
+ Time::HiRes::sleep(0.1); # slow down, just in case
+ } else {
+ $self->disconnect;
+ if ($! == ENOTCONN || $! == EPIPE ||
+ $! == ECONNRESET || $! == ECONNABORTED) {
+ $self->connect or die "Connect failed: $!";
+ } else {
+ die "Error writing to redis socket: $!";
+ }
+ }
+ }
+ 1;
+}
+
+# Send a redis command with arguments, returning a redis reply.
+#
+sub call {
+ my $self = shift;
+
+ my $buff = '*' . scalar(@_) . "\015\012";
+ $buff .= '$' . length($_) . "\015\012" . $_ . "\015\012" for @_;
+
+ $self->_write_buff(\$buff);
+ local($/) = "\015\012";
+ my $arr_ref = $self->_response(1);
+ $arr_ref && $arr_ref->[0];
+}
+
+# Append a redis command with arguments to a batch.
+#
+sub b_call {
+ my $self = shift;
+
+ my $bufref = $self->{outbuf};
+ $$bufref .= '*' . scalar(@_) . "\015\012";
+ $$bufref .= '$' . length($_) . "\015\012" . $_ . "\015\012" for @_;
+ ++ $self->{batch_size};
+}
+
+# Send a batch of commands, returning an arrayref of redis replies,
+# each array element corresponding to one command in a batch.
+#
+sub b_results {
+ my $self = $_[0];
+ my $batch_size = $self->{batch_size};
+ return if !$batch_size;
+ my $bufref = $self->{outbuf};
+ $self->_write_buff($bufref);
+ $$bufref = ''; $self->{batch_size} = 0;
+ local($/) = "\015\012";
+ $self->_response($batch_size);
+}
+
+1;
+
+__END__
+=head1 NAME
+
+Redis::TinyRedis - client side of the Redis protocol
+
+=head1 SYNOPSIS
+
+EXAMPLE:
+
+ use Redis::TinyRedis;
+
+ sub on_connect {
+ my($r) = @_;
+ # $r->call('AUTH', 'xyz');
+ $r->call('SELECT', 3);
+ $r->call('CLIENT', 'SETNAME', "test[$$]");
+ 1;
+ }
+
+# my $server = '/tmp/redis.sock';
+ my $server = '[::1]:6379';
+
+ my $r = Redis::TinyRedis->new(server => $server,
+ on_connect => \&on_connect);
+ $r or die "Error connecting to a Redis server: $!";
+
+ $r->call('SET', 'key123', 'val123'); # will die on error
+ $r->call('SET', 'key456', 'val456'); # will die on error
+
+ my $v = $r->call('GET', 'key123');
+ if (defined $v) { printf("got %s\n", $v) }
+ else { printf("key not in a database\n") }
+
+ my @keys = ('key123', 'key456', 'keynone');
+ my $values = $r->call('MGET', @keys);
+ printf("got %s => %s\n", $_, shift @$values // 'UNDEF') for @keys;
+
+ # batching (pipelining) multiple commands saves on round-trips
+ $r->b_call('DEL', 'keyxxx');
+ $r->b_call('HINCRBY', 'keyxxx', 'cnt1', 5);
+ $r->b_call('HINCRBY', 'keyxxx', 'cnt2', 1);
+ $r->b_call('HINCRBY', 'keyxxx', 'cnt2', 2);
+ $r->b_call('EXPIRE', 'keyxxx', 120);
+ $r->b_results; # collect response ignoring results, dies on error
+
+ my $counts = $r->call('HMGET', 'keyxxx', 'cnt1', 'cnt2', 'cnt3');
+ printf("count %s\n", $_ // 'UNDEF') for @$counts;
+
+ # Lua server side scripting
+ my $lua_results = $r->call('EVAL',
+ 'return redis.call("HGETALL", KEYS[1])', 1, 'keyxxx');
+ printf("%s\n", join(', ', @$lua_results));
+
+ # traversing all keys
+ for (my $cursor = 0; ; ) {
+ my $pair = $r->call('SCAN', $cursor, 'COUNT', 20);
+ ($cursor, my $elements) = @$pair;
+ printf("key: %s\n", $_) for @$elements;
+ last if !$cursor;
+ }
+
+ # another batch of commands
+ $r->b_call('DEL', $_) for @keys;
+ my $results = $r->b_results; # collect response
+ printf("delete status for %s: %d\n", $_, shift @$results) for @keys;
+
+ # monitor activity on a database through Redis keyspace notifications
+ $r->call('CONFIG', 'SET', 'notify-keyspace-events', 'KEA');
+ $r->call('PSUBSCRIBE', '__key*__:*');
+ for (1..20) {
+ my $msg = $r->call; # collect one message at a time
+ printf("%s\n", join(", ",@$msg));
+ }
+ $r->call('UNSUBSCRIBE');
+ $r->call('CONFIG', 'SET', 'notify-keyspace-events', '');
+
+ undef $r; # DESTROY cleanly closes a connection to a redis server
+
+=head1 DESCRIPTION
+
+This is a Perl module Redis::TinyRedis implementing a client side of
+the Redis protocol, i.e. a unified request protocol as introduced
+in Redis 1.2. Design priorities were speed, simplicity, error checking.
+
+=head1 METHODS
+
+=head2 new
+
+Initializes a Redis::TinyRedis object and established a connection
+to a Redis server. Returns a Redis::TinyRedis object if the connection
+was successfully established (by calling a connect() method implicitly),
+or false otherwise, leaving a failure reason in $! .
+
+=over 4
+
+=item B<server>
+
+Specifies a socket where a Redis server is listening. If a string
+starts with a '/' an absolute path to a Unix socket is assumed,
+otherwise it is interpreted as an INET or INET6 socket specification
+in a syntax as recognized by a C<PeerAddr> option of the underlying
+socket module (IO::Socket::IP, or IO::Socket::INET6, or IO::Socket::INET),
+e.g. '127.0.0.1:6379' or '[::1]:6379' or 'localhost::6379'.
+Port number must be explicitly specified.
+
+A default is '127.0.0.1:6379'.
+
+=item B<on_connect>
+
+Specifies an optional callback subroutine, to be called by a connect()
+method after each successful establishment of a connection to a redis server.
+Useful as a provider of a Redis client authentication or for database
+index selection.
+
+The on_connect() callback is given a Redis::TinyRedis object as its
+argument. This object also carries all arguments that were given in
+a call to new() when it was created, including any additional options
+unrecognized and ignored by Redis::TinyRedis->new().
+
+An on_connect() callback subroutine must not use batched calls
+(b_call / b_results), but may use the call() method.
+
+=back
+
+=head2 connect
+
+Establishes a connection to a Redis server. Returns a socket object,
+or undef if the connection failed, leaving error status in $! .
+The connect() method is called implicitly by new(), or by call() or
+b_results() if a connection was dropped due to some previous failure.
+It may be called explicitly by an application, possibly combined with
+a disconnect() method, to give more control to the application.
+
+=head2 disconnect
+
+Closes a connection to a Redis server if it is established,
+does nothing if a connection is not established. The connection
+will be re-established by subsequent calls to connect() or call()
+or b_results().
+
+Closing a connection is implied by a DESTROY method, so dropping
+references to a Redis::TinyRedis object also cleanly disconnects
+an established session with a Redis server.
+
+=head2 call
+
+Sends a redis command with arguments, returning a redis reply.
+The first argument is expected to be a name of a Redis command,
+followed by its arguments according to Redis documentation.
+
+The command will die if a Redis server returns an error reply.
+It may also die if it needs to implicitly re-establish a connection
+and the connect() call fails.
+
+The returned value is an integer if a Redis server returns an integer
+reply, is a status string if a status reply is returned, is undef if a
+null bulk reply is returned, is a string in case of a bulk reply, and
+is a reference to an array of results in case of a multi-bulk reply.
+
+=head2 b_call
+
+Appends a redis command with arguments to a batch.
+The first argument is expected to be a name of a Redis command,
+followed by its arguments according to Redis documentation.
+
+=head2 b_results
+
+Sends a batch of commands, then resets the batch. Returns a reference
+to an array of redis replies, each array element corresponding to one
+command in a batch.
+
+The command will die if a Redis server returns an error reply.
+It may also die if it needs to implicitly re-establish a connection
+and the connect() call fails.
+
+Returns a reference to an array of results, each one corresponding
+to one command of a batch. A data type of each array element is the
+same as described in a call() method.
+
+=head1 AUTHOR
+
+Mark Martinec, C<< <Mark.Martinec at ijs.si> >>
+
+=head1 BUGS
+
+Please send bug reports to the author.
+
+=head1 LICENSE AND COPYRIGHT
+
+Copyright (c) 2013-2014 Mark Martinec
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions
+are met:
+1. Redistributions of source code must retain the above copyright notice,
+ this list of conditions and the following disclaimer.
+2. Redistributions in binary form must reproduce the above copyright notice,
+ this list of conditions and the following disclaimer in the documentation
+ and/or other materials provided with the distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS
+BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+POSSIBILITY OF SUCH DAMAGE.
+
+The views and conclusions contained in the software and documentation are
+those of the authors and should not be interpreted as representing official
+policies, either expressed or implied, of the Jozef Stefan Institute.
+
+(the above license is the 2-clause BSD license, also known as
+ a "Simplified BSD License", and pertains to this program only)
+
+=cut
diff --git a/amavis-mc b/amavis-mc
index d739a48..88d9701 100755
--- a/amavis-mc
+++ b/amavis-mc
@@ -4,35 +4,38 @@
# This is amavis-mc, a master (of ceremonies) processes to supervise
# supporting service processes (such as amavis-services) used by amavisd-new.
#
-# Author: Mark Martinec <mark.martinec at ijs.si>
-# Copyright (C) 2012,2013 Mark Martinec, All Rights Reserved.
+# Author: Mark Martinec <Mark.Martinec at ijs.si>
#
-# Redistribution and use in source and binary forms, with or without
-# modification, are permitted provided that the following conditions are met:
+# Copyright (c) 2012-2014, Mark Martinec
+# All rights reserved.
#
-# * Redistributions of source code must retain the above copyright notice,
-# this list of conditions and the following disclaimer.
-# * Redistributions in binary form must reproduce the above copyright notice,
-# this list of conditions and the following disclaimer in the documentation
-# and/or other materials provided with the distribution.
-# * Neither the name of the author, nor the name of the "Jozef Stefan"
-# Institute, nor the names of contributors may be used to endorse or
-# promote products derived from this software without specific prior
-# written permission.
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright notice,
+# this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright notice,
+# this list of conditions and the following disclaimer in the documentation
+# and/or other materials provided with the distribution.
#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
-# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
-# OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
-# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
-# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
-# OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
-# WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
-# OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
-# ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS
+# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+# POSSIBILITY OF SUCH DAMAGE.
#
-#(the license above is the new BSD license, and pertains to this program only)
+# The views and conclusions contained in the software and documentation are
+# those of the authors and should not be interpreted as representing official
+# policies, either expressed or implied, of the Jozef Stefan Institute.
+
+# (the above license is the 2-clause BSD license, also known as
+# a "Simplified BSD License", and pertains to this program only)
#
# Patches and problem reports are welcome.
# The latest version of this program is available at:
@@ -45,12 +48,12 @@ use warnings;
use warnings FATAL => qw(utf8 void);
no warnings 'uninitialized';
-use vars qw($VERSION); $VERSION = 2.008001;
+use vars qw($VERSION); $VERSION = 2.008002;
use vars qw($myproduct_name $myversion_id $myversion_date $myversion);
BEGIN {
$myproduct_name = 'amavis-mc';
- $myversion_id = '2.8.1'; $myversion_date = '20130321';
+ $myversion_id = '2.9.0'; $myversion_date = '20140506';
$myversion = "$myproduct_name-$myversion_id ($myversion_date)";
}
diff --git a/amavis-mc_init.sh b/amavis-mc_init.sh
index 3ea5ec4..bdf1b39 100755
--- a/amavis-mc_init.sh
+++ b/amavis-mc_init.sh
@@ -13,7 +13,9 @@
. /etc/rc.subr
name="amavis_mc"
-rcvar=`set_rcvar`
+desc="Amavis status monitoring and SNMP statistics services"
+rcvar="amavis_mc_enable"
+
command="/usr/local/sbin/amavis-mc"
command_interpreter="/usr/bin/perl"
pidfile="/var/amavis/amavis-mc.pid"
@@ -22,9 +24,9 @@ required_files="/usr/local/sbin/amavis-services"
load_rc_config $name
-amavis_mc_enable=${amavis_mc_enable:-"NO"}
-amavis_mc_flags=${amavis_mc_flags:-"-P ${pidfile}"}
-amavis_mc_user=${amavis_mc_user:-"vscan"}
-amavis_mc_group=${amavis_mc_group:-"vscan"}
+: ${amavis_mc_enable:="NO"}
+: ${amavis_mc_flags:="-P ${pidfile}"}
+: ${amavis_mc_user:="vscan"}
+: ${amavis_mc_group:="vscan"}
run_rc_command "$1"
diff --git a/amavis-services b/amavis-services
index 2c03ce9..37d80de 100755
--- a/amavis-services
+++ b/amavis-services
@@ -3,35 +3,38 @@
#------------------------------------------------------------------------------
# This is amavis-services, a set of supervisor processes for amavisd-new.
#
-# Author: Mark Martinec <mark.martinec at ijs.si>
-# Copyright (C) 2012,2013 Mark Martinec, All Rights Reserved.
+# Author: Mark Martinec <Mark.Martinec at ijs.si>
#
-# Redistribution and use in source and binary forms, with or without
-# modification, are permitted provided that the following conditions are met:
+# Copyright (c) 2012-2014, Mark Martinec
+# All rights reserved.
#
-# * Redistributions of source code must retain the above copyright notice,
-# this list of conditions and the following disclaimer.
-# * Redistributions in binary form must reproduce the above copyright notice,
-# this list of conditions and the following disclaimer in the documentation
-# and/or other materials provided with the distribution.
-# * Neither the name of the author, nor the name of the "Jozef Stefan"
-# Institute, nor the names of contributors may be used to endorse or
-# promote products derived from this software without specific prior
-# written permission.
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright notice,
+# this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright notice,
+# this list of conditions and the following disclaimer in the documentation
+# and/or other materials provided with the distribution.
#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
-# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
-# OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
-# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
-# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
-# OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
-# WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
-# OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
-# ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS
+# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+# POSSIBILITY OF SUCH DAMAGE.
#
-#(the license above is the new BSD license, and pertains to this program only)
+# The views and conclusions contained in the software and documentation are
+# those of the authors and should not be interpreted as representing official
+# policies, either expressed or implied, of the Jozef Stefan Institute.
+
+# (the above license is the 2-clause BSD license, also known as
+# a "Simplified BSD License", and pertains to this program only)
#
# Patches and problem reports are welcome.
# The latest version of this program is available at:
@@ -44,7 +47,7 @@ use warnings;
use warnings FATAL => qw(utf8 void);
no warnings 'uninitialized';
-use vars qw($VERSION); $VERSION = 2.008001;
+use vars qw($VERSION); $VERSION = 2.008002;
use Errno qw(ESRCH ENOENT);
use POSIX qw(strftime);
@@ -57,7 +60,7 @@ use vars qw($syslog_ident $syslog_facility);
use vars qw($inner_sock_specs $outer_sock_specs $snmp_sock_specs);
BEGIN {
$myproduct_name = 'amavis-services';
- $myversion_id = '2.8.1'; $myversion_date = '20130321';
+ $myversion_id = '2.9.0'; $myversion_date = '20140506';
$myversion = "$myproduct_name-$myversion_id ($myversion_date)";
}
diff --git a/amavisd b/amavisd
index 159e680..f721756 100755
--- a/amavisd
+++ b/amavisd
@@ -1,5 +1,8 @@
#!/usr/bin/perl -T
-#!/usr/bin/perl -T -d:NYTProf
+
+### profiling:
+### #!/usr/bin/perl -d:NYTProf
+### NYTPROF=start=no:addpid=1:forkdepth=1 amavisd -m 5 foreground
#------------------------------------------------------------------------------
# This is amavisd-new.
@@ -11,7 +14,7 @@
# on amavisd-snapshot-20020300).
#
# All work since amavisd-snapshot-20020300:
-# Copyright (C) 2002-2013 Mark Martinec,
+# Copyright (C) 2002-2014 Mark Martinec,
# All Rights Reserved.
# with contributions from the amavis-user mailing list and individuals,
# as acknowledged in the release notes.
@@ -63,6 +66,7 @@
# Amavis::DbgLog
# Amavis::Timing
# Amavis::Util
+# Amavis::JSON
# Amavis::ProcControl
# Amavis::rfc2821_2822_Tools
# Amavis::Lookup::RE
@@ -70,8 +74,8 @@
# Amavis::Lookup::Opaque
# Amavis::Lookup::OpaqueRef
# Amavis::Lookup::Label
-# Amavis::Lookup::SQLfield (just the new() method)
-# Amavis::Lookup::LDAPattr (just the new() method)
+# Amavis::Lookup::SQLfield (just the new() method declared here)
+# Amavis::Lookup::LDAPattr (just the new() method declared here)
# Amavis::Lookup
# Amavis::Expand
# Amavis::TempDir
@@ -128,6 +132,8 @@
# Amavis::Tools
#------------------------------------------------------------------------------
+use sigtrap qw(stack-trace BUS SEGV EMT FPE ILL SYS);
+
use strict;
use re 'taint';
use warnings;
@@ -199,9 +205,10 @@ sub fetch_modules($$@) {
local $_ = $m;
$_ .= /^auto::/ ? '.al' : '.pm' if !m{^/} && !m{\.(?:pm|pl|al|ix)\z};
s{::}{/}g;
- # eval { my_require $_ } #more informative on err, but some problems reported
- eval { require $_ }
- or do {
+ eval {
+ require $_;
+ # my_require $_; # more informative on err, but some problems reported
+ } or do {
my $eval_stat = $@ ne '' ? $@ : "errno=$!"; chomp $eval_stat;
push(@missing,$m);
$eval_stat =~ s/^/ /mgs; # indent
@@ -277,7 +284,7 @@ use constant CC_VIRUS => 9;
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
%EXPORT_TAGS = (
'dynamic_confvars' => # per- policy bank settings
@@ -307,7 +314,7 @@ BEGIN {
$undecipherable_subject_tag $localpart_is_case_sensitive
$recipient_delimiter $replace_existing_extension
$hdr_encoding $bdy_encoding $hdr_encoding_qb
- $allow_disclaimers
+ $allow_disclaimers $outbound_disclaimers_only
$prepend_header_fields_hdridx
$allow_fixing_improper_header
$allow_fixing_improper_header_folding $allow_fixing_long_header_lines
@@ -323,9 +330,10 @@ BEGIN {
@altermime_args_disclaimer @disclaimer_options_bysender_maps
%signed_header_fields @dkim_signature_options_bysender_maps
$enable_dkim_verification $enable_dkim_signing $dkim_signing_service
- $dkim_minimum_key_bits $enable_ldap
+ $dkim_minimum_key_bits $enable_ldap $enable_ip_repu $redis_logging_key
- @local_domains_maps @mynetworks_maps @client_ipaddr_policy
+ @local_domains_maps
+ @mynetworks_maps @client_ipaddr_policy @ip_repu_ignore_maps
@forward_method_maps @newvirus_admin_maps @banned_filename_maps
@spam_quarantine_bysender_to_maps
@spam_tag_level_maps @spam_tag2_level_maps @spam_tag3_level_maps
@@ -345,7 +353,7 @@ BEGIN {
@remove_existing_spam_headers_maps
@sa_userconf_maps @sa_username_maps
- %final_destiny_by_ccat %forward_method_maps_by_ccat
+ %final_destiny_maps_by_ccat %forward_method_maps_by_ccat
%lovers_maps_by_ccat %defang_maps_by_ccat %subject_tag_maps_by_ccat
%quarantine_method_by_ccat %quarantine_to_maps_by_ccat
%notify_admin_templ_by_ccat %notify_recips_templ_by_ccat
@@ -385,8 +393,8 @@ BEGIN {
$MIN_EXPANSION_QUOTA $MIN_EXPANSION_FACTOR
$MAX_EXPANSION_QUOTA $MAX_EXPANSION_FACTOR
$database_sessions_persistent $lookup_maps_imply_sql_and_ldap
- @lookup_sql_dsn @storage_sql_dsn
- @storage_redis_dsn $storage_redis_ttl
+ @lookup_sql_dsn @storage_sql_dsn @storage_redis_dsn
+ $storage_redis_ttl $redis_logging_queue_size_limit
$sql_schema_version $timestamp_fmt_mysql
$sql_quarantine_chunksize_max $sql_allow_8bit_address
$sql_lookups_no_at_means_domain $ldap_lookups_no_at_means_domain
@@ -407,7 +415,7 @@ BEGIN {
$dspam $sa_spawned
)],
'platform' => [qw(
- $can_truncate $unicode_aware $my_pid
+ $profiling $can_truncate $unicode_aware $my_pid
$AF_INET6 $have_inet4 $have_inet6 $io_socket_module_name
&D_TEMPFAIL &D_REJECT &D_BOUNCE &D_DISCARD &D_PASS
&CC_CATCHALL &CC_CLEAN &CC_MTA &CC_OVERSIZED &CC_BADH
@@ -462,7 +470,8 @@ BEGIN {
# not be visible in other modules, but may be referenced through
# @*_maps variables for backward compatibility
[qw(
- %local_domains @local_domains_acl $local_domains_re @mynetworks
+ %local_domains @local_domains_acl $local_domains_re
+ @mynetworks @ip_repu_ignore_networks
%bypass_virus_checks @bypass_virus_checks_acl $bypass_virus_checks_re
%bypass_spam_checks @bypass_spam_checks_acl $bypass_spam_checks_re
%bypass_banned_checks @bypass_banned_checks_acl $bypass_banned_checks_re
@@ -544,7 +553,7 @@ sub cr($) {
$name, $current_policy_bank{'policy_bank_name'}));
}
}
- !defined $var ? undef : !ref $var ? \$var : $var;
+ ref $var ? $var : defined $var ? \$var : undef;
}
# return a ref to a config variable value (which is supposed to be an array),
@@ -560,7 +569,7 @@ sub ca($) {
$name, $current_policy_bank{'policy_bank_name'}));
}
}
- !defined $var ? [] : !ref $var ? [$var] : $var;
+ ref $var ? $var : defined $var ? [$var] : [];
}
sub deprecate_var($$$) {
@@ -667,12 +676,12 @@ BEGIN { # init_primary: version, $unicode_aware, base policy bank
$myprogram_name = $0; # typically 'amavisd'
local $1; $myprogram_name =~ s{([^/]*)\z}{$1}s;
$myproduct_name = 'amavisd-new';
- $myversion_id = '2.8.1'; $myversion_date = '20130628';
+ $myversion_id = '2.9.0'; $myversion_date = '20140509';
$myversion = "$myproduct_name-$myversion_id ($myversion_date)";
$myversion_id_numeric = # x.yyyzzz, allows numerical compare, like Perl $]
sprintf('%8.6f', $1 + ($2 + $3/1000)/1000)
- if $myversion_id =~ /^(\d+)(?:\.(\d*)(?:\.(\d*))?)?(.*)$/;
+ if $myversion_id =~ /^(\d+)(?:\.(\d*)(?:\.(\d*))?)?(.*)$/s;
$sql_schema_version = $myversion_id_numeric;
$unicode_aware =
@@ -682,7 +691,7 @@ BEGIN { # init_primary: version, $unicode_aware, base policy bank
for my $tag (@EXPORT_TAGS{'dynamic_confvars', 'legacy_dynamic_confvars'}) {
for my $v (@$tag) {
local($1,$2);
- if ($v !~ /^([%\$\@])(.*)\z/) { die "Unsupported variable type: $v" }
+ if ($v !~ /^([%\$\@])(.*)\z/s) { die "Unsupported variable type: $v" }
else {
no strict 'refs'; my($type,$name) = ($1,$2);
$current_policy_bank{$name} = $type eq '$' ? \${"Amavis::Conf::$name"}
@@ -709,6 +718,9 @@ BEGIN {
# Create debugging output - true: log to stderr; false: log to syslog/file
$DEBUG = 0;
+ # Is Devel::NYTProf profiler loaded?
+ $profiling = 1 if DB->UNIVERSAL::can('enable_profile');
+
# In case of trouble, allow preserving temporary files for forensics
$allow_preserving_evidence = 1;
@@ -760,10 +772,24 @@ BEGIN {
# $enable_dkim_signing = undef;
# $enable_dkim_verification = undef;
- $reputation_factor = 0.2; # a value between 0 and 1, controlling the amount
- # of 'bending' of a calculated spam score towards a fixed score assigned
- # to a signing domain (its 'reputation') through @signer_reputation_maps;
- # the formula is: adjusted_spam_score = f*reputation + (1-f)*spam_score;
+ $enable_ip_repu = 1; # ignored when @storage_redis_dsn is empty
+
+ # a key (string) for a redis list serving as a queue of json events
+ # for logstash / elasticsearch use; undef or empty or '0' disables
+ # logging of events to redis
+ $redis_logging_key = undef; # e.g. "amavis-log";
+
+ # a limit on the length of a redis list - new log events will be dropped
+ # while the queue size limit is exceeded; undef or 0 disables logging;
+ # reasonable value: 100000, takes about 250 MB of memory in a redis server
+ # when noone is pulling events from the list
+ $redis_logging_queue_size_limit = undef;
+
+ $reputation_factor = 0.2; # DKIM reputation: a value between 0 and 1,
+ # controlling the amount of 'bending' of a calculated spam score
+ # towards a fixed score assigned to a signing domain (its 'reputation')
+ # through @signer_reputation_maps; the formula is:
+ # adjusted_spam_score = f*reputation + (1-f)*spam_score
# which has the same semantics as auto_whitelist_factor in SpamAssassin AWL
# keep SQL, LDAP and Redis sessions open when idle
@@ -815,13 +841,14 @@ BEGIN {
$mail_id_size_bits = 72; # 24, 48, 72, 96
# redis data (penpals) expiration - time-to-live in seconds of stored items
- $storage_redis_ttl = 16*24*60*60; # 16 days
+ $storage_redis_ttl = 16*24*60*60; # 16 days (only affects penpals data)
$sql_store_info_for_all_msgs = 1;
$penpals_bonus_score = undef; # maximal (positive) score value by which spam
# score is lowered when sender is known to have previously received mail
# from our local user from this mail system. Zero or undef disables
- # pen pals lookups in SQL tables msgs and msgrcpt, and is a default.
+ # pen pals lookups in Redis or in SQL tables msgs and msgrcpt, and
+ # is a default.
$penpals_halflife = 7*24*60*60; # exponential decay time constant in seconds;
# pen pal bonus is halved for each halflife period since the last mail
# sent by a local user to a current message's sender
@@ -831,7 +858,7 @@ BEGIN {
# when (SA_score - $penpals_bonus_score > $penpals_threshold_high)
# pen pals lookup will not be performed to save time, as it could not
# influence blocking of spam even at maximal penpals bonus (age=0);
- # usual choice for value would be kill level or other reasonably high
+ # usual choice for value would be a kill level or other reasonably high
# value; undef lets the threshold be ignored and is a default (useful
# for testing and statistics gathering);
@@ -891,9 +918,10 @@ BEGIN {
: $have_inet6 ? '[::1]' : '127.0.0.1';
}
@inet_acl = qw( 127.0.0.1 [::1] ); # allow SMTP access only from localhost
- @mynetworks = qw( 127.0.0.0/8 [::1] [fe80::]/10 [fc00::]/7
- 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 169.254.0.0/16 );
- $originating = 0; # a boolean, initially reflects @mynetworks,
+ @mynetworks = qw( 127.0.0.0/8 [::1] 169.254.0.0/16 [fe80::]/10
+ 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16
+ [fc00::]/7 ); # consider also RFC 6598: 100.64.0.0/10
+ $originating = 0; # a boolean, initially reflects @mynetworks match,
# but may be modified later through a policy bank
$forward_method = $have_inet6 && !$have_inet4 ? 'smtp:[::1]:10025'
@@ -950,7 +978,7 @@ BEGIN {
# encoding (encoding in MIME terminology)
$hdr_encoding_qb = 'Q'; # quoted-printable (default)
- #$hdr_encoding_qb = 'B'; # base64 (usual for far east charsets)
+# $hdr_encoding_qb = 'B'; # base64 (usual for far east charsets)
$smtpd_recipient_limit = 1100; # max recipients (RCPT TO) - sanity limit
@@ -1140,16 +1168,16 @@ BEGIN {
$final_spam_destiny = D_PASS;
$final_bad_header_destiny = D_PASS;
- # If decided to pass viruses (or spam) to certain recipients using
- # %lovers_maps_by_ccat, or by %final_destiny_by_ccat resulting in D_PASS,
- # one may set the corresponding %addr_extension_maps_by_ccat to some string,
- # and the recipient address will have this string appended as an address
- # extension to a local-part (mailbox part) of the address. This extension
- # can be used by a final local delivery agent for example to place such mail
- # in different folder. Leaving this variable undefined or an empty string
- # prevents appending address extension. Recipients which do not match
- # @local_domains_maps are not affected (i.e. non-local recipients (=outbound
- # mail) do not get address extension appended).
+ # If decided to pass viruses (or spam) to certain recipients
+ # by %final_destiny_maps_by_ccat yielding a D_PASS, or %lovers_maps_by_ccat
+ # yielding a true, one may set the corresponding %addr_extension_maps_by_ccat
+ # to some string, and the recipient address will have this string appended
+ # as an address extension to a local-part (mailbox part) of the address.
+ # This extension can be used by a final local delivery agent for example
+ # to place such mail in different folder. Leaving this variable undefined
+ # or an empty string prevents appending address extension. Recipients
+ # which do not match @local_domains_maps are not affected (i.e. non-local
+ # recipients (=outbound mail) do not get address extension appended).
#
# LDAs usually default to stripping away address extension if no special
# handling for it is specified, so having this option enabled normally
@@ -1294,7 +1322,9 @@ BEGIN {
CC_SPAMMY, 'Spammy', # tag2_level
CC_SPAMMY.',1','Spammy3', # tag3_level
CC_SPAM, 'Spam', # kill_level
- CC_UNCHECKED, 'Unchecked',
+ CC_UNCHECKED, 'Unchecked',
+ CC_UNCHECKED.',1', 'UncheckedEncrypted',
+ CC_UNCHECKED.',2', 'UncheckedOverLimits',
CC_BANNED, 'Banned',
CC_VIRUS, 'Virus',
);
@@ -1466,7 +1496,8 @@ BEGIN {
[qr/^UTF.* Unicode text\b/i => 'txt'],
[qr/^'diff' output text\b/ => 'txt'],
[qr/^GNU message catalog\b/ => 'mo'],
- [qr/^PGP encrypted data\b/ => 'pgp'],
+ [qr/^(?:PGP|GPG) encrypted data\b/ => ['pgp','pgp.enc'] ],
+ [qr/^PGP message\b/ => ['pgp','pgp.enc'] ],
[qr/^PGP armored data( signed)? message\b/ => ['pgp','pgp.asc'] ],
[qr/^PGP armored\b/ => ['pgp','pgp.asc'] ],
@@ -1525,6 +1556,7 @@ BEGIN {
[qr/^lzma compressed\b/ => 'lzma'],
[qr/^lrz compressed\b/ => 'lrz'], #***(untested)
[qr/^lzop compressed\b/ => 'lzo'],
+ [qr/^LZ4 compressed\b/ => 'lz4'],
[qr/^compress'd/ => 'Z'],
[qr/^Zip archive\b/i => 'zip'],
[qr/^7-zip archive\b/i => '7z'],
@@ -1626,6 +1658,7 @@ BEGIN {
['lrz', \&Amavis::Unpackers::do_uncompress,
['lrzip -q -k -d -o -', 'lrzcat -q -k'] ],
['lzo', \&Amavis::Unpackers::do_uncompress, \$unlzop],
+ ['lz4', \&Amavis::Unpackers::do_uncompress, ['lz4c -d'] ],
['rpm', \&Amavis::Unpackers::do_uncompress, \$rpm2cpio],
# ['rpm2cpio.pl', 'rpm2cpio'] ],
[['cpio','tar'], \&Amavis::Unpackers::do_pax_cpio, \$pax],
@@ -1646,7 +1679,7 @@ BEGIN {
[['zip','kmz'], \&Amavis::Unpackers::do_7zip, ['7za', '7z'] ],
[['zip','kmz'], \&Amavis::Unpackers::do_unzip],
['7z', \&Amavis::Unpackers::do_7zip, ['7zr', '7za', '7z'] ],
- [[qw(7z zip gz bz2 Z tar)],
+ [[qw(gz bz2 Z tar)],
\&Amavis::Unpackers::do_7zip, ['7za', '7z'] ],
[[qw(xz lzma jar cpio arj rar swf lha iso cab deb rpm)],
\&Amavis::Unpackers::do_7zip, '7z' ],
@@ -1659,6 +1692,7 @@ BEGIN {
\%local_domains, \@local_domains_acl, \$local_domains_re);
@mynetworks_maps = (\@mynetworks);
@client_ipaddr_policy = map(($_,'MYNETS'), @mynetworks_maps);
+ @ip_repu_ignore_maps = (\@ip_repu_ignore_networks);
@bypass_virus_checks_maps = (
\%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re);
@@ -1697,7 +1731,7 @@ BEGIN {
@map_full_type_to_short_type_maps = (\$map_full_type_to_short_type_re);
# @banned_filename_maps = ( {'.' => [$banned_filename_re]} );
# @banned_filename_maps = ( {'.' => 'DEFAULT'} );#names mapped by %banned_rules
- @banned_filename_maps = ( 'DEFAULT' ); # same as previous, but shorter
+ @banned_filename_maps = ( 'DEFAULT' ); # same as above, but shorter
@viruses_that_fake_sender_maps = (\$viruses_that_fake_sender_re, 1);
@spam_tag_level_maps = (\$sa_tag_level_deflt); # CC_CLEAN,1
@spam_tag2_level_maps = (\$sa_tag2_level_deflt); # CC_SPAMMY
@@ -1725,19 +1759,23 @@ BEGIN {
# @debug_recipient_maps = ();
@remove_existing_spam_headers_maps = (\$remove_existing_spam_headers);
- # new variables, no backward compatibility needed, empty by default
+ # new variables, no backward compatibility needed, empty by default:
# @score_sender_maps, @author_to_policy_bank_maps, @signer_reputation_maps,
# @message_size_limit_maps
# build backward-compatible settings hashes
- %final_destiny_by_ccat = (
+ #
+ %final_destiny_maps_by_ccat = (
+ # value is normally a list of by-recipient lookup tables, but for compa-
+ # tibility with old %final_destiny_by_ccat a value may also be a scalar
CC_VIRUS, sub { c('final_virus_destiny') },
CC_BANNED, sub { c('final_banned_destiny') },
CC_UNCHECKED, sub { c('final_unchecked_destiny') },
CC_SPAM, sub { c('final_spam_destiny') },
CC_BADH, sub { c('final_bad_header_destiny') },
- CC_MTA.',1', D_TEMPFAIL,
- CC_MTA.',2', D_REJECT,
+ CC_MTA.',1', D_TEMPFAIL, # MTA response was 4xx
+ CC_MTA.',2', D_REJECT, # MTA response was 5xx
+ CC_MTA, D_TEMPFAIL,
CC_OVERSIZED, D_BOUNCE,
CC_CATCHALL, D_PASS,
);
@@ -1749,7 +1787,9 @@ BEGIN {
# a multiline message will produce a valid multiline SMTP response
CC_VIRUS, 'id=%n - INFECTED: %V',
CC_BANNED, 'id=%n - BANNED: %F',
- CC_UNCHECKED, 'id=%n - UNCHECKED',
+ CC_UNCHECKED.',1', 'id=%n - UNCHECKED: encrypted',
+ CC_UNCHECKED.',2', 'id=%n - UNCHECKED: over limits',
+ CC_UNCHECKED, 'id=%n - UNCHECKED',
CC_SPAM, 'id=%n - spam',
CC_SPAMMY.',1', 'id=%n - spammy (tag3)',
CC_SPAMMY, 'id=%n - spammy',
@@ -1778,6 +1818,7 @@ BEGIN {
CC_BADH, sub { ca('bad_header_lovers_maps') },
);
%defang_maps_by_ccat = (
+ # compatible with legacy %defang_by_ccat: value may be a scalar
CC_VIRUS, sub { c('defang_virus') },
CC_BANNED, sub { c('defang_banned') },
CC_UNCHECKED, sub { c('defang_undecipherable') },
@@ -1990,8 +2031,10 @@ sub OpaqueRef { Amavis::Lookup::OpaqueRef->new(@_) }
# @forward_method_maps = ( OpaqueRef(\$forward_method) );
@forward_method_maps = ( sub { Opaque(c('forward_method')) } );
-# compatibility with old names
-use vars qw(%defang_by_ccat $sql_partition_tag $DO_SYSLOG $LOGFILE);
+# retain compatibility with old names
+use vars qw(%final_destiny_by_ccat %defang_by_ccat
+ $sql_partition_tag $DO_SYSLOG $LOGFILE);
+*final_destiny_by_ccat = \%final_destiny_maps_by_ccat;
*defang_by_ccat = \%defang_maps_by_ccat;
*sql_partition_tag = \$partition_tag;
*DO_SYSLOG = \$do_syslog;
@@ -2021,6 +2064,7 @@ use vars qw(%defang_by_ccat $sql_partition_tag $DO_SYSLOG $LOGFILE);
[ qr'^HTML/Bankish' => 0.1 ], # F-Prot
[ qr'^PORCUPINE_JUNK' => 0.1 ],
[ qr'^PORCUPINE_PHISHING' => 0.1 ],
+ [ qr'^Porcupine\.Junk' => 0.1 ],
[ qr'-SecuriteInfo\.com(\.|\z)' => undef ], # keep as infected
[ qr'^MBL_NA\.UNOFFICIAL' => 0.1 ], # false positives
[ qr'^MBL_' => undef ], # keep as infected
@@ -2035,7 +2079,7 @@ use vars qw(%defang_by_ccat $sql_partition_tag $DO_SYSLOG $LOGFILE);
sub label_default_maps() {
for my $varname (qw(
@disclaimer_options_bysender_maps @dkim_signature_options_bysender_maps
- @local_domains_maps @mynetworks_maps
+ @local_domains_maps @mynetworks_maps @ip_repu_ignore_maps
@forward_method_maps @newvirus_admin_maps @banned_filename_maps
@spam_quarantine_bysender_to_maps
@spam_tag_level_maps @spam_tag2_level_maps @spam_tag3_level_maps
@@ -2191,7 +2235,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&init &amavis_log_id &collect_log_stats
&log_to_stderr &log_fd &open_log &close_log &write_log);
@@ -2218,7 +2262,7 @@ use vars qw($current_actual_syslog_ident $current_actual_syslog_facility);
use vars qw($log_lines $log_retries %log_entries_by_level %log_status_counts);
use vars qw($log_prio_debug $log_prio_info $log_prio_notice
$log_prio_warning $log_prio_err $log_prio_crit);
-BEGIN { # saves a few ms later by avoiding a subroutine call
+BEGIN { # saves a few ms by avoiding a subroutine call later
$log_prio_debug = LOG_DEBUG;
$log_prio_info = LOG_INFO;
$log_prio_notice = LOG_NOTICE;
@@ -2318,7 +2362,7 @@ sub close_log() {
sub write_log($$) {
my($level,$errmsg) = @_;
return if $within_write_log;
- $within_write_log = 1;
+ $within_write_log++;
my $am_id = !defined $current_amavis_log_id ? ''
: "($current_amavis_log_id) ";
# my $old_locale = POSIX::setlocale(LC_TIME,'C'); # English dates required!
@@ -2378,6 +2422,7 @@ sub write_log($$) {
flock($loghandle,LOCK_EX) or die "Can't lock a log file: $!";
seek($loghandle,0,2) or die "Can't position log file to its tail: $!";
$loghandle->print($s) or die "Error writing to log file: $!";
+ # we have autoflush on, so unlocking here is safe
flock($loghandle,LOCK_UN) or die "Can't unlock a log file: $!";
}
}
@@ -2394,7 +2439,7 @@ use re 'taint';
BEGIN {
use vars qw(@ISA $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
import Amavis::Conf qw(:platform $TEMPBASE);
import Amavis::Log qw(write_log);
}
@@ -2491,7 +2536,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&init §ion_time &report &get_time_so_far
&get_rusage &rusage_report);
@@ -2555,7 +2600,8 @@ sub report() {
my $dt_c = $t <= $t00 ? 0 : $t-$t00; # handle possible clock jumps
my $dtp = $dt >= $total ? 100 : $dt*100.0/$total; # this event
my $dtp_c = $dt_c >= $total ? 100 : $dt_c*100.0/$total; # cumulative
- push(@sections, sprintf('%s: %.0f (%.0f%%)%.0f',
+ my $fmt = $dt >= 0.005 ? "%.0f" : "%.1f";
+ push(@sections, sprintf("%s: $fmt (%.0f%%)%.0f",
$section, $dt*1000, $dtp, $dtp_c));
$t0 = $t;
}
@@ -2645,12 +2691,13 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
- @EXPORT_OK = qw(&untaint &untaint_inplace
- &min &max &minmax &unique_list &unique_ref
+ @EXPORT_OK = qw(&untaint &untaint_inplace &min &max &minmax
+ &unique_list &unique_ref &format_time_interval
&safe_encode &safe_encode_ascii &safe_encode_utf8
- &safe_decode &q_encode &orcpt_encode &orcpt_decode
+ &safe_decode &safe_decode_latin1
+ &q_encode &orcpt_encode &orcpt_decode
&xtext_encode &xtext_decode &proto_encode &proto_decode
&ll &do_log &do_log_safe &snmp_count &snmp_count64
&snmp_counters_init &snmp_counters_get &snmp_initial_oids
@@ -2686,13 +2733,15 @@ use MIME::Base64;
use Encode; # Perl 5.8 UTF-8 support
use Scalar::Util qw(tainted);
-use vars qw($enc_ascii $enc_utf8 $enc_tainted
+use vars qw($enc_ascii $enc_utf8 $enc_latin1 $enc_tainted
$enc_taintsafe $enc_is_utf8_buggy);
BEGIN {
- $enc_ascii = Encode::find_encoding('ascii');
- $enc_utf8 = Encode::find_encoding('UTF-8');
+ $enc_ascii = Encode::find_encoding('ascii');
+ $enc_utf8 = Encode::find_encoding('UTF-8');
+ $enc_latin1 = Encode::find_encoding('ISO-8859-1');
$enc_ascii or die "Amavis::Util: unknown encoding 'ascii'";
$enc_utf8 or die "Amavis::Util: unknown encoding 'UTF-8'";
+ $enc_latin1 or die "Amavis::Util: unknown encoding 'ISO-8859-1'";
$enc_tainted = substr($ENV{PATH}.$ENV{HOME}, 0,0); # tainted empty string
# Encode::is_utf8 is always false on tainted in Perl 5.8, Perl bug #32687
$enc_is_utf8_buggy = 1 if $] < 5.010;
@@ -2762,7 +2811,7 @@ sub minmax(@) {
# and duplicates removed
#
sub unique_list(@) {
- my $r = @_ == 1 && ref($_[0]) ? $_[0] : \@_; # accept list, or a list ref
+ my $r = @_ == 1 && ref($_[0]) ? $_[0] : \@_; # accepts list, or a list ref
my %seen; my(@result) = grep(defined($_) && !$seen{$_}++, @$r);
@result;
}
@@ -2770,11 +2819,21 @@ sub unique_list(@) {
# same as unique, except that it returns a ref to the resulting list
#
sub unique_ref(@) {
- my $r = @_ == 1 && ref($_[0]) ? $_[0] : \@_; # accept list, or a list ref
+ my $r = @_ == 1 && ref($_[0]) ? $_[0] : \@_; # accepts list, or a list ref
my %seen; my(@result) = grep(defined($_) && !$seen{$_}++, @$r);
\@result;
}
+sub format_time_interval($) {
+ my $t = $_[0];
+ return 'undefined' if !defined $t;
+ my $sign = ''; if ($t < 0) { $sign = '-'; $t = - $t };
+ my $dd = int($t / (24*3600)); $t = $t - $dd*(24*3600);
+ my $hh = int($t / 3600); $t = $t - $hh*3600;
+ my $mm = int($t / 60); $t = $t - $mm*60;
+ sprintf("%s%d %d:%02d:%02d", $sign, $dd, $hh, $mm, int($t+0.5));
+}
+
# A wrapper for Encode::encode, avoiding a bug in Perl 5.8.0 which causes
# Encode::encode to loop and fill memory when given a tainted string.
# Also works around a CPAN bug #64642 in module Encode:
@@ -2812,6 +2871,12 @@ sub safe_encode_utf8($) {
$enc_tainted . $enc_utf8->encode(untaint($_[0]), 0);
}
+sub safe_decode_latin1($) {
+# my $str = $_[0];
+ return undef if !defined $_[0]; # must return undef even in a list context!
+ $enc_latin1->decode($_[0]);
+}
+
sub safe_decode($$;$) {
# my($encoding,$str,$check) = @_;
my $encoding = shift;
@@ -3042,7 +3107,7 @@ sub stir_random() {
}
# generate a reasonably unique (long-term) id based on collected entropy.
-# The result is a pair of (mostly public) mail_id, and a secret id,
+# The result is a pair of a (mostly public) mail_id, and a secret id,
# where mail_id == b64(md5(secret_bin)). The secret id could be used to
# authorize releasing quarantined mail. Both the mail_id and secret id are
# strings of characters [A-Za-z0-9-_], with an additional restriction
@@ -3078,9 +3143,13 @@ sub generate_mail_id() {
# retry on less than 7% of cases
do_log(5,'generate_mail_id retry: %s', $id_b64);
}
- my $secret_b64 = encode_base64($secret_bin,''); # $mail_id_size_bits/6 chars
- $secret_bin = 'X' x length($secret_bin); # can't hurt to be conservative
- $id_b64 =~ tr{+/}{-_}; # base64 -> RFC 4648 base64url [A-Za-z0-9-_]
+ $id_b64 =~ tr{+/}{-_}; # base64 -> RFC 4648 base64url [A-Za-z0-9-_]
+ if (!wantarray) { # not interested in secret
+ $secret_bin = 'X' x length($secret_bin); # can't hurt to wipe out
+ return $id_b64;
+ }
+ my $secret_b64 = encode_base64($secret_bin,''); # $mail_id_size_bits/6 chars
+ $secret_bin = 'X' x length($secret_bin); # can't hurt to wipe out
$secret_b64 =~ tr{+/}{-_}; # base64 -> RFC 4648 base64url [A-Za-z0-9-_]
# do_log(5,'generate_mail_id: %s %s', $id_b64, $secret_b64);
($id_b64, $secret_b64);
@@ -3137,7 +3206,7 @@ sub snmp_initial_oids() {
['sysObjectID', 'OID', '1.3.6.1.4.1.15312.2'],
# iso.org.dod.internet.private.enterprise.ijs.amavisd-new
['sysUpTime', 'INT', int(time)], # to be converted to TIM
- # later it must be converted to timeticks (10ms since start)
+ # later it must be converted to timeticks (10ms units since start)
['sysContact', 'STR', $snmp_contact],
['sysName', 'STR', $myhostname],
['sysLocation', 'STR', $snmp_location],
@@ -3894,6 +3963,66 @@ sub collect_equal_delivery_recips($$$) {
1;
#
+package Amavis::JSON;
+use strict;
+use re 'taint';
+
+# serialize a data structure to JSON, RFC 4627
+
+BEGIN {
+ require Exporter;
+ use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
+ $VERSION = '2.320';
+ @ISA = qw(Exporter);
+ @EXPORT_OK = qw(&boolean &numeric);
+}
+use subs @EXPORT_OK;
+
+our %jesc = ( # JSON escaping
+ "\x22" => '\\"', "\x5C" => '\\\\',
+ "\x08" => '\\b', "\x09" => '\\t',
+ "\x0A" => '\\n', "\x0C" => '\\f', "\x0D" => '\\r',
+ "\x{2028}" => '\\u2028', "\x{2029}" => '\\u2029' );
+
+our($FALSE, $TRUE) = ('false', 'true');
+sub boolean { bless($_[0] ? \$TRUE : \$FALSE) }
+sub numeric { my $value = $_[0]; bless(\$value) }
+
+# serialize a data structure to JSON, RFC 4627
+#
+sub encode($); # prototype
+sub encode($) {
+ my $val = $_[0];
+ my $ref = ref $val;
+ local $1;
+ if ($ref) {
+ if ($ref eq 'ARRAY') {
+ return '[' . join(',', map(encode($_), @$val)) . ']';
+ } elsif ($ref eq 'HASH') {
+ return '{' .
+ join(',',
+ map {
+ my $k = $_;
+ $k =~ s{ ([\x00-\x1f\x7f\x{2028}\x{2029}"\\]) }
+ { exists $jesc{$1} ? $jesc{$1}
+ : sprintf('\\u%.4X',ord($1)) }xgse;
+ '"' . $k . '":' . encode($val->{$_});
+ } sort keys %$val
+ ) . '}';
+ } elsif ($ref->isa('Amavis::JSON')) { # numeric or boolean type
+ return defined $$val ? $$val : 'null';
+ }
+ # fall through, encode other refs as strings, helps debugging
+ }
+ return 'null' if !defined $val;
+ $val =~ s{ ([\x00-\x1f\x7f\x{2028}\x{2029}"\\]) }
+ { exists $jesc{$1} ? $jesc{$1} : sprintf('\\u%.4X',ord($1)) }xgse;
+ return '"' . $val . '"';
+}
+
+1;
+
+#
package Amavis::ProcControl;
use strict;
use re 'taint';
@@ -3901,7 +4030,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&exit_status_str &proc_status_ok &kill_proc &cloexec
&run_command &run_command_consumer &run_as_subprocess
@@ -4448,13 +4577,13 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
@EXPORT = qw(
&rfc2822_timestamp &rfc2822_utc_timestamp
&iso8601_timestamp &iso8601_utc_timestamp
&iso8601_week &iso8601_yearweek &iso8601_year_and_week &iso8601_weekday
- &format_time_interval &make_received_header_field &parse_received
+ &make_received_header_field &parse_received
&fish_out_ip_from_received &parse_message_id
&split_address &split_localpart &replace_addr_fields &make_query_keys
"e_rfc2821_local &qquote_rfc2821_local
@@ -4531,7 +4660,7 @@ sub rfc2822_utc_timestamp($) {
# provide date-time timestamp (local time) as specified in ISO 8601 (EN 28601)
#
sub iso8601_timestamp($;$$$) {
- my($t,$suppress_zone,$dtseparator,$with_field_separators) = @_;
+ my($t, $suppress_zone, $dtseparator, $with_field_separators) = @_;
# can't use %z because some systems do not support it (is treated as %Z)
my $fmt = $with_field_separators ? "%Y-%m-%dT%H:%M:%S" : "%Y%m%dT%H%M%S";
$fmt =~ s/T/$dtseparator/ if defined $dtseparator;
@@ -4543,11 +4672,13 @@ sub iso8601_timestamp($;$$$) {
# Given a Unix numeric time (seconds since 1970-01-01T00:00Z),
# provide date-time timestamp (UTC) as specified in ISO 8601 (EN 28601)
#
-sub iso8601_utc_timestamp($;$$$) {
- my($t,$suppress_zone,$dtseparator,$with_field_separators) = @_;
+sub iso8601_utc_timestamp($;$$$$) {
+ my($t, $suppress_zone, $dtseparator,
+ $with_field_separators, $with_fraction) = @_;
my $fmt = $with_field_separators ? "%Y-%m-%dT%H:%M:%S" : "%Y%m%dT%H%M%S";
$fmt =~ s/T/$dtseparator/ if defined $dtseparator;
- my $s = strftime($fmt,gmtime(int($t)));
+ my $s = strftime($fmt, gmtime(int($t)));
+ $s .= sprintf(".%03d", int(1000*($t-int($t))+0.5)) if $with_fraction;
$s .= 'Z' unless $suppress_zone;
$s;
}
@@ -4594,16 +4725,6 @@ sub iso8601_weekday($) { # 1..7, Mo=1
my $unix_time = $_[0]; ((localtime($unix_time))[6] + 6) % 7 + 1;
}
-sub format_time_interval($) {
- my $t = $_[0];
- return 'undefined' if !defined $t;
- my $sign = ''; if ($t < 0) { $sign = '-'; $t = - $t };
- my $dd = int($t / (24*3600)); $t = $t - $dd*(24*3600);
- my $hh = int($t / 3600); $t = $t - $hh*3600;
- my $mm = int($t / 60); $t = $t - $mm*60;
- sprintf("%s%d %d:%02d:%02d", $sign,$dd,$hh,$mm,int($t+0.5));
-}
-
sub make_received_header_field($$) {
my($msginfo, $folded) = @_;
my $conn = $msginfo->conn_obj;
@@ -4807,10 +4928,10 @@ sub split_localpart($$) {
my $extension; local($1,$2);
if ($localpart =~ /^(postmaster|mailer-daemon|double-bounce)\z/i) {
# do not split these, regardless of what the delimiter is
- } elsif ($delimiter eq '-' && $owner_request_special &&
+ } elsif (index($delimiter,'-') >= 0 && $owner_request_special &&
$localpart =~ /^owner-.|.-request\z/si) {
# don't split owner-foo or foo-request
- } elsif ($localpart =~ /^(.+?)(\Q$delimiter\E.*)\z/s) {
+ } elsif ($localpart =~ /^(.+?)([\Q$delimiter\E].*)\z/s) {
($localpart, $extension) = ($1, $2); # extension includes a delimiter
# do not split the address if the result would have a null localpart
}
@@ -5398,7 +5519,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
import Amavis::Util qw(ll do_log fmt_struct);
}
@@ -5521,7 +5642,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION $have_patricia);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&lookup_ip_acl &ip_to_vec &normalize_ip_addr);
import Amavis::Util qw(ll do_log);
@@ -5578,7 +5699,8 @@ BEGIN {
# - an IP address represented as a 128-bit vector (a string)
# - network mask derived from prefix length, a 128-bit vector (string)
# - prefix length as an integer (0..128)
-# - interface scope (for link-local addresses), undef if non-scoped
+# - zone_id, e.g. an interface scope for link-local addresses,
+# undef if not specified (implies a default zone_id 0, RFC 4007 sect. 11)
#
sub ip_to_vec($;$) {
my($ip,$allow_mask) = @_;
@@ -5592,7 +5714,7 @@ sub ip_to_vec($;$) {
if ($ipa =~ s/^IPv6://i) { $have_ipv6 = 1 }
elsif ($ipa =~ /:[0-9a-f]*:/i) { $have_ipv6 = 1 }
- # RFC 4007: IPv6 Scoped Address Architecture
+ # RFC 4007: IPv6 Scoped Address Architecture, sect 11: textual representation
# RFC 6874 A <zone_id> SHOULD contain only ASCII characters
# classified as "unreserved" for use in URIs [RFC 3986]
# RFC 3986: unreserved = ALPHA / DIGIT / "-" / "." / "_" / "~"
@@ -5604,7 +5726,7 @@ sub ip_to_vec($;$) {
my(@d) = ($2,$3,$4,$5);
!grep($_ > 255, @d)
or die "Invalid decimal field value in IPv6 address: [$ip]\n";
- $ipa = $2 . sprintf('%02x%02x:%02x%02x', @d);
+ $ipa = $1 . sprintf('%02x%02x:%02x%02x', @d);
} elsif (!$have_ipv6 &&
$ipa =~ m{^ \d{1,3} (?: \. \d{1,3}){0,3} \z}xs) { # IPv4
my(@d) = split(/\./,$ipa,-1);
@@ -5642,7 +5764,7 @@ sub ip_to_vec($;$) {
}
@ip_fields >= 8 or die "IPv6 address [$ip] contains fewer than 8 fields\n";
@ip_fields <= 8 or die "IPv6 address [$ip] contains more than 8 fields\n";
- !grep(!/^[0-9a-zA-Z]{1,4}\z/, @ip_fields) # this is quite slow
+ !grep(!/^[0-9a-fA-F]{1,4}\z/, @ip_fields) # this is quite slow
or die "Invalid syntax of IPv6 address: [$ip]\n";
my $vec = pack('n8', map(hex($_), at ip_fields));
if (!defined($ip_len)) {
@@ -5687,7 +5809,7 @@ sub normalize_ip_addr($) {
elsif ($ip =~ /:[0-9a-f]*:/i) { $have_ipv6 = 1 }
if ($have_ipv6) {
local($1);
- $scope = $1 if $ip =~ s/ ( % [A-Z0-9._~-]* ) \z//xsi; # scoped address
+ $scope = $1 if $ip =~ s/ % ( [A-Z0-9._~-]* ) \z//xsi; # scoped address
if ($ip !~ /^[0:]+:ffff:/i) { # triage for IPv4-mapped
$ip = lc $ip; # lowercase: RFC 5952
} else { # looks like an IPv4-mapped address
@@ -5705,14 +5827,14 @@ sub normalize_ip_addr($) {
}
}
}
- $ip .= $scope if defined $scope;
+ $ip .= '%'.$scope if $scope; # defined, nonempty and nonzero
$ip;
}
# lookup_ip_acl() performs a lookup for an IPv4 or IPv6 address against a list
# of lookup tables, each may be a constant, or a ref to an access control
# list or a ref to an associative array (hash) of network or host addresses.
-# Interface scope (for link-local addresses) is ignored.
+# Interface zone_id (e.g. scope for link-local addresses) is ignored.
#
# IP address is compared to each member of an access list in turn,
# the first match wins (terminates the search), and its value decides
@@ -5720,7 +5842,7 @@ sub normalize_ip_addr($) {
# Falling through without a match produces a false (undef).
#
# The presence of a character '!' prepended to a list member decides
-# whether the result will be true (without a '!') or false (with '!')
+# whether the result will be true (without a '!') or false (with a '!')
# in case this list member matches and terminates the search.
#
# Because search stops at the first match, it only makes sense
@@ -5728,8 +5850,8 @@ sub normalize_ip_addr($) {
#
# For IPv4 a network address can be specified in classless notation
# n.n.n.n/k, or using a mask n.n.n.n/m.m.m.m . Missing mask implies /32,
-# i.e. a host address. For IPv6 addresses all RFC 4291 forms are allowed.
-# See also comments at ip_to_vec().
+# i.e. a host address. For IPv6 addresses all RFC 4291 forms are allowed
+# and the /k specifies a prefix length. See also comments at ip_to_vec().
#
# Although not a special case, it is good to remember that '::/0'
# always matches any IPv4 or IPv6 address (even syntactically invalid address).
@@ -5749,7 +5871,7 @@ sub normalize_ip_addr($) {
# and return true.
#
# If the supplied lookup table is a hash reference, match a canonical
-# IP address: dot-quad IPv4, or preferred IPv6 form, against hash keys.
+# IP address: dot-quad IPv4, or a preferred IPv6 form, against hash keys.
# For IPv4 addresses a simple classful subnet specification is allowed in
# hash keys by truncating trailing bytes from the looked up IPv4 address.
# A syntactically invalid IP address cannot match any hash entry.
@@ -5863,7 +5985,7 @@ sub lookup_ip_acl($@) {
# Create a pre-parsed object from a list of IP networks, which
# may be used as an argument to lookup_ip_acl to speed up its searches.
-# Interface scope (for link-local addresses) is ignored.
+# Interface zone_id (e.g. scope for link-local addresses) is ignored.
#
sub new($@) {
my($class, at nets) = @_;
@@ -6007,7 +6129,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&lookup &lookup2 &lookup_hash &lookup_acl);
import Amavis::Util qw(ll do_log fmt_struct unique_list);
@@ -6295,7 +6417,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&expand &tokenize);
import Amavis::Util qw(ll do_log);
@@ -6597,7 +6719,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform :confvars c cr ca);
import Amavis::Timing qw(section_time);
@@ -7132,7 +7254,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
}
use Errno qw(EIO);
@@ -7268,7 +7390,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform);
import Amavis::Util qw(ll do_log min max minmax);
@@ -7748,7 +7870,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
}
@@ -7782,7 +7904,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform);
import Amavis::Util qw(setting_by_given_contents_category_all
@@ -7790,7 +7912,7 @@ BEGIN {
}
sub new # NOTE: this class is a list, not a hash
- { my $class = $_[0]; bless [(undef) x 41], $class }
+ { my $class = $_[0]; bless [(undef) x 42], $class }
# subs to set or access individual elements of a n-tuple by name
sub recip_addr # unquoted recipient envelope e-mail address
@@ -7805,77 +7927,79 @@ sub recip_maddr_id # maddr.id field from SQL corresponding to recip_addr_smtp
{ @_<2 ? shift->[4] : ($_[0]->[4] = $_[1]) }
sub recip_maddr_id_orig # maddr.id field from SQL corresponding to dsn_orcpt
{ @_<2 ? shift->[5] : ($_[0]->[5] = $_[1]) }
-sub recip_penpals_age # penpals age in seconds if logging to SQL is enabled
+sub recip_penpals_related # mail_id of a previous correspondence
{ @_<2 ? shift->[6] : ($_[0]->[6] = $_[1]) }
-sub recip_penpals_score # penpals score (info, also added to spam_level)
+sub recip_penpals_age # penpals age in seconds if SQL or Redis is enabled
{ @_<2 ? shift->[7] : ($_[0]->[7] = $_[1]) }
-sub dsn_notify # ESMTP RCPT command NOTIFY option (DSN-RFC 3461, listref)
+sub recip_penpals_score # penpals score (info, also added to spam_level)
{ @_<2 ? shift->[8] : ($_[0]->[8] = $_[1]) }
-sub dsn_orcpt # ESMTP RCPT command ORCPT option (DSN-RFC 3461, encoded)
+sub dsn_notify # ESMTP RCPT command NOTIFY option (DSN-RFC 3461, listref)
{ @_<2 ? shift->[9] : ($_[0]->[9] = $_[1]) }
-sub dsn_suppress_reason # if defined disable sending DSN and supply a reason
+sub dsn_orcpt # ESMTP RCPT command ORCPT option (DSN-RFC 3461, encoded)
{ @_<2 ? shift->[10] : ($_[0]->[10] = $_[1]) }
-sub recip_destiny # D_REJECT, D_BOUNCE, D_DISCARD, D_PASS
+sub dsn_suppress_reason # if defined disable sending DSN and supply a reason
{ @_<2 ? shift->[11] : ($_[0]->[11] = $_[1]) }
-sub recip_done # false: not done, true: done (1: faked, 2: truly sent)
+sub recip_destiny # D_REJECT, D_BOUNCE, D_DISCARD, D_PASS
{ @_<2 ? shift->[12] : ($_[0]->[12] = $_[1]) }
-sub recip_smtp_response # RFC 5321 response (3-digit + enhanced resp + text)
+sub recip_done # false: not done, true: done (1: faked, 2: truly sent)
{ @_<2 ? shift->[13] : ($_[0]->[13] = $_[1]) }
-sub recip_remote_mta_smtp_response # smtp response as issued by remote MTA
+sub recip_smtp_response # RFC 5321 response (3-digit + enhanced resp + text)
{ @_<2 ? shift->[14] : ($_[0]->[14] = $_[1]) }
-sub recip_remote_mta # remote MTA that issued the smtp response
+sub recip_remote_mta_smtp_response # smtp response as issued by remote MTA
{ @_<2 ? shift->[15] : ($_[0]->[15] = $_[1]) }
-sub recip_tagged # message was tagged by address extension or Subject or X-Spam
+sub recip_remote_mta # remote MTA that issued the smtp response
{ @_<2 ? shift->[16] : ($_[0]->[16] = $_[1]) }
-sub recip_mbxname # mailbox name or file when known (local:, bsmtp: or sql:)
+sub recip_tagged # message was tagged by address extension or Subject or X-Spam
{ @_<2 ? shift->[17] : ($_[0]->[17] = $_[1]) }
-sub recip_whitelisted_sender # recip considers this sender whitelisted
+sub recip_mbxname # mailbox name or file when known (local:, bsmtp: or sql:)
{ @_<2 ? shift->[18] : ($_[0]->[18] = $_[1]) }
-sub recip_blacklisted_sender # recip considers this sender blacklisted
+sub recip_whitelisted_sender # recip considers this sender whitelisted
{ @_<2 ? shift->[19] : ($_[0]->[19] = $_[1]) }
-sub bypass_virus_checks # boolean: virus checks to be bypassed for this recip
+sub recip_blacklisted_sender # recip considers this sender blacklisted
{ @_<2 ? shift->[20] : ($_[0]->[20] = $_[1]) }
-sub bypass_banned_checks # bool: ban checks are to be bypassed for this recip
+sub bypass_virus_checks # boolean: virus checks to be bypassed for this recip
{ @_<2 ? shift->[21] : ($_[0]->[21] = $_[1]) }
-sub bypass_spam_checks # boolean: spam checks are to be bypassed for this recip
+sub bypass_banned_checks # bool: ban checks are to be bypassed for this recip
{ @_<2 ? shift->[22] : ($_[0]->[22] = $_[1]) }
-sub banned_parts # banned part descriptions (ref to a list of banned parts)
+sub bypass_spam_checks # boolean: spam checks are to be bypassed for this recip
{ @_<2 ? shift->[23] : ($_[0]->[23] = $_[1]) }
-sub banned_parts_as_attr # banned part descriptions - newer syntax (listref)
+sub banned_parts # banned part descriptions (ref to a list of banned parts)
{ @_<2 ? shift->[24] : ($_[0]->[24] = $_[1]) }
-sub banning_rule_key # matching banned rules (lookup table keys) (ref to list)
+sub banned_parts_as_attr # banned part descriptions - newer syntax (listref)
{ @_<2 ? shift->[25] : ($_[0]->[25] = $_[1]) }
-sub banning_rule_comment #comments (or whole expr) from banning_rule_key regexp
+sub banning_rule_key # matching banned rules (lookup table keys) (ref to list)
{ @_<2 ? shift->[26] : ($_[0]->[26] = $_[1]) }
-sub banning_reason_short # just one banned part leaf name with a rule comment
+sub banning_rule_comment #comments (or whole expr) from banning_rule_key regexp
{ @_<2 ? shift->[27] : ($_[0]->[27] = $_[1]) }
-sub banning_rule_rhs # a right-hand side of matching rules (a ref to a list)
+sub banning_reason_short # just one banned part leaf name with a rule comment
{ @_<2 ? shift->[28] : ($_[0]->[28] = $_[1]) }
-sub mail_body_mangle # mail body is being modified (and how) (e.g. defanged)
+sub banning_rule_rhs # a right-hand side of matching rules (a ref to a list)
{ @_<2 ? shift->[29] : ($_[0]->[29] = $_[1]) }
-sub contents_category # sorted listref of "major,minor" strings(category types)
+sub mail_body_mangle # mail body is being modified (and how) (e.g. defanged)
{ @_<2 ? shift->[30] : ($_[0]->[30] = $_[1]) }
-sub blocking_ccat # category type most responsible for blocking msg, or undef
+sub contents_category # sorted listref of "major,minor" strings(category types)
{ @_<2 ? shift->[31] : ($_[0]->[31] = $_[1]) }
-sub user_id # listref of recipient IDs from a lookup, e.g. SQL field users.id
+sub blocking_ccat # category type most responsible for blocking msg, or undef
{ @_<2 ? shift->[32] : ($_[0]->[32] = $_[1]) }
-sub user_policy_id # recipient's policy ID, e.g. SQL field users.policy_id
+sub user_id # listref of recipient IDs from a lookup, e.g. SQL field users.id
{ @_<2 ? shift->[33] : ($_[0]->[33] = $_[1]) }
-sub courier_control_file # path to control file containing this recipient
+sub user_policy_id # recipient's policy ID, e.g. SQL field users.policy_id
{ @_<2 ? shift->[34] : ($_[0]->[34] = $_[1]) }
-sub courier_recip_index # index of recipient within control file
+sub courier_control_file # path to control file containing this recipient
{ @_<2 ? shift->[35] : ($_[0]->[35] = $_[1]) }
-sub delivery_method # delivery method, or empty for implicit delivery (milter)
+sub courier_recip_index # index of recipient within control file
{ @_<2 ? shift->[36] : ($_[0]->[36] = $_[1]) }
-sub spam_level # spam score as returned by spam scanners, ham near 0, spam 5
+sub delivery_method # delivery method, or empty for implicit delivery (milter)
{ @_<2 ? shift->[37] : ($_[0]->[37] = $_[1]) }
-sub spam_tests # a listref of r/o stringrefs, each: t1=score1,t2=score2,..
+sub spam_level # spam score as returned by spam scanners, ham near 0, spam 5
{ @_<2 ? shift->[38] : ($_[0]->[38] = $_[1]) }
+sub spam_tests # a listref of r/o stringrefs, each: t1=score1,t2=score2,..
+ { @_<2 ? shift->[39] : ($_[0]->[39] = $_[1]) }
# per-recipient spam info - when undefined consult a per-message counterpart
sub spam_report # SA terse report of tests hit (for header section reports)
- { @_<2 ? shift->[39] : ($_[0]->[39] = $_[1]) }
-sub spam_summary # SA summary of tests hit for standard body reports
{ @_<2 ? shift->[40] : ($_[0]->[40] = $_[1]) }
+sub spam_summary # SA summary of tests hit for standard body reports
+ { @_<2 ? shift->[41] : ($_[0]->[41] = $_[1]) }
sub recip_final_addr { # return recip_addr_modified if set, else recip_addr
my $self = shift;
@@ -7971,7 +8095,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform);
import Amavis::rfc2821_2822_Tools qw(rfc2822_timestamp quote_rfc2821_local
@@ -8014,6 +8138,8 @@ sub mail_id # long-term unique id of the message on this system
{ @_<2 ? shift->{mail_id} : ($_[0]->{mail_id} = $_[1]) }
sub secret_id # secret string to grant access to a message with mail_id
{ @_<2 ? $_[0]->{secret_id} : ($_[0]->{secret_id} = $_[1]) }
+sub parent_mail_id # original mail_id for msgs generated by amavis (DSN,notif)
+ { @_<2 ? shift->{parent_mail_id} : ($_[0]->{parent_mail_id} = $_[1]) }
sub attachment_password # scrambles a potentially dangerous released mail
{ @_<2 ? shift->{release_pwd}: ($_[0]->{release_pwd} = $_[1]) }
sub msg_size # ESMTP SIZE value, later corrected to actual size,RFC 1870
@@ -8086,6 +8212,8 @@ sub orig_header # orig.h.sect, arrayref of h.fields, with folding & trailing LF
{ @_<2 ? shift->{orig_header}: ($_[0]->{orig_header} = $_[1]) }
sub orig_header_size # size of original header, incl. a separator line,RFC 1870
{ @_<2 ? shift->{orig_hdr_s} : ($_[0]->{orig_hdr_s} = $_[1]) }
+sub references # References & In-Reply-To message IDs, array
+ { @_<2 ? shift->{refs} : ($_[0]->{refs} = $_[1]) }
sub orig_body_size # size of original body (in bytes), RFC 1870
{ @_<2 ? shift->{orig_bdy_s} : ($_[0]->{orig_bdy_s} = $_[1]) }
sub body_start_pos # byte offset into a msg where mail body starts (if known)
@@ -8094,6 +8222,8 @@ sub body_digest # digest of a message body (e.g. MD5, SHA1, SHA256), hex
{ @_<2 ? shift->{body_digest}: ($_[0]->{body_digest} = $_[1]) }
sub ip_addr_trace # IP addresses in 'Received from' hdr flds, top-down, array
{ @_<2 ? shift->{iptrace} : ($_[0]->{iptrace} = $_[1]) }
+sub ip_addr_trace_public # public IP addresses in 'Received from' hdr flds
+ { @_<2 ? shift->{iptracepub} : ($_[0]->{iptracepub} = $_[1]) }
sub is_mlist # mail is from a mailing list (boolean/string)
{ @_<2 ? shift->{is_mlist} : ($_[0]->{is_mlist} = $_[1]) }
sub is_auto # mail is an auto-response (boolean/string)
@@ -8135,9 +8265,13 @@ sub actions_performed # listref, summarized actions & SMTP status, for logging
sub virusnames # a ref to a list of virus names detected, or undef
{ @_<2 ? shift->{virusnames} : ($_[0]->{virusnames} = $_[1]) }
sub spam_report # SA terse report of tests hit (for header section reports)
- { @_<2 ? shift->{spam_report} :($_[0]->{spam_report} = $_[1])}
+ { @_<2 ? shift->{spam_report} : ($_[0]->{spam_report} = $_[1])}
sub spam_summary # SA summary of tests hit for standard body reports
- { @_<2 ? shift->{spam_summary}:($_[0]->{spam_summary} = $_[1])}
+ { @_<2 ? shift->{spam_summary} :($_[0]->{spam_summary} = $_[1])}
+sub ip_repu_score # IP reputation score (info, also added to spam_level)
+ { @_<2 ? shift->{ip_repu_score} :($_[0]->{ip_repu_score} = $_[1])}
+sub time_elapsed # elapsed times by section - associative array ref
+ { @_<2 ? shift->{elapsed} : ($_[0]->{elapsed} = $_[1])}
# new style of providing additional information from checkers
sub supplementary_info { # holds a hash of tag/value pairs, such as SA get_tag
@@ -8219,9 +8353,11 @@ sub header_field_signed_by {
#
sub get_header_field2 {
my($self, $field_name, $j) = @_;
+ my $orig_hfields = $self->orig_header_fields;
+ return if !$orig_hfields;
my($field_ind, $field, $all_fields, $hfield_indices);
- $hfield_indices = # arrayref of h.field indices for a given h.field name
- $self->orig_header_fields->{lc $field_name} if defined $field_name;
+ # arrayref of header field indices for a given h.field name
+ $hfield_indices = $orig_hfields->{lc $field_name} if defined $field_name;
$all_fields = $self->orig_header;
if (defined $field_name) {
if (!defined $hfield_indices) {
@@ -8279,7 +8415,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&hdr);
import Amavis::Conf qw(:platform c cr ca);
@@ -8592,7 +8728,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
@EXPORT = qw(&mail_dispatch);
import Amavis::Conf qw(:platform :confvars c cr ca);
@@ -8650,7 +8786,7 @@ sub mail_dispatch($$$;$) {
# each entry will be using the same protocol name, otherwise behaviour
# is unspecified - so just obtain the protocol name from the first entry
#
- my(%protocols,$any_tempfail);
+ my(%protocols, $any_tempfail);
for my $r (@$per_recip_data) {
if (!$dsn_per_recip_capable) {
my $recip_smtp_response = $r->recip_smtp_response; # any 4xx code ?
@@ -8746,7 +8882,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&first_received_from &oldest_public_ip_addr_from_received);
import Amavis::Conf qw(:platform c cr ca);
@@ -8754,7 +8890,7 @@ BEGIN {
import Amavis::rfc2821_2822_Tools qw(
split_address parse_received fish_out_ip_from_received);
import Amavis::Lookup qw(lookup lookup2);
- import Amavis::Lookup::IP qw(lookup_ip_acl normalize_ip_addr);
+ import Amavis::Lookup::IP qw(normalize_ip_addr);
}
use subs @EXPORT_OK;
@@ -8781,17 +8917,10 @@ sub first_received_from($) {
sub oldest_public_ip_addr_from_received($) {
my($msginfo) = @_;
my $received_from_ip;
- my $ip_trace_ref = $msginfo->ip_addr_trace; # top-down trace
- if ($ip_trace_ref) {
- for my $ip (reverse @$ip_trace_ref) { # bottom-up
- if (defined $ip && $ip ne '') {
- my($is_public,$fullkey,$err) =
- lookup_ip_acl($ip, @Amavis::public_networks_maps);
- if ($is_public && !$err) { $received_from_ip = $ip; last }
- }
- }
- }
- do_log(5, "oldest_public_ip_addr_from_received: %s", $received_from_ip);
+ my $ip_trace_ref = $msginfo->ip_addr_trace_public; # top-down trace
+ $received_from_ip = $ip_trace_ref->[-1] if $ip_trace_ref; # last is oldest
+ do_log(5, "oldest_public_ip_addr_from_received: %s", $received_from_ip)
+ if defined $received_from_ip;
$received_from_ip;
}
@@ -8805,7 +8934,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&consumed_bytes);
import Amavis::Conf qw(c cr ca
@@ -8894,7 +9023,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
import Amavis::Util qw(ll do_log);
}
@@ -8946,12 +9075,12 @@ sub size
{ @_<2 ? shift->{size} : ($_[0]->{size} = $_[1]) };
sub exists
{ @_<2 ? shift->{exists} : ($_[0]->{exists} = $_[1]) };
-sub attributes # listref of characters representing attributes
+sub attributes # a string of characters representing attributes
{ @_<2 ? shift->{attr} : ($_[0]->{attr} = $_[1]) };
sub attributes_add { # U=undecodable, C=crypted, D=directory,S=special,L=link
- my $self = shift; my $a = $self->{attr} || [];
- for my $arg (@_) { push(@$a,$arg) if $arg ne '' && !grep($_ eq $arg, @$a) }
+ my $self = shift; my $a = $self->{attr}; $a = '' if !defined $a;
+ for my $arg (@_) { $a .= $arg if $arg ne '' && index($a,$arg) < 0 }
$self->{attr} = $a;
};
@@ -8982,7 +9111,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter MIME::Parser::Filer); # subclass of MIME::Parser::Filer
}
# This package will be used by mime_decode().
@@ -9024,7 +9153,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&check_header_validity &check_for_banned_names);
import Amavis::Util qw(ll do_log min max minmax untaint untaint_inplace
@@ -9204,9 +9333,7 @@ sub check_for_banned_names($) {
for (@$n) {if (defined($_) && $_ ne '')
{my $m=$_; $m=~s/[\t\n]/ /g; push(@k,"N=$m")} }
$n = $p->attributes;
- $n = [$n] if !ref($n);
- for (@$n) {if (defined($_) && $_ ne '')
- {my $m=$_; $m=~s/[\t\n]/ /g; push(@k,"A=$m")} }
+ if (defined $n && $n ne '') { push(@k,"A=$_") for split(/ */,$n) }
push(@descr, join("\t", at k));
push(@descr_trad, [map { local($1,$2);
/^([a-zA-Z0-9])=(.*)\z/s; my($key_what,$key_val) = ($1,$2);
@@ -9329,12 +9456,13 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&mime_decode);
import Amavis::Conf qw(:platform c cr ca $TEMPBASE $MAXFILES);
import Amavis::Timing qw(section_time);
- import Amavis::Util qw(snmp_count untaint ll do_log safe_decode
+ import Amavis::Util qw(snmp_count untaint ll do_log
+ safe_decode safe_decode_latin1
safe_encode safe_encode_ascii safe_encode_utf8);
import Amavis::Unpackers::NewFilename qw(consumed_bytes);
}
@@ -9432,8 +9560,8 @@ sub mime_traverse($$$$$) {
if (defined $part) {
$part->mime_placement($placement);
$part->type_declared($mt eq $et ? $mt : [$mt, $et]);
- $part->attributes_add('U','C') if $mt =~ m{/encrypted}i ||
- $et =~ m{/encrypted}i;
+ $part->attributes_add('U','C') if $mt =~ m{/.*encrypted}si ||
+ $et =~ m{/.*encrypted}si;
my %rn_seen;
my @rn; # recommended file names, both raw and RFC 2047 / RFC 2231 decoded
for my $attr_name ('content-disposition.filename', 'content-type.name') {
@@ -9444,9 +9572,12 @@ sub mime_traverse($$$$$) {
my(@chunks) = MIME::Words::decode_mimewords($val_raw);
for my $pair (@chunks) {
my($data,$encoding) = @$pair;
- $encoding = 'ISO-8859-1' if !defined $encoding || $encoding eq '';
- $encoding =~ s/\*[^*]*\z//; # strip RFC 2231 language suffix
- $val_dec .= safe_decode($encoding,$data);
+ if (!defined $encoding || $encoding eq '') {
+ $val_dec .= safe_decode_latin1($data);
+ } else {
+ $encoding =~ s/\*[^*]*\z//s; # strip RFC 2231 language suffix
+ $val_dec .= safe_decode($encoding,$data);
+ }
}
1;
} or do {
@@ -9554,7 +9685,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter MIME::Body); # subclass of MIME::Body
import Amavis::Util qw(ll do_log);
}
@@ -9619,7 +9750,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&delivery_status_notification &delivery_short_report
&build_mime_entity &defanged_mime_entity
@@ -9627,7 +9758,8 @@ BEGIN {
import Amavis::Util qw(ll do_log sanitize_str min max minmax
safe_encode safe_encode_ascii safe_encode_utf8
untaint untaint_inplace make_password
- orcpt_decode xtext_decode ccat_split ccat_maj);
+ orcpt_decode xtext_decode ccat_split ccat_maj
+ generate_mail_id);
import Amavis::Timing qw(section_time);
import Amavis::Conf qw(:platform :confvars c cr ca);
import Amavis::ProcControl qw(exit_status_str proc_status_ok
@@ -9731,7 +9863,7 @@ sub wrap_message_into_archive($$) {
# an overweight cannon unnecessary here: the password is used as a
# scrambler only, protecting against accidental opening of a file,
# so there is no security issue here
- $password = 'X' x length($password); # can't hurt to hide it
+ $password = 'X' x length($password); # can't hurt to wipe out
my($proc_fh,$pid) = run_command(undef,undef, at command);
my($r,$status) = collect_results($proc_fh,$pid,'zip',16384,[0]);
undef $proc_fh; undef $pid;
@@ -9897,7 +10029,7 @@ sub build_mime_entity($$$$$$$) {
if ($msg_format eq 'attach' && # not 'arf' and not 'dsn'
defined $password && $password ne '') {
# attach as a ZIP archive
- $password = 'X' x length($password); # can't hurt to hide it
+ $password = 'X' x length($password); # can't hurt to wipe out
do_log(4, "build_mime_entity: attaching entire original message as zip");
my $archive_fn = wrap_message_into_archive($msginfo,\@prefix_lines);
local($1); $archive_fn =~ m{([^/]*)\z}; my $att_filename = $1;
@@ -10377,11 +10509,14 @@ sub delivery_status_notification($$$;$$$$) { # ..._or_report
$notification->rx_time($dsn_time);
$notification->log_id($msginfo->log_id);
$notification->partition_tag($msginfo->partition_tag);
+ $notification->parent_mail_id($msginfo->mail_id);
+ $notification->mail_id(scalar generate_mail_id());
$notification->conn_obj($msginfo->conn_obj);
$notification->originating(
($request_type eq 'dsn' || $request_type eq 'report') ? 1 : 0);
# $notification->body_type('7BIT');
$notification->mail_text($report_entity);
+ $notification->add_contents_category(CC_CLEAN,0);
$notification->sender($mailfrom);
$notification->sender_smtp(qquote_rfc2821_local($mailfrom));
$notification->auth_submitter('<>');
@@ -10396,7 +10531,7 @@ sub delivery_status_notification($$$;$$$$) { # ..._or_report
if ($request_type eq 'dsn' || $request_type eq 'report') {
$bcc = $msginfo->setting_by_contents_category(cr('dsn_bcc_by_ccat'));
}
- $notification->recips([(defined $notif_recips ? @$notif_recips
+ $notification->recips([($notif_recips ? @$notif_recips
: map($_->recip_addr, @$per_recip_data)),
defined $bcc && $bcc ne '' ? $bcc : () ], 1);
my $notif_m = c('notify_method');
@@ -10406,7 +10541,7 @@ sub delivery_status_notification($$$;$$$$) { # ..._or_report
# that DSN is not to be sent, or it is believed the bounce would not reach
# the correct sender (faked sender with viruses or spam);
# $notification is undef if DSN is not needed
- ($notification,$suppressed);
+ ($notification, $suppressed);
}
# Return a triple of arrayrefs of quoted recipient addresses (the first lists
@@ -10550,7 +10685,7 @@ sub msg_from_quarantine($$$) {
my $fh = $msginfo->mail_text;
my $sender_override = $msginfo->sender;
my $recips_data_override = $msginfo->per_recip_data;
- my $quarantine_id = $msginfo->mail_id;
+ my $quarantine_id = $msginfo->parent_mail_id;
$quarantine_id = '' if !defined $quarantine_id;
my $reporting = $request_type eq 'report';
my $release_m;
@@ -10562,7 +10697,8 @@ sub msg_from_quarantine($$$) {
$release_m = c('notify_method') if !defined $release_m || $release_m eq '';
$release_m ne '' or die "release_method and notify_method are unspecified";
}
- $msginfo->originating(0); # let's make it explicit; disables DKIM signing
+ $msginfo->originating(1); # (also enables DKIM signing)
+ $msginfo->add_contents_category(CC_CLEAN,0);
$msginfo->auth_submitter('<>');
$msginfo->auth_user(c('amavis_auth_user'));
$msginfo->auth_pass(c('amavis_auth_pass'));
@@ -10724,7 +10860,9 @@ sub msg_from_quarantine($$$) {
$hdr_edits->add_header('Resent-Date', # time of the release
rfc2822_timestamp($msginfo->rx_time));
$hdr_edits->add_header('Resent-Message-ID',
- sprintf('<QRR%s@%s>', $msginfo->mail_id||'', c('myhostname')) );
+ sprintf('<%s-%s@%s>',
+ $msginfo->parent_mail_id||'', $msginfo->mail_id||'',
+ c('myhostname')) );
}
$hdr_edits->add_header('Received', make_received_header_field($msginfo,1),1);
my $bcc = $msginfo->setting_by_contents_category(cr('always_bcc_by_ccat'));
@@ -10782,7 +10920,7 @@ use re 'taint';
BEGIN {
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
import Amavis::Conf qw(:platform :sa :confvars c cr ca);
import Amavis::Util qw(untaint untaint_inplace
min max minmax unique_list unique_ref
@@ -10790,14 +10928,16 @@ BEGIN {
dump_captured_log log_capture_enabled am_id
sanitize_str sanitize_str_inplace debug_oneshot
safe_encode safe_encode_ascii safe_encode_utf8
- safe_decode proto_decode
- add_entropy stir_random generate_mail_id make_password
+ safe_decode safe_decode_latin1 proto_decode
+ format_time_interval add_entropy stir_random
+ generate_mail_id make_password
prolong_timer get_deadline waiting_for_client
switch_to_my_time switch_to_client_time
snmp_counters_init snmp_count dynamic_destination
ccat_split ccat_maj cmp_ccat cmp_ccat_maj
setting_by_given_contents_category_all
- setting_by_given_contents_category orcpt_encode);
+ setting_by_given_contents_category
+ orcpt_encode orcpt_decode);
import Amavis::ProcControl qw(exit_status_str proc_status_ok
cloexec run_command collect_results);
import Amavis::Log qw(open_log close_log collect_log_stats);
@@ -10857,7 +10997,7 @@ use vars qw($child_init_hook_was_called);
# but with SMTP or LMTP input there may be more than one
# message passed during a single SMTP session
use vars qw(@config_files); # configuration files provided by -c or defaulted
-use vars qw($MSGINFO);
+use vars qw($MSGINFO $report_ref);
use vars qw($av_output @virusname @detecting_scanners @av_scanners_results
$banned_filename_any $banned_filename_all @bad_headers);
@@ -10870,7 +11010,7 @@ use vars qw($sql_storage); # Amavis::Out::SQL::Log object
use vars qw($sql_lookups $sql_wblist); # Amavis::Lookup::SQL objects
use vars qw($ldap_connection); # Amavis::LDAP::Connection object
use vars qw($ldap_lookups); # Amavis::Lookup::LDAP object
-use vars qw($redis_storage); # Amavis::Redis object
+use vars qw($redis_storage); # Amavis::Redis object: penpals & repu
use vars qw($dns_resolver); # a reusable Net::DNS::Resolver object
use vars qw($warm_restart); # 1: warm (reload), 0: cold start (restart)
use vars qw(@public_networks_maps);
@@ -10902,13 +11042,13 @@ sub macro_tests {
$r = $per_recip_data->[$recip_index] if $recip_index >= 0;
if (defined $r) {
my $spam_tests = $r->spam_tests;
- @s = split(/,/, join(',',map($$_,@$spam_tests))) if defined $spam_tests;
+ @s = split(/,/, join(',',map($$_,@$spam_tests))) if $spam_tests;
}
} else {
my(%all_spam_tests);
for my $r (@$per_recip_data) {
my $spam_tests = $r->spam_tests;
- if (defined $spam_tests) {
+ if ($spam_tests) {
$all_spam_tests{$_} = 1 for split(/,/,join(',',map($$_,@$spam_tests)));
}
}
@@ -11109,10 +11249,10 @@ sub init_builtin_macros() {
ccat =>
sub { # somewhat expensive! #**
my($name,$attr,$which) = @_;
- $attr = lc($attr); # name | major | minor | <empty>
- # | is_blocking | is_nonblocking
- # | is_blocked_by_nonmain
- $which = lc($which); # main | blocking | auto
+ $attr = lc $attr; # name | major | minor | <empty>
+ # | is_blocking | is_nonblocking
+ # | is_blocked_by_nonmain
+ $which = lc $which; # main | blocking | auto
my $result = ''; my $blocking_ccat = $MSGINFO->blocking_ccat;
if ($attr eq 'is_blocking') {
$result = defined($blocking_ccat) ? 1 : '';
@@ -11138,8 +11278,8 @@ sub init_builtin_macros() {
$which ne 'main' && defined $blocking_ccat)
? $blocking_ccat : $MSGINFO->contents_category);
$result = $attr eq 'major' ? $maj
- : $attr eq 'minor' ? sprintf("%d",$min)
- : sprintf("(%d,%d)",$maj,$min);
+ : $attr eq 'minor' ? sprintf('%d',$min)
+ : sprintf('(%d,%d)',$maj,$min);
}
$result;
},
@@ -11179,8 +11319,9 @@ sub init_builtin_macros() {
},
n => sub {$MSGINFO->log_id}, # amavis internal task id (in log and nanny)
i => sub {$MSGINFO->mail_id}, # long-term unique mail id on this system
- mail_id => sub {$MSGINFO->mail_id}, # synonym for %i, base64url (RFC 4648)
secret_id => sub {$MSGINFO->secret_id}, # mail_id's counterpart, base64url
+ mail_id => sub {$MSGINFO->mail_id}, # synonym for %i, base64url (RFC 4648)
+ parent_mail_id => sub {$MSGINFO->parent_mail_id},
log_id => sub {$MSGINFO->log_id}, # synonym for %n
MAILID => sub {$MSGINFO->mail_id}, # synonym for %i (no equivalent in SA)
LOGID => sub {$MSGINFO->log_id}, # synonym for %n (no equivalent in SA)
@@ -11263,12 +11404,9 @@ sub init_builtin_macros() {
[ map(defined $_ ? sanitize_str($_) : 'x', @$ip_trace) ];
},
ip_trace_public => sub { # all public IP addresses in the Received trace
- my $ip_trace = $MSGINFO->ip_addr_trace;
+ my $ip_trace = $MSGINFO->ip_addr_trace_public;
return if !$ip_trace;
- [ map(do { my($public,$key,$err) =
- lookup_ip_acl($_, @Amavis::public_networks_maps);
- $public && !$err ? sanitize_str($_) : () },
- grep(defined $_, @$ip_trace)) ];
+ [ map(defined $_ ? sanitize_str($_) : 'x', @$ip_trace) ];
},
t => sub { # first (oldest) entry in the Received trace
sanitize_str(first_received_from($MSGINFO)) },
@@ -11293,7 +11431,7 @@ sub init_builtin_macros() {
remote_mta_smtp_response =>
sub { unique_ref(map($_->recip_remote_mta_smtp_response,
@{$MSGINFO->per_recip_data})) },
- REMOTEHOSTADDR => # where the request was sent from
+ REMOTEHOSTADDR => # where the request came from
sub { my $c = $MSGINFO->conn_obj; !$c ? '' : $c->client_ip },
REMOTEHOSTNAME =>
sub { my $c = $MSGINFO->conn_obj;
@@ -11368,6 +11506,8 @@ sub init_builtin_macros() {
join('', map { my $s=$_; $s=~s{"}{""}g; '"'.$s.'"' } @_)},
uquote => sub {my $nm=shift;
join('', map { my $s=$_; $s=~s{[ \t]+}{_}g; $s } @_)},
+ rot13 => sub {my($name,$s) = @_; # obfuscation (Caesar cipher)
+ $s=~tr/a-zA-Z/n-za-mN-ZA-M/; $s },
hexenc => sub {my $nm=shift; join('', map(unpack('H*',$_), @_))},
b64encode => sub {my $nm=shift; join(' ', map(encode_base64($_,''), at _))},
b64enc => sub {my $nm=shift; # preferred over b64encode
@@ -11411,6 +11551,11 @@ sub init_builtin_macros() {
}
$str;
},
+ report_json => sub {
+ return if !$report_ref; # ugly globals
+ structured_report_update_time($report_ref);
+ safe_encode_utf8(Amavis::JSON::encode($report_ref));
+ },
# macros f, T, C, B will be defined for each notification as appropriate
# (representing From:, To:, Cc:, and Bcc: respectively)
# remaining free letters: wxEGIJKLMYZ
@@ -11495,9 +11640,15 @@ sub init_preparse_ip_lookups() {
for my $bank_name (keys %policy_bank) {
my $r = $policy_bank{$bank_name}{'inet_acl'};
- if (ref($r) eq 'ARRAY') # should be a ref to a single IP lookup table
- { $policy_bank{$bank_name}{'inet_acl'} = Amavis::Lookup::IP->new(@$r) }
-
+ if (ref($r) eq 'ARRAY') { # should be a ref to an IP lookup table
+ $policy_bank{$bank_name}{'inet_acl'} = Amavis::Lookup::IP->new(@$r);
+ }
+ $r = $policy_bank{$bank_name}{'ip_repu_ignore_maps'}; # listref of tables
+ if (ref($r) eq 'ARRAY') { # should be an array, test just to make sure
+ for my $table (@$r) { # replace plain lists with pre-parsed objects
+ $table = Amavis::Lookup::IP->new(@$table) if ref($table) eq 'ARRAY';
+ }
+ }
$r = $policy_bank{$bank_name}{'client_ipaddr_policy'}; # listref of pairs
if (ref($r) eq 'ARRAY') { # should be an array, test just to make sure
my $odd = 1;
@@ -11583,13 +11734,12 @@ sub after_chroot_init() {
Net::Server NetAddr::IP Net::DNS Net::SSLeay Net::Patricia Net::LDAP
Mail::ClamAV Mail::SpamAssassin Mail::DKIM::Verifier Mail::DKIM::Signer
Mail::SPF Mail::SPF::Query URI Razor2::Client::Version
- DBI DBD::mysql DBD::Pg DBD::SQLite BerkeleyDB DB_File Redis
+ DBI DBD::mysql DBD::Pg DBD::SQLite BerkeleyDB DB_File
ZMQ ZMQ::LibZMQ2 ZMQ::LibZMQ3 ZeroMQ SAVI Anomy::Sanitizer));
do_log(0, "Module %-19s %s", $m, eval{$m->VERSION} || '?');
}
do_log(0,"Amavis::ZMQ code %s loaded", $extra_code_zmq ?'':" NOT");
do_log(0,"Amavis::DB code %s loaded", $extra_code_db ?'':" NOT");
- do_log(0,"Redis code %s loaded", $extra_code_redis ?'':" NOT");
do_log(0,"SQL base code %s loaded", $extra_code_sql_base ?'':" NOT");
do_log(0,"SQL::Log code %s loaded", $extra_code_sql_log ?'':" NOT");
do_log(0,"SQL::Quarantine %s loaded", $extra_code_sql_quar ?'':" NOT");
@@ -11659,17 +11809,17 @@ sub load_policy_bank($;$) {
} elsif (ref($new_bank_ref->{$k}) ne 'HASH' ||
ref($current_policy_bank{$k}) ne 'HASH') {
$current_policy_bank{$k} = $new_bank_ref->{$k};
- # do_log(5,"loading policy bank %s, curr{%s} replaced by %s",
+ # do_log(5,'loading policy bank %s, curr{%s} replaced by %s',
# $policy_bank_name, $k, $current_policy_bank{$k}) if $do_log5;
} else { # new hash to be merged into or replacing an existing hash
if ($new_bank_ref->{$k}{REPLACE}) { # replace the entire hash
$current_policy_bank{$k} = { %{$new_bank_ref->{$k}} }; # copy of new
- do_log(5,"loading policy bank %s, curr{%s} hash replaced",
+ do_log(5,'loading policy bank %s, curr{%s} hash replaced',
$policy_bank_name, $k) if $do_log5;
} else { # merge field-by-field, old fields missing in new are retained
$current_policy_bank{$k} = { %{$current_policy_bank{$k}} }; # copy
while (my($key,$val) = each %{$new_bank_ref->{$k}}) {
- do_log(5,"loading policy bank %s, curr{%s}{%s} = %s, %s",
+ do_log(5,'loading policy bank %s, curr{%s}{%s} = %s, %s',
$policy_bank_name, $k, $key, $val,
!exists($current_policy_bank{$k}{$key}) ? 'new'
: 'replaces '.$current_policy_bank{$k}{$key}
@@ -11683,8 +11833,8 @@ sub load_policy_bank($;$) {
$current_policy_bank{'policy_bank_path'} =
($cpbp eq '' ? '' : $cpbp.'/') . $policy_bank_name;
update_current_log_level();
- do_log(2,'loaded policy bank "%s"%s', $policy_bank_name,
- $cpbp eq '' ? '' : " over \"$cpbp\"");
+ ll(3) && do_log(3,'loaded policy bank "%s"%s', $policy_bank_name,
+ $cpbp eq '' ? '' : " over \"$cpbp\"");
}
}
@@ -11859,6 +12009,8 @@ sub child_init_hook {
$child_init_hook_was_called = 1;
do_log(5, "entered child_init_hook");
$my_pid = $$; $0 = c('myprogram_name') . ' (virgin child)';
+# DB::enable_profile(sprintf("/tmp/nytprof-amavis-%s-%d.out",
+# $my_pid, int rand 1000000)) if $profiling;
stir_random();
log_capture_enabled(1) if $enable_log_capture;
# reset log counters inherited from a master process
@@ -11947,6 +12099,7 @@ sub child_init_hook {
};
add_entropy($inherited_entropy, Time::HiRes::gettimeofday, rand());
Amavis::Timing::go_idle('vir');
+# DB::disable_profile() if $profiling;
}
### user customizable Net::Server hook
@@ -11955,6 +12108,8 @@ sub post_accept_hook {
my $self = $_[0];
local $SIG{CHLD} = 'DEFAULT';
# do_log(5, "entered post_accept_hook");
+ DB::enable_profile(sprintf("/tmp/nytprof-amavis-%s-%d.out",
+ $my_pid, int rand 1000000)) if $profiling;
if (!$child_init_hook_was_called) {
# this can happen with base Net::Server (not PreFork nor PreForkSiple)
do_log(5, "post_accept_hook: invoking child_init_hook which was skipped");
@@ -12341,6 +12496,8 @@ sub post_process_request_hook {
dump_captured_log(1, c('enable_log_capture_dump'));
# workaround: Net::Server 0.91 forgets to disconnect session
if (Net::Server->VERSION == 0.91) { close STDIN; close STDOUT }
+# DB::disable_profile() if $profiling;
+ DB::finish_profile() if $profiling;
}
### Child is about to be terminated
@@ -12459,12 +12616,14 @@ sub generate_unique_mail_id($) {
for (my $attempt = 5; ;) { # sanity limit on retries
($mail_id,$secret_id) = generate_mail_id();
$msginfo->secret_id($secret_id);
- $secret_id = 'X' x length($secret_id); # can't hurt to be conservative
+ $secret_id = 'X' x length($secret_id); # can't hurt to wipe out
$msginfo->mail_id($mail_id); # assign a long-term unique id to the msg
my $is_unique = 1;
- if ($redis_storage) {
- # attempt to save a message placeholder to Redis, ensuring it is unique
+ if ($redis_storage && $msginfo->originating) {
+ # attempt to save a message placeholder to Redis, ensuring it is unique;
+ # don't bother to save info on incoming messages, saves Redis storage
+ # while still offering necessary data for a penpals function
$redis_storage->save_info_preliminary($msginfo) or ($is_unique=0);
}
if ($is_unique && $sql_storage) {
@@ -12472,6 +12631,7 @@ sub generate_unique_mail_id($) {
$sql_storage->save_info_preliminary($msginfo) or ($is_unique=0);
}
last if $is_unique;
+
if (--$attempt <= 0) {
do_log(-2,"ERROR sql_storage: too many retries ".
"on storing preliminary, info not saved");
@@ -12593,6 +12753,13 @@ sub collect_some_info($) {
if @rfc2822_resent_sender;
}
+ my $refs_in_reply_to = $msginfo->get_header_field_body('in-reply-to');
+ my $refs_references = $msginfo->get_header_field_body('references');
+ my(@refs) = grep(defined $_, $refs_in_reply_to, $refs_references);
+ @refs = parse_message_id(join(' ', at refs)) if @refs;
+ do_log(4, 'references: %s', join(', ', at refs)) if @refs;
+ $msginfo->references(\@refs);
+
my $mail_size = $msginfo->msg_size; # use corrected ESMTP size if avail.
if (!defined($mail_size) || $mail_size <= 0) { # not yet known?
$mail_size = $msginfo->orig_header_size + $msginfo->orig_body_size;
@@ -12609,7 +12776,12 @@ sub collect_some_info($) {
unshift(@ip_trace, $cl_ip);
}
}
+ my @ip_trace_public =
+ map(do { my($public,$key,$err) = lookup_ip_acl($_, @public_networks_maps);
+ $public && !$err ? $_ : () },
+ grep(defined $_ && $_ ne '', @ip_trace) );
$msginfo->ip_addr_trace(\@ip_trace);
+ $msginfo->ip_addr_trace_public(\@ip_trace_public);
# check for mailing lists, bulk mail and auto-responses
my $is_mlist; # mail from a mailing list
@@ -12699,8 +12871,10 @@ sub collect_some_info($) {
sub check_mail($$) {
my($msginfo, $dsn_per_recip_capable) = @_;
- my $which_section = 'check_init'; my(%elapsed,$t0_sect);
- $elapsed{'TimeElapsedReceiving'} = Time::HiRes::time - $msginfo->rx_time;
+ my $which_section = 'check_init';
+ my $t0_sect;
+ my $elapsed = {}; $msginfo->time_elapsed($elapsed);
+ $elapsed->{'TimeElapsedReceiving'} = Time::HiRes::time - $msginfo->rx_time;
my $point_of_no_return = 0; # past the point where mail or DSN was sent
my $mail_id = $msginfo->mail_id; # typically undef at this stage
my $am_id = $msginfo->log_id;
@@ -12711,7 +12885,8 @@ sub check_mail($$) {
my($smtp_resp, $exit_code, $preserve_evidence);
my $custom_object;
my $hold; # set to some string causes the message to be placed on hold
- # (frozen) by MTA. This can be used in cases when we stumble
+ # (frozen) by MTA (if configured to understand the inserted
+ # header field). This can be used in cases when we stumble
# across some permanent problem making us unable to decide
# if the message is to be really delivered.
# is any mail component password protected or otherwise non-decodable?
@@ -12750,7 +12925,7 @@ sub check_mail($$) {
# compute body digest, measure mail size, check for 8-bit data, get entropy
get_body_digest($msginfo, $Amavis::Conf::mail_digest_algorithm);
- $which_section = 'check_init3';
+ $which_section = 'collect_info';
collect_some_info($msginfo);
if (!defined($msginfo->client_addr)) { # fetch missing IP addr from header
@@ -12766,6 +12941,7 @@ sub check_mail($$) {
}
}
}
+ section_time($which_section);
$which_section = 'check_init4';
my $mail_size = $msginfo->msg_size; # use corrected ESMTP size
@@ -12777,13 +12953,15 @@ sub check_mail($$) {
# section_time($which_section);
if (!defined $mail_id && ($sql_store_info_for_all_msgs || !$sql_storage)) {
- $which_section = 'gen_mail_id';
+ $which_section = 'reg_proc';
$zmq_obj->register_proc(2,0,'G',$am_id) if $zmq_obj;
$snmp_db->register_proc(2,0,'G',$am_id) if $snmp_db;
+ # section_time($which_section);
+ $which_section = 'gen_mail_id';
# create a mail_id unique to a database and save preliminary info to SQL
generate_unique_mail_id($msginfo);
$mail_id = $msginfo->mail_id;
- section_time($which_section) if $sql_storage;
+ section_time($which_section) if $sql_storage; # || $redis_storage
}
$which_section = "custom-new";
@@ -12791,7 +12969,7 @@ sub check_mail($$) {
my $old_orig = c('originating');
# may load policy banks
$custom_object = Amavis::Custom->new($conn,$msginfo);
- my $new_orig = c('originating'); # may have changed by a p.b.load
+ my $new_orig = c('originating'); # may have changed by a pol. bank load
$msginfo->originating($new_orig) if ($old_orig?1:0) != ($new_orig?1:0);
update_current_log_level(); 1;
} or do {
@@ -12803,6 +12981,22 @@ sub check_mail($$) {
do_log(5,"Custom hooks enabled"); section_time($which_section);
}
+ if ($redis_storage && c('enable_ip_repu')) {
+ $which_section = 'redis_ip_repu';
+ my($score, $worst_ip) =
+ $redis_storage->query_and_update_ip_reputation($msginfo);
+ if ($score && $score >= 0.5) {
+ $msginfo->ip_repu_score($score);
+ my $spam_test = sprintf('AM.IP_BAD_%s=%.1f', $worst_ip, $score);
+ for my $r (@{$msginfo->per_recip_data}) {
+ $r->spam_level( ($r->spam_level || 0) + $score);
+ $r->spam_tests([]) if !$r->spam_tests;
+ unshift(@{$r->spam_tests}, \$spam_test);
+ }
+ }
+ section_time($which_section);
+ }
+
my $cl_ip = $msginfo->client_addr;
my($os_fingerprint_obj,$os_fingerprint);
my $os_fingerprint_method = c('os_fingerprint_method');
@@ -12821,6 +13015,7 @@ sub check_mail($$) {
0.050, $cl_ip, $msginfo->client_port, $dst_ip, $dst_port,
defined $mail_id ? $mail_id : sprintf("%08x",rand(0x7fffffff)) );
}
+
my $sender = $msginfo->sender;
my(@recips) = map($_->recip_addr, @{$msginfo->per_recip_data});
my $rfc2822_sender = $msginfo->rfc2822_sender;
@@ -12978,23 +13173,34 @@ sub check_mail($$) {
if ($bounce_header_fields_ref &&
exists $bounce_header_fields_ref->{'message-id'}) {
$bounce_msgid = $bounce_header_fields_ref->{'message-id'};
+ if (defined $bounce_msgid && $bounce_msgid ne '') {
+ my $refs = $msginfo->references;
+ if (!$refs) { $refs = []; $msginfo->references($refs) }
+ push(@$refs, $bounce_msgid);
+ }
}
prolong_timer($which_section);
}
$which_section = "parts_decode_ext";
snmp_count('OpsDec');
- ($hold,$any_undecipherable) =
+ my($any_encrypted,$over_levels);
+ ($hold, $any_undecipherable, $any_encrypted, $over_levels) =
Amavis::Unpackers::decompose_mail($msginfo->mail_tempdir,
$file_generator_object);
- if ($hold ne '' || $any_undecipherable) {
+ $any_undecipherable ||= ($any_encrypted || $over_levels);
+ if ($any_undecipherable) {
$msginfo->add_contents_category(CC_UNCHECKED,0);
+ $msginfo->add_contents_category(CC_UNCHECKED,1) if $any_encrypted;
+ $msginfo->add_contents_category(CC_UNCHECKED,2) if $over_levels;
for my $r (@{$msginfo->per_recip_data}) {
- $r->add_contents_category(CC_UNCHECKED,0)
- if !$r->bypass_virus_checks;
+ next if $r->bypass_virus_checks;
+ $r->add_contents_category(CC_UNCHECKED,0);
+ $r->add_contents_category(CC_UNCHECKED,1) if $any_encrypted;
+ $r->add_contents_category(CC_UNCHECKED,2) if $over_levels;
}
}
- $elapsed{'TimeElapsedDecoding'} = Time::HiRes::time - $t0_sect;
+ $elapsed->{'TimeElapsedDecoding'} = Time::HiRes::time - $t0_sect;
}
my $bphcm = ca('bypass_header_checks_maps');
@@ -13073,7 +13279,7 @@ sub check_mail($$) {
# special case to make available a complete mail file for inspection
if ((defined $mime_err && $mime_err ne '') ||
!defined($msginfo->mime_entity) ||
- lookup2(0,'MAIL',\@keep_decoded_original_maps) ||
+ lookup2(0, 'MAIL', \@keep_decoded_original_maps) ||
$any_undecipherable && lookup2(0,'MAIL-UNDECIPHERABLE',
\@keep_decoded_original_maps)) {
if (!defined($msginfo->mail_text_fn)) {
@@ -13122,7 +13328,7 @@ sub check_mail($$) {
$virus_checking_failed = $eval_stat;
$virus_checking_failed = 1 if !$virus_checking_failed;
};
- $elapsed{'TimeElapsedVirusCheck'} = Time::HiRes::time - $t0_sect;
+ $elapsed->{'TimeElapsedVirusCheck'} = Time::HiRes::time - $t0_sect;
snmp_count('OpsVirusCheck');
if ($virus_presence_checked && @virusname && $snmp_db) {
@@ -13262,15 +13468,21 @@ sub check_mail($$) {
my $any_pass = 0; my $prelim_blocking_ccat;
for my $r (@{$msginfo->per_recip_data}) {
my $final_destiny = D_PASS;
+ my $recip = $r->recip_addr;
my(@fd_tuples) = $r->setting_by_main_contents_category_all(
- cr('final_destiny_by_ccat'), cr('lovers_maps_by_ccat'));
+ cr('final_destiny_maps_by_ccat'),
+ cr('lovers_maps_by_ccat'));
for my $tuple (@fd_tuples) {
- my($cc, $fd, $lovers_map_ref) = @$tuple;
- if (!defined($fd) || $fd == D_PASS) {
+ my($cc, $fd_map_ref, $lovers_map_ref) = @$tuple;
+ my $fd = !ref $fd_map_ref ? $fd_map_ref # compatibility
+ : lookup2(0, $recip, $fd_map_ref,
+ Label => 'Destiny1');
+ if (!defined $fd || $fd == D_PASS) {
+ $fd = D_PASS; # keep D_PASS
} elsif (defined($lovers_map_ref) &&
- lookup2(0, $r->recip_addr, $lovers_map_ref,
- Label=>'Lovers1')) {
- } else {
+ lookup2(0, $recip, $lovers_map_ref, Label => 'Lovers1')) {
+ $fd = D_PASS; # D_PASS for content lovers
+ } else { # $fd != D_PASS, blocked
$prelim_blocking_ccat = $cc; $final_destiny = $fd;
last;
}
@@ -13305,7 +13517,7 @@ sub check_mail($$) {
};
$msginfo->checks_performed->{S} = 1;
prolong_timer($which_section);
- $elapsed{'TimeElapsedSpamCheck'} = Time::HiRes::time - $t0_sect;
+ $elapsed->{'TimeElapsedSpamCheck'} = Time::HiRes::time - $t0_sect;
snmp_count('OpsSpamCheck');
$spam_presence_checked = 1;
}
@@ -13350,7 +13562,7 @@ sub check_mail($$) {
if (!$redis_storage &&
!(defined $sql_storage && $sql_store_info_for_all_msgs)) {
- # pen pals disabled - data on past mail transactions not available
+ # pen pals disabled - data on past mail transactions unavailable
} elsif ($msginfo->is_in_contents_category(CC_VIRUS)) {
# pen pals disabled, not needed for infected messages
} else {
@@ -13367,47 +13579,53 @@ sub check_mail($$) {
# spam, can't get below threshold_high even under best circumstances
do_log(5,"penpals: high score, penpals won't help");
} elsif ($sender ne '' && !$msginfo->originating &&
- lookup2(0,$sender, ca('local_domains_maps'))) {
+ lookup2(0, $sender, ca('local_domains_maps'))) {
# no bonus to unauthent. senders from outside claiming a local domain
do_log(5,"penpals: local sender from outside, ignored: %s", $sender);
} else {
$t0_sect = Time::HiRes::time;
$zmq_obj->register_proc(2,0,'P',$am_id) if $zmq_obj; # penpals
$snmp_db->register_proc(2,0,'P',$am_id) if $snmp_db;
+ my $refs = $msginfo->references;
my $sid = $msginfo->sender_maddr_id;
+ section_time("pre-penpals");
+
+ if ($redis_storage) {
+ # does all recipient queries in one go
+ my $ok = eval { $redis_storage->penpals_find($msginfo, $refs) };
+ section_time("penpals-redis") if $ok;
+ }
+
for my $r (@{$msginfo->per_recip_data}) {
next if $r->recip_done; # already dealt with
my $recip = $r->recip_addr;
- my $rid = $r->recip_maddr_id;
- if (defined($rid) && $sid ne $rid && $r->recip_is_local) {
+ if ($r->recip_is_local && lc($sender) ne lc($recip)) {
# inbound or internal_to_internal, except self_to_self
- my $refs_str = $msginfo->get_header_field_body('in-reply-to') .
- $msginfo->get_header_field_body('references');
- my(@refs) = $refs_str eq '' ? () : parse_message_id($refs_str);
- push(@refs, $bounce_msgid) if defined $bounce_msgid &&
- $bounce_msgid ne '';
- do_log(4,'penpals: references: %s', join(', ', at refs)) if @refs;
- # NOTE: swap $rid and $sid as args here, as we are now checking
- # for a potential reply mail - whether the current recipient has
- # recently sent any mail to the sender of the current mail:
- my($pp_age_sql, $pp_age_redis, $pp_mail_id, $pp_subj);
- if ($sql_storage) {
- ($pp_age_sql, $pp_mail_id, $pp_subj) =
- $sql_storage->penpals_find($rid, $sid,
- \@refs, $msginfo->rx_time);
- }
- if ($redis_storage) {
- my($pp_mail_id_redis, $pp_sid, $pp_rid, $pp_mid);
- ($pp_age_redis, $pp_mail_id_redis, $pp_sid, $pp_rid, $pp_mid) =
- $redis_storage->penpals_find($r->recip_addr_smtp,
- $msginfo->sender_smtp,
- \@refs, $msginfo->rx_time);
- $pp_mail_id = $pp_mail_id_redis if !defined $pp_mail_id;
+
+ my $pp_mail_id = $r->recip_penpals_related;
+ my $pp_age = $r->recip_penpals_age;
+ my $pp_subj;
+ my $rid = $r->recip_maddr_id;
+ if ($sql_storage && defined $sid && defined $rid) {
+ # NOTE: swap $rid and $sid as args in a query here, as we are
+ # now checking for a potential reply mail - whether the current
+ # recipient has recently sent any mail to the sender of the
+ # current mail:
+ my($pp_age_sql, $pp_mail_id_sql, $pp_subj_sql) =
+ $sql_storage->penpals_find($rid, $sid, $refs, $msginfo);
+ if (defined $pp_age_sql) {
+ if (!defined $pp_age || $pp_age_sql < $pp_age) {
+ $pp_age = $pp_age_sql; $pp_mail_id = $pp_mail_id_sql;
+ $r->recip_penpals_age($pp_age);
+ $r->recip_penpals_related($pp_mail_id);
+ }
+ $pp_subj = $pp_subj_sql;
+ }
+ section_time("penpals-sql");
}
- $pp_age = min($pp_age_sql, $pp_age_redis);
+
$msginfo->checks_performed->{P} = 1;
if (defined $pp_age) { # found info about previous correspondence
- $r->recip_penpals_age($pp_age); # save the information
my $weight = exp(-($pp_age/$pp_halflife) * log(2));
# weight is a factor between 1 and 0, representing
# exponential decay: weight(t) = 1 / 2^(t/halflife)
@@ -13416,7 +13634,7 @@ sub check_mail($$) {
$r->recip_penpals_score($adj);
$r->spam_level( ($r->spam_level || 0) + $adj);
{ my $spam_tests = 'AM.PENPAL=' . (0+sprintf("%.3f",$adj));
- if (!defined($r->spam_tests)) {
+ if (!$r->spam_tests) {
$r->spam_tests([ \$spam_tests ]);
} else {
unshift(@{$r->spam_tests}, \$spam_tests);
@@ -13437,8 +13655,8 @@ sub check_mail($$) {
}
}
}
- section_time($which_section);
- $elapsed{'TimeElapsedPenPals'} = Time::HiRes::time - $t0_sect;
+ # section_time($which_section);
+ $elapsed->{'TimeElapsedPenPals'} = Time::HiRes::time - $t0_sect;
}
}
@@ -13479,7 +13697,7 @@ sub check_mail($$) {
for my $r (@{$msginfo->per_recip_data}) {
$r->spam_level( ($r->spam_level || 0) + $bounce_killer_score);
my $spam_tests = 'AM.BOUNCE=' . $bounce_killer_score;
- if (!defined($r->spam_tests)) {
+ if (!$r->spam_tests) {
$r->spam_tests([ \$spam_tests ]);
} else {
unshift(@{$r->spam_tests}, \$spam_tests);
@@ -13524,9 +13742,6 @@ sub check_mail($$) {
}
my $blacklisted = $r->recip_blacklisted_sender;
my $whitelisted = $r->recip_whitelisted_sender;
- # penpals_score is already accounted for in spam_level,
- # it is provided here separately for informational/logging purposes
- my $penpals_score = $r->recip_penpals_score; # is zero or negative!
my $do_tag = !$bypassed && (
$blacklisted || !defined $tag_level || $tag_level eq '' ||
($spam_level + ($whitelisted?-10:0) >= $tag_level));
@@ -13572,15 +13787,22 @@ sub check_mail($$) {
# determine true reason for blocking,considering lovers and final_destiny
my $blocking_ccat; my $final_destiny = D_PASS; my $to_be_mangled;
my(@fd_tuples) = $r->setting_by_main_contents_category_all(
- cr('final_destiny_by_ccat'), cr('lovers_maps_by_ccat'),
- cr('defang_maps_by_ccat') );
+ cr('final_destiny_maps_by_ccat'),
+ cr('lovers_maps_by_ccat'),
+ cr('defang_maps_by_ccat') );
for my $tuple (@fd_tuples) {
- my($cc, $fd, $lovers_map_ref, $mangle_map_ref) = @$tuple;
- if (!defined($fd) || $fd == D_PASS) {
- do_log(5, "final_destiny (ccat=%s) is PASS, recip %s", $cc,$recip);
+ my($cc, $fd_map_ref, $lovers_map_ref, $mangle_map_ref) = @$tuple;
+ my $fd = !ref $fd_map_ref ? $fd_map_ref # compatibility
+ : lookup2(0, $recip, $fd_map_ref,
+ Label => 'Destiny2');
+ if (!defined $fd || $fd == D_PASS) {
+ ll(5) && do_log(5, "final_destiny (ccat=%s) is PASS, recip %s",
+ $cc, $recip);
+ $fd = D_PASS; # keep D_PASS
} elsif (defined($lovers_map_ref) &&
- lookup2(0,$recip,$lovers_map_ref, Label=>'Lovers2')) {
- do_log(5, "contents lover (ccat=%s) %s", $cc,$recip);
+ lookup2(0, $recip, $lovers_map_ref, Label => 'Lovers2')) {
+ ll(5) && do_log(5, "contents lover (ccat=%s) %s", $cc, $recip);
+ $fd = D_PASS; # change to D_PASS for content lovers
} elsif ($fd == D_BOUNCE &&
($sender eq '' || defined($msginfo->is_bulk)) &&
ccat_maj($cc) == CC_BADH) {
@@ -13592,8 +13814,9 @@ sub check_mail($$) {
do_log(1, "allow bad header section from %s<%s> -> <%s>: %s",
!defined($is_bulk) ? '' : "($is_bulk) ",
$sender, $recip, $bad_headers[0]);
- } else {
- $blocking_ccat = $cc; $final_destiny = $fd;
+ $fd = D_PASS; # change D_BOUNCE to D_PASS for CC_BADH
+ } else { # $fd != D_PASS, blocked
+ $blocking_ccat = $cc; $final_destiny = $fd;
my $cc_main = $r->contents_category;
$cc_main = $cc_main->[0] if $cc_main;
if ($blocking_ccat eq $cc_main) {
@@ -13644,6 +13867,9 @@ sub check_mail($$) {
@rfc2822_from, $rfc2822_sender, $sender))) {
$to_be_mangled = 0; # not for foreign 'Sender:' or 'From:'
do_log(5,"will not add disclaimer, sender not local");
+ } elsif (c('outbound_disclaimers_only') && $r->recip_is_local) {
+ $to_be_mangled = 0;
+ do_log(5, "will not add disclaimer, recipient is local");
}
}
} else { # defanging (not disclaiming)
@@ -13660,7 +13886,9 @@ sub check_mail($$) {
}
}
- if ($penpals_score < 0) {
+ # penpals_score is already accounted for in spam_level
+ my $penpals_score = $r->recip_penpals_score; # is zero or negative!
+ if ($penpals_score && $penpals_score < 0) {
# only for logging and statistics purposes
my($do_tag2_nopp, $do_tag3_nopp, $do_kill_nopp) =
map { !$whitelisted &&
@@ -13759,7 +13987,7 @@ sub check_mail($$) {
# do_quarantine($msginfo, undef, ['sender-quarantine'], 'local:user-%m'
# ) if lookup(0,$sender, ['user1 at domain','user2 at domain']);
# section_time($which_section);
- $elapsed{'TimeElapsedQuarantineAndNotify'} = Time::HiRes::time - $t0_sect;
+ $elapsed->{'TimeElapsedQuarantineAndNotify'} = Time::HiRes::time - $t0_sect;
if (defined $hold && $hold ne '')
{ do_log(-1, "NOTICE: HOLD reason: %s", $hold) }
@@ -13915,10 +14143,15 @@ sub check_mail($$) {
$r->blocking_ccat($blocking_ccat);
$msginfo->blocking_ccat($blocking_ccat)
if !defined($msginfo->blocking_ccat);
+ my $fd_map_ref =
+ $r->setting_by_contents_category(cr('final_destiny_maps_by_ccat'));
my $final_destiny =
- $r->setting_by_contents_category(cr('final_destiny_by_ccat'));
+ !ref $fd_map_ref ? $fd_map_ref # compatibility
+ : lookup2(0, $r->recip_addr, $fd_map_ref, Label => 'Destiny3');
+ $final_destiny = D_PASS if !defined $final_destiny;
if ($final_destiny == D_PASS) {
- $final_destiny = D_REJECT; # impossible to pass, change to reject
+ # impossible to pass, change to tempfail or reject
+ $final_destiny = $smtp_resp =~ /^5/ ? D_REJECT : D_TEMPFAIL;
}
$r->recip_destiny($final_destiny);
local($1,$2);
@@ -13950,7 +14183,7 @@ sub check_mail($$) {
# note that 5xx status rejects may later be converted to bounces
}
$msginfo->header_edits($hdr_edits); # restore original edits just in case
- $elapsed{'TimeElapsedForwarding'} = Time::HiRes::time - $t0_sect;
+ $elapsed->{'TimeElapsedForwarding'} = Time::HiRes::time - $t0_sect;
}
# AM.PDP or AM.CL (milter)
@@ -13994,6 +14227,7 @@ sub check_mail($$) {
};
section_time($which_section);
}
+
$which_section = "delivery-notification"; $t0_sect = Time::HiRes::time;
# generate a delivery status notification according to RFC 3462 & RFC 3464
my($notification,$suppressed) = delivery_status_notification(
@@ -14012,10 +14246,11 @@ sub check_mail($$) {
} elsif (defined $notification) { # dsn needed, send delivery notification
mail_dispatch($notification, 'Dsn', 0);
my($n_smtp_resp, $n_exit_code, $n_dsn_needed) =
- one_response_for_all($notification, 0); # check status
+ one_response_for_all($notification, 0); # check status
if ($n_smtp_resp =~ /^2/ && !$n_dsn_needed) { # dsn successful?
$msginfo->dsn_sent(1); # mark the message as bounced
$point_of_no_return = 2; # now past the point where DSN was sent
+ build_and_save_structured_report($notification,'DSN');
} elsif ($n_smtp_resp =~ /^4/) {
die sprintf("temporarily unable to send DSN to <%s>: %s",
$msginfo->sender, $n_smtp_resp);
@@ -14029,7 +14264,10 @@ sub check_mail($$) {
}
# $notification->purge;
}
+ prolong_timer($which_section);
+ $elapsed->{'TimeElapsedDSN'} = Time::HiRes::time - $t0_sect;
+ $which_section = "snmp-counters"; $t0_sect = Time::HiRes::time;
{ # increment appropriate InMsgsStatus* SNMP counters and do some sanity
# checking along the way; also sets $msginfo->actions_performed
#
@@ -14114,9 +14352,22 @@ sub check_mail($$) {
ll(3) && do_log(3, 'status counters: InMsgsStatus{%s}',
join(',', @which_list));
}
-
prolong_timer($which_section);
- $elapsed{'TimeElapsedDSN'} = Time::HiRes::time - $t0_sect;
+
+ # merge similar timing entries
+ $elapsed->{'TimeElapsedSending'} = 0;
+ $elapsed->{'TimeElapsedSending'} +=
+ delete $elapsed->{$_} for ('TimeElapsedQuarantineAndNotify',
+ 'TimeElapsedForwarding', 'TimeElapsedDSN');
+
+ $which_section = 'report';
+ eval { # protect the new code just in case
+ # structured_report returns a string as perl characters (not octets)
+ $report_ref = structured_report($msginfo); 1;
+ } or do {
+ chomp $@; do_log(-1,"structured_report failed: %s", $@);
+ };
+ section_time($which_section);
# generate customized log report at log level 0 - this is usually the
# only log entry interesting to administrators during normal operation
@@ -14280,16 +14531,20 @@ sub check_mail($$) {
if ($redis_storage && defined $msginfo->mail_id) {
$which_section = 'redis-update';
- # save final information to Redis (if enabled)
- $redis_storage->save_info_final($msginfo);
+ # save final information to Redis
+ eval {
+ $redis_storage->save_info_final($msginfo,$report_ref); 1;
+ } or do {
+ chomp $@; do_log(-1, "redis: save final information failed: %s", $@);
+ };
+ section_time($which_section);
}
+
if ($sql_storage && defined $msginfo->mail_id) {
# save final information to SQL (if enabled)
$which_section = 'sql-update';
- my $ds = $msginfo->dsn_sent;
- $ds = !$ds ? 'N' : $ds==1 ? 'Y' : $ds==2 ? 'q' : '?';
for (my $attempt=5; $attempt>0; ) { # sanity limit on retries
- if ($sql_storage->save_info_final($msginfo,$ds)) {
+ if ($sql_storage->save_info_final($msginfo,$report_ref)) {
last;
} elsif (--$attempt <= 0) {
do_log(-2,"ERROR sql_storage: too many retries ".
@@ -14302,6 +14557,7 @@ sub check_mail($$) {
};
section_time($which_section);
}
+
if (ll(2)) { # log SpamAssassin timing report if available
my $sa_tim = $msginfo->supplementary_info('TIMING');
if (defined $sa_tim && $sa_tim ne '') {
@@ -14314,6 +14570,7 @@ sub check_mail($$) {
do_log(2, "TIMING-SA %s", $sa_tim);
}
}
+
if ($snmp_db || $zmq_obj) {
$which_section = 'update_snmp';
my($log_lines, $log_entries_by_level_ref,
@@ -14342,15 +14599,12 @@ sub check_mail($$) {
do_log(3,"Syslog retries: %d x %s", $log_status_counts_ref->{$_}, $_)
for (keys %$log_status_counts_ref);
}
- $elapsed{'TimeElapsedSending'} += # merge similar timing entries
- delete $elapsed{$_} for ('TimeElapsedQuarantineAndNotify',
- 'TimeElapsedForwarding', 'TimeElapsedDSN');
snmp_count( ['entropy',0,'STR'] );
- $elapsed{'TimeElapsedTotal'} = Time::HiRes::time - $msginfo->rx_time;
+ $elapsed->{'TimeElapsedTotal'} = Time::HiRes::time - $msginfo->rx_time;
# Will end up as SNMPv2-TC TimeInterval (INTEGER), units of 0.01 seconds,
# but we keep it in milliseconds in the bdb database!
# Note also the use of C32 instead of INT, we want cumulative time.
- snmp_count([$_, int(1000*$elapsed{$_}+0.5), 'C32']) for (keys %elapsed);
+ snmp_count([$_, int(1000*$elapsed->{$_}+0.5), 'C32']) for keys %$elapsed;
$snmp_db->update_snmp_variables if $snmp_db;
$zmq_obj->update_snmp_variables if $zmq_obj;
section_time($which_section);
@@ -14384,7 +14638,7 @@ sub check_mail($$) {
}
};
-# if ($hold ne '') {
+# if (defined $hold && $hold ne '') {
# do_log(-1, "NOTICE: Evidence is to be preserved: %s", $hold);
# $preserve_evidence = 1 if $allow_preserving_evidence;
# }
@@ -14392,7 +14646,26 @@ sub check_mail($$) {
do_log(0, "DEBUG_ONESHOT CAUSES EVIDENCE TO BE PRESERVED");
$preserve_evidence = 1; # regardless of $allow_preserving_evidence
}
-
+ if ($redis_storage &&
+ $redis_logging_queue_size_limit && c('redis_logging_key') ) {
+ if ($report_ref) { # already have it
+ # last-minute update of the "elapsed" field
+ structured_report_update_time($report_ref);
+ } else { # prepare the log report
+ eval { # protect the new code just in case
+ # structured_report returns a string as perl characters (not octets)
+ $report_ref = structured_report($msginfo); 1;
+ } or do {
+ chomp $@; do_log(-1, "structured_report failed: %s", $@);
+ };
+ }
+ eval {
+ $redis_storage->save_structured_report($report_ref,
+ c('redis_logging_key'), $redis_logging_queue_size_limit); 1;
+ } or do {
+ chomp $@; do_log(-1, "redis: save structured report: %s", $@);
+ };
+ }
$zmq_obj->register_proc(1,0,'.') if $zmq_obj; # content checking done
$snmp_db->register_proc(1,0,'.') if $snmp_db;
do_log(-1, "signal: %s", join(', ',keys %got_signals)) if %got_signals;
@@ -14400,6 +14673,377 @@ sub check_mail($$) {
($smtp_resp, $exit_code, $preserve_evidence);
}
+# ROT13 obfuscation (Caesar cipher)
+# (possibly useful as a weak privacy measure when analyzing logs)
+#
+sub rot13 {
+ my $str = $_[0];
+ $str =~ tr/a-zA-Z/n-za-mN-ZA-M/;
+ $str;
+}
+
+# Assemble a structured report, suitable for JSON serialization,
+# useful in save_info_final()
+#
+sub structured_report($;$) {
+ my($msginfo, $notification_type) = @_;
+
+ my(@recipients); # per-recipient records
+ my(@rcpt_to_list); # list of recipient addresses
+ my(@queued_as_list); # list of unique MTA queue IDs of forwarded mail
+ my(@smtp_status_code_list); # list of unique SMTP responses
+ my(@destiny_list); # list of destiny names
+ my(@mail_id_related); # list of related mail_id's according to penpals
+ my(%spam_test_names);
+ my $true = Amavis::JSON::boolean(1);
+ local($1,$2);
+
+ for my $r (@{$msginfo->per_recip_data}) {
+ my $recip_smtp = $r->recip_addr_smtp;
+ if (defined $recip_smtp) {
+ $recip_smtp =~ s/^<(.*)>\z/$1/s;
+ $recip_smtp =~ s/(\@[^@]+)\z/lc $1/se; # lowercase a domain part
+ }
+ push(@rcpt_to_list, $recip_smtp);
+ my $orig_addr = $r->dsn_orcpt; # RCPT command ORCPT option, RFC 3461
+ if (defined $orig_addr) {
+ $orig_addr = orcpt_decode($orig_addr);
+ $orig_addr =~ s/(\@[^@]+)\z/lc $1/se; # lowercase a domain part
+ if (defined $recip_smtp && $orig_addr eq $recip_smtp) {
+ undef $orig_addr; # is redundant
+ }
+ }
+ my $dest = $r->recip_destiny;
+ my $resp = $r->recip_smtp_response;
+ my $rem_smtp_resp = $r->recip_remote_mta_smtp_response;
+ my($queued_as, $resp_code, $resp_code_enh);
+ $queued_as = $1 if defined $rem_smtp_resp &&
+ $rem_smtp_resp =~ /\bqueued as ([0-9A-Za-z]+)$/;
+ ($resp_code, $resp_code_enh) = ($1,$2)
+ if $resp =~ /^(\d{3}) (?: [ \t]+ ([245] \. \d{1,3} \. \d{1,3}) \b)? /xs;
+ my $d = $resp=~/^4/ ? 'TEMPFAIL'
+ : ($dest==D_BOUNCE && $resp=~/^5/) ? 'BOUNCE'
+ : ($dest!=D_BOUNCE && $resp=~/^5/) ? 'REJECT'
+ : ($dest==D_DISCARD) ? 'DISCARD'
+ : ($dest==D_PASS && ($resp=~/^2/ || !$r->recip_done))
+ ? ($notification_type ? $notification_type : 'PASS') : '?';
+ push(@destiny_list, $d);
+ push(@smtp_status_code_list, $resp_code);
+ push(@queued_as_list, $queued_as) if defined $queued_as;
+ my $rid = $r->recip_maddr_id; # may be undefined
+ my $o_rid = $r->recip_maddr_id_orig; # may be undefined
+ my $banning_reason_short = $r->banning_reason_short;
+ my $spam_level = $r->spam_level;
+ my $user_policy_id = $r->user_policy_id;
+ my $ccat_blk_name =
+ $r->setting_by_blocking_contents_category(\%ccat_display_names);
+ my $ccat_main_name =
+ $r->setting_by_main_contents_category(\%ccat_display_names);
+ if (!defined $ccat_main_name ||
+ # ($ccat_main_name =~ /^(?:Clean|CatchAll)\z/s) ||
+ (defined $ccat_blk_name && $ccat_main_name eq $ccat_blk_name)) {
+ # not worth reporting main ccat if the same as blocking ccat (or clean?)
+ undef $ccat_main_name;
+ }
+ my $spam_tests = $r->spam_tests; # arrayref of scalar refs
+ if ($spam_tests) {
+ for my $test_name_val (split(/,/,join(',',map($$_,@$spam_tests)))) {
+ my($tname, $tscore) = split(/=/, $test_name_val, 2);
+ $spam_test_names{$tname} = max($tscore, $spam_test_names{$tname});
+ }
+ }
+ my $penpals_age = $r->recip_penpals_age; # penpals age in seconds, or undef
+ my $penpals_related = $r->recip_penpals_related;
+ push(@mail_id_related, $penpals_related) if defined $penpals_related;
+
+ my(%recip) = (
+ rcpt_to => $recip_smtp,
+ defined $orig_addr ? (rcpt_to_orig => $orig_addr) : (),
+ defined $rid ? (rid => $rid) : (),
+ defined $o_rid ? (rid_orig => Amavis::JSON::numeric($o_rid)) : (),
+ rcpt_is_local => Amavis::JSON::boolean($r->recip_is_local),
+ defined $user_policy_id ? (sql_user_policy_id => $user_policy_id) : (),
+ action => $d, # i.e. destiny
+ defined $resp ? (smtp_response => $resp) : (),
+ defined $resp_code ? (smtp_code => $resp_code) : (),
+ # defined $resp_code_enh ? (smtp_code_enh => $resp_code_enh) : (),
+ defined $queued_as ? (queued_as => $queued_as) : (),
+ !defined $spam_level ? ()
+ : (spam_score => Amavis::JSON::numeric(sprintf("%.3f",$spam_level))),
+ $r->recip_blacklisted_sender ? (blacklisted => $true) : (),
+ $r->recip_whitelisted_sender ? (whitelisted => $true) : (),
+ $r->bypass_virus_checks ? (bypass_virus_checks => $true) : (),
+ $r->bypass_banned_checks ? (bypass_banned_checks => $true) : (),
+ $r->bypass_spam_checks ? (bypass_spam_checks => $true) : (),
+ defined $ccat_blk_name ? (ccat_blocking => $ccat_blk_name) : (),
+ defined $ccat_main_name ? (ccat_main => $ccat_main_name) : (),
+ $banning_reason_short ? (banning_reason => $banning_reason_short) : (),
+ defined $penpals_related ? (mail_id_related => $penpals_related) : (),
+ !defined $penpals_age ? ()
+ : (penpals_age => Amavis::JSON::numeric(int($penpals_age))),
+ # recip_tagged # was tagged by address extension or Subject or X-Spam
+ );
+ for my $key (keys %recip) {
+ next if ref $recip{$key};
+ next if $recip{$key} !~ /[\x{a0}-\x{ff}]/s; # not upper half codepoints
+ # garbage-in/garbage-out, but at least ensure characters are valid
+ $recip{$key} = safe_decode_latin1($recip{$key}); # assumes ISO-8859-1
+ }
+ push(@recipients, \%recip);
+ }
+
+ my $sender_smtp = $msginfo->sender_smtp;
+ $sender_smtp =~ s/^<(.*)>\z/$1/s;
+ $sender_smtp =~ s/(\@[^@]+)\z/lc $1/se; # lowercase a domain part
+ $sender_smtp = '<>' if $sender_smtp eq ''; # looks more natural in the log
+
+ my $rfc2822_from = $msginfo->rfc2822_from; # undef, scalar or listref
+ my $rfc2822_sender = $msginfo->rfc2822_sender; # undef or scalar
+ my $rfc2822_to = $msginfo->rfc2822_to; # undef, scalar or listref
+ my $rfc2822_cc = $msginfo->rfc2822_cc; # undef, scalar or listref
+
+ my $q_type = $msginfo->quar_type;
+ # only keep the first quarantine type used (e.g. ignore archival quar.)
+ $q_type = $q_type->[0] if ref $q_type;
+
+ my $q_to = $msginfo->quarantined_to; # ref to a list of quar. locations
+ if (!defined($q_to) || !@$q_to) { $q_to = undef }
+ else {
+ $q_to = $q_to->[0]; # keep only the first quarantine location
+ $q_to =~ s{^\Q$QUARANTINEDIR\E/}{}; # strip directory name
+ }
+
+ my($min_spam_level, $max_spam_level) =
+ minmax(map($_->spam_level, @{$msginfo->per_recip_data}));
+
+ my(@test_names_spam_topdown) =
+ sort { $spam_test_names{$b} <=> $spam_test_names{$a} }
+ grep($spam_test_names{$_} > 0, keys %spam_test_names);
+
+ my(@test_names_ham_bottomup) =
+ sort { $spam_test_names{$a} <=> $spam_test_names{$b} }
+ grep($spam_test_names{$_} < 0, keys %spam_test_names);
+
+ my $useragent = $msginfo->get_header_field_body('user-agent');
+ $useragent = $msginfo->get_header_field_body('x-mailer') if !$useragent;
+ $useragent =~ s/^\s*(.*?)\s*\z/$1/s if $useragent;
+ my $m_id = $msginfo->get_header_field_body('message-id');
+ $m_id = join(' ', parse_message_id($m_id))
+ if defined $m_id && $m_id ne ''; # strip CFWS
+ my $refs = $msginfo->references;
+ my $subj = $msginfo->get_header_field_body('subject');
+ my $from = $msginfo->get_header_field_body('from'); # raw full field
+ for ($subj,$from) { # character set decoding, unfolding
+ chomp; s/\n(?=[ \t])//gs; s/^[ \t]+//s; s/[ \t]+\z//s; # unfold, trim
+ eval { # convert to UTF-8 octets
+ $_ = safe_decode('MIME-Header',$_); 1; # to characters
+ } or do {
+ my $eval_stat = $@ ne '' ? $@ : "errno=$!"; chomp $eval_stat;
+ do_log(1,"structured_report INFO: header field ".
+ "not decodable, assuming Latin1: %s", $eval_stat);
+ };
+ }
+
+ my($conn,$src_ip,$dst_ip,$dst_port);
+ $conn = $msginfo->conn_obj;
+ if ($conn) { # MTA -> amavisd
+ $src_ip = $conn->client_ip; # immediate client IP addr, i.e. our MTA
+ $dst_ip = $conn->socket_ip; # IP address of our receiving socket
+ $dst_port = $conn->socket_port; # port number of our receiving socket
+ }
+ my $client_addr = $msginfo->client_addr; # SMTP client -> MTA
+ my $client_port = $msginfo->client_port; # SMTP client -> MTA
+ my $ip_trace_public = $msginfo->ip_addr_trace_public; # "Received" trace
+ my $checks_performed = $msginfo->checks_performed;
+ $checks_performed = join(' ', grep($checks_performed->{$_},
+ qw(V S H B F P D))) if $checks_performed;
+ my $actions_performed = $msginfo->actions_performed;
+ $actions_performed = join(' ', @$actions_performed) if $actions_performed;
+ @destiny_list = unique_list(\@destiny_list);
+ my $partition_tag = $msginfo->partition_tag;
+ my $sid = $msginfo->sender_maddr_id;
+ my $policy_bank_path = c('policy_bank_path');
+ my $is_mlist = $msginfo->is_mlist;
+ $is_mlist =~ s/^ml:(?=.)//s if $is_mlist; # strip ml: prefix
+ my $os_fp = $msginfo->client_os_fingerprint;
+ my $dsn_sent = $msginfo->dsn_sent;
+ my $queue_id = $msginfo->queue_id;
+ @queued_as_list = unique_list(\@queued_as_list);
+ @smtp_status_code_list = unique_list(\@smtp_status_code_list);
+
+ my $dkim_author_sig = $msginfo->dkim_author_sig;
+ my $dkim_sigs_new_ref = $msginfo->dkim_signatures_new;
+ my $dkim_sigs_ref = $msginfo->dkim_signatures_valid;
+ my(@dkim_sigs_valid, @dkim_sigs_new);
+ @dkim_sigs_valid =
+ unique_list(map($_->domain, @$dkim_sigs_ref)) if $dkim_sigs_ref;
+ @dkim_sigs_new =
+ unique_list(map($_->domain, @$dkim_sigs_new_ref)) if $dkim_sigs_new_ref;
+
+ my $vn = $msginfo->virusnames;
+ undef $vn if $vn && !@$vn;
+ my(%scanners_report); # per-scanner report of virus names found
+ if ($vn) {
+ for (@av_scanners_results) {
+ my($av, $status, @virus_names) = @$_;
+ my $scanner = $av && $av->[0];
+ if ($status && defined $scanner) {
+ $scanner =~ tr/"/'/; # sanitize scanner name for json
+ $scanner =~ tr/\x00-\x1f\x7f\\/ /;
+ $scanners_report{$scanner} = \@virus_names;
+ }
+ }
+ }
+
+ my $rx_time = $msginfo->rx_time;
+ my $mjd = $rx_time/86400 + 40587; # Modified Julian Day, float
+ my($iso8601_year, $iso8601_wn) = iso8601_year_and_week($rx_time);
+
+ my(%elapsed);
+ if (!$notification_type) {
+ my $elapsed_ref = $msginfo->time_elapsed;
+ if ($elapsed_ref) {
+ while (my($k,$v) = each(%$elapsed_ref)) {
+ next if $k eq 'TimeElapsedPenPals'; # quick, don't bother
+ $k =~ s/^TimeElapsed//;
+ $elapsed{$k} = $v; # cast to numeric later down
+ }
+ }
+ }
+
+ my(%result) = (
+ type => 'amavis',
+ host => c('myhostname'),
+ log_id => $msginfo->log_id,
+ # secret_id => $msginfo->secret_id,
+ mail_id => $msginfo->mail_id,
+ !defined $msginfo->parent_mail_id ? () :
+ (mail_id_parent => $msginfo->parent_mail_id),
+ @mail_id_related ? (mail_id_related => \@mail_id_related) : (),
+ defined $src_ip ? (src_ip => $src_ip) : (),
+ defined $dst_ip ? (dst_ip => $dst_ip) : (),
+ $dst_port ? (dst_port => Amavis::JSON::numeric($dst_port)) : (),
+ defined $client_addr ? (client_ip => $client_addr) : (),
+ $client_port ? (client_port => Amavis::JSON::numeric($client_port)) : (),
+ defined $partition_tag ? (partition => $partition_tag) : (),
+ defined $queue_id && $queue_id ne '' ? (queue_id => $queue_id) : (),
+ defined $sid ? (sid => $sid) : (),
+ mail_from => $sender_smtp,
+ !defined $rfc2822_sender ? ()
+ : (sender => $rfc2822_sender),
+ !defined $rfc2822_from ? ()
+ : (author => [ ref $rfc2822_from ? @$rfc2822_from : $rfc2822_from ]),
+ !defined $rfc2822_to ? ()
+ : (to_addr => [ ref $rfc2822_to ? @$rfc2822_to : $rfc2822_to ]),
+ !defined $rfc2822_cc ? ()
+ : (cc_addr => [ ref $rfc2822_cc ? @$rfc2822_cc : $rfc2822_cc ]),
+ # defined $from ? (from_raw => $from) : (),
+ defined $subj ? (subject => $subj) : (),
+ defined $subj ? (subject_rot13 => rot13($subj)) : (),
+ defined $m_id ? (message_id => $m_id) : (),
+ $refs && @$refs ? (references => [ @$refs ]) : (),
+ defined $useragent ? (user_agent => $useragent) : (),
+ !defined $policy_bank_path ? ()
+ : (policy_banks => [ split(m{/}, $policy_bank_path) ]),
+ ref $ip_trace_public ? (ip_trace => [ @$ip_trace_public ]) : (),
+ !$msginfo->msg_size ? ()
+ : (size => Amavis::JSON::numeric(0+$msginfo->msg_size)),
+ !$msginfo->body_digest ? ()
+ : (digest_body => $msginfo->body_digest),
+ content_type => # blocking ccat if blocked, main ccat otherwise
+ $msginfo->setting_by_contents_category(\%ccat_display_names),
+ defined $q_to ? (quarantine => $q_to) : (),
+ defined $q_type ? (quar_type => $q_type) : (),
+ !defined $max_spam_level ? ()
+ : (spam_score => Amavis::JSON::numeric(sprintf("%.3f",$max_spam_level))),
+ $notification_type ? () : (dsn_sent => Amavis::JSON::boolean($dsn_sent==1)),
+ originating => Amavis::JSON::boolean($msginfo->originating),
+ defined $os_fp && $os_fp ne '' ? (os_fp => $os_fp) : (),
+ defined $actions_performed ? (actions_performed => $actions_performed): (),
+ defined $checks_performed ? (checks_performed => $checks_performed) : (),
+ $vn ? (virusnames => unique_ref($vn)) : (),
+ $vn ? (av_scan => \%scanners_report) : (),
+ # %spam_test_names ? (tests => { %spam_test_names }) : (),
+ !%spam_test_names ? () : (
+ tests => [ sort keys %spam_test_names ], # alphabetically
+ tests_spam => \@test_names_spam_topdown, # > 0, largest first
+ tests_ham => \@test_names_ham_bottomup, # < 0, smallest first
+ ),
+ $msginfo->is_auto ? (is_auto_resp => $true) : (), # is an auto-response
+ $msginfo->is_mlist? (is_mlist => $true) : (), # is a mailing list
+ $msginfo->is_bulk ? (is_bulk => $true) : (), # bulk or m.list or auto-resp
+ @dkim_sigs_valid ? (dkim_valid_sig => \@dkim_sigs_valid) : (),
+ @dkim_sigs_new ? (dkim_new_sig => \@dkim_sigs_new) : (),
+ defined $dkim_author_sig ? (dkim_author_sig => $dkim_author_sig) : (),
+ !@smtp_status_code_list ? () : (smtp_code => \@smtp_status_code_list),
+ !@queued_as_list ? () : (queued_as => \@queued_as_list),
+ action => \@destiny_list,
+ rcpt_num => Amavis::JSON::numeric(scalar @rcpt_to_list), # num. of recips
+ rcpt_to => \@rcpt_to_list, # list of recipient addresses
+ recipients => \@recipients, # list of hashes
+ message => # a brief report
+ sprintf("%s %s %s %s -> %s",
+ $msginfo->log_id, join(',', @destiny_list),
+ $msginfo->setting_by_contents_category(\%ccat_display_names),
+ $msginfo->sender_smtp,
+ join(',', map($_->recip_addr_smtp,
+ @{$msginfo->per_recip_data}))),
+ time_unix => # UNIX time to millisecond precision
+ Amavis::JSON::numeric(sprintf("%.3f", $rx_time)),
+ # time_mjd => # Modified Julian Day to millisecond precision
+ # Amavis::JSON::numeric(sprintf("%14.8f", $mjd)),
+ '@timestamp' => iso8601_utc_timestamp($rx_time,undef,undef,1,1),
+ time_iso_week_date => sprintf("%04d-W%02d-%d",
+ $iso8601_year, # ISO week-numbering year
+ $iso8601_wn, # ISO week number 1..53
+ iso8601_weekday($rx_time)), # 1..7, Mo=1, localtime
+ !%elapsed ? () : (elapsed => \%elapsed),
+ );
+ for my $key (keys %result) {
+ next if ref $result{$key};
+ # already decoded characters in the following fields
+ next if $key eq 'subject' || $key eq 'subject_rot13' || $key eq 'from';
+ next if $result{$key} !~ /[\x{a0}-\x{ff}]/s; # not upper half codepoints
+ # garbage-in/garbage-out, but at least ensure characters are valid
+ $result{$key} = safe_decode_latin1($result{$key}); # assumes ISO-8859-1
+ }
+ if (%elapsed) {
+ # last-minute update of total elapsed time, cast to numeric
+ my $el = $result{elapsed};
+ $el->{Total} = get_time_so_far();
+ $el->{Amavis} = $el->{Total}-($el->{SpamCheck}||0)-($el->{VirusCheck}||0);
+ $el->{$_} = Amavis::JSON::numeric(sprintf("%.3f",$el->{$_})) for keys %$el;
+ }
+ \%result;
+}
+
+# Last-minute update of total elapsed time
+#
+sub structured_report_update_time($) {
+ my $report_ref = $_[0];
+ if ($report_ref->{elapsed}) {
+ # just Total, does not adjust $report_ref->{elapsed}{Amavis}
+ $report_ref->{elapsed}{Total} =
+ Amavis::JSON::numeric(sprintf("%.3f", get_time_so_far()));
+ }
+ $report_ref;
+}
+
+sub build_and_save_structured_report($;$) {
+ my($msginfo, $notification_type) = @_;
+ if ($redis_storage &&
+ $redis_logging_queue_size_limit && c('redis_logging_key') ) {
+ eval { # protect the new code just in case
+ $redis_storage->save_structured_report(
+ structured_report($msginfo, $notification_type),
+ c('redis_logging_key'), $redis_logging_queue_size_limit);
+ 1;
+ } or do {
+ chomp $@; do_log(-1,"structured report failed: %s", $@);
+ };
+ }
+}
+
# Ensure we have $msginfo->$entity defined when we expect we'll need it,
#
sub ensure_mime_entity($) {
@@ -14697,9 +15341,9 @@ sub add_forwarding_header_edits_common($$$$$$) {
if ($allowed_hdrs && $allowed_hdrs->{lc('X-Amavis-Hold')}) {
# discard existing X-Amavis-Hold header field, only allow our own
$hdr_edits->delete_header('X-Amavis-Hold');
- if ($hold ne '') {
+ if (defined $hold && $hold ne '') {
$hdr_edits->add_header('X-Amavis-Hold', $hold);
- do_log(-1, "Inserting header field: X-Amavis-Hold: %s", $hold);
+ do_log(0, "Inserting header field: X-Amavis-Hold: %s", $hold);
}
}
if (c('enable_dkim_verification') &&
@@ -14865,7 +15509,7 @@ sub add_forwarding_header_edits_per_recip($$$$$$$) {
: $blacklisted ? 64
: 0+$spam_level) if $slc ne '';
my $spam_tests = $r->spam_tests;
- $spam_tests = !defined $spam_tests ?'' : join(',',map($$_,@$spam_tests));
+ $spam_tests = !$spam_tests ? '' : join(',',map($$_,@$spam_tests));
# allow header field wrapping at any comma
my $s = $spam_tests; $s =~ s/,/,\n /g;
$full_spam_status = sprintf(
@@ -15112,7 +15756,7 @@ sub add_forwarding_header_edits_per_recip($$$$$$$) {
cr('addr_extension_maps_by_ccat'));
my $ext = !ref($ext_map) ? undef : lookup2(0,$recip,$ext_map);
if ($ext ne '') {
- $ext = $delim . $ext;
+ $ext = substr($delim,0,1) . $ext;
my $orig_extension; my($localpart,$domain) = split_address($recip);
($localpart,$orig_extension) = split_localpart($localpart,$delim)
if c('replace_existing_extension'); # strip existing extension
@@ -15369,7 +16013,7 @@ sub prepare_modified_mail($$$$) {
"do not open unless you know what you are doing");
}
if ($ccm==CC_UNCHECKED) {
- if ($hold ne '') {
+ if (defined $hold && $hold ne '') {
push(@explanation,
"WARNING: NOT CHECKED FOR VIRUSES (mail bomb?):\n $hold");
} elsif ($any_undecipherable) {
@@ -15425,6 +16069,8 @@ sub do_quarantine($$$$;@) {
$quar_msg->rx_time($msginfo->rx_time); # copy the reception time
$quar_msg->log_id($msginfo->log_id); # use the same log_id
$quar_msg->partition_tag($msginfo->partition_tag); # same partition_tag
+ $quar_msg->parent_mail_id($msginfo->mail_id);
+ $quar_msg->mail_id(scalar generate_mail_id());
$quar_msg->conn_obj($msginfo->conn_obj);
$quar_msg->mail_id($msginfo->mail_id); # use the same mail_id
$quar_msg->body_type($msginfo->body_type); # use the same BODY= type
@@ -15527,7 +16173,7 @@ sub do_quarantine($$$$;@) {
one_response_for_all($quar_msg, 0); # check status
if ($n_smtp_resp =~ /^2/ && !$n_dsn_needed) { # ok
@snmp_id = ('Other') if !@snmp_id;
- for (unique_list(@snmp_id)) {
+ for (unique_list(\@snmp_id)) {
snmp_count('QuarMsgs'.$_);
snmp_count( ['QuarMsgsSize'.$_, $quar_msg->msg_size, 'C64'] );
}
@@ -15622,7 +16268,7 @@ sub prepare_header_edits_for_quarantine($) {
$do_tag2_any = 1 if $do_tag2;
$do_kill_any = 1 if $do_kill;
my $spam_tests = $r->spam_tests;
- if (defined $spam_tests) {
+ if ($spam_tests) {
$all_spam_tests{$_} = 1 for split(/,/, join(',',map($$_,@$spam_tests)));
}
}
@@ -16006,8 +16652,11 @@ sub do_notify_and_quarantine($$) {
$notification->rx_time($msginfo->rx_time); # copy the reception time
$notification->log_id($msginfo->log_id); # copy log id
$notification->partition_tag($msginfo->partition_tag); # same partition_tag
+ $notification->parent_mail_id($msginfo->mail_id);
+ $notification->mail_id(scalar generate_mail_id());
$notification->conn_obj($msginfo->conn_obj);
$notification->originating(1);
+ $notification->add_contents_category(CC_CLEAN,0);
$notification->sender($mailfrom_admin);
$notification->sender_smtp($mailfrom_admin_q);
$notification->auth_submitter($mailfrom_admin_q);
@@ -16034,6 +16683,7 @@ sub do_notify_and_quarantine($$) {
my($n_smtp_resp, $n_exit_code, $n_dsn_needed) =
one_response_for_all($notification, 0); # check status
if ($n_smtp_resp =~ /^2/ && !$n_dsn_needed) { # ok
+ build_and_save_structured_report($notification,'NOTIF');
} elsif ($n_smtp_resp =~ /^4/) {
die "temporarily unable to notify admin: $n_smtp_resp";
} else {
@@ -16091,8 +16741,11 @@ sub do_notify_and_quarantine($$) {
$notification->rx_time($msginfo->rx_time); # copy the reception time
$notification->log_id($msginfo->log_id); # copy log id
$notification->partition_tag($msginfo->partition_tag); # same partition
+ $notification->parent_mail_id($msginfo->mail_id);
+ $notification->mail_id(scalar generate_mail_id());
$notification->conn_obj($msginfo->conn_obj);
$notification->originating(1);
+ $notification->add_contents_category(CC_CLEAN,0);
$notification->sender($mailfrom_recip);
$notification->sender_smtp($mailfrom_recip_q);
$notification->auth_submitter($mailfrom_recip_q);
@@ -16130,7 +16783,8 @@ sub do_notify_and_quarantine($$) {
mail_dispatch($notification, 'Notif', 0);
my($n_smtp_resp, $n_exit_code, $n_dsn_needed) =
one_response_for_all($notification, 0); # check status
- if ($n_smtp_resp =~ /^2/ && !$n_dsn_needed) { # ok
+ if ($n_smtp_resp =~ /^2/ && !$n_dsn_needed) { # ok
+ build_and_save_structured_report($notification,'NOTIF');
} elsif ($n_smtp_resp =~ /^4/) {
die "temporarily unable to notify recipient rec: $n_smtp_resp";
} else {
@@ -16172,9 +16826,13 @@ sub get_body_digest($$) {
do_log(-1, "Failed to create a Net::DNS::Resolver object");
$dns_resolver = 0; # defined but false
} else {
- # RFC 2460 (for IPv6) requires that a minimal MTU is 1280 bytes
- my $payload_size = 1280 - 40; # less 40 bytes for a basic IP header
- # RFC 2671, RFC 2671bis - EDNS0, set requestor's UDP payload size
+ # RFC 2460 (for IPv6) requires that a minimal MTU is 1280 bytes,
+ # less 40 bytes for a basic IP header = 1240;
+ # RFC 3226: minimum of 1220 for RFC 2535 compliant servers
+ # RFC 6891: choosing between 1280 and 1410 bytes for IP (v4 or v6)
+ # over Ethernet would be reasonable.
+ my $payload_size = 1220; # a conservative default
+ # RFC 6891 (ex RFC 2671) - EDNS0, set requestor's UDP payload size
$dns_resolver->udppacketsize($payload_size) if $payload_size > 512;
ll(5) && do_log(5, "DNS resolver created, UDP payload size %s, NS: %s",
$dns_resolver->udppacketsize,
@@ -16555,8 +17213,9 @@ sub find_external_programs($) {
for my $f (@{ca('decoders')}) {
next if !defined $f || !ref $f; # empty, skip
my $short_types = $f->[0];
- if (!defined $short_types || (ref $short_types && !@$short_types))
- { undef $f; next }
+ if (!defined $short_types || (ref $short_types && !@$short_types)) {
+ undef $f; next;
+ }
my(@tried, at found); my $any = 0;
for my $d (@$f[2..$#$f]) { # all but the first two elements are programs
# find the program, allow one level of indirection
@@ -17464,7 +18123,8 @@ eval {
kill('HUP',$amavisd_pid) or $! == ESRCH
or die "Can't SIGHUP amavisd[$amavisd_pid]: $!";
my $msg = "Signalling a SIGHUP to a running daemon [$amavisd_pid]";
- do_log(2,"%s",$msg); print STDOUT "$msg\n";
+ do_log(2,"%s",$msg);
+ # print STDOUT "$msg\n";
exit(0);
} elsif ($cmd =~ /^(?:restart|stop)\z/) { # stop or restart
defined $amavisd_pid or die "The amavisd daemon is not running\n";
@@ -17556,9 +18216,11 @@ $spamcontrol_obj->init_pre_chroot if $spamcontrol_obj;
if ($daemonize) { # log warnings and uncaught errors
$SIG{'__DIE__' } =
- sub { if (!$^S) { my $m = $_[0]; chomp($m); do_log(-1,"_DIE: %s", $m) } };
+ sub { return if $^S || !defined $^S;
+ my $m = $_[0]; chomp($m); do_log(-1,"_DIE: %s", $m);
+ };
$SIG{'__WARN__'} =
- sub { my $m = $_[0]; chomp($m); do_log(2,"_WARN: %s",$m) };
+ sub { my $m = $_[0]; chomp($m); do_log(2,"_WARN: %s", $m) };
# use Data::Dumper;
# my $m2 = Carp::longmess(); do_log(2,"%s",Dumper($m2));
}
@@ -17629,11 +18291,14 @@ my(@bind_to);
do_log(2,"bind to %s", join(', ', at listen_sockets));
}
+# better catch and report potential Redis problems early before forking
if ($extra_code_redis && @storage_redis_dsn) {
- # better to catch and report potential Redis problems early before forking
- $redis_storage = Amavis::Redis->new(@storage_redis_dsn);
- $redis_storage->connect;
- undef $redis_storage; # disconnect
+ eval {
+ my $redis_storage_tmp = Amavis::Redis->new(@storage_redis_dsn);
+ $redis_storage_tmp->connect; undef $redis_storage_tmp; 1;
+ } or do {
+ warn "Redis error, starting anyway: $@";
+ };
}
# DESTROY a ZMQ context (if any) of the main process,
@@ -17699,7 +18364,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform $myversion $myhostname
$nanny_details_level);
@@ -17961,7 +18626,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform $myversion $myhostname
$nanny_details_level);
@@ -18219,7 +18884,7 @@ use warnings FATAL => qw(utf8 void);
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
import Amavis::Conf qw($db_home $daemon_chroot_dir);
import Amavis::Util qw(untaint ll do_log);
@@ -18308,7 +18973,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
import Amavis::Util qw(ll do_log);
import Amavis::Conf qw($trim_trailing_space_in_lookup_result_fields);
@@ -18433,7 +19098,7 @@ use warnings FATAL => qw(utf8 void);
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform :confvars c cr ca);
import Amavis::Timing qw(section_time);
@@ -18654,11 +19319,11 @@ BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION
$have_sasl $ldap_sys_default);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
$have_sasl = eval { require Authen::SASL };
import Amavis::Conf qw(:platform :confvars c cr ca);
- import Amavis::Util qw(ll do_log);
+ import Amavis::Util qw(ll do_log do_log_safe);
import Amavis::Timing qw(section_time);
}
@@ -18876,7 +19541,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
import Amavis::Util qw(ll do_log);
import Amavis::Conf qw($trim_trailing_space_in_lookup_result_fields);
@@ -19001,7 +19666,7 @@ BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION
$ldap_sys_default @ldap_attrs @mv_ldap_attrs);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform :confvars c cr ca);
import Amavis::Timing qw(section_time);
@@ -19214,14 +19879,15 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform :confvars c cr ca);
import Amavis::Util qw(ll do_log debug_oneshot dump_captured_log
untaint snmp_counters_init read_file
snmp_count proto_encode proto_decode orcpt_encode
switch_to_my_time switch_to_client_time
- am_id new_am_id add_entropy rmdir_recursively);
+ am_id new_am_id add_entropy rmdir_recursively
+ generate_mail_id);
import Amavis::Lookup qw(lookup lookup2);
import Amavis::Lookup::IP qw(lookup_ip_acl normalize_ip_addr);
import Amavis::Timing qw(section_time);
@@ -19354,6 +20020,8 @@ sub preprocess_policy_query($$) {
$msginfo->rx_time($now);
$msginfo->log_id(am_id());
$msginfo->conn_obj($conn);
+ $msginfo->originating(1);
+ $msginfo->add_contents_category(CC_CLEAN,0);
add_entropy(%$attr_ref);
# amavisd -> amavis-helper protocol query consists of any number of
@@ -19480,7 +20148,8 @@ sub preprocess_policy_query($$) {
# RFC 4648 base64url: 62 -, 63 _
$mail_id =~ m{^ [A-Za-z0-9] [A-Za-z0-9_+-]* ={0,2} \z}xs
or die "Invalid mail_id '$mail_id'";
- $msginfo->mail_id(untaint($mail_id));
+ $msginfo->parent_mail_id(untaint($mail_id));
+ $msginfo->mail_id(scalar generate_mail_id());
if (!exists($attr_ref->{'secret_id'}) || $attr_ref->{'secret_id'} eq '') {
die "Secret_id is required, but missing" if c('auth_required_release');
} else {
@@ -19579,8 +20248,8 @@ sub preprocess_policy_query($$) {
} else { # mail checking or releasing from a file
do_log(5, "preprocess_policy_query: opening mail '%s'", $fname);
# set new amavis message id
- new_am_id( ($fname =~ m{amavis-(milter-)?([^/ \t]+)}s ? $2 : undef) )
- if !$releasing;
+ new_am_id( ($fname =~ m{amavis-(milter-)?([^/ \t]+)}s ? $2 : undef),
+ $Amavis::child_invocation_count ) if !$releasing;
# file created by amavis helper program or other client, just open it
my(@stat_list) = lstat($fname); my $errn = @stat_list ? 0 : 0+$!;
if ($errn == ENOENT) { die "File $fname does not exist" }
@@ -19608,13 +20277,13 @@ sub preprocess_policy_query($$) {
$msginfo->mail_text($fh); # save file handle to object
$msginfo->log_id(am_id());
}
- if ($ampdp) {
- do_log(1, "Request: %s %s %s: %s -> %s", $attr_ref->{'request'},
+ if ($ampdp && ll(3)) {
+ do_log(3, "Request: %s %s %s: %s -> %s", $attr_ref->{'request'},
$attr_ref->{'mail_id'}, $msginfo->mail_tempdir,
$msginfo->sender_smtp,
join(',', map($_->recip_addr_smtp, @recips)) );
} else {
- do_log(1, "Request: %s(%s): %s %s %s: %s[%s] <%s> -> <%s>",
+ do_log(3, "Request: %s(%s): %s %s %s: %s[%s] <%s> -> <%s>",
@$attr_ref{qw(request protocol_state mail_id protocol_name
queue_id client_name client_address sender recipient)});
}
@@ -19642,11 +20311,12 @@ sub check_ampdp_policy($$$$) {
# treat unknown client IP addr as 0.0.0.0, from "This" Network, RFC 1700
$cl_ip_tmp = '0.0.0.0' if !defined($cl_ip) || $cl_ip eq '';
my(@cp) = @{ca('client_ipaddr_policy')};
- do_log(-1,"\@client_ipaddr_policy must contain pairs, ".
- "number of elements is not even") if @cp % 2 != 0;
+ do_log(-1,'@client_ipaddr_policy must contain pairs, '.
+ 'number of elements is not even') if @cp % 2 != 0;
+ my $labeler = Amavis::Lookup::Label->new('client_ipaddr_policy');
while (@cp) {
my $lookup_table = shift(@cp); my $policy_name = shift(@cp);
- if (lookup_ip_acl($cl_ip_tmp, $lookup_table)) {
+ if (lookup_ip_acl($cl_ip_tmp, $labeler, $lookup_table)) {
if (defined $policy_name && $policy_name ne '') {
$policy_name_requested = $policy_name;
$cl_ip_mynets = 1 if $policy_name eq 'MYNETS'; # compatibility
@@ -19693,10 +20363,10 @@ sub check_ampdp_policy($$$$) {
} elsif ($msginfo->client_delete) {
do_log(4, "AM.PDP: deletion of %s is client's responsibility", $tempdir);
} elsif ($preserve_evidence) {
- do_log(-1,"AM.PDP: tempdir is to be PRESERVED: %s", $tempdir);
+ do_log(-1,'AM.PDP: tempdir is to be PRESERVED: %s', $tempdir);
} else {
my $fname = $msginfo->mail_text_fn;
- do_log(4, "AM.PDP: tempdir and file being removed: %s, %s",
+ do_log(4, 'AM.PDP: tempdir and file being removed: %s, %s',
$tempdir,$fname);
unlink($fname) or die "Can't remove file $fname: $!" if $fname ne '';
# must step out of the directory which is about to be deleted,
@@ -19816,7 +20486,7 @@ sub check_ampdp_policy($$$$) {
push(@response, proto_encode('return_value','continue'));
}
push(@response, proto_encode('exit_code',sprintf("%d",$exit_code)));
- ll(2) && do_log(2, "mail checking ended: %s", join("\n", at response));
+ ll(3) && do_log(3, 'mail checking ended: %s', join("\n", at response));
dump_captured_log(1, c('enable_log_capture_dump'));
%current_policy_bank = %baseline_policy_bank; # restore bank settings
@response;
@@ -19854,6 +20524,7 @@ sub dispatch_from_quarantine($$$) {
push(@response, proto_encode('setreply','250','2.5.0',
"No recipients, nothing to do"));
} else {
+ Amavis::build_and_save_structured_report($msginfo,'SEND');
for my $r (@$per_recip_data) {
local($1,$2,$3); my($smtp_s,$smtp_es,$msg);
my $resp = $r->recip_smtp_response;
@@ -19885,7 +20556,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform :confvars c cr ca);
import Amavis::Util qw(ll do_log do_log_safe untaint
@@ -20193,13 +20864,19 @@ sub process_smtp_request($$$$) {
my(%baseline_policy_bank) = %current_policy_bank;
$conn->appl_proto($self->{proto} = $lmtp ? 'LMTP' : 'SMTP');
+ my $final_oversized_destiny_all_pass = 1;
+ my $oversized_fd_map_ref =
+ setting_by_given_contents_category(CC_OVERSIZED,
+ cr('final_destiny_maps_by_ccat'));
+ my $oversized_lovers_map_ref =
+ setting_by_given_contents_category(CC_OVERSIZED,
+ cr('lovers_maps_by_ccat'));
# system-wide message size limit, if any
- my $final_oversized_destiny = setting_by_given_contents_category(
- CC_OVERSIZED, cr('final_destiny_by_ccat'));
my $message_size_limit = c('smtpd_message_size_limit');
if ($enforce_smtpd_message_size_limit_64kb_min &&
- $message_size_limit && $message_size_limit < 65536)
- { $message_size_limit = 65536 } # RFC 5321 requires at least 64k
+ $message_size_limit && $message_size_limit < 65536) {
+ $message_size_limit = 65536; # RFC 5321 requires at least 64k
+ }
my $smtpd_greeting_banner_tmp = c('smtpd_greeting_banner');
$smtpd_greeting_banner_tmp =~
s{ \$ (?: \{ ([^\}]+) \} |
@@ -20247,18 +20924,22 @@ sub process_smtp_request($$$$) {
# do_log(2,"%s",$msg); $self->smtp_resp(1,"421 4.3.0 ".$msg); #flush!
# $terminating=1; last;
# };
+
$tls_security_level && lc($tls_security_level) ne 'may' &&
!$self->{ssl_active} && !/^(?:NOOP|EHLO|STARTTLS|QUIT)\z/ && do {
$self->smtp_resp(1,"530 5.7.0 Must issue a STARTTLS command first",
1,$cmd);
last;
};
+
# lc($tls_security_level) eq 'verify' && !/^QUIT\z/ && do {
# $self->smtp_resp(1,"554 5.7.0 Command refused due to lack of security",
# 1,$cmd);
# last;
# };
+
/^NOOP\z/ && do { $self->smtp_resp(1,"250 2.0.0 Ok $_"); last }; #flush!
+
/^QUIT\z/ && do {
if ($args ne '') {
$self->smtp_resp(1,"501 5.5.4 Error: QUIT does not accept arguments",
@@ -20281,10 +20962,12 @@ sub process_smtp_request($$$$) {
}
last;
};
+
/^(?:RSET|HELO|EHLO|LHLO|STARTTLS)\z/ && do {
# explicit or implicit session reset
$sender_unq = $sender_quo = undef; @recips = (); $got_rcpt = 0;
undef $max_recip_size_limit; undef $msginfo; # forget previous
+ $final_oversized_destiny_all_pass = 1;
%current_policy_bank = %baseline_policy_bank; # restore bank settings
%xforward_args = ();
if (/^(?:RSET|STARTTLS)\z/ && $args ne '') {
@@ -20348,6 +21031,7 @@ sub process_smtp_request($$$$) {
};
last;
};
+
/^XFORWARD\z/ && do { # Postfix extension
if (defined $sender_unq) {
$self->smtp_resp(1,"503 5.5.1 Error: XFORWARD not allowed ".
@@ -20381,11 +21065,13 @@ sub process_smtp_request($$$$) {
$self->smtp_resp(1,"250 2.5.0 Ok $_") if !$bad;
last;
};
+
/^HELP\z/ && do {
$self->smtp_resp(0,"214 2.0.0 See $myproduct_name home page at:\n".
"http://www.ijs.si/software/amavisd/");
last;
};
+
/^AUTH\z/ && @{ca('auth_mech_avail')} && do { # RFC 4954 (ex RFC 2554)
if ($args !~ /^([^ ]+)(?: ([^ ]*))?\z/is) {
$self->smtp_resp(1,"501 5.5.2 Syntax: AUTH mech [initresp]",1,$cmd);
@@ -20446,6 +21132,7 @@ sub process_smtp_request($$$$) {
}
last;
};
+
/^VRFY\z/ && do {
if ($args eq '') {
$self->smtp_resp(1,"501 5.5.2 Syntax: VRFY address", 1,$cmd); #flush!
@@ -20454,6 +21141,7 @@ sub process_smtp_request($$$$) {
}
last;
};
+
/^MAIL\z/ && do { # begin new SMTP transaction
if (defined $sender_unq) {
$self->smtp_resp(1,"503 5.5.1 Error: nested MAIL command", 1, $cmd);
@@ -20497,9 +21185,10 @@ sub process_smtp_request($$$$) {
my(@cp) = @{ca('client_ipaddr_policy')};
do_log(-1,"\@client_ipaddr_policy must contain pairs, ".
"number of elements is not even") if @cp % 2 != 0;
+ my $labeler = Amavis::Lookup::Label->new('client_ipaddr_policy');
while (@cp) {
my $lookup_table = shift(@cp); my $policy_name = shift(@cp);
- if (lookup_ip_acl($cl_ip_tmp, $lookup_table)) {
+ if (lookup_ip_acl($cl_ip_tmp, $labeler, $lookup_table)) {
if (defined $policy_name && $policy_name ne '') {
$policy_name_requested = $policy_name;
$cl_ip_mynets = 1 if $policy_name eq 'MYNETS'; # compatibility
@@ -20594,14 +21283,6 @@ sub process_smtp_request($$$$) {
if (!defined($msg) && defined $dsn_ret && $dsn_ret!~/^(FULL|HDRS)\z/) {
$msg = "501 5.5.4 Syntax error in MAIL parameter RET: $dsn_ret";
}
- if (!defined($msg) && defined($size) &&
- $message_size_limit && $size > $message_size_limit &&
- $final_oversized_destiny == D_REJECT) {
- $msg = "552 5.3.4 Declared message size ($size B) ".
- "exceeds fixed size limit";
- $msg_nopenalize = 1;
- do_log(0, "%s REJECT 'MAIL FROM': %s", $self->{proto},$msg);
- }
if (!defined $msg) {
$sender_quo = $addr; $sender_unq = unquote_rfc2821_local($addr);
$addr = $1 if $addr =~ /^<(.*)>\z/s;
@@ -20634,6 +21315,7 @@ sub process_smtp_request($$$$) {
$self->smtp_resp(0,$msg, !$msg_nopenalize && $msg=~/^5/ ? 1 : 0, $cmd);
last;
};
+
/^RCPT\z/ && do {
if (!defined($sender_unq)) {
$self->smtp_resp(1,"503 5.5.1 Need MAIL command before RCPT",1,$cmd);
@@ -20650,7 +21332,7 @@ sub process_smtp_request($$$$) {
$self->smtp_resp(0,"501 5.5.2 Syntax: RCPT TO:<address>",1,$cmd);
last;
}
- my($addr,$opt) = ($1,$2); my($notify,$orcpt);
+ my($addr_smtp,$opt) = ($1,$2); my($notify,$orcpt);
my $msg; my $msg_nopenalize = 0;
for (split(' ',$opt)) {
if (!/^ ( [A-Za-z0-9] [A-Za-z0-9-]* ) =
@@ -20673,29 +21355,33 @@ sub process_smtp_request($$$$) {
}
last if defined $msg;
}
- my $addr_unq = unquote_rfc2821_local($addr);
- my $requoted = qquote_rfc2821_local($addr_unq);
- if ($requoted ne $addr) { # check for valid canonical quoting
+ my $addr = unquote_rfc2821_local($addr_smtp);
+ my $requoted = qquote_rfc2821_local($addr);
+ if ($requoted ne $addr_smtp) { # check for valid canonical quoting
do_log(0, "WARN: address modified (recip): %s -> %s",
- $addr, $requoted);
+ $addr_smtp, $requoted);
# RFC 3461: If no ORCPT parameter was present in the RCPT command
# when the message was received, an ORCPT parameter MAY be added
# to the RCPT command when the message is relayed. If an ORCPT
# parameter is added by the relaying MTA, it MUST contain the
# recipient address from the RCPT command used when the message
# was received by that MTA
- $orcpt = orcpt_encode($addr) if !defined $orcpt;
+ $orcpt = orcpt_encode($addr_smtp) if !defined $orcpt;
}
- if (lookup2(0,$addr_unq, ca('debug_recipient_maps'))) {
+ if (lookup2(0,$addr, ca('debug_recipient_maps'))) {
debug_oneshot(1, $self->{proto} . "< $cmd");
}
- my $recip_size_limit; my $mslm = ca('message_size_limit_maps');
- $recip_size_limit = lookup2(0,$addr_unq,$mslm) if @$mslm;
- if ($enforce_smtpd_message_size_limit_64kb_min &&
- $recip_size_limit && $recip_size_limit < 65536)
- { $recip_size_limit = 65536 } # RFC 5321 requires at least 64k
- if ($recip_size_limit > $max_recip_size_limit)
- { $max_recip_size_limit = $recip_size_limit }
+ my $mslm = ca('message_size_limit_maps');
+ my $recip_size_limit;
+ $recip_size_limit = lookup2(0,$addr,$mslm) if @$mslm;
+ if ($recip_size_limit) {
+ # RFC 5321 requires at least 64k
+ $recip_size_limit = 65536
+ if $recip_size_limit < 65536 &&
+ $enforce_smtpd_message_size_limit_64kb_min;
+ $max_recip_size_limit = $recip_size_limit
+ if $recip_size_limit > $max_recip_size_limit;
+ }
my $mail_size = $msginfo->msg_size;
if (!defined($msg) && defined($notify)) {
my(@v) = split(/,/,uc($notify),-1);
@@ -20711,35 +21397,61 @@ sub process_smtp_request($$$$) {
}
$notify = \@v; # replace a string with a listref of items
}
- if (!defined($msg) && defined($mail_size) &&
- $recip_size_limit && $mail_size > $recip_size_limit &&
- $final_oversized_destiny == D_REJECT) {
- $msg = "552 5.3.4 Declared message size ($mail_size B) ".
- "exceeds size limit for recipient $addr";
- $msg_nopenalize = 1;
- do_log(0, "%s REJECT 'RCPT TO': %s", $self->{proto},$msg);
+ if (!defined($msg) && $recip_size_limit) {
+ # check mail size if known, update $final_oversized_destiny_all_pass
+ my $fd = !ref $oversized_fd_map_ref ? $oversized_fd_map_ref # compat
+ : lookup2(0, $addr, $oversized_fd_map_ref, Label => 'Destiny4');
+ if (!defined $fd || $fd == D_PASS) {
+ $fd = D_PASS; # keep D_PASS
+ } elsif (defined($oversized_lovers_map_ref) &&
+ lookup2(0, $addr, $oversized_lovers_map_ref,
+ Label => 'Lovers4')) {
+ $fd = D_PASS; # D_PASS for oversized lovers
+ } else { # $fd != D_PASS, blocked if oversized
+ if ($final_oversized_destiny_all_pass) {
+ $final_oversized_destiny_all_pass = 0; # not PASS for all recips
+ do_log(5, 'Not a D_PASS on oversized for all recips: %s', $addr);
+ }
+ }
+ # check declared mail size here if known, otherwise we'll check
+ # the actual mail size after the message is received
+ if (defined $mail_size && $mail_size > $recip_size_limit) {
+ $msg = $fd == D_TEMPFAIL ? '452 4.3.4' :
+ $fd == D_PASS ? '250 2.3.4' : '552 5.3.4';
+ $msg .= " Declared message size ($mail_size B) ".
+ "exceeds size limit for recipient $addr_smtp";
+ $msg_nopenalize = 1;
+ do_log(0, "%s %s 'RCPT TO': %s", $self->{proto},
+ $fd == D_TEMPFAIL ? 'TEMPFAIL' :
+ $fd == D_PASS ? 'PASS' : 'REJECT',
+ $msg);
+ }
}
if (!defined($msg) && $got_rcpt > $smtpd_recipient_limit) {
$msg = "452 4.5.3 Too many recipients";
}
if (!defined $msg) {
+ $msg = "250 2.1.5 Recipient $addr_smtp OK";
+ }
+ if ($msg =~ /^2/) {
my $recip_obj = Amavis::In::Message::PerRecip->new;
- $recip_obj->recip_addr($addr_unq);
- $recip_obj->recip_addr_smtp($addr);
+ $recip_obj->recip_addr($addr);
+ $recip_obj->recip_addr_smtp($addr_smtp);
$recip_obj->recip_destiny(D_PASS); # default is Pass
$recip_obj->dsn_notify($notify) if defined $notify;
$recip_obj->dsn_orcpt($orcpt) if defined $orcpt;
push(@recips,$recip_obj);
- $msg = "250 2.1.5 Recipient $addr OK";
}
$self->smtp_resp(0,$msg, !$msg_nopenalize && $msg=~/^5/ ? 1 : 0, $cmd);
last;
};
+
/^DATA\z/ && $args ne '' && do {
$self->smtp_resp(1,"501 5.5.4 Error: DATA does not accept arguments",
1,$cmd); #flush
last;
};
+
/^DATA\z/ && !@recips && do {
if (!defined($sender_unq)) {
$self->smtp_resp(1,"503 5.5.1 Need MAIL command before DATA",1,$cmd);
@@ -20754,11 +21466,13 @@ sub process_smtp_request($$$$) {
}
last;
};
+
# /^DATA\z/ && uc($msginfo->body_type) eq "BINARYMIME" && do { # RFC 3030
# $self->smtp_resp(1,"503 5.5.1 DATA is incompatible with BINARYMIME",
# 0,$cmd); #flush!
# last;
# };
+
/^DATA\z/ && do {
# set timer to the initial value, MTA timer starts here
if ($message_size_limit) { # enforce system-wide size limit
@@ -20785,7 +21499,7 @@ sub process_smtp_request($$$$) {
' SIZE='.$msginfo->msg_size,
!defined $msginfo->body_type ? () : ' BODY='.$msginfo->body_type,
!defined $msginfo->auth_submitter ||
- $msginfo->auth_submitter eq '<>' ? ():
+ $msginfo->auth_submitter eq '<>' ? () :
' AUTH='.$msginfo->auth_submitter,
!defined $msginfo->dsn_ret ? () : ' RET='.$msginfo->dsn_ret,
!defined $msginfo->dsn_envid ? () :
@@ -20804,14 +21518,14 @@ sub process_smtp_request($$$$) {
my $fh = $self->{tempdir}->fh;
# the copy_smtp_data() will use syswrite, flush buffer just in case
if ($fh) { $fh->flush or die "Can't flush mail file: $!" }
- if (!$max_recip_size_limit || $final_oversized_destiny == D_PASS) {
- # no message size limit enforced
+ if (!$max_recip_size_limit || $final_oversized_destiny_all_pass) {
+ # no message size limit enforced, faster
($size,$oversized) = $self->copy_smtp_data($fh, \$out_str, undef);
} else { # enforce size limit
do_log(5,"enforcing size limit %s during DATA",
$max_recip_size_limit);
- ($size,$oversized) =
- $self->copy_smtp_data($fh, \$out_str, $max_recip_size_limit);
+ ($size,$oversized) = $self->copy_smtp_data($fh, \$out_str,
+ $max_recip_size_limit);
};
switch_to_my_time('rx data-end');
$complete = !$self->{within_data_transfer};
@@ -20831,7 +21545,7 @@ sub process_smtp_request($$$$) {
$eval_stat = $@ ne '' ? $@ : "errno=$!";
};
if ( defined $eval_stat || !$complete || # err or connection broken
- ($oversized && $final_oversized_destiny == D_REJECT) ) {
+ ($oversized && !$final_oversized_destiny_all_pass) ) {
chomp $eval_stat if defined $eval_stat;
# on error, either send: '421 Shutting down',
# or: '451 Aborted, error in processing' and NOT shut down!
@@ -20944,6 +21658,7 @@ sub process_smtp_request($$$$) {
my $sa_rusage = $msginfo->supplementary_info('RUSAGE-SA');
$sender_unq = $sender_quo = undef; @recips = (); $got_rcpt = 0;
undef $max_recip_size_limit; undef $msginfo; # forget previous
+ $final_oversized_destiny_all_pass = 1;
%xforward_args = ();
section_time('dump_captured_log') if log_capture_enabled();
dump_captured_log(1, c('enable_log_capture_dump'));
@@ -20956,7 +21671,7 @@ sub process_smtp_request($$$$) {
if ($sa_rusage && @$sa_rusage) {
local $1; my $sa_cpu_sum = 0; $sa_cpu_sum += $_ for @$sa_rusage;
$am_timing_report =~ # ugly hack
- s{cpu ([0-9.]+) ms\]}
+ s{\bcpu ([0-9.]+) ms\]}
{sprintf("cpu %s ms, AM-cpu %.0f ms, SA-cpu %.0f ms]",
$1, $1 - $sa_cpu_sum*1000, $sa_cpu_sum*1000) }se;
}
@@ -20968,6 +21683,7 @@ sub process_smtp_request($$$$) {
$Amavis::last_task_completed_at = Time::HiRes::time;
last;
}; # DATA
+
/^(?:EXPN|TURN|ETRN|SEND|SOML|SAML)\z/ && do {
$self->smtp_resp(1,"502 5.5.1 Error: command $_ not implemented",
0,$cmd);
@@ -21100,7 +21816,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform);
import Amavis::Util qw(ll do_log min max minmax);
@@ -21300,7 +22016,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&rundown_stale_sessions);
import Amavis::Conf qw(:platform c cr ca $smtp_connection_cache_enable);
@@ -21664,7 +22380,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
@EXPORT = qw(&mail_via_smtp);
import Amavis::Conf qw(:platform c cr ca $smtp_connection_cache_enable);
@@ -21709,9 +22425,9 @@ sub enhance_smtp_response($$$$$) {
$smtp_resp = sprintf('No resp. to %s', $cmd_name);
} elsif ($smtp_resp !~ /^[245]\d{2}/) {
$smtp_resp = sprintf('Bad resp. to %s: %s', $cmd_name,$smtp_resp);
- } elsif ($smtp_resp=~/^ (\d{3}) (\ |-|\z) [ \t]*
- ([245] \. \d{1,3} \. \d{1,3})?
- \s* (.*) \z/xs) {
+ } elsif ($smtp_resp =~ /^ (\d{3}) (\ |-|\z) [ \t]*
+ ([245] \. \d{1,3} \. \d{1,3})?
+ \s* (.*) \z/xs) {
($resp_code, $resp_more, $resp_enhcode, $resp_msg) = ($1, $2, $3, $4);
if (!defined $resp_enhcode && $resp_code =~ /^[245]/) {
my $c = substr($resp_code,0,1);
@@ -21738,8 +22454,11 @@ sub mail_via_smtp(@) {
# needs it can be any of: false: 0
# or true: 'Quar', 'Dsn', 'Notif', 'AV', 'Arf'
my $which_section = 'fwd_init';
- my $logmsg = sprintf("%s from %s", $initial_submission?'SEND':'FWD',
- $msginfo->sender_smtp);
+ my $id = $msginfo->parent_mail_id;
+ $id = $msginfo->mail_id . (defined $id ? "($id)" : "");
+ my $logmsg = sprintf("%s %s from %s", $id,
+ ($initial_submission ? 'SEND' : 'FWD'),
+ $msginfo->sender_smtp);
my($per_recip_data_ref, $proto_sockname) =
collect_equal_delivery_recips($msginfo, $filter, qr/^(?:smtp|lmtp):/i);
if (!$per_recip_data_ref || !@$per_recip_data_ref) {
@@ -22349,7 +23068,7 @@ sub mail_via_smtp(@) {
# but a value of 'AV' is supplied by av_smtp_client to allow a forwarding
# method to distinguish it from ordinary submissions
my $ll = ($smtp_response =~ /^2/ || $initial_submission eq 'AV') ? 1 : -1;
- ll($ll) && do_log($ll, "%s -> %s,%s %s", $logmsg,
+ ll($ll) && do_log($ll, "%s -> %s, %s %s", $logmsg,
join(',', qquote_rfc2821_local(
map($_->recip_final_addr, @per_recip_data))),
join(' ', map { my $v=$from_options{$_}; defined($v)?"$_=$v":"$_" }
@@ -22399,7 +23118,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
@EXPORT = qw(&mail_via_pipe);
import Amavis::Conf qw(:platform c cr ca);
@@ -22428,7 +23147,10 @@ sub mail_via_pipe(@) {
: ('', 'Submit', 'ProtoPipe', 'ProtoPipeSubmit',
'Submit'.$initial_submission);
snmp_count('OutMsgs'.$_) for @snmp_vars;
- my $logmsg = sprintf("%s via PIPE: %s", ($initial_submission?'SEND':'FWD'),
+ my $id = $msginfo->parent_mail_id;
+ $id = $msginfo->mail_id . (defined $id ? "($id)" : "");
+ my $logmsg = sprintf("%s %s via PIPE from %s", $id,
+ ($initial_submission ? 'SEND' : 'FWD'),
$msginfo->sender_smtp);
my($per_recip_data_ref, $proto_sockname) =
collect_equal_delivery_recips($msginfo, $filter, qr/^pipe:/i);
@@ -22600,7 +23322,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
@EXPORT = qw(&mail_via_bsmtp);
import Amavis::Conf qw(:platform $QUARANTINEDIR c cr ca);
@@ -22798,7 +23520,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&mail_to_local_mailbox);
import Amavis::Conf qw(:platform c cr ca
@@ -23118,10 +23840,11 @@ sub mail_to_local_mailbox(@) {
$failed = 1;
die $eval_stat if $eval_stat =~ /^timed out\b/; # resignal timeout
};
- if ($ux) {
- $mp->flush or die "Can't flush mailbox file $mbxname: $!";
- flock($mp,LOCK_UN) or die "Can't unlock mailbox $mbxname: $!";
- }
+ # if ($ux) {
+ # # explicit unlocking is unnecessary, close will do a flush & unlock
+ # $mp->flush or die "Can't flush mailbox file $mbxname: $!";
+ # flock($mp,LOCK_UN) or die "Can't unlock mailbox $mbxname: $!";
+ # }
$mp->close or die "Error closing $mbxname: $!";
undef $mp;
if (!$failed) {
@@ -23165,7 +23888,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform);
import Amavis::Util qw(ll do_log);
@@ -23260,6 +23983,207 @@ sub collect_response {
__DATA__
#^L
+package Amavis::TinyRedis;
+
+use strict;
+use re 'taint';
+use warnings;
+
+use Errno qw(EINTR EAGAIN EPIPE ENOTCONN ECONNRESET ECONNABORTED);
+use IO::Socket::UNIX;
+use Time::HiRes ();
+
+use vars qw($VERSION);
+BEGIN {
+ $VERSION = '1.000';
+ import Amavis::Conf qw(:platform); # $io_socket_module_name
+}
+
+sub new {
+ my($class, %args) = @_;
+ my $self = bless { args => {%args} }, $class;
+ my $outbuf = ''; $self->{outbuf} = \$outbuf;
+ $self->{batch_size} = 0;
+ $self->{server} = $args{server} || $args{sock} || '127.0.0.1:6379';
+ $self->{on_connect} = $args{on_connect};
+ return if !$self->connect;
+ $self;
+}
+
+sub DESTROY {
+ my $self = $_[0];
+ local($@, $!, $_);
+ undef $self->{sock};
+}
+
+sub disconnect {
+ my $self = $_[0];
+ local($@, $!);
+ undef $self->{sock};
+}
+
+sub connect {
+ my $self = $_[0];
+
+ $self->disconnect;
+ my $sock;
+ my $server = $self->{server};
+ if ($server =~ m{^/}) {
+ $sock = IO::Socket::UNIX->new(
+ Peer => $server, Type => SOCK_STREAM);
+ } else {
+ $sock = $io_socket_module_name->new(
+ PeerAddr => $server, Proto => 'tcp');
+ }
+ if ($sock) {
+ $self->{sock} = $sock;
+
+ $self->{sock_fd} = $sock->fileno; $self->{fd_mask} = '';
+ vec($self->{fd_mask}, $self->{sock_fd}, 1) = 1;
+
+ # an on_connect() callback must not use batched calls!
+ $self->{on_connect}->($self) if $self->{on_connect};
+ }
+ $sock;
+}
+
+# Receive, parse and return $cnt consecutive redis replies as a list.
+#
+sub _response {
+ my($self, $cnt) = @_;
+
+ my $sock = $self->{sock};
+ if (!$sock) {
+ $self->connect or die "Connect failed: $!";
+ $sock = $self->{sock};
+ };
+
+ my @list;
+
+ for (1 .. $cnt) {
+
+ my $result = <$sock>;
+ if (!defined $result) {
+ $self->disconnect;
+ die "Error reading from Redis server: $!";
+ }
+ chomp $result;
+ my $resp_type = substr($result, 0, 1, '');
+
+ if ($resp_type eq '$') { # bulk reply
+ if ($result < 0) {
+ push(@list, undef); # null bulk reply
+ } else {
+ my $data = ''; my $ofs = 0; my $len = $result + 2;
+ while ($len > 0) {
+ my $nbytes = read($sock, $data, $len, $ofs);
+ if (!$nbytes) {
+ $self->disconnect;
+ defined $nbytes or die "Error reading from Redis server: $!";
+ die "Redis server closed connection";
+ }
+ $ofs += $nbytes; $len -= $nbytes;
+ }
+ chomp $data;
+ push(@list, $data);
+ }
+
+ } elsif ($resp_type eq ':') { # integer reply
+ push(@list, 0+$result);
+
+ } elsif ($resp_type eq '+') { # status reply
+ push(@list, $result);
+
+ } elsif ($resp_type eq '*') { # multi-bulk reply
+ push(@list, $result < 0 ? undef : $self->_response(0+$result) );
+
+ } elsif ($resp_type eq '-') { # error reply
+ die "$result\n";
+
+ } else {
+ die "Unknown Redis reply: $resp_type ($result)";
+ }
+ }
+ \@list;
+}
+
+sub _write_buff {
+ my($self, $bufref) = @_;
+
+ if (!$self->{sock}) { $self->connect or die "Connect failed: $!" };
+ my $nwrite;
+ for (my $ofs = 0; $ofs < length($$bufref); $ofs += $nwrite) {
+ # to reliably detect a disconnect we need to check for an input event
+ # using a select; checking status of syswrite is not sufficient
+ my($rout, $wout, $inbuff); my $fd_mask = $self->{fd_mask};
+ my $nfound = select($rout=$fd_mask, $wout=$fd_mask, undef, undef);
+ defined $nfound && $nfound >= 0 or die "Select failed: $!";
+ if (vec($rout, $self->{sock_fd}, 1) &&
+ !sysread($self->{sock}, $inbuff, 1024)) {
+ # eof, try reconnecting
+ $self->connect or die "Connect failed: $!";
+ }
+ local $SIG{PIPE} = 'IGNORE'; # don't signal on a write to a widowed pipe
+ $nwrite = syswrite($self->{sock}, $$bufref, length($$bufref)-$ofs, $ofs);
+ next if defined $nwrite;
+ $nwrite = 0;
+ if ($! == EINTR || $! == EAGAIN) { # no big deal, try again
+ Time::HiRes::sleep(0.1); # slow down, just in case
+ } else {
+ $self->disconnect;
+ if ($! == ENOTCONN || $! == EPIPE ||
+ $! == ECONNRESET || $! == ECONNABORTED) {
+ $self->connect or die "Connect failed: $!";
+ } else {
+ die "Error writing to redis socket: $!";
+ }
+ }
+ }
+ 1;
+}
+
+# Send a redis command with arguments, returning a redis reply.
+#
+sub call {
+ my $self = shift;
+
+ my $buff = '*' . scalar(@_) . "\015\012";
+ $buff .= '$' . length($_) . "\015\012" . $_ . "\015\012" for @_;
+
+ $self->_write_buff(\$buff);
+ local($/) = "\015\012";
+ my $arr_ref = $self->_response(1);
+ $arr_ref && $arr_ref->[0];
+}
+
+# Append a redis command with arguments to a batch.
+#
+sub b_call {
+ my $self = shift;
+
+ my $bufref = $self->{outbuf};
+ $$bufref .= '*' . scalar(@_) . "\015\012";
+ $$bufref .= '$' . length($_) . "\015\012" . $_ . "\015\012" for @_;
+ ++ $self->{batch_size};
+}
+
+# Send a batch of commands, returning an arrayref of redis replies,
+# each array element corresponding to one command in a batch.
+#
+sub b_results {
+ my $self = $_[0];
+ my $batch_size = $self->{batch_size};
+ return if !$batch_size;
+ my $bufref = $self->{outbuf};
+ $self->_write_buff($bufref);
+ $$bufref = ''; $self->{batch_size} = 0;
+ local($/) = "\015\012";
+ $self->_response($batch_size);
+}
+
+1;
+
+
package Amavis::Redis;
use strict;
use re 'taint';
@@ -23270,57 +24194,94 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform :confvars c cr ca);
import Amavis::rfc2821_2822_Tools;
- import Amavis::Util qw(ll do_log do_log_safe min max minmax untaint);
+ import Amavis::Util qw(ll do_log do_log_safe min max minmax untaint
+ safe_encode_utf8 format_time_interval unique_list
+ snmp_count);
+ import Amavis::Lookup::IP qw(lookup_ip_acl normalize_ip_addr);
+ import Amavis::Timing qw(section_time);
}
-use Redis;
-
sub new {
- my($class, @storage_redis_dsn) = @_;
- bless { redis_dsn => \@storage_redis_dsn }, $class;
+ my($class, @redis_dsn) = @_;
+ bless { redis_dsn => \@redis_dsn }, $class;
}
sub disconnect {
my $self = $_[0];
# do_log(5, "redis: disconnect");
- undef $self->{redis}; $self->{connected} = 0;
+ $self->{connected} = 0; undef $self->{redis};
+}
+
+sub on_connect {
+ my($self, $r) = @_;
+ my $db_id = $self->{db_id} || 0;
+ do_log(5, "redis: on_connect, db_id %d", $db_id);
+ eval {
+ $r->call('SELECT', $db_id) eq 'OK' ? 1 : 0;
+ } or do {
+ if ($@ =~ /\bNOAUTH\b/) {
+ defined $self->{password}
+ or die "Redis server requires authentication, no password provided";
+ $r->call('AUTH', $self->{password});
+ $r->call('SELECT', $db_id);
+ } else {
+ chomp $@; die "Redis error: $@";
+ }
+ };
+ $r->call('CLIENT', 'SETNAME', 'amavis['.$$.']');
+ 1;
}
sub connect {
my $self = $_[0];
# do_log(5, "redis: connect");
$self->disconnect if $self->{connected};
+ $self->{redis} = $self->{db_id} = $self->{ttl} = undef;
- my($r, $dsn, $db_index, $ttl, $err); undef $self->{redis};
+ my($r, $err, $dsn, %options);
my $dsn_list_ref = $self->{redis_dsn};
for my $j (1 .. @$dsn_list_ref) {
$dsn = $dsn_list_ref->[0];
- my %options = ref $dsn eq 'HASH' ? %$dsn : ();
+ %options = ref $dsn eq 'HASH' ? %$dsn : ();
# expiration time (time-to-live) is 16 days by default
- $ttl = exists $options{ttl} ? delete $options{ttl} : $storage_redis_ttl;
- $db_index = delete $options{db_id};
+ $self->{ttl} = exists $options{ttl} ? $options{ttl} : $storage_redis_ttl;
+ $self->{db_id} = $options{db_id};
+ if (defined $options{password}) {
+ $self->{password} = $options{password};
+ $options{password} = '(hidden)'; # for logging purposes
+ }
undef $err;
- eval { $r = Redis->new(encoding => undef, %options) }
- or do { undef $r; $err = $@; chomp $err; $err =~ s{\s+at /.*}{}s };
- last if $r;
- if ($j < @$dsn_list_ref) {
- do_log(0, "Error connecting to a redis server %s: %s; trying next",
- join(' ',%$dsn), $err);
- push(@$dsn_list_ref, shift @$dsn_list_ref);
- }
- }
- $r or die sprintf("Error connecting to a redis server %s: %s\n",
- join(' ',%$dsn), $err);
- $self->{redis} = $r; $self->{ttl} = $ttl;
- do_log(5, "redis: connected to: %s, db_index %s, ttl %s s",
- !%$dsn ? 'default server' : join(' ',%$dsn),
- $db_index||0, $ttl||'x');
- $r->select($db_index) if $db_index;
+ eval {
+ my %opt = %options; delete @opt{qw(ttl db_id password)};
+ $r = Amavis::TinyRedis->new(on_connect => sub { $self->on_connect(@_) },
+ %opt);
+ $r or die "Error: $!";
+ } or do {
+ undef $r; $err = $@; chomp $err;
+ };
+ $self->{redis} = $r;
+ last if $r; # success, done
+ if ($j < @$dsn_list_ref) { # not all tried yet
+ do_log(0, "Can't connect to a redis server, %s: %s; trying next",
+ join(' ',%options), $err);
+ push(@$dsn_list_ref, shift @$dsn_list_ref); # rotate left
+ }
+ }
+ if (!$r) {
+ $self->{redis} = $self->{db_id} = $self->{ttl} = undef;
+ die sprintf("Can't connect to a redis server %s: %s\n",
+ join(' ',%options), $err);
+ }
$self->{connected} = 1;
+ ll(5) && do_log(5, "redis: connected to: %s, ttl %s s",
+ !defined $options{server} ? 'default server'
+ : join(' ',%options),
+ $self->{ttl}||'x');
+ section_time("redis-connect");
$self->load_lua_programs;
$r;
}
@@ -23328,56 +24289,117 @@ sub connect {
sub DESTROY {
my $self = $_[0]; local($@,$!,$_);
do_log_safe(5,"Amavis::Redis DESTROY called");
+ # ignore potential errors during DESTROY of a Redis object
+ eval { $self->{connected} = 0; undef $self->{redis} };
}
-# find a penpals record which proves that a local user sid really sent a
-# mail to a recipient rid some time ago. Returns an interval time in seconds
+# find a penpals record which proves that a local user (sender) really sent a
+# mail to a given recipient some time ago. Returns an interval time in seconds
# since the last such mail was sent by our local user to a specified recipient
# (or undef if information is not available). If @$message_id_list is a
# nonempty list of Message-IDs as found in References header field, the query
-# also provides previous outgoing messages with a matching Message-ID but
+# also finds previous outgoing messages with a matching Message-ID but
# possibly to recipients different from what the mail was originally sent to.
#
sub penpals_find {
- my($self, $sender, $recip, $message_id_list, $now) = @_;
+ my($self, $msginfo, $message_id_list) = @_;
+
+ my $sender = $msginfo->sender;
+ $message_id_list = [] if !$message_id_list;
+ return if !@$message_id_list && $sender eq '';
+
+ # inbound or internal_to_internal, except self_to_self
+ my(@per_recip_data) = grep(!$_->recip_done && $_->recip_is_local &&
+ lc($sender) ne lc($_->recip_addr),
+ @{$msginfo->per_recip_data});
+ return if !@per_recip_data;
# do_log(5, "redis: penpals_find");
- local($1); $sender =~ s/^<(.*)>\z/$1/s; $recip =~ s/^<(.*)>\z/$1/s;
- $self->connect if !$self->{connected};
+ snmp_count('PenPalsAttempts');
+
+ my $sender_smtp = $msginfo->sender_smtp;
+ local($1); $sender_smtp =~ s/^<(.*)>\z/$1/s;
+ my(@recip_addresses) =
+ map { my $a = $_->recip_addr_smtp; $a =~ s/^<(.*)>\z/$1/s; lc $a }
+ @per_recip_data;
+
+ # NOTE: swap recipient and sender in a query here, as we are
+ # now checking for a potential reply mail - whether the current
+ # recipient has recently sent any mail to the sender of the
+ # current mail:
+
+ # no need for cryptographical strength, just checking for protocol errors
+ my $nonce = $msginfo->mail_id;
my $result;
+ my @args = (
+ 0, sprintf("%.0f",$msginfo->rx_time), $nonce, lc $sender_smtp,
+ scalar @recip_addresses, @recip_addresses,
+ scalar @$message_id_list, @$message_id_list,
+ );
eval {
- $result = $self->{redis}->evalsha($self->{lua_query}, 0,
- lc $sender, lc $recip, @$message_id_list);
+ $self->connect if !$self->{connected};
+ $result =
+ $self->{redis}->call('EVALSHA', $self->{lua_query_penpals}, @args);
1;
- } or do { # Lua program probably not cached, define again and re-try
- $@ =~ s{\s+at /.*}{}s;
- if ($@ !~ /^\Q[evalsha] NOSCRIPT\E/)
- { $self->disconnect; die "Redis LUA error: $@\n" }
- $self->load_lua_programs;
- $result = $self->{redis}->evalsha($self->{lua_query}, 0,
- lc $sender, lc $recip, @$message_id_list);
+ } or do { # Lua function probably not cached, define again and re-try
+ if ($@ !~ /^NOSCRIPT/) {
+ $self->disconnect; undef $result; chomp $@;
+ do_log(-1, "Redis Lua error: %s", $@);
+ } else {
+ $self->load_lua_programs;
+ $result =
+ $self->{redis}->call('EVALSHA', $self->{lua_query_penpals}, @args);
+ }
};
- my($ref_mail_id, $send_time, $sid, $rid, @mid);
- ($ref_mail_id, $send_time, $sid, $rid, @mid) =
- map(!defined $_ || $_ eq '' ? undef : $_, @$result) if ref $result;
-
- my $age;
- if (!defined $ref_mail_id) {
- ll(4) && do_log(4, "penpals: (redis) not found (%s,%s)%s%s",
- defined $sid ? $sid : $sender,
- defined $rid ? $rid : $recip,
- !@mid ? '' : ', refs: '.join(',',map($_||'x', at mid)),
- !@$message_id_list ? '' : '; '.join(', ',@$message_id_list) );
+
+ my $ok = 1;
+ if (!$result || !@$result) {
+ $ok = 0; $self->disconnect;
+ do_log(0, "redis: penpals_find - no results");
} else {
- $age = max(0, $now - $send_time);
- ll(3) && do_log(3, "penpals: (redis) found (%s,%s) %s age %s (%d s)%s",
- defined $sid ? $sid : $sender,
- defined $rid ? $rid : $recip,
- $ref_mail_id,
- format_time_interval($age), $age,
- !@mid ? '' : ', refs: '.join(',',map($_||'x', at mid)) );
+ my $r_nonce = pop(@$result);
+ if (!defined($r_nonce) || $r_nonce ne $nonce) {
+ # redis protocol falling out of step?
+ $ok = 0; $self->disconnect;
+ do_log(-1,"redis: penpals_find - nonce mismatch, expected %s, got %s",
+ $nonce, defined $r_nonce ? $r_nonce : 'UNDEF');
+ }
+ }
+ if ($ok && (@$result != @per_recip_data)) {
+ $ok = 0; $self->disconnect;
+ do_log(-1,"redis: penpals_find - number of results expected %d, got %d",
+ scalar @per_recip_data, scalar @$result);
+ }
+ if ($ok) {
+ for my $r (@per_recip_data) {
+ my $result_entry = shift @$result;
+ next if !$result_entry;
+ my($sid, $rid, $send_time, $best_ref_mail_id, $report) = @$result_entry;
+ if (!$send_time) { # undef or empty (or zero)
+ snmp_count('PenPalsMisses');
+ ll(4) && do_log(4, "penpals: (redis) not found (%s,%s)%s%s",
+ $sid ? $sid : $r->recip_addr_smtp,
+ $rid ? $rid : $msginfo->sender_smtp,
+ !$report ? '' : ', refs: '.$report,
+ !@$message_id_list ? '' :
+ '; '.join(', ',@$message_id_list) );
+ } else { # found a previous related correspondence
+ snmp_count('PenPalsHits');
+ my $age = max(0, $msginfo->rx_time - $send_time);
+ $r->recip_penpals_age($age);
+ $r->recip_penpals_related($best_ref_mail_id);
+ ll(3) && do_log(3, "penpals: (redis) found (%s,%s) age %s%s",
+ $sid ? $sid : $r->recip_addr_smtp,
+ $rid ? $rid : $msginfo->sender_smtp,
+ format_time_interval($age),
+ !$report ? '' : ', refs: '.$report );
+ # $age and $best_ref_mail_id are not logged explicitly,
+ # as they can be seen in the first entry of a lua query report
+ # (i.e. the last string)
+ }
+ }
}
- ($age, $ref_mail_id, $sid, $rid, \@mid);
+ $ok;
}
sub save_info_preliminary {
@@ -23385,31 +24407,195 @@ sub save_info_preliminary {
my $mail_id = $msginfo->mail_id;
defined $mail_id or die "save_info_preliminary: mail_id still undefined";
- my $ok;
+
+ # connect() sets the default ttl if missing, need to call it before do_log
$self->connect if !$self->{connected};
- do_log(5, "redis: save_info_preliminary: %s, %s, ttl %s s",
- $mail_id, int $msginfo->rx_time, $self->{ttl}||'x');
+ ll(5) && do_log(5, "redis: save_info_preliminary: %s, %s, ttl %s s",
+ $mail_id, int $msginfo->rx_time, $self->{ttl}||'x');
+
+ # use Lua to do HSETNX *and* EXPIRE atomically, otherwise we risk inserting
+ # a key with no expiration time if redis server goes down inbetween
+ my $added;
+ my $r = $self->{redis};
+ my(@args) = (1, $mail_id, int $msginfo->rx_time,
+ $self->{ttl} ? int $self->{ttl} : 0);
eval {
- $ok = $self->{redis}->evalsha($self->{lua_save_prelim}, 1, $mail_id,
- $self->{ttl}||'', int $msginfo->rx_time);
+ $added = $r->call('EVALSHA', $self->{lua_save_info_preliminary}, @args);
1;
- } or do { # Lua program probably not cached, define again and re-try
- $@ =~ s{\s+at /.*}{}s;
- if ($@ !~ /^\Q[evalsha] NOSCRIPT\E/)
- { $self->disconnect; die "Redis LUA error: $@\n" }
- $self->load_lua_programs;
- $ok = $self->{redis}->evalsha($self->{lua_save_prelim}, 1, $mail_id,
- $self->{ttl}||'', int $msginfo->rx_time);
+ } or do { # Lua function probably not cached, define again and re-try
+ if ($@ !~ /^NOSCRIPT/) {
+ $self->disconnect; chomp $@;
+ do_log(-1, "Redis Lua error: %s", $@);
+ } else {
+ $self->load_lua_programs;
+ $added = $r->call('EVALSHA', $self->{lua_save_info_preliminary}, @args);
+ }
};
- $ok;
+ $self->disconnect if !$database_sessions_persistent;
+ $added; # 1 if added successfully, false otherwise
}
-sub save_info_final {
+sub query_and_update_ip_reputation {
my($self, $msginfo) = @_;
+ my $ip_trace_ref = $msginfo->ip_addr_trace_public;
+ return if !$ip_trace_ref;
+ my @ip_trace = unique_list($ip_trace_ref);
+ return if !@ip_trace;
+
+ # Irwin-Hall distribution - approximates normal distribution
+ # n = 4, mean = n/2, variance = n/12, sigma = sqrt(n/12) =~ 0.577
+ my $normal_random = (rand() + rand() + rand() + rand() - 2) / 0.577;
+
+ my(@args) = (scalar @ip_trace, map("ip:$_", at ip_trace),
+ sprintf("%.3f", $msginfo->rx_time),
+ sprintf("%.6f", $normal_random) );
+ my($r, $ip_stats);
+ eval {
+ $self->connect if !$self->{connected};
+ $r = $self->{redis};
+ $ip_stats = $r->call('EVALSHA', $self->{lua_query_and_update_ip}, @args);
+ 1;
+ } or do { # Lua function probably not cached, define again and re-try
+ if ($@ !~ /^NOSCRIPT/) {
+ $self->disconnect; chomp $@;
+ do_log(-1, "Redis Lua error: %s", $@);
+ } else {
+ $self->load_lua_programs;
+ $ip_stats = $r->call('EVALSHA', $self->{lua_query_and_update_ip}, @args);
+ }
+ };
+ my($highest_score, $worst_ip);
+ for my $entry (!$ip_stats ? () : @$ip_stats) {
+ my($ip, $n_all, $s, $h, $b, $tfirst, $tlast, $ttl) = @$entry;
+ $ip =~ s/^ip://s; # strip key prefix
+ # the current event is not yet counted nor classified
+ if ($n_all <= 0) {
+ do_log(5, "redis: IP %s ttl: %.1f h", $ip, $ttl/3600);
+ } else {
+ my $n_other = $n_all - ($s + $h + $b);
+ if ($n_other < 0) { $n_all = $s + $h + $b; $n_other = 0 } # just in case
+ my $bad_content_ratio = ($s+$b) / $n_all;
+ # gains strength by the number of samples, watered down by share of ham
+ my $score = !($s+$b) ? 0 : 0.9 * ($n_all**0.36) * exp(-6 * $h/$n_all);
+
+ my $ip_ignore;
+ if ($score >= 0.05) {
+ # it is cheaper to do a redis/lookup unconditionally,
+ # then ditch an ignored IP address later if necessary
+ my($key, $err);
+ ($ip_ignore, $key, $err) =
+ lookup_ip_acl($ip, @{ca('ip_repu_ignore_maps')});
+ undef $ip_ignore if $err;
+ }
+ my $ll = ($score <= 0 || $ip_ignore) ? 3 : 2; # log level
+ if (ll($ll)) {
+ my $rxtime = $msginfo->rx_time;
+ do_log($ll, "redis: IP %s age: %s%s, ttl: %.1f h, %s, %s%s",
+ $ip, format_time_interval($rxtime-$tfirst),
+ defined $tlast ? ', last: '.format_time_interval($rxtime-$tlast) :'',
+ $ttl/3600,
+ $n_other ?
+ ($b ? "s/h/bv/?: $s/$h/$b/$n_other" : "s/h/?: $s/$h/$n_other")
+ : ($b ? "s/h/bv: $s/$h/$b" : "s/h: $s/$h"),
+ $score <= 0 ? 'clean' : sprintf("%.0f%%, score: %.1f",
+ 100*$bad_content_ratio, $score),
+ $ip_ignore ? ' =>0 ip_repu_ignore' : '');
+ }
+ $score = 0 if $ip_ignore || $score < 0.05;
+ if (!defined $highest_score || $score > $highest_score) {
+ $highest_score = $score; $worst_ip = $ip;
+ }
+ }
+ }
+ $self->disconnect if !$database_sessions_persistent;
+ ($highest_score, $worst_ip);
+}
+
+sub save_structured_report {
+ my($self, $report_ref, $log_key, $queue_size_limit) = @_;
+ return if !$report_ref;
+ $self->connect if !$self->{connected};
+ my $r = $self->{redis};
+ my $report_json = safe_encode_utf8(Amavis::JSON::encode($report_ref));
+ do_log(5, "redis: structured_report: %s %s", $log_key, $report_json);
+ $r->b_call("RPUSH", $log_key, $report_json);
+ # keep most recent - queue size limit in case noone is pulling events
+ $r->b_call("LTRIM", $log_key, -$queue_size_limit, -1) if $queue_size_limit;
+ my $res = $r->b_results; # errors will be signalled
+ ll(5) && do_log(5, "redis: save_structured_report, ".
+ "%d bytes, q_lim=%s, q_size=%s",
+ length $report_json, $queue_size_limit || 0,
+ $res ? join(', ',@$res) : '?');
+ 1;
+}
+
+sub save_info_final {
+ my($self, $msginfo, $report_ref) = @_;
+
+ $self->connect if !$self->{connected};
+ my $r = $self->{redis};
+
+ if (c('enable_ip_repu')) {
+ my $rigm = ca('ip_repu_ignore_maps');
+ my $ip_trace_ref = $msginfo->ip_addr_trace_public;
+ my @ip_trace;
+ @ip_trace = grep { my($ignore, $key, $err) = lookup_ip_acl($_, @$rigm);
+ !$ignore || $err;
+ } unique_list($ip_trace_ref) if $ip_trace_ref;
+ if (@ip_trace) {
+ my $content =
+ $msginfo->is_in_contents_category(CC_VIRUS) ? 'b' :
+ $msginfo->is_in_contents_category(CC_BANNED) ? 'b' : undef;
+ if (!defined $content) { # test for ham or spam
+ my($min, $max);
+ for my $r (@{$msginfo->per_recip_data}) {
+ my $spam_level = $r->spam_level;
+ next if !defined $spam_level;
+ $max = $spam_level if !defined $max || $spam_level > $max;
+ $min = $spam_level if !defined $min || $spam_level < $min;
+ }
+ if (defined $min) {
+ my $ip_repu_score = $msginfo->ip_repu_score || 0; # positive or 0
+ # avoid self-reinforcing feedback in the IP reputation auto-learning,
+ # use the score without the past IP reputation contribution
+ if ($max - $ip_repu_score < 0.5) { $content = 'h' }
+ elsif ($min - $ip_repu_score >= 5) { $content = 's' }
+ }
+ }
+ if (!defined $content) {
+ # just increment the total counter
+ $r->b_call("HINCRBY", "ip:$_", 'n', 1) for @ip_trace;
+ $r->b_results;
+ if (ll(5)) { do_log(5,"redis: IP INCR %s", $_) for @ip_trace }
+ } else {
+ # content type is known
+ for (@ip_trace) {
+ $r->b_call("HINCRBY", "ip:$_", 'n', 1);
+ $r->b_call("HINCRBY", "ip:$_", $content, 1);
+ }
+ my $counts = $r->b_results;
+ if (ll(5) && $counts) {
+ do_log(5,"redis: IP INCR %s n=%d, %s=%d",
+ $_, shift @$counts, $content, shift @$counts) for @ip_trace;
+ }
+ }
+ }
+ }
+
+ if (!$msginfo->originating) {
+ # don't bother saving info on incoming messages, saves Redis storage
+ # while still offering necessary data for a pen pals function
+ $self->disconnect if !$database_sessions_persistent;
+ return;
+ }
+
my $mail_id = $msginfo->mail_id;
defined $mail_id or die "save_info_preliminary: mail_id still undefined";
+ my $sender_smtp = $msginfo->sender_smtp;
+ local($1); $sender_smtp =~ s/^<(.*)>\z/$1/s;
+
my(@recips); # only recipients which did receive a message
for my $r (@{$msginfo->per_recip_data}) {
my($dest, $resp) = ($r->recip_destiny, $r->recip_smtp_response);
@@ -23417,38 +24603,54 @@ sub save_info_final {
my $addr_smtp = $r->recip_addr_smtp;
next if !defined $addr_smtp;
local($1); $addr_smtp =~ s/^<(.*)>\z/$1/s;
+ # don't remember messages sent to self
+ next if lc($sender_smtp) eq lc($addr_smtp);
+ # don't remember problematic outgoing messages, even if delivered
+ next if $r->is_in_contents_category(CC_VIRUS) ||
+ $r->is_in_contents_category(CC_BANNED) ||
+ $r->is_in_contents_category(CC_SPAM) || # kill_level
+ $r->is_in_contents_category(CC_SPAMMY); # tag2_level
push(@recips, lc $addr_smtp);
}
- my $ok;
- my $sender_smtp = $msginfo->sender_smtp;
- local($1); $sender_smtp =~ s/^<(.*)>\z/$1/s;
my $m_id = $msginfo->get_header_field_body('message-id');
$m_id = join(' ',parse_message_id($m_id))
if defined $m_id && $m_id ne ''; # strip CFWS
- my(@args) = map(defined $_ ? $_ : '', # can't have nil in a Lua table
- ($self->{ttl}, $msginfo->log_id, $m_id,
- $msginfo->client_addr, lc $sender_smtp, @recips) );
+ my(@args) = map(defined $_ ? $_ : '', # avoid nil in a Lua table
+ ($self->{ttl}, $msginfo->log_id,
+ $m_id, $msginfo->client_addr, lc $sender_smtp, @recips) );
if (!@recips) {
do_log(5,"redis: save_info_final: %s deleted", $mail_id);
} elsif (ll(5)) {
- do_log(5,"redis: save_info_final: %s, %d of %d, %s", $mail_id,
- scalar @recips, scalar @{$msginfo->per_recip_data},
+ do_log(5,"redis: save_info_final: %s, passed %d of %d recips, %s",
+ $mail_id, scalar @recips, scalar @{$msginfo->per_recip_data},
join(', ', at args));
}
- $self->connect if !$self->{connected};
+ my $result;
eval {
- $ok = $self->{redis}->evalsha($self->{lua_save_final},
- 1, $mail_id, @args);
+ $result = $r->call('EVALSHA', $self->{lua_save_final},
+ 1, $mail_id, @args);
1;
- } or do { # Lua program probably not cached, define again and re-try
- $@ =~ s{\s+at /.*}{}s;
- if ($@ !~ /^\Q[evalsha] NOSCRIPT\E/)
- { $self->disconnect; die "Redis LUA error: $@\n" }
- $self->load_lua_programs;
- $ok = $self->{redis}->evalsha($self->{lua_save_final},
- 1, $mail_id, @args);
+ } or do { # Lua function probably not cached, define again and re-try
+ if ($@ !~ /^NOSCRIPT/) {
+ $self->disconnect; undef $result; chomp $@;
+ do_log(-1, "Redis Lua error: %s", $@);
+ } else {
+ $self->load_lua_programs;
+ $result = $r->call('EVALSHA', $self->{lua_save_final},
+ 1, $mail_id, @args);
+ }
};
+
+ my $ok = 1;
+ my $r_nonce = $result;
+ if (!defined($r_nonce) || $r_nonce ne $mail_id) {
+ # redis protocol falling out of step?
+ $ok = 0; $self->disconnect;
+ do_log(-1,"redis: save_info_final - nonce mismatch, expected %s, got %s",
+ $mail_id, defined $r_nonce ? $r_nonce : 'UNDEF');
+ }
+ # $r->call("EVAL", 'collectgarbage()', 0);
$self->disconnect if !$database_sessions_persistent;
$ok;
}
@@ -23456,171 +24658,331 @@ sub save_info_final {
sub load_lua_programs($) {
my $self = $_[0];
do_log(5, "redis: load_lua_programs");
+ my $r = $self->{redis};
eval {
- $self->{lua_save_prelim} = $self->{redis}->script_load(<<'END');
---LUA_SAVE_PRELIM
- local mail_id = KEYS[1]
- local ttl, rx_time = ARGV[1], ARGV[2]
- if ttl and (ttl == "" or tonumber(ttl) <= 0) then ttl = nil end
- -- ensure the mail_id is unique, fail otherwise
- local added = redis.call("HSETNX", mail_id, "time", rx_time)
- if added == 1 and ttl then redis.call("EXPIRE", mail_id, ttl) end
- -- returns 1 if ok (is unique, added to a db), 0 if collision
- return added
+ $self->{lua_save_info_preliminary} = $r->call('SCRIPT', 'LOAD', <<'END');
+--LUA_SAVE_INFO_PRELIMINARY
+ local rcall, tonumber = redis.call, tonumber
+ local mail_id, rx_time, ttl = KEYS[1], ARGV[1], ARGV[2]
+
+ -- ensure the mail_id is unique, report false otherwise
+ local added = rcall("HSETNX", mail_id, "time", rx_time)
+ if added == 1 and ttl and tonumber(ttl) > 0 then
+ if rcall("EXPIRE", mail_id, ttl) ~= 1 then
+ return { err = "Failed to set EXPIRE on key " .. mail_id }
+ end
+ end
+ return added -- 1:yes, 0:no,failed
END
- 1;
} or do {
- $@ =~ s{\s+at /.*}{}s;
- $self->disconnect; die "Redis LUA error - lua_save_prelim: $@\n"
+ $self->disconnect; die "Redis LUA error - lua_save_info_preliminary: $@\n"
};
eval {
- $self->{lua_save_final} = $self->{redis}->script_load(<<'END');
+ $self->{lua_save_final} = $r->call('SCRIPT', 'LOAD', <<'END');
--LUA_SAVE_FINAL
local mail_id = KEYS[1]
- local ttl, log_id, msgid, client_addr, sender = unpack(ARGV,1,5)
+ local rcall = redis.call
+ local ARGV = ARGV
- if #ARGV < 6 then -- not delivered to any recipient
- redis.call("DEL", mail_id) -- delete the record, won't be on any use
+ -- not delivered to any recipient, just delete the useless record
+ if #ARGV < 6 then
+ rcall("DEL", mail_id)
else
- if ttl and (ttl == "" or tonumber(ttl) <= 0) then ttl = nil end
- local addresses = {}
- addresses[sender] = 0
+ local ttl, log_id, msgid, client_addr, sender = unpack(ARGV,1,5)
+ local tonumber, unpack = tonumber, unpack
+ if not tonumber(ttl) or tonumber(ttl) <= 0 then ttl = nil end
+
+ local addresses = { [sender] = true }
-- remaining arguments 6 to #ARGV are recipient addresses
- for r = 6, #ARGV do addresses[ARGV[r]] = 0 end
+ for r = 6, #ARGV do addresses[ARGV[r]] = true end
- -- create mail address -> id mappings
- for addr, addr_id in pairs(addresses) do
+ -- create mail address -> id mapping
+ for addr in pairs(addresses) do
local addr_key = "a:" .. addr
- addr_id = redis.call("GET", addr_key)
- if addr_id and ttl then
- redis.call("EXPIRE", addr_key, ttl) -- found, extend its lifetime
- else -- not found, assign a new id and store the email address
- addr_id = redis.call("INCR", "last.id.addr") -- get next id
+ local addr_id
+ if not ttl then
+ addr_id = rcall("GET", addr_key)
+ else
+ -- to avoid potential race between GET and EXPIRE, set EXPIRE first
+ local refreshed = rcall("EXPIRE", addr_key, ttl)
+ if refreshed == 1 then addr_id = rcall("GET", addr_key) end
+ end
+ if not addr_id then
+ -- not found, assign a new id and store the email address
+ addr_id = rcall("INCR", "last.id.addr") -- get next id, starts at 1
+ addr_id = tostring(addr_id)
local ok
if ttl then
- ok = redis.call("SET", addr_key, addr_id, "EX", ttl, "NX")
+ ok = rcall("SET", addr_key, addr_id, "EX", ttl, "NX")
else
- ok = redis.call("SET", addr_key, addr_id, "NX")
+ ok = rcall("SET", addr_key, addr_id, "NX")
end
if not ok then
- -- shouldn't happen, Lua script runs atomically, but anyway...
- addr_id = redis.call("GET", addr_key) -- collision, retry
+ -- shouldn't happen, Lua program runs atomically, but anyway...
+ addr_id = rcall("GET", addr_key) -- collision, retry
end
end
addresses[addr] = addr_id
end
- -- create Message-ID -> id mapping
+ -- create a Message-ID -> id mapping
local msgid_key = "m:" .. msgid
- local msgid_id = redis.call("GET", msgid_key)
- if msgid_id and ttl then -- unlikely duplicate Message-ID, but anyway...
- redis.call("EXPIRE", msgid_key, ttl) -- extend its lifetime
+ local msgid_id = rcall("GET", msgid_key)
+ if msgid_id then -- unlikely duplicate Message-ID, but anyway...
+ if ttl then rcall("EXPIRE", msgid_key, ttl) end -- extend its lifetime
else
- msgid_id = redis.call("INCR", "last.id.msgid") -- get next id
+ msgid_id = rcall("INCR", "last.id.msgid") -- get next id, starts at 1
+ msgid_id = tostring(msgid_id)
local ok
if ttl then
- ok = redis.call("SET", msgid_key, msgid_id, "EX", ttl, "NX")
+ ok = rcall("SET", msgid_key, msgid_id, "EX", ttl, "NX")
else
- ok = redis.call("SET", msgid_key, msgid_id, "NX")
+ ok = rcall("SET", msgid_key, msgid_id, "NX")
end
if not ok then
- -- shouldn't happen, Lua script runs atomically, but anyway...
- msgid_id = redis.call("GET", msgid_key) -- collision, retry
+ -- shouldn't happen, Lua program runs atomically, but anyway...
+ msgid_id = rcall("GET", msgid_key) -- collision, retry
end
end
-- store additional information to an existing mail_id record
local sender_id = addresses[sender]
- redis.call("HSET", mail_id, "log", log_id)
- -- redis.call("HMSET", mail_id, "log", log_id,
+ rcall("HSET", mail_id, "log", log_id)
+ -- rcall("HMSET", mail_id, "log", log_id,
-- "msgid", msgid_id, "ip", client_addr, "sender", sender_id)
+
+ -- store relations: sender/msgid and sender/recipient pairs
local mapkeys = { "sm:" .. sender_id .. "::" .. msgid_id }
for r = 6, #ARGV do
local recip_id = addresses[ARGV[r]]
- table.insert(mapkeys, "sr:" .. sender_id .. ":" .. recip_id)
- table.insert(mapkeys, "srm:" .. sender_id .. ":" .. recip_id ..
- ":" .. msgid_id)
+ -- only the most recent sr:* record is kept, older are overwritten
+ mapkeys[#mapkeys+1] = "sr:" .. sender_id .. ":" .. recip_id
+ -- mapkeys[#mapkeys+1] = "srm:" .. sender_id .. ":" .. recip_id ..
+ -- ":" .. msgid_id
end
if not ttl then
- for j = 1, #mapkeys do redis.call("SET", mapkeys[j], mail_id) end
+ for _,k in ipairs(mapkeys) do rcall("SET", k, mail_id) end
else
- for j = 1, #mapkeys do
- redis.call("SET", mapkeys[j], mail_id, "EX", ttl)
- end
+ for _,k in ipairs(mapkeys) do rcall("SET", k, mail_id, "EX", ttl) end
end
end
- return 1
+
+ return mail_id
END
} or do {
- $@ =~ s{\s+at /.*}{}s;
$self->disconnect; die "Redis LUA error - lua_save_final: $@\n"
};
eval {
- $self->{lua_query} = $self->{redis}->script_load(<<'END');
---LUA_QUERY
- local q_keys = { "a:" .. ARGV[1], "a:" .. ARGV[2] } -- sender, recipient
- for j = 3, #ARGV do table.insert(q_keys, "m:" .. ARGV[j]) end --Message-ID
- local mid, sid, rid = {}, nil, nil
- local q_result = redis.call("MGET", unpack(q_keys))
- if q_result then
- sid, rid = q_result[1], q_result[2]
- for j = 3, #ARGV do table.insert(mid, q_result[j]) end
+ $self->{lua_query_and_update_ip} = $r->call('SCRIPT', 'LOAD', <<'END');
+--LUA_QUERY_AND_UPDATE_IP
+ local rcall, tonumber, unpack, floor, sprintf =
+ redis.call, tonumber, unpack, math.floor, string.format
+ local KEYS, ARGV = KEYS, ARGV
+ local rx_time, normal_random = ARGV[1], tonumber(ARGV[2])
+
+ local results = {}
+ for j = 1, #KEYS do
+ local ipkey = KEYS[j] -- an IP address, prefixed by "ip:"
+ local tfirst, tlast -- Unix times of creation and last access
+ local n, s, h, b -- counts: all, spam, ham, banned+virus
+ local age, ttl -- time since creation, time to live in seconds
+ local ip_addr_data =
+ rcall("HMGET", ipkey, 'tl', 'tf', 'n', 's', 'h', 'b')
+ if ip_addr_data then
+ tlast, tfirst, n, s, h, b = unpack(ip_addr_data,1,6)
+ end
+ if not tlast then -- does not exist, a new entry is needed
+ n = 0; tfirst = rx_time; ttl = 3*3600 -- 3 hours for new entries
+ rcall("HMSET", ipkey, 'tf', rx_time, 'tl', rx_time, 'n', '0')
+ else -- a record for this IP address exists, collect its counts and age
+ n = tonumber(n) or 0
+ local rx_time_n, tfirst_n, tlast_n =
+ tonumber(rx_time), tonumber(tfirst), tonumber(tlast)
+ if rx_time_n and tfirst_n and tlast_n then -- valid numbers
+ age = rx_time_n - tfirst_n -- time since entry creation
+ local dt = rx_time_n - tlast_n -- time since last occurrence
+ ttl = 3600 * (n >= 8 and 80 or (3 + n^2.2)) -- 4 to 80 hours
+ if ttl < 1.5 * dt then ttl = 1.5 * dt end
+ else -- just in case - ditch a record with invalid fields
+ n = 0; tfirst = rx_time; ttl = 3*3600
+ rcall("DEL", ipkey);
+ rcall("HMSET", ipkey, 'tf', rx_time, 'n', '0')
+ end
+ rcall("HMSET", ipkey, 'tl', rx_time) -- update its last-seen time
+ end
+ -- the 's', 'h', 'b' and 'n' counts will be updated later
+ if normal_random then
+ -- introduce some randomness, don't let spammers depend on a fixed ttl
+ ttl = ttl * (1 + normal_random * 0.2)
+ if ttl < 4000 then ttl = 4000 end -- no less than 1h 7min
+ end
+ -- set time-to-live in seconds, capped at 3 days, integer
+ if age and (age + ttl > 3*24*3600) then ttl = 3*24*3600 - age end
+ if ttl < 1 then
+ rcall("DEL", ipkey); ttl = 0
+ else
+ rcall("EXPIRE", ipkey, floor(ttl))
+ end
+ results[#results+1] = { ipkey, n or 0, s or 0, h or 0, b or 0,
+ tfirst or "", tlast or "", ttl }
end
+ return results
+END
+ } or do {
+ $self->disconnect; die "Redis LUA error - lua_query_and_update_ip: $@\n"
+ };
- local mail_id, rx_time
- local q_keys = {}
- if sid then
- if rid then
- -- try full sender/recipient/Message-ID tuples first
- for j = 1, #mid do
- if mid[j] then
- table.insert(q_keys, "srm:" .. sid .. ":" .. rid .. ":" .. mid[j])
- end
- end
+ eval {
+ $self->{lua_query_penpals} = $r->call('SCRIPT', 'LOAD', <<'END');
+--LUA_QUERY_PENPALS
+ local tonumber, unpack, sprintf = tonumber, unpack, string.format
+ local rcall = redis.call
+ local ARGV = ARGV
+
+ local now, nonce, recipient = ARGV[1], ARGV[2], ARGV[3]
+ local senders_count = tonumber(ARGV[4])
+ local senders_argv_ofs = 5
+ local messageid_argv_ofs = senders_argv_ofs + senders_count + 1
+ local messageid_count = tonumber(ARGV[messageid_argv_ofs - 1])
+
+ local q_keys1 = {}
+ -- current sender as a potential previous recipient
+ if recipient == '' then recipient = nil end -- nothing ever sent to "<>"
+ if recipient then
+ q_keys1[#q_keys1+1] = "a:" .. recipient
+ end
+ for j = 1, senders_count do
+ q_keys1[#q_keys1+1] = "a:" .. ARGV[senders_argv_ofs + j - 1]
+ end
+ for j = 1, messageid_count do
+ q_keys1[#q_keys1+1] = "m:" .. ARGV[messageid_argv_ofs + j - 1]
+ end
+
+ -- map e-mail addresses and referenced Message-IDs to internal id numbers
+ local q_result = rcall("MGET", unpack(q_keys1))
+ q_keys1 = nil
+
+ local rid -- internal id of a recipient's e-mail addresses
+ local mids = {} -- internal ids corresponding to referenced "Message-ID"s
+ local senders = {}
+ if q_result then
+ local k = 0;
+ if recipient then -- nonempty e-mail address, i.e. not "<>"
+ k = k+1; rid = q_result[k]
end
- -- next try sender/Message-ID pairs without a recipient
- for j = 1, #mid do
- if mid[j] then
- table.insert(q_keys, "sm:" .. sid .. "::" .. mid[j])
- end
+ for j = 1, senders_count do
+ k = k+1;
+ if not q_result[k] then senders[j] = false -- non-nil
+ else senders[j] = { sid = q_result[k] } end
end
- if rid then
- -- as a last resort, try a sender/recipient pair without a Message-ID
- table.insert(q_keys, "sr:" .. sid .. ":" .. rid)
+ for j = 1, messageid_count do
+ k = k+1; if q_result[k] then mids[q_result[k]] = true end
+ end
+ end
+ q_result = nil
+
+ -- prepare query keys to find a closest-matching previous e-mail message
+ -- for each sender
+ local q_keys2, belongs_to_sender, on_hit_txt = {}, {}, {}
+ for _, s in ipairs(senders) do
+ if s then
+ -- try sender/Message-ID pairs without a recipient
+ for m in pairs(mids) do
+ local nxt = #q_keys2 + 1
+ q_keys2[nxt] = "sm:" .. s.sid .. "::" .. m
+ on_hit_txt[nxt] = "mid=" .. m
+ belongs_to_sender[nxt] = s
+ end
+ -- try a sender/recipient pair without a Message-ID ref
+ if rid then
+ local nxt = #q_keys2 + 1
+ q_keys2[nxt] = "sr:" .. s.sid .. ":" .. rid
+ on_hit_txt[nxt] = "rid=" .. rid
+ belongs_to_sender[nxt] = s
+ end
end
+ end
+
+ -- get an internal id (or nil) of a matching mail_id for each query key
+ local q_result2
+ if #q_keys2 >= 1 then q_result2 = rcall("MGET", unpack(q_keys2)) end
+
+ local msginfo = {} -- data about a message mail_id (e.g. its rx_time)
+ if q_result2 then
+ for j = 1, #q_keys2 do
+ local rx_time_n
+ local mail_id = q_result2[j]
+
+ if not mail_id then
+ -- no matching mail_id
+ elseif msginfo[mail_id] then -- already looked-up
+ rx_time_n = msginfo[mail_id].rx_time_n
+ else -- not yet looked-up
+ msginfo[mail_id] = {}
+ -- see if a record for this mail_id exists, find its timestamp
+ rx_time_n = tonumber(rcall("HGET", mail_id, "time"))
+ msginfo[mail_id].rx_time_n = rx_time_n
+ end
+
+ if rx_time_n then -- exists and is a valid number
+ local s = belongs_to_sender[j]
+ if not s.hits then s.hits = {} end
+ if not s.hits[mail_id] then
+ s.hits[mail_id] = on_hit_txt[j]
+ else
+ s.hits[mail_id] = s.hits[mail_id] .. " " .. on_hit_txt[j]
+ end
- if next(q_keys) then
- local q_result2 = redis.call("MGET", unpack(q_keys))
- if q_result2 then
- for j = 1, #q_result2 do -- pick the first
- if q_result2[j] then mail_id = q_result2[j]; break end
+ -- for each sender manage a sorted list of mail_ids found
+ if not s.mail_id_list then
+ s.mail_id_list = { mail_id }
+ else
+ -- keep sender's mail_id_list sorted by rx_time, highest first
+ local mail_id_list = s.mail_id_list
+ local first_smaller_ind
+ for j = 1, #mail_id_list do
+ if msginfo[mail_id_list[j]].rx_time_n <= rx_time_n then
+ first_smaller_ind = j; break
+ end
+ end
+ table.insert(mail_id_list,
+ first_smaller_ind or #mail_id_list+1, mail_id)
end
end
- if mail_id then
- rx_time = unpack(redis.call("HMGET", mail_id, "time"))
+ end
+ end
+
+ local results = {} -- one entry for each sender, followed by a nonce
+ for _, s in ipairs(senders) do
+ if not s or not s.mail_id_list then -- no matching mail_id
+ results[#results+1] = { s and s.sid or "", rid }
+ else -- some matches for this sender, compile a report
+ local report = {}; local mail_id_list = s.mail_id_list
+ for _, mail_id in ipairs(mail_id_list) do -- first is best
+ report[#report+1] = sprintf("%s (%.0f s) %s", mail_id,
+ tonumber(now) - msginfo[mail_id].rx_time_n,
+ s.hits and s.hits[mail_id] or "")
end
+ results[#results+1] =
+ { s.sid or "", rid or "", msginfo[mail_id_list[1]].rx_time_n,
+ mail_id_list[1], table.concat(report,", ") }
end
end
- if not mail_id then mail_id = "" end
- if not rx_time then rx_time = "" end
- if not sid then sid = "" end
- if not rid then rid = "" end
- for j = 1, #mid do if not mid[j] then mid[j] = "" end end
- return { mail_id, rx_time, sid, rid, unpack(mid) }
+ results[#results+1] = nonce
+ return results
END
1;
} or do {
- $@ =~ s{\s+at /.*}{}s;
- $self->disconnect; die "Redis LUA error - lua_query: $@\n"
+ $self->disconnect; die "Redis LUA error - lua_query_penpals: $@\n"
};
- ll(5) && do_log(5, "redis: prelim %s, final %s, query %s",
- map(substr($_,0,10),
- @$self{qw(lua_save_prelim lua_save_final lua_query)}));
+ ll(5) && do_log(5, "redis: SHA fingerprints: final %s, query %s",
+ map(substr($_,0,10), @$self{qw(lua_save_final lua_query)}));
+ section_time("redis-load");
1;
}
@@ -23638,7 +25000,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform c cr ca);
import Amavis::Util qw(ll do_log do_log_safe);
@@ -23919,12 +25281,12 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform :confvars c cr ca);
import Amavis::rfc2821_2822_Tools;
import Amavis::Util qw(ll do_log do_log_safe min max minmax
- untaint untaint_inplace
+ untaint untaint_inplace format_time_interval
add_entropy sanitize_str safe_decode
safe_encode safe_encode_ascii safe_encode_utf8
snmp_count orcpt_decode ccat_split ccat_maj);
@@ -24028,20 +25390,21 @@ sub find_or_save_addr {
($id, $existed);
}
-# find a penpals record which proves that a local user sid really sent a
-# mail to a recipient rid some time ago. Returns an interval time in seconds
+# find a penpals record which proves that a local user (sid) really sent a
+# mail to a recipient (rid) some time ago. Returns an interval time in seconds
# since the last such mail was sent by our local user to a specified recipient
# (or undef if information is not available). If @$message_id_list is a
# nonempty list of Message-IDs as found in References header field, the query
-# also provides previous outgoing messages with a matching Message-ID but
+# also finds previous outgoing messages with a matching Message-ID but
# possibly to recipients different from what the mail was originally sent to.
#
sub penpals_find {
- my($self, $sid,$rid,$message_id_list, $now) = @_;
+ my($self, $sid,$rid,$message_id_list, $msginfo) = @_;
my($a_ref,$found,$age,$send_time,$ref_mail_id,$ref_subj,$ref_mid,$ref_rid);
my $conn_h = $self->{conn_h}; my $sql_cl_r = cr('sql_clause');
my $sel_penpals = $sql_cl_r->{'sel_penpals'};
my $sel_penpals_msgid = $sql_cl_r->{'sel_penpals_msgid'};
+ $message_id_list = [] if !$message_id_list;
if (defined($sel_penpals_msgid) && @$message_id_list && defined($sid)) {
# list of refs to Message-ID is nonempty, try reference or recipient match
my $n = scalar(@$message_id_list); # number of keys
@@ -24096,8 +25459,8 @@ sub penpals_find {
ll(4) && do_log(4, "penpals: (sql) not found (%s,%s)%s", $sid,$rid,
!@$message_id_list ? '' : ' refs: '.join(", ",@$message_id_list));
} else {
- $age = max(0, $now - $send_time);
- ll(3) && do_log(3, "penpals: (sql) found (%s,%s) %s age %s (%d s)",
+ $age = max(0, $msginfo->rx_time - $send_time);
+ ll(3) && do_log(3, "penpals: (sql) found (%s,%s) %s age %s (%.0f s)",
$sid, $rid, $ref_mail_id,
format_time_interval($age), $age);
}
@@ -24204,9 +25567,11 @@ sub save_info_preliminary {
}
sub save_info_final {
- my($self, $msginfo,$dsn_sent) = @_;
+ my($self, $msginfo, $report_ref) = @_;
my $mail_id = $msginfo->mail_id;
defined $mail_id or die "save_info_final: mail_id still undefined";
+ my $dsn_sent = $msginfo->dsn_sent;
+ $dsn_sent = !$dsn_sent ? 'N' : $dsn_sent==1 ? 'Y' : $dsn_sent==2 ? 'q' : '?';
my $sid = $msginfo->sender_maddr_id;
my $conn_h = $self->{conn_h}; my($sql_cl_r) = cr('sql_clause');
my $ins_msg = $sql_cl_r->{'ins_msg'};
@@ -24219,7 +25584,7 @@ sub save_info_final {
} else {
$conn_h->begin_work; # SQL transaction starts
eval {
- my(%content_short_name) = ( # as written to a SQL record
+ my(%ccat_short_name) = ( # as written to a SQL record
CC_VIRUS,'V', CC_BANNED,'B', CC_UNCHECKED,'U',
CC_SPAM,'S', CC_SPAMMY,'Y', CC_BADH.",2",'M', CC_BADH,'H',
CC_OVERSIZED,'O', CC_MTA,'T', CC_CLEAN,'C', CC_CATCHALL,'?');
@@ -24240,7 +25605,7 @@ sub save_info_final {
: ($dest==D_PASS && ($resp=~/^2/ || !$r->recip_done)) ? 'PASS'
: ($dest==D_DISCARD) ? 'DISCARD' : '?';
my $r_content_type =
- $r->setting_by_contents_category(\%content_short_name);
+ $r->setting_by_contents_category(\%ccat_short_name);
for ($r_content_type) { $_ = ' ' if !defined $_ || /^ *\z/ }
substr($resp,255) = '' if length($resp) > 255;
$resp =~ s/[^\040-\176]/?/gs; # just in case, only need 7 bit printbl
@@ -24306,7 +25671,7 @@ sub save_info_final {
s/[^\040-\176]/?/gs; # only use 7 bit printable, compatible with UTF-8
}
my $content_type =
- $msginfo->setting_by_contents_category(\%content_short_name);
+ $msginfo->setting_by_contents_category(\%ccat_short_name);
my $checks_performed = $msginfo->checks_performed;
$checks_performed = !ref $checks_performed ? ''
: join('', grep($checks_performed->{$_}, qw(V S H B F P D)));
@@ -24372,7 +25737,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
import Amavis::Util qw(ll do_log untaint min max minmax);
}
@@ -24671,7 +26036,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
@EXPORT = qw(&mail_via_sql);
import Amavis::Conf qw(:platform c cr ca $sql_quarantine_chunksize_max);
@@ -24808,12 +26173,12 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform :confvars c cr ca);
import Amavis::Util qw(ll untaint min max minmax unique_list do_log
add_entropy proto_decode rmdir_recursively
- prolong_timer get_deadline);
+ prolong_timer get_deadline generate_mail_id);
import Amavis::ProcControl qw(exit_status_str proc_status_ok
run_command run_as_subprocess
collect_results collect_results_structured);
@@ -24989,7 +26354,7 @@ sub sophos_savi_internal {
my $mime_option_value = 0;
if (ref($part) && ($part->type_short eq 'MAIL' ||
lc($part->type_declared) eq 'message/rfc822')) {
- do_log(2, "%s: $query - enabling option Mime", $av_name);
+ do_log(2, "%s: %s - enabling option Mime", $av_name, $query);
$mime_option_value = 1;
}
my $error = $savi_obj->set('Mime', $mime_option_value);
@@ -25286,6 +26651,8 @@ sub av_smtp_client($$$$) {
$test_msg->rx_time($msginfo->rx_time); # copy the reception time
$test_msg->log_id($msginfo->log_id); # use the same log_id
$test_msg->partition_tag($msginfo->partition_tag); # same partition_tag
+ $test_msg->parent_mail_id($msginfo->mail_id);
+ $test_msg->mail_id(scalar generate_mail_id());
$test_msg->conn_obj($msginfo->conn_obj);
$test_msg->mail_id($msginfo->mail_id); # use the same mail_id
$test_msg->body_type($msginfo->body_type); # use the same BODY= type
@@ -25391,8 +26758,14 @@ sub ask_daemon_internal {
};
prolong_timer('ask_daemon_internal');
last if !defined $eval_stat; # mission accomplished
+
# error handling (the most interesting error codes are EPIPE and ENOTCONN)
chomp $eval_stat; my $err = "$!"; my $errn = 0+$!;
+
+ # close socket through its DESTROY method, ignoring status
+ $st_sock{$socketname} = $sock = undef;
+ $st_socket_created{$socketname} = 0;
+
if (Time::HiRes::time >= $deadline) {
die "ask_daemon_internal: Exceeded allowed time";
}
@@ -25410,11 +26783,6 @@ sub ask_daemon_internal {
sleep($dly); # slow down a possible runaway
}
}
- if ($st_socket_created{$socketname}) {
- # prepare for a retry, implicit close through DESTROY ignoring status
- $st_sock{$socketname} = $sock = undef;
- $st_socket_created{$socketname} = 0;
- }
# leave good socket as the first entry in the list
# so that it will be tried first when needed again
if (@$socket_specs > 1) {
@@ -25587,7 +26955,7 @@ sub run_av(@) {
}
# exact matching on command arguments, no substring matches
for my $j (0..$#query_template) {
- if (ref($query_expanded[$j])) { # placeholders collecting fnames
+ if (ref $query_expanded[$j]) { # placeholders collecting fnames
my $arg = $query_template[$j];
my $repl = $arg eq '{}/*' ? "$tempdir/parts/$f"
: $arg eq '*' ? $f : undef;
@@ -25797,7 +27165,7 @@ sub virus_scan($$) {
my(%seen);
for my $r (@{$msginfo->per_recip_data}) {
my $spam_tests = $r->spam_tests;
- if (defined $spam_tests) {
+ if ($spam_tests) {
local($1,$2);
for (split(/,/, join(',',map($$_,@$spam_tests)))) {
$seen{$1} = $2 if /^AV\.([^=]*)=([0-9.+-]+)\z/;
@@ -25824,7 +27192,7 @@ sub virus_scan($$) {
(0..$#$this_vn) ));
for my $r (@{$msginfo->per_recip_data}) {
$r->spam_level( ($r->spam_level || 0) + $spam_level );
- if (!defined($r->spam_tests)) {
+ if (!$r->spam_tests) {
$r->spam_tests([ \$spam_tests ]);
} else {
push(@{$r->spam_tests}, \$spam_tests);
@@ -25968,7 +27336,7 @@ use IO::File qw(O_RDONLY O_WRONLY O_RDWR O_APPEND O_CREAT O_EXCL);
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform c cr ca);
import Amavis::Util qw(ll do_log min max minmax untaint untaint_inplace
@@ -26083,6 +27451,7 @@ sub unlock {
if ($lock_fh) {
my $scanner_name = $scanner_obj->{scanner_name};
do_log(5, "releasing a lock for %s", $scanner_name);
+ # close would unlock automatically, but let's check for locking mistakes
flock($lock_fh, LOCK_UN)
or die "Can't release a lock for $scanner_name: $!";
$lock_fh->close or die "Can't close a lock file for $scanner_name: $!";
@@ -26329,7 +27698,7 @@ sub white_black_list($$$$) {
if ($boost) { # defined and nonzero
$r->spam_level( ($r->spam_level || 0) + $boost);
my $spam_tests = 'AM.WBL=' . (0+sprintf("%.3f",$boost));
- if (!defined($r->spam_tests)) {
+ if (!$r->spam_tests) {
$r->spam_tests([ \$spam_tests ]);
} else {
unshift(@{$r->spam_tests}, \$spam_tests);
@@ -26369,7 +27738,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform :confvars :sa c cr ca);
import Amavis::Util qw(ll do_log sanitize_str min max minmax
@@ -26448,10 +27817,8 @@ sub check_or_learn {
$per_recip_data = [] if !$per_recip_data;
for my $r (@$per_recip_data) {
my $spam_tests = $r->spam_tests;
- if (defined $spam_tests) {
- push(@av_tests,
- grep(/^AV\..+=/, split(/,/, join(',',map($$_,@$spam_tests)))));
- }
+ push(@av_tests, grep(/^AV\..+=/,
+ split(/,/, join(',',map($$_,@$spam_tests))))) if $spam_tests;
}
$prefix .= sprintf("X-Amavis-AV-Status: %s\n",
sanitize_str(join(',', at av_tests))) if @av_tests;
@@ -26800,7 +28167,7 @@ sub check_or_learn {
$msginfo->supplementary_info('SCORE-'.$scanner_name, $spam_score);
for my $r (@$per_recip_data) {
$r->spam_level( ($r->spam_level || 0) + $spam_score );
- if (!defined($r->spam_tests)) {
+ if (!$r->spam_tests) {
$r->spam_tests([ \$spam_tests ]);
} else {
push(@{$r->spam_tests}, \$spam_tests);
@@ -26825,7 +28192,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform :confvars :sa c cr ca);
import Amavis::Util qw(ll do_log sanitize_str min max minmax get_deadline);
@@ -26872,10 +28239,8 @@ sub check {
$per_recip_data = [] if !$per_recip_data;
for my $r (@$per_recip_data) {
my $spam_tests = $r->spam_tests;
- if (defined $spam_tests) {
- push(@av_tests,
- grep(/^AV\..+=/, split(/,/, join(',',map($$_,@$spam_tests)))));
- }
+ push(@av_tests, grep(/^AV\..+=/,
+ split(/,/, join(',',map($$_,@$spam_tests))))) if $spam_tests;
}
$hdr_prefix .= sprintf("X-Amavis-AV-Status: %s\n",
sanitize_str(join(',', at av_tests))) if @av_tests;
@@ -27017,7 +28382,7 @@ sub check {
: $attr{'spam'} =~ /^False/ ? 'Ham' : 'Unknown');
for my $r (@$per_recip_data) {
$r->spam_level( ($r->spam_level || 0) + $spam_level );
- if (!defined($r->spam_tests)) {
+ if (!$r->spam_tests) {
$r->spam_tests([ \$sa_tests ]);
} else {
push(@{$r->spam_tests}, \$sa_tests);
@@ -27039,7 +28404,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
# let a 'require' understand that this module is already loaded:
$INC{'Mail/SpamAssassin/Logger/Amavislog.pm'} = 'amavisd';
@@ -27076,7 +28441,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform :confvars :sa $daemon_user c cr ca);
import Amavis::Util qw(ll do_log do_log_safe sanitize_str prolong_timer
@@ -27894,10 +29259,8 @@ sub check {
my(@av_tests);
for my $r (@{$msginfo->per_recip_data}) {
my $spam_tests = $r->spam_tests;
- if ($spam_tests) {
- push(@av_tests,
- grep(/^AV[.:].+=/, split(/,/, join(',',map($$_,@$spam_tests)))));
- }
+ push(@av_tests, grep(/^AV[.:].+=/,
+ split(/,/, join(',',map($$_,@$spam_tests))))) if $spam_tests;
}
$prefix .= sprintf("X-Amavis-AV-Status: %s\n",
sanitize_str(join(',', at av_tests))) if @av_tests;
@@ -27991,10 +29354,10 @@ sub check {
for my $r (@r_list) {
$r->spam_level( ($r->spam_level || 0) + $spam_level );
$r->spam_report($spam_report); $r->spam_summary($spam_summary);
- if (!defined($r->spam_tests)) {
+ if (!$r->spam_tests) {
$r->spam_tests([ \$sa_tests ]);
} else {
- # comes last: here we use push, unlike elsewhere which may do unshift
+ # comes last: here we use push, unlike elsewhere where may do unshift
push(@{$r->spam_tests}, \$sa_tests);
}
if ($dkim_adsp_suppress) {
@@ -28036,7 +29399,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&init &decompose_part &determine_file_types);
import Amavis::Util qw(untaint min max minmax ll do_log snmp_count
@@ -28058,9 +29421,6 @@ use Time::HiRes ();
use File::Basename qw(basename);
use Compress::Zlib 1.35; # avoid security vulnerability in <= 1.34
use Archive::Zip 1.14 qw(:CONSTANTS :ERROR_CODES);
- # avoid an exploitable security hole in Convert::UUlib 1.04 and older!
-use Convert::UUlib 1.05 qw(:constants); # 1.08 or newer is preferred!
-use Convert::TNEF; #***
# recursively descend into a directory $dir containing potentially unsafe
# files with unpredictable names, soft links, etc., rename each regular
@@ -28201,9 +29561,9 @@ sub determine_file_types($$) {
: '; (' . join(', ',@$type_short) . ')'
) );
$part->type_long($type_long); $part->type_short($type_short);
- $part->attributes_add('C') # simpleminded
- if !ref($type_short) ? $type_short eq 'pgp' # encrypted?
- : grep($_ eq 'pgp', @$type_short);
+ $part->attributes_add('C')
+ if !ref($type_short) ? $type_short eq 'pgp.enc' # encrypted?
+ : grep($_ eq 'pgp.enc', @$type_short);
$index++;
}
}
@@ -28243,12 +29603,14 @@ sub determine_file_types($$) {
sub decompose_mail($$) {
my($tempdir,$file_generator_object) = @_;
- my $hold; my(@parts); my $depth = 1; my $any_undecipherable = 0;
+ my $hold; my(@parts); my $depth = 1;
+ my($any_undecipherable, $any_encrypted, $over_levels) = (0,0,0);
my $which_section = "parts_decode";
# fetch all not-yet-visited part names, and start a new cycle
TIER:
while (@parts = @{$file_generator_object->parts_list}) {
if ($MAXLEVELS > 0 && $depth > $MAXLEVELS) {
+ $over_levels = 1;
$hold = "Maximum decoding depth ($MAXLEVELS) exceeded";
last;
}
@@ -28295,15 +29657,22 @@ TIER:
determine_file_types($tempdir, \@parts);
}
for my $part (@parts) {
- if ($part->exists && !defined($hold))
- { $hold = decompose_part($part, $tempdir) }
- $any_undecipherable++ if grep($_ eq 'U', @{ $part->attributes || [] });
+ if ($part->exists && !defined($hold)) {
+ my($hold_tmp, $over_levels_tmp) = decompose_part($part, $tempdir);
+ $hold = $hold_tmp if $hold_tmp;
+ $over_levels ||= $over_levels_tmp;
+ }
+ my $attr = $part->attributes;
+ if (defined $attr) {
+ $any_undecipherable++ if index($attr, 'U') >= 0;
+ $any_encrypted++ if index($attr, 'C') >= 0;
+ }
}
last TIER if defined $hold;
$depth++;
}
section_time($which_section); prolong_timer($which_section);
- ($hold, $any_undecipherable);
+ ($hold, $any_undecipherable, $any_encrypted, $over_levels);
}
# Decompose one part
@@ -28314,7 +29683,7 @@ sub decompose_part($$) {
# 0 - truly atomic or unknown or archiver failure; consider atomic
# 1 - some archive, successfully unpacked, result replaces original
# 2 - probably unpacked, but keep the original (eg self-extracting archive)
- my $hold; my $eval_stat; my $sts = 0; my $any_called = 0;
+ my $hold; my $eval_stat; my($sts, $any_called, $over_levels) = (0,0,0);
eval {
my $type_short = $part->type_short;
my(@ts) = !defined $type_short ? ()
@@ -28337,7 +29706,7 @@ sub decompose_part($$) {
my $ll = -1;
if ($eval_stat =~ /\bExceeded storage quota\b.*\bbytes by/ ||
$eval_stat =~ /\bMaximum number of files\b.*\bexceeded/) {
- $hold = $eval_stat; $ll = 1;
+ $hold = $eval_stat; $ll = 1; $over_levels = 1;
}
do_log($ll,"Decoding of %s (%s) failed, leaving it unpacked: %s",
$part->base_name, $part->type_long, $eval_stat);
@@ -28362,7 +29731,7 @@ sub decompose_part($$) {
['atomic','archive, unpacked','source retained']->[$sts]);
section_time('decompose_part') if $any_called;
die $eval_stat if $eval_stat =~ /^timed out\b/; # resignal timeout
- $hold;
+ ($hold, $over_levels);
}
# a trivial wrapper around mime_decode() to adjust arguments and result
@@ -28383,12 +29752,26 @@ sub do_mime_decode($$) {
# if ASCII text, try multiple decoding methods as provided by UUlib
# (uuencoded, xxencoded, BinHex, yEnc, Base64, Quoted-Printable)
#
+use vars qw($have_uulib_module);
sub do_ascii($$) {
my($part, $tempdir) = @_;
ll(4) && do_log(4,"do_ascii: Decoding part %s", $part->base_name);
-
+ if (!defined $have_uulib_module) {
+ eval {
+ require Convert::UUlib && ($have_uulib_module = 1);
+ # avoid an exploitable security hole in Convert::UUlib 1.04 and older
+ Convert::UUlib->VERSION(1.05); # 1.08 or newer is preferred!
+ $have_uulib_module;
+ } or do {
+ $have_uulib_module = 0;
+ chomp $@; $@ =~ s/ \(you may need to install the .*\z//i;
+ do_log(5,"do_ascii: module Convert::UULIB unavailable: %s", $@);
+ };
+ }
+ return 0 if !$have_uulib_module;
snmp_count('OpsDecByUUlibAttempt');
- # prevent uunconc.c/UUDecode() from trying to create temp file in '/'
+
+ # prevent uunconc.c/UUDecode() from trying to create a temp file in '/'
my $old_env_tmpdir = $ENV{TMPDIR}; $ENV{TMPDIR} = "$tempdir/parts";
my $any_errors = 0; my $any_decoded = 0;
@@ -28417,11 +29800,15 @@ sub do_ascii($$) {
prolong_timer('do_ascii_pre'); # restart timer
$sts = Convert::UUlib::Initialize();
$sts = 0 if !defined $sts; # avoid Use of uninit. value in numeric eq (==)
- $sts==RET_OK or die "Convert::UUlib::Initialize failed: ".
- Convert::UUlib::strerror($sts);
- my $uulib_version = Convert::UUlib::GetOption(OPT_VERSION);
- !Convert::UUlib::SetOption(OPT_IGNMODE,1) or die "bad uulib OPT_IGNMODE";
- # !Convert::UUlib::SetOption(OPT_DESPERATE,1) or die "bad uulib OPT_DESPERATE";
+ $sts == Convert::UUlib::RET_OK()
+ or die "Convert::UUlib::Initialize failed: ".
+ Convert::UUlib::strerror($sts);
+ my $uulib_version =
+ Convert::UUlib::GetOption(Convert::UUlib::OPT_VERSION());
+ !Convert::UUlib::SetOption(Convert::UUlib::OPT_IGNMODE(), 1)
+ or die "bad uulib OPT_IGNMODE";
+ # !Convert::UUlib::SetOption(Convert::UUlib::OPT_DESPERATE(), 1)
+ # or die "bad uulib OPT_DESPERATE";
if (defined $action) {
$action->safe(0); # bypass safe Perl signals
POSIX::sigaction(SIGALRM,$action) or die "Can't set ALRM handler: $!";
@@ -28432,11 +29819,12 @@ sub do_ascii($$) {
$action->safe(1); # re-establish safe signal handling
POSIX::sigaction(SIGALRM,$action) or die "Can't set ALRM handler: $!";
}
- if ($sts != RET_OK) {
+ if ($sts != Convert::UUlib::RET_OK()) {
my $errmsg = Convert::UUlib::strerror($sts) . ": $!";
$errmsg .= ", (???"
- . Convert::UUlib::strerror(Convert::UUlib::GetOption(OPT_ERRNO))."???)"
- if $sts == RET_IOERR;
+ . Convert::UUlib::strerror(
+ Convert::UUlib::GetOption(Convert::UUlib::OPT_ERRNO()))."???)"
+ if $sts == Convert::UUlib::RET_IOERR();
die "Convert::UUlib::LoadFile (uulib V$uulib_version) failed: $errmsg";
}
ll(4) && do_log(4,"do_ascii: Decoding part %s (%d items), uulib V%s",
@@ -28450,7 +29838,7 @@ sub do_ascii($$) {
$j, $uu->state, Convert::UUlib::strencoding($uu->uudet),
($uu->mimetype ne '' ? ", mimetype=" . $uu->mimetype : ''),
$uu->size, $uu->filename);
- if (!($uu->state & FILE_OK)) {
+ if (!($uu->state & Convert::UUlib::FILE_OK())) {
$any_errors = 1;
do_log(1,"do_ascii: Convert::UUlib info: %s not decodable, %s",
$j,$uu->state);
@@ -28483,10 +29871,11 @@ sub do_ascii($$) {
my $size = 0 + (-s _);
$newpart_obj->size($size);
consumed_bytes($size, 'do_ascii');
- if ($sts == RET_OK && $errn==0) {
+ if ($sts == Convert::UUlib::RET_OK() && $errn==0) {
$any_decoded = 1;
do_log(4,"do_ascii: RET_OK%s", $statmsg) if defined $statmsg;
- } elsif ($sts == RET_NODATA || $sts == RET_NOEND) {
+ } elsif ($sts == Convert::UUlib::RET_NODATA() ||
+ $sts == Convert::UUlib::RET_NOEND()) {
$any_errors = 1;
do_log(-1,"do_ascii: Convert::UUlib error: %s%s",
Convert::UUlib::strerror($sts), $statmsg);
@@ -28494,7 +29883,8 @@ sub do_ascii($$) {
$any_errors = 1;
my $errmsg = Convert::UUlib::strerror($sts) . ":: $err_decode";
$errmsg .= ", " . Convert::UUlib::strerror(
- Convert::UUlib::GetOption(OPT_ERRNO) ) if $sts == RET_IOERR;
+ Convert::UUlib::GetOption(Convert::UUlib::OPT_ERRNO()) )
+ if $sts == Convert::UUlib::RET_IOERR();
die("Convert::UUlib failed: " . $errmsg . $statmsg);
}
}
@@ -28611,7 +30001,7 @@ sub do_unzip($$;$$) {
$retval;
}
-# use external decompressor program from the compress/gzip/bzip2/xz family
+# use external decompressor program from the compress/gzip/bzip2/xz/lz4 family
#
sub do_uncompress($$$) {
my($part, $tempdir, $decompressor) = @_;
@@ -28638,6 +30028,7 @@ sub do_uncompress($$$) {
$_ eq 'lzma' and $name=~s/\.lzma\z// || $name=~s/\.tlz\z/.tar/;
$_ eq 'lrz' and $name=~s/\.lrz\z//;
$_ eq 'lzo' and $name=~s/\.lzo\z//;
+ $_ eq 'lz4' and $name=~s/\.lz4\z//;
$_ eq 'rpm' and $name=~s/\.rpm\z/.cpio/;
}
push(@rn,$name) if !grep($_ eq $name, @rn);
@@ -29241,7 +30632,7 @@ sub do_unarj($$$;$) {
else { do_log(0, "unarj: error extracting: %s",exit_status_str($rv,$err)) }
# add attributes to the parent object, because we didn't remember names
# of its scrambled members
- $part->attributes_add('U') if $skippedcount;
+ $part->attributes_add('U') if $encryptedcount || $skippedcount;
$part->attributes_add('C') if $encryptedcount;
my $errn = lstat("$tempdir/parts/arj") ? 0 : 0+$!;
if ($errn != ENOENT) {
@@ -29311,10 +30702,22 @@ sub do_tnef_ext($$$) {
# use Convert-TNEF
#
+use vars qw($have_tnef_module);
sub do_tnef($$) {
my($part, $tempdir) = @_;
do_log(4, "Extracting from TNEF encapsulation (int) %s", $part->base_name);
+ if (!defined $have_tnef_module) {
+ eval {
+ require Convert::TNEF && ($have_tnef_module = 1);
+ } or do {
+ $have_tnef_module = 0;
+ chomp $@; $@ =~ s/ \(you may need to install the .*\z//i;
+ do_log(5,"module Convert::TNEF unavailable: %s", $@);
+ };
+ }
+ return 0 if !$have_tnef_module;
snmp_count('OpsDecByTnef');
+
my $tnef = Convert::TNEF->read_in($part->full_name,
{output_dir=>"$tempdir/parts", buffer_size=>16384, ignore_checksum=>1});
defined $tnef or die "Convert::TNEF failed: ".$Convert::TNEF::errstr;
@@ -29745,7 +31148,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&dkim_key_postprocess &generate_authentication_results
&dkim_make_signatures &adjust_score_by_signer_reputation
@@ -29754,9 +31157,9 @@ BEGIN {
%dkim_signing_keys_by_domain
@dkim_signing_keys_list @dkim_signing_keys_storage);
import Amavis::Util qw(min max minmax untaint ll do_log unique_list
- get_deadline proto_encode proto_decode);
+ format_time_interval get_deadline proto_encode proto_decode);
import Amavis::rfc2821_2822_Tools qw(split_address quote_rfc2821_local
- qquote_rfc2821_local format_time_interval);
+ qquote_rfc2821_local);
import Amavis::Timing qw(section_time);
import Amavis::Lookup qw(lookup lookup2);
}
@@ -29824,7 +31227,7 @@ sub dkim_key_postprocess() {
my $regexp = $domain;
$regexp =~ s/\*{2,}/*/gs; # collapse successive wildcards
# '*' is a wildcard, quote the rest
- $regexp =~ s{ ([@#/.^$|*+?(){}\[\]\\]) }{$1 eq '*' ? '.*' : '\\'.$1}gex;
+ $regexp =~ s{ ([@\#/.^\$|*+?(){}\[\]\\]) }{$1 eq '*' ? '.*' : '\\'.$1}gex;
$regexp = '^' . $regexp . '\\z'; # implicit anchors
$regexp =~ s/^\^\.\*//s; # remove leading anchor if redundant
$regexp =~ s/\.\*\\z\z//s; # remove trailing anchor if redundant
@@ -30672,7 +32075,7 @@ sub adjust_score_by_signer_reputation($) {
$r->spam_level($new_level);
my $spam_tests = 'AM.DKIM_REPUT=' .
(0+sprintf("%.3f", $new_level-$spam_level));
- if (!defined($r->spam_tests)) {
+ if (!$r->spam_tests) {
$r->spam_tests([ \$spam_tests ]);
} else {
unshift(@{$r->spam_tests}, \$spam_tests);
@@ -30687,7 +32090,7 @@ sub adjust_score_by_signer_reputation($) {
}
}
-# check if we have a valid author domain signature and do
+# check if we have a valid author domain signature, and do
# other DKIM pre-processing; called from collect_some_dkim()
#
sub collect_some_dkim_info($) {
@@ -30891,7 +32294,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.318';
+ $VERSION = '2.320';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&show_or_test_dkim_public_keys &generate_dkim_private_key
&convert_dkim_keys_file);
@@ -31133,7 +32536,7 @@ sub convert_dkim_keys_file($) {
} else {
$sender_pattern =~ s/\*{2,}/*/gs; # collapse successive wildcards
$sender_pattern =~ # '*' is a wildcard, quote the rest
- s{ ([@#/.^$|*+?(){}\[\]\\]) }{ $1 eq '*' ? '.*' : '\\'.$1 }gex;
+ s{ ([@\#/.^\$|*+?(){}\[\]\\]) }{ $1 eq '*' ? '.*' : '\\'.$1 }gex;
$sender_pattern = '^' . $sender_pattern . '\\z'; # implicit anchors
# remove trailing first, leading next, preferring /^.*\z/ -> /^/, not /\z/
$sender_pattern =~ s/\.\*\\z\z//s; # remove trailing anchor if redundant
@@ -31176,8 +32579,9 @@ __DATA__
[?%#D|#|Passed #
[? [:ccat|major] |#
OTHER|CLEAN|MTA-BLOCKED|OVERSIZED|BAD-HEADER-[:ccat|minor]|SPAMMY|SPAM|\
-UNCHECKED|BANNED (%F)|INFECTED (%V)] {[:actions_performed]}#
-, [? %p ||%p ][?%a||[?%l||LOCAL ][:client_addr_port] ][?%e||\[%e\] ]%s -> [%D|,]#
+UNCHECKED[?[:ccat|minor]||-ENCRYPTED|]|BANNED (%F)|INFECTED (%V)]#
+ {[:actions_performed]}#
+,[?%p|| %p][?%a||[?%l|| LOCAL] [:client_addr_port]][?%e|| \[%e\]] %s -> [%D|,]#
[? %q ||, quarantine: %q]#
[? %Q ||, Queue-ID: %Q]#
[? %m ||, Message-ID: %m]#
@@ -31198,8 +32602,9 @@ UNCHECKED|BANNED (%F)|INFECTED (%V)] {[:actions_performed]}#
[?%#O|#|Blocked #
[? [:ccat|major|blocking] |#
OTHER|CLEAN|MTA-BLOCKED|OVERSIZED|BAD-HEADER-[:ccat|minor]|SPAMMY|SPAM|\
-UNCHECKED|BANNED (%F)|INFECTED (%V)] {[:actions_performed]}#
-, [? %p ||%p ][?%a||[?%l||LOCAL ][:client_addr_port] ][?%e||\[%e\] ]%s -> [%O|,]#
+UNCHECKED[?[:ccat|minor]||-ENCRYPTED|]|BANNED (%F)|INFECTED (%V)]#
+ {[:actions_performed]}#
+,[?%p|| %p][?%a||[?%l|| LOCAL] [:client_addr_port]][?%e|| \[%e\]] %s -> [%O|,]#
[? %q ||, quarantine: %q]#
[? %Q ||, Queue-ID: %Q]#
[? %m ||, Message-ID: %m]#
@@ -31225,8 +32630,9 @@ __DATA__
[?%#D|#|Passed #
[? [:ccat|major] |#
OTHER|CLEAN|MTA-BLOCKED|OVERSIZED|BAD-HEADER-[:ccat|minor]|SPAMMY|SPAM|\
-UNCHECKED|BANNED (%F)|INFECTED (%V)] {[:actions_performed]}#
-, [? %p ||%p ][?%a||[?%l||LOCAL ][:client_addr_port] ][?%e||\[%e\] ]%s -> [%D|,]#
+UNCHECKED[?[:ccat|minor]||-ENCRYPTED|]|BANNED (%F)|INFECTED (%V)]#
+ {[:actions_performed]}#
+,[?%p|| %p][?%a||[?%l|| LOCAL] [:client_addr_port]][?%e|| \[%e\]] %s -> [%D|,]#
, ([ip_trace_public|%x| < ])#
[? [:tls_in] ||, tls: [:tls_in]]#
[? %q ||, quarantine: %q]#
@@ -31272,8 +32678,9 @@ UNCHECKED|BANNED (%F)|INFECTED (%V)] {[:actions_performed]}#
[?%#O|#|Blocked #
[? [:ccat|major|blocking] |#
OTHER|CLEAN|MTA-BLOCKED|OVERSIZED|BAD-HEADER-[:ccat|minor]|SPAMMY|SPAM|\
-UNCHECKED|BANNED (%F)|INFECTED (%V)] {[:actions_performed]}#
-, [? %p ||%p ][?%a||[?%l||LOCAL ][:client_addr_port] ][?%e||\[%e\] ]%s -> [%O|,]#
+UNCHECKED[?[:ccat|minor]||-ENCRYPTED|]|BANNED (%F)|INFECTED (%V)]#
+ {[:actions_performed]}#
+,[?%p|| %p][?%a||[?%l|| LOCAL] [:client_addr_port]][?%e|| \[%e\]] %s -> [%O|,]#
, ([ip_trace_public|%x| < ])#
[? [:tls_in] ||, tls: [:tls_in]]#
[? %q ||, quarantine: %q]#
diff --git a/amavisd-agent b/amavisd-agent
index 0e5565c..1ebe2bb 100755
--- a/amavisd-agent
+++ b/amavisd-agent
@@ -4,35 +4,38 @@
# This is amavisd-agent, a demo program to display
# SNMP-like counters updated by amavisd-new.
#
-# Author: Mark Martinec <mark.martinec at ijs.si>
-# Copyright (C) 2004-2009 Mark Martinec, All Rights Reserved.
+# Author: Mark Martinec <Mark.Martinec at ijs.si>
#
-# Redistribution and use in source and binary forms, with or without
-# modification, are permitted provided that the following conditions are met:
+# Copyright (c) 2004-2014, Mark Martinec
+# All rights reserved.
#
-# * Redistributions of source code must retain the above copyright notice,
-# this list of conditions and the following disclaimer.
-# * Redistributions in binary form must reproduce the above copyright notice,
-# this list of conditions and the following disclaimer in the documentation
-# and/or other materials provided with the distribution.
-# * Neither the name of the author, nor the name of the "Jozef Stefan"
-# Institute, nor the names of contributors may be used to endorse or
-# promote products derived from this software without specific prior
-# written permission.
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright notice,
+# this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright notice,
+# this list of conditions and the following disclaimer in the documentation
+# and/or other materials provided with the distribution.
#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
-# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
-# OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
-# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
-# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
-# OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
-# WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
-# OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
-# ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS
+# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+# POSSIBILITY OF SUCH DAMAGE.
#
-#(the license above is the new BSD license, and pertains to this program only)
+# The views and conclusions contained in the software and documentation are
+# those of the authors and should not be interpreted as representing official
+# policies, either expressed or implied, of the Jozef Stefan Institute.
+
+# (the above license is the 2-clause BSD license, also known as
+# a "Simplified BSD License", and pertains to this program only)
#
# Patches and problem reports are welcome.
# The latest version of this program is available at:
@@ -55,7 +58,7 @@ my($db_home) = # DB databases directory
my($wakeuptime) = 10; # -w, sleep time in seconds, may be fractional
my($repeatcount); # -c, repeat count (when defined)
-use vars qw($VERSION); $VERSION = 2.700;
+use vars qw($VERSION); $VERSION = 2.701;
use vars qw(%values %virus_by_name);
use vars qw(%virus_by_os %spam_by_os %ham_by_os);
use vars qw(%history $avg_int $uptime);
diff --git a/amavisd-nanny b/amavisd-nanny
index ddb95de..80b84dc 100755
--- a/amavisd-nanny
+++ b/amavisd-nanny
@@ -4,35 +4,38 @@
# This is amavisd-nanny, a program to show the status
# and keep an eye on the health of child processes in amavisd-new.
#
-# Author: Mark Martinec <mark.martinec at ijs.si>
-# Copyright (C) 2004-2009 Mark Martinec, All Rights Reserved.
+# Author: Mark Martinec <Mark.Martinec at ijs.si>
#
-# Redistribution and use in source and binary forms, with or without
-# modification, are permitted provided that the following conditions are met:
+# Copyright (c) 2004-2014, Mark Martinec
+# All rights reserved.
#
-# * Redistributions of source code must retain the above copyright notice,
-# this list of conditions and the following disclaimer.
-# * Redistributions in binary form must reproduce the above copyright notice,
-# this list of conditions and the following disclaimer in the documentation
-# and/or other materials provided with the distribution.
-# * Neither the name of the author, nor the name of the "Jozef Stefan"
-# Institute, nor the names of contributors may be used to endorse or
-# promote products derived from this software without specific prior
-# written permission.
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright notice,
+# this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright notice,
+# this list of conditions and the following disclaimer in the documentation
+# and/or other materials provided with the distribution.
#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
-# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
-# OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
-# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
-# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
-# OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
-# WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
-# OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
-# ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS
+# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+# POSSIBILITY OF SUCH DAMAGE.
#
-#(the license above is the new BSD license, and pertains to this program only)
+# The views and conclusions contained in the software and documentation are
+# those of the authors and should not be interpreted as representing official
+# policies, either expressed or implied, of the Jozef Stefan Institute.
+
+# (the above license is the 2-clause BSD license, also known as
+# a "Simplified BSD License", and pertains to this program only)
#
# Patches and problem reports are welcome.
# The latest version of this program is available at:
@@ -49,7 +52,7 @@ use POSIX qw(strftime);
use Time::HiRes ();
use BerkeleyDB;
-use vars qw($VERSION); $VERSION = 1.400;
+use vars qw($VERSION); $VERSION = 1.401;
my($idlettl) = 3*60*60; # idle children are sent a SIGTERM
# after this many seconds
diff --git a/amavisd-new-courier.patch b/amavisd-new-courier.patch
index 3c0ff55..de0033b 100644
--- a/amavisd-new-courier.patch
+++ b/amavisd-new-courier.patch
@@ -1,27 +1,20 @@
---- amavisd.ori 2013-06-28 20:41:55.000000000 +0200
-+++ amavisd 2013-06-28 20:56:59.839127460 +0200
-@@ -104,5 +104,5 @@
+--- amavisd.ori 2014-05-07 16:50:03.712143074 +0200
++++ amavisd 2014-05-07 16:50:36.219141034 +0200
+@@ -108,5 +108,5 @@
# Amavis::In::AMPDP
# Amavis::In::SMTP
-#( Amavis::In::Courier )
+# Amavis::In::Courier
# Amavis::Out::SMTP::Protocol
# Amavis::Out::SMTP::Session
-@@ -223,5 +223,5 @@
+@@ -230,5 +230,5 @@
fetch_modules('REQUIRED BASIC MODULES', 1, qw(
Exporter POSIX Fcntl Socket Errno Carp Time::HiRes
- IO::Handle IO::File IO::Socket IO::Socket::UNIX
+ IO::Handle IO::File IO::Socket IO::Socket::UNIX IO::Select
IO::Stringy Digest::MD5 Unix::Syslog File::Basename
Compress::Zlib MIME::Base64 MIME::QuotedPrint MIME::Words
-@@ -11695,5 +11695,5 @@
- #
- sub post_configure_hook {
--# umask(0007); # affect protection of Unix sockets created by Net::Server
-+ umask(0007); # affect protection of Unix sockets created by Net::Server
- }
-
-@@ -11722,4 +11722,34 @@
+@@ -11840,4 +11840,18 @@
### Net::Server hook
+### This hook takes place immediately after the "->run()" method is called.
@@ -38,11 +31,26 @@
+}
+
+### Net::Server hook
-+### This hook occurs just after the bind process and just before any
-+### chrooting, change of user, or change of group occurs. At this point
+ ### Occurs in the parent (master) process after (possibly) opening a log file,
+ ### creating pid file, reopening STDIN/STDOUT to /dev/null and daemonizing;
+@@ -11845,5 +11859,5 @@
+ #
+ sub post_configure_hook {
+-# umask(0007); # affect protection of Unix sockets created by Net::Server
++ umask(0007); # affect protection of Unix sockets created by Net::Server
+ }
+
+@@ -11864,9 +11878,18 @@
+ ### Net::Server hook
+ ### Occurs in the parent (master) process after binding to sockets,
+-### but before chrooting and dropping privileges
+-#
++### but before chrooting and dropping privileges. At this point
+### the process will still be running as the user who started the server.
-+sub post_bind_hook {
+ sub post_bind_hook {
+ my ($self) = @_;
+ umask(0027); # restore our preferred umask
+ set_sockets_access() if defined $warm_restart && !$warm_restart;
+ if (c('protocol') eq 'COURIER') {
+ # Allow courier to write to the socket
+ chmod(0660, $unix_socketname);
@@ -51,12 +59,9 @@
+ # Watch for courierfilter telling us to shut down
+ $self->{server}->{select}->add($self->{courierfilter_pipe});
+ }
-+}
-+
-+### Net::Server hook
- ### This hook occurs in the parent (master) process after chroot,
- ### after change of user, and change of group has occurred.
-@@ -11774,4 +11804,15 @@
+ }
+
+@@ -11924,4 +11947,15 @@
}
$spamcontrol_obj->init_pre_fork if $spamcontrol_obj;
+ if ($courierfilter_shutdown) {
@@ -72,7 +77,7 @@
+ }
my(@modules_extra) = grep(!exists $modules_basic{$_}, keys %INC);
if (@modules_extra) {
-@@ -12238,5 +12279,7 @@
+@@ -12393,5 +12427,7 @@
$ampdp_in_obj->process_policy_request($sock, $conn, \&check_mail, 0);
} elsif ($suggested_protocol eq 'COURIER') {
- die "unavailable support for protocol: $suggested_protocol";
@@ -81,7 +86,7 @@
+ $courier_in_obj->process_courier_request($sock, $conn, \&check_mail);
} elsif ($suggested_protocol eq 'QMQPqq') {
die "unavailable support for protocol: $suggested_protocol";
-@@ -12344,4 +12387,24 @@
+@@ -12501,4 +12537,24 @@
}
+### Net::Server hook
@@ -106,7 +111,7 @@
+
### Child is about to be terminated
### user customizable Net::Server hook
-@@ -17003,4 +17066,9 @@
+@@ -17662,4 +17718,9 @@
undef $Amavis::Conf::log_verbose_templ;
+# courierfilter shutdown needs can_read_hook, added in Net::Server 0.90
@@ -116,14 +121,14 @@
+
if (defined $desired_user && $daemon_user ne '') {
local($1);
-@@ -17650,4 +17718,6 @@
+@@ -18315,4 +18376,6 @@
host => $bind_to[0], # default bind, redundant, merged to @listen_sockets
listen => $listen_queue_size, # undef for a default
+ # need to set multi_port for can_read_hook
+ multi_port => $courierfilter_shutdown ? 1 : undef,
max_servers => $max_servers, # number of pre-forked children
!defined($min_servers) ? ()
-@@ -21085,5 +21155,424 @@
+@@ -21801,5 +21864,424 @@
no warnings 'uninitialized';
-BEGIN { die "Code not available for module Amavis::In::Courier" }
diff --git a/amavisd-new-qmqpqq.patch b/amavisd-new-qmqpqq.patch
index 08b4683..62e60fc 100644
--- a/amavisd-new-qmqpqq.patch
+++ b/amavisd-new-qmqpqq.patch
@@ -1,36 +1,36 @@
---- amavisd.ori 2013-06-28 20:41:55.000000000 +0200
-+++ amavisd 2013-06-28 20:58:06.896128236 +0200
-@@ -105,4 +105,5 @@
+--- amavisd.ori 2014-05-07 16:50:03.712143074 +0200
++++ amavisd 2014-05-07 16:51:36.275142228 +0200
+@@ -109,4 +109,5 @@
# Amavis::In::SMTP
#( Amavis::In::Courier )
+# Amavis::In::QMQPqq
# Amavis::Out::SMTP::Protocol
# Amavis::Out::SMTP::Session
-@@ -4625,4 +4626,5 @@
+@@ -4746,4 +4747,5 @@
$myproduct_name,
$conn->socket_port eq '' ? 'unix socket' : "port ".$conn->socket_port);
+ # must not use proto name QMQPqq in 'with'
$s .= "\n with $smtp_proto" if $smtp_proto=~/^(ES|S|L)MTPS?A?\z/i; #RFC 3848
$s .= "\n id $id" if defined $id && $id ne '';
-@@ -10837,4 +10839,5 @@
+@@ -10977,4 +10979,5 @@
$extra_code_sql_lookup $extra_code_ldap
$extra_code_in_ampdp $extra_code_in_smtp $extra_code_in_courier
+ $extra_code_in_qmqpqq
$extra_code_out_smtp $extra_code_out_pipe
$extra_code_out_bsmtp $extra_code_out_local $extra_code_p0f
-@@ -10864,4 +10867,5 @@
+@@ -11004,4 +11007,5 @@
# Amavis::In::AMPDP, Amavis::In::SMTP and In::Courier objects
use vars qw($ampdp_in_obj $smtp_in_obj $courier_in_obj);
+use vars qw($qmqpqq_in_obj); # Amavis::In::QMQPqq object
use vars qw($sql_dataset_conn_lookups); # Amavis::Out::SQL::Connection object
-@@ -11599,4 +11603,5 @@
+@@ -11749,4 +11753,5 @@
do_log(0,"SMTP-in proto code %s loaded", $extra_code_in_smtp ?'':" NOT");
do_log(0,"Courier proto code %s loaded", $extra_code_in_courier ?'':" NOT");
+ do_log(0,"QMQPqq-in proto code %s loaded", $extra_code_in_qmqpqq ?'':" NOT");
do_log(0,"SMTP-out proto code %s loaded", $extra_code_out_smtp ?'':" NOT");
do_log(0,"Pipe-out proto code %s loaded", $extra_code_out_pipe ?'':" NOT");
-@@ -12240,5 +12245,9 @@
+@@ -12395,5 +12400,9 @@
die "unavailable support for protocol: $suggested_protocol";
} elsif ($suggested_protocol eq 'QMQPqq') {
- die "unavailable support for protocol: $suggested_protocol";
@@ -41,20 +41,20 @@
+ $qmqpqq_in_obj->process_qmqpqq_request($sock,$conn,\&check_mail);
} elsif ($suggested_protocol eq 'TCP-LOOKUP') { #postfix maps, experimental
process_tcp_lookup_request($sock, $conn);
-@@ -12361,4 +12370,6 @@
+@@ -12518,4 +12527,6 @@
do_log_safe(5,"child_finish_hook: invoking DESTROY methods");
undef $smtp_in_obj; undef $ampdp_in_obj; undef $courier_in_obj;
+ undef $qmqpqq_in_obj;
+ undef $qmqpqq_in_obj;
undef $sql_storage; undef $sql_wblist; undef $sql_lookups;
undef $sql_dataset_conn_lookups; undef $sql_dataset_conn_storage;
-@@ -16797,4 +16808,5 @@
+@@ -17456,4 +17467,5 @@
$extra_code_sql_lookup, $extra_code_ldap,
$extra_code_in_ampdp, $extra_code_in_smtp, $extra_code_in_courier,
+ $extra_code_in_qmqpqq,
$extra_code_out_smtp, $extra_code_out_pipe,
$extra_code_out_bsmtp, $extra_code_out_local,
-@@ -17154,5 +17166,11 @@
+@@ -17813,5 +17825,11 @@
undef $extra_code_in_courier;
}
- if ($needed_protocols_in{'QMQPqq'}) { die "In::QMQPqq code not available" }
@@ -67,7 +67,7 @@
+ }
}
-@@ -21091,4 +21109,276 @@
+@@ -21807,4 +21825,276 @@
__DATA__
#
+package Amavis::In::QMQPqq;
@@ -344,8 +344,8 @@
+#
package Amavis::Out::SMTP::Protocol;
use strict;
---- amavisd.conf.ori 2012-08-30 17:00:16.744096000 +0200
-+++ amavisd.conf 2013-06-28 20:58:06.897128501 +0200
+--- amavisd.conf.ori 2014-05-07 16:50:20.541141753 +0200
++++ amavisd.conf 2014-05-07 16:51:36.277142196 +0200
@@ -56,6 +56,6 @@
# option(s) -p overrides $inet_socket_port and $unix_socketname
diff --git a/amavisd-release b/amavisd-release
index 9fdbf39..c185730 100755
--- a/amavisd-release
+++ b/amavisd-release
@@ -5,6 +5,44 @@
# It uses AM.PDP protocol to send a request to amavisd daemon to release
# a quarantined mail message.
#
+# Author: Mark Martinec <Mark.Martinec at ijs.si>
+#
+# Copyright (c) 2005-2014, Mark Martinec
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright notice,
+# this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright notice,
+# this list of conditions and the following disclaimer in the documentation
+# and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS
+# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+# POSSIBILITY OF SUCH DAMAGE.
+#
+# The views and conclusions contained in the software and documentation are
+# those of the authors and should not be interpreted as representing official
+# policies, either expressed or implied, of the Jozef Stefan Institute.
+
+# (the above license is the 2-clause BSD license, also known as
+# a "Simplified BSD License", and pertains to this program only)
+#
+# Patches and problem reports are welcome.
+# The latest version of this program is available at:
+# http://www.ijs.si/software/amavisd/
+#------------------------------------------------------------------------------
+
# Usage:
# amavisd-release mail_file secret_id [alt_recip1 alt_recip2 ..]
#
@@ -12,7 +50,7 @@
# $interface_policy{'SOCK'} = 'AM.PDP';
# $policy_bank{'AM.PDP'} = { protocol=>'AM.PDP' };
# $unix_socketname = '/var/amavis/amavisd.sock';
-#or:
+# or:
# $interface_policy{'9998'} = 'AM.PDP';
# $policy_bank{'AM.PDP'} = { protocol=>'AM.PDP' };
# $inet_socket_port = [10024,9998];
@@ -30,41 +68,7 @@
# secret_id widens the right to release to anyone who can connect to amavisd
# socket (Unix or inet). Access to the socket therefore needs to be restricted
# using socket protection (unix socket) or @inet_acl (for inet socket).
-#
-# Author: Mark Martinec <mark.martinec at ijs.si>
-# Copyright (C) 2005-2012 Mark Martinec, All Rights Reserved.
-#
-# Redistribution and use in source and binary forms, with or without
-# modification, are permitted provided that the following conditions are met:
-#
-# * Redistributions of source code must retain the above copyright notice,
-# this list of conditions and the following disclaimer.
-# * Redistributions in binary form must reproduce the above copyright notice,
-# this list of conditions and the following disclaimer in the documentation
-# and/or other materials provided with the distribution.
-# * Neither the name of the author, nor the name of the "Jozef Stefan"
-# Institute, nor the names of contributors may be used to endorse or
-# promote products derived from this software without specific prior
-# written permission.
-#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
-# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
-# OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
-# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
-# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
-# OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
-# WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
-# OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
-# ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-#
-#(the license above is the new BSD license, and pertains to this program only)
-#
-# Patches and problem reports are welcome.
-# The latest version of this program is available at:
-# http://www.ijs.si/software/amavisd/
-#------------------------------------------------------------------------------
+
use warnings;
use warnings FATAL => 'utf8';
@@ -74,7 +78,7 @@ use re 'taint';
use IO::Socket;
use Time::HiRes ();
-use vars qw($VERSION); $VERSION = 2.001;
+use vars qw($VERSION); $VERSION = 2.002;
use vars qw($log_level $socketname $have_inet4 $have_inet6 $have_socket_ip);
BEGIN {
diff --git a/amavisd-signer b/amavisd-signer
index 75f9f14..f154646 100755
--- a/amavisd-signer
+++ b/amavisd-signer
@@ -6,50 +6,53 @@
# and provides two services: choosing a signing key, and signing a
# message digest with a chosen DKIM private key.
#
-# Using a separate signing service (which may run under a dedicated UID or
-# GID or as root, having exclusive access to private keys) releaves amavisd
-# process from needing to have access to private keys. Separating roles can
-# provide improved protection for DKIM private keys, and/or can provide more
-# flexibility in choosing a signing key.
+# Author: Mark Martinec <Mark.Martinec at ijs.si>
#
-# Usage:
-# amavisd-signer &
-#
-# Author: Mark Martinec <mark.martinec at ijs.si>
-# Copyright (C) 2010 Mark Martinec, All Rights Reserved.
+# Copyright (c) 2010-2014, Mark Martinec
+# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
-# modification, are permitted provided that the following conditions are met:
-#
-# * Redistributions of source code must retain the above copyright notice,
-# this list of conditions and the following disclaimer.
-# * Redistributions in binary form must reproduce the above copyright notice,
-# this list of conditions and the following disclaimer in the documentation
-# and/or other materials provided with the distribution.
-# * Neither the name of the author, nor the name of the "Jozef Stefan"
-# Institute, nor the names of contributors may be used to endorse or
-# promote products derived from this software without specific prior
-# written permission.
-#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
-# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
-# OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
-# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
-# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
-# OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
-# WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
-# OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
-# ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-#
-#(the license above is the new BSD license, and pertains to this program only)
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright notice,
+# this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright notice,
+# this list of conditions and the following disclaimer in the documentation
+# and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS
+# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+# POSSIBILITY OF SUCH DAMAGE.
+#
+# The views and conclusions contained in the software and documentation are
+# those of the authors and should not be interpreted as representing official
+# policies, either expressed or implied, of the Jozef Stefan Institute.
+
+# (the above license is the 2-clause BSD license, also known as
+# a "Simplified BSD License", and pertains to this program only)
#
# Patches and problem reports are welcome.
# The latest version of this program is available at:
# http://www.ijs.si/software/amavisd/
#------------------------------------------------------------------------------
+# Using a separate signing service (which may run under a dedicated UID or
+# GID or as root, having exclusive access to private keys) releaves amavisd
+# process from needing to have access to private keys. Separating roles can
+# provide improved protection for DKIM private keys, and/or can provide more
+# flexibility in choosing a signing key.
+#
+# Usage:
+# amavisd-signer &
+
package AmavisSigner;
use strict;
@@ -77,7 +80,7 @@ use vars qw(
$syslog_ident $syslog_facility
);
-$VERSION = 1.000; # 20100730
+$VERSION = 1.001; # 20100730
#
# Please adjust the following settings as necessary:
diff --git a/amavisd-snmp-subagent b/amavisd-snmp-subagent
index e8332a9..d4c313a 100755
--- a/amavisd-snmp-subagent
+++ b/amavisd-snmp-subagent
@@ -4,34 +4,37 @@
# This program implements a SNMP AgentX (RFC 2741) subagent for amavisd-new.
#
# Author: Mark Martinec <Mark.Martinec at ijs.si>
-# Copyright (C) 2009,2010,2011 Mark Martinec, All Rights Reserved.
#
-# Redistribution and use in source and binary forms, with or without
-# modification, are permitted provided that the following conditions are met:
+# Copyright (c) 2009-2014, Mark Martinec
+# All rights reserved.
#
-# * Redistributions of source code must retain the above copyright notice,
-# this list of conditions and the following disclaimer.
-# * Redistributions in binary form must reproduce the above copyright notice,
-# this list of conditions and the following disclaimer in the documentation
-# and/or other materials provided with the distribution.
-# * Neither the name of the author, nor the name of the "Jozef Stefan"
-# Institute, nor the names of contributors may be used to endorse or
-# promote products derived from this software without specific prior
-# written permission.
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright notice,
+# this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright notice,
+# this list of conditions and the following disclaimer in the documentation
+# and/or other materials provided with the distribution.
#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
-# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
-# OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
-# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
-# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
-# OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
-# WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
-# OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
-# ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS
+# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+# POSSIBILITY OF SUCH DAMAGE.
#
-#(the license above is the new BSD license, and pertains to this program only)
+# The views and conclusions contained in the software and documentation are
+# those of the authors and should not be interpreted as representing official
+# policies, either expressed or implied, of the Jozef Stefan Institute.
+
+# (the above license is the 2-clause BSD license, also known as
+# a "Simplified BSD License", and pertains to this program only)
#
# Patches and problem reports are welcome.
# The latest version of this program is available at:
@@ -53,11 +56,11 @@ use IO::File qw(O_RDONLY O_WRONLY O_RDWR O_CREAT O_EXCL);
use Unix::Syslog qw(:macros :subs);
use BerkeleyDB;
-use vars qw($VERSION); $VERSION = 1.006;
+use vars qw($VERSION); $VERSION = 1.007;
use vars qw($myversion $myproduct_name $myversion_id $myversion_date);
$myproduct_name = 'amavis-agentx';
-$myversion_id = '1.6'; $myversion_date = '20120627';
+$myversion_id = '1.7'; $myversion_date = '20140127';
$myversion = "$myproduct_name-$myversion_id ($myversion_date)";
my($agent_name) = $myproduct_name;
diff --git a/amavisd-snmp-subagent-zmq b/amavisd-snmp-subagent-zmq
index 1544ef0..d239e84 100755
--- a/amavisd-snmp-subagent-zmq
+++ b/amavisd-snmp-subagent-zmq
@@ -4,34 +4,37 @@
# This program implements an SNMP AgentX (RFC 2741) subagent for amavisd-new.
#
# Author: Mark Martinec <Mark.Martinec at ijs.si>
-# Copyright (C) 2012,2013 Mark Martinec, All Rights Reserved.
#
-# Redistribution and use in source and binary forms, with or without
-# modification, are permitted provided that the following conditions are met:
+# Copyright (c) 2012-2014, Mark Martinec
+# All rights reserved.
#
-# * Redistributions of source code must retain the above copyright notice,
-# this list of conditions and the following disclaimer.
-# * Redistributions in binary form must reproduce the above copyright notice,
-# this list of conditions and the following disclaimer in the documentation
-# and/or other materials provided with the distribution.
-# * Neither the name of the author, nor the name of the "Jozef Stefan"
-# Institute, nor the names of contributors may be used to endorse or
-# promote products derived from this software without specific prior
-# written permission.
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright notice,
+# this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright notice,
+# this list of conditions and the following disclaimer in the documentation
+# and/or other materials provided with the distribution.
#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
-# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
-# OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
-# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
-# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
-# OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
-# WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
-# OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
-# ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS
+# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+# POSSIBILITY OF SUCH DAMAGE.
#
-#(the license above is the new BSD license, and pertains to this program only)
+# The views and conclusions contained in the software and documentation are
+# those of the authors and should not be interpreted as representing official
+# policies, either expressed or implied, of the Jozef Stefan Institute.
+
+# (the above license is the 2-clause BSD license, also known as
+# a "Simplified BSD License", and pertains to this program only)
#
# Patches and problem reports are welcome.
# The latest version of this program is available at:
@@ -58,14 +61,14 @@ use NetSNMP::default_store qw(:all);
use NetSNMP::agent qw(:all);
use NetSNMP::agent::default_store (':all');
-use vars qw($VERSION); $VERSION = 2.008001;
+use vars qw($VERSION); $VERSION = 2.008002;
use vars qw($myversion $myproduct_name $myversion_id $myversion_date);
use vars qw($syslog_ident $syslog_facility $log_level);
use vars qw($zmq_ctx $zmq_sock $snmp_sock_specs $agentx_sock_specs);
$myproduct_name = 'amavis-agentx';
-$myversion_id = '2.8.1'; $myversion_date = '20130321';
+$myversion_id = '2.9.0'; $myversion_date = '20140506';
$myversion = "$myproduct_name-$myversion_id ($myversion_date)";
my $agent_name = $myproduct_name;
diff --git a/amavisd-snmp-subagent_init.sh b/amavisd-snmp-subagent_init.sh
new file mode 100755
index 0000000..4336eb4
--- /dev/null
+++ b/amavisd-snmp-subagent_init.sh
@@ -0,0 +1,24 @@
+#!/bin/sh
+
+# PROVIDE: amavisd_snmp_subagent
+# REQUIRE: DAEMON
+# KEYWORD: shutdown
+
+. /etc/rc.subr
+
+name="amavisd_snmp_subagent"
+desc="Amavis SNMP AgentX"
+rcvar="amavisd_snmp_subagent_enable"
+
+command=/usr/local/sbin/amavisd-snmp-subagent-zmq
+command_interpreter=/usr/bin/perl
+
+load_rc_config $name
+
+: ${amavisd_snmp_subagent_enable:="NO"}
+: ${amavisd_snmp_subagent_pidfile:="/var/run/amavisd-snmp-subagent.pid"}
+: ${amavisd_snmp_subagent_flags:="-P ${amavisd_snmp_subagent_pidfile}"}
+
+pidfile=${amavisd_snmp_subagent_pidfile}
+
+run_rc_command "$1"
diff --git a/amavisd-status b/amavisd-status
index 86851cf..cefbf38 100755
--- a/amavisd-status
+++ b/amavisd-status
@@ -4,35 +4,38 @@
# This is amavisd-status, a program to show status of child processes
# in amavisd-new.
#
-# Author: Mark Martinec <mark.martinec at ijs.si>
-# Copyright (C) 2012,2013 Mark Martinec, All Rights Reserved.
+# Author: Mark Martinec <Mark.Martinec at ijs.si>
#
-# Redistribution and use in source and binary forms, with or without
-# modification, are permitted provided that the following conditions are met:
+# Copyright (c) 2012-2014, Mark Martinec
+# All rights reserved.
#
-# * Redistributions of source code must retain the above copyright notice,
-# this list of conditions and the following disclaimer.
-# * Redistributions in binary form must reproduce the above copyright notice,
-# this list of conditions and the following disclaimer in the documentation
-# and/or other materials provided with the distribution.
-# * Neither the name of the author, nor the name of the "Jozef Stefan"
-# Institute, nor the names of contributors may be used to endorse or
-# promote products derived from this software without specific prior
-# written permission.
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright notice,
+# this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright notice,
+# this list of conditions and the following disclaimer in the documentation
+# and/or other materials provided with the distribution.
#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
-# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
-# OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
-# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
-# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
-# OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
-# WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
-# OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
-# ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS
+# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+# POSSIBILITY OF SUCH DAMAGE.
#
-#(the license above is the new BSD license, and pertains to this program only)
+# The views and conclusions contained in the software and documentation are
+# those of the authors and should not be interpreted as representing official
+# policies, either expressed or implied, of the Jozef Stefan Institute.
+
+# (the above license is the 2-clause BSD license, also known as
+# a "Simplified BSD License", and pertains to this program only)
#
# Patches and problem reports are welcome.
# The latest version of this program is available at:
@@ -49,9 +52,13 @@ use Errno qw(ESRCH ENOENT);
use POSIX qw(strftime);
use Time::HiRes ();
-use vars qw($VERSION); $VERSION = 2.008001;
+use vars qw($VERSION); $VERSION = 2.008002;
+use vars qw($myversion $myproduct_name $myversion_id $myversion_date);
use vars qw($outer_sock_specs);
+$myproduct_name = 'amavisd-status';
+$myversion_id = '2.9.0'; $myversion_date = '20140506';
+$myversion = "$myproduct_name-$myversion_id ($myversion_date)";
### USER CONFIGURABLE:
diff --git a/amavisd-submit b/amavisd-submit
index 97d40aa..d95c752 100755
--- a/amavisd-submit
+++ b/amavisd-submit
@@ -7,51 +7,53 @@
# AM.PDP protocol with the amavisd daemon. See README.protocol for the
# description of AM.PDP protocol.
#
-# Usage:
-# amavisd-submit sender recip1 recip2 recip3 ... <email.msg
-# (should run under the same GID as amavisd, to make files accessible to it)
-#
-# To be placed in amavisd.conf:
-# $interface_policy{'SOCK'} = 'AM.PDP';
-# $policy_bank{'AM.PDP'} = { protocol=>'AM.PDP' };
-# $unix_socketname = '/var/amavis/amavisd.sock';
+# Author: Mark Martinec <Mark.Martinec at ijs.si>
#
-#
-# Author: Mark Martinec <mark.martinec at ijs.si>
-# Copyright (C) 2004,2010,2013 Mark Martinec, All Rights Reserved.
+# Copyright (c) 2004,2010-2014, Mark Martinec
+# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
-# modification, are permitted provided that the following conditions are met:
-#
-# * Redistributions of source code must retain the above copyright notice,
-# this list of conditions and the following disclaimer.
-# * Redistributions in binary form must reproduce the above copyright notice,
-# this list of conditions and the following disclaimer in the documentation
-# and/or other materials provided with the distribution.
-# * Neither the name of the author, nor the name of the "Jozef Stefan"
-# Institute, nor the names of contributors may be used to endorse or
-# promote products derived from this software without specific prior
-# written permission.
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright notice,
+# this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright notice,
+# this list of conditions and the following disclaimer in the documentation
+# and/or other materials provided with the distribution.
#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
-# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
-# OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
-# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
-# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
-# OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
-# WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
-# OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
-# ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS
+# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+# POSSIBILITY OF SUCH DAMAGE.
#
-#(the license above is the new BSD license, and pertains to this program only)
+# The views and conclusions contained in the software and documentation are
+# those of the authors and should not be interpreted as representing official
+# policies, either expressed or implied, of the Jozef Stefan Institute.
+
+# (the above license is the 2-clause BSD license, also known as
+# a "Simplified BSD License", and pertains to this program only)
#
# Patches and problem reports are welcome.
# The latest version of this program is available at:
# http://www.ijs.si/software/amavisd/
#------------------------------------------------------------------------------
+# Usage:
+# amavisd-submit sender recip1 recip2 recip3 ... <email.msg
+# (should run under the same GID as amavisd, to make files accessible to it)
+#
+# To be placed in amavisd.conf:
+# $interface_policy{'SOCK'} = 'AM.PDP';
+# $policy_bank{'AM.PDP'} = { protocol=>'AM.PDP' };
+# $unix_socketname = '/var/amavis/amavisd.sock';
+
use warnings;
use warnings FATAL => 'utf8';
no warnings 'uninitialized';
@@ -63,7 +65,7 @@ use File::Temp ();
use Time::HiRes ();
BEGIN {
- use vars qw($VERSION); $VERSION = 2.100;
+ use vars qw($VERSION); $VERSION = 2.101;
use vars qw($log_level $socketname $tempbase $io_socket_module_name);
diff --git a/amavisd.conf b/amavisd.conf
index 1f3cc83..a09597f 100644
--- a/amavisd.conf
+++ b/amavisd.conf
@@ -109,6 +109,9 @@ $sa_local_tests_only = 0; # only tests which do not require internet access?
# ['DBI:mysql:database=mail;host=host2', 'username2', 'password2'],
# ["DBI:SQLite:dbname=$MYHOME/sql/mail_prefs.sqlite", '', ''] );
# @storage_sql_dsn = @lookup_sql_dsn; # none, same, or separate database
+# @storage_redis_dsn = ( {server=>'127.0.0.1:6379', db_id=>1} );
+# $redis_logging_key = 'amavis-log';
+# $redis_logging_queue_size_limit = 300000; # about 250 MB / 100000
# $timestamp_fmt_mysql = 1; # if using MySQL *and* msgs.time_iso is TIMESTAMP;
# defaults to 0, which is good for non-MySQL or if msgs.time_iso is CHAR(16)
diff --git a/amavisd.conf-default b/amavisd.conf-default
index 20f7a70..a6d5896 100644
--- a/amavisd.conf-default
+++ b/amavisd.conf-default
@@ -41,8 +41,8 @@ use strict;
# $nanny_details_level = 1; # verbosity: 0, 1, 2
# @additional_perl_modules = ();
# @local_domains_maps=(\%local_domains,\@local_domains_acl,\$local_domains_re);
-# @mynetworks = qw( 127.0.0.0/8 [::1] [fe80::]/10 [fc00::]/7
-# 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 169.254.0.0/16 );
+# @mynetworks = qw( 127.0.0.0/8 [::1] 169.254.0.0/16 [fe80::]/10
+# 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 [fc00::]/7 );
# @mynetworks_maps = (\@mynetworks);
# @client_ipaddr_policy = map { $_ => 'MYNETS' } @mynetworks_maps;
@@ -290,6 +290,7 @@ use strict;
# $defang_all = undef; # mostly for testing
# $allow_disclaimers = undef;
+# $outbound_disclaimers_only = undef;
# $enable_anomy_sanitizer = 0;
# @anomy_sanitizer_args = (); # a config file or list of var=value pairs
# $altermime = 'altermime'; # a path to the program
@@ -519,8 +520,13 @@ use strict;
# $trim_trailing_space_in_lookup_result_fields = 0;
# $lookup_maps_imply_sql_and_ldap = 1;
+# @storage_redis_dsn = (); # Redis server(s) for pen pals, IP reput, JSON log
# $storage_redis_ttl = 16*24*60*60;
-# @storage_redis_dsn = (); # Redis server(s) for pen pals, or empty
+# $enable_ip_repu = 1;
+# @ip_repu_ignore_networks = ();
+# @ip_repu_ignore_maps = (\@ip_repu_ignore_networks);
+# $redis_logging_key = undef;
+# $redis_logging_queue_size_limit = undef;
# @lookup_sql_dsn = (); # SQL data source name for lookups, or empty
# @storage_sql_dsn = (); # SQL data source name for log/quarantine, or empty
@@ -623,7 +629,7 @@ use strict;
## MAPPING A CONTENTS CATEGORY TO A SETTING CHOSEN
-# %final_destiny_by_ccat = (
+# %final_destiny_maps_by_ccat = (
# CC_VIRUS, sub { c('final_virus_destiny') },
# CC_BANNED, sub { c('final_banned_destiny') },
# CC_UNCHECKED, sub { c('final_unchecked_destiny') },
@@ -828,7 +834,7 @@ use strict;
## $undecipherable_subject_tag $localpart_is_case_sensitive
## $recipient_delimiter $replace_existing_extension
## $hdr_encoding $bdy_encoding $hdr_encoding_qb
- ## $allow_disclaimers
+ ## $allow_disclaimers $outbound_disclaimers_only
## $prepend_header_fields_hdridx
## $allow_fixing_improper_header
## $allow_fixing_improper_header_folding $allow_fixing_long_header_lines
@@ -844,9 +850,10 @@ use strict;
## @altermime_args_disclaimer @disclaimer_options_bysender_maps
## %signed_header_fields @dkim_signature_options_bysender_maps
## $enable_dkim_verification $enable_dkim_signing $dkim_signing_service
- ## $dkim_minimum_key_bits $enable_ldap
+ ## $dkim_minimum_key_bits $enable_ldap $enable_ip_repu $redis_logging_key
##
- ## @local_domains_maps @mynetworks_maps @client_ipaddr_policy
+ ## @local_domains_maps
+ ## @mynetworks_maps @client_ipaddr_policy @ip_repu_ignore_maps
## @forward_method_maps @newvirus_admin_maps @banned_filename_maps
## @spam_quarantine_bysender_to_maps
## @spam_tag_level_maps @spam_tag2_level_maps @spam_tag3_level_maps
@@ -866,7 +873,7 @@ use strict;
## @remove_existing_spam_headers_maps
## @sa_userconf_maps @sa_username_maps
##
- ## %final_destiny_by_ccat %forward_method_maps_by_ccat
+ ## %final_destiny_maps_by_ccat %forward_method_maps_by_ccat
## %lovers_maps_by_ccat %defang_maps_by_ccat %subject_tag_maps_by_ccat
## %quarantine_method_by_ccat %quarantine_to_maps_by_ccat
## %notify_admin_templ_by_ccat %notify_recips_templ_by_ccat
diff --git a/p0f-analyzer.pl b/p0f-analyzer.pl
index 118094e..e445d8f 100755
--- a/p0f-analyzer.pl
+++ b/p0f-analyzer.pl
@@ -6,34 +6,37 @@
# over UDP from some program (like amavisd-new) about collected data.
#
# Author: Mark Martinec <Mark.Martinec at ijs.si>
-# Copyright (C) 2006,2012,2013 Mark Martinec, All Rights Reserved.
#
-# Redistribution and use in source and binary forms, with or without
-# modification, are permitted provided that the following conditions are met:
+# Copyright (c) 2006,2012-2014, Mark Martinec
+# All rights reserved.
#
-# * Redistributions of source code must retain the above copyright notice,
-# this list of conditions and the following disclaimer.
-# * Redistributions in binary form must reproduce the above copyright notice,
-# this list of conditions and the following disclaimer in the documentation
-# and/or other materials provided with the distribution.
-# * Neither the name of the author, nor the name of the "Jozef Stefan"
-# Institute, nor the names of contributors may be used to endorse or
-# promote products derived from this software without specific prior
-# written permission.
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright notice,
+# this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright notice,
+# this list of conditions and the following disclaimer in the documentation
+# and/or other materials provided with the distribution.
#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
-# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
-# OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
-# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
-# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
-# OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
-# WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
-# OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
-# ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS
+# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+# POSSIBILITY OF SUCH DAMAGE.
#
-#(the license above is the new BSD license, and pertains to this program only)
+# The views and conclusions contained in the software and documentation are
+# those of the authors and should not be interpreted as representing official
+# policies, either expressed or implied, of the Jozef Stefan Institute.
+
+# (the above license is the 2-clause BSD license, also known as
+# a "Simplified BSD License", and pertains to this program only)
#
# Patches and problem reports are welcome.
# The latest version of this program is available at:
@@ -47,7 +50,7 @@
use Socket;
use IO::File qw(O_RDONLY);
use vars qw($VERSION);
- $VERSION = '1.501';
+ $VERSION = '1.502';
# Example usage with p0f v3:
# p0f -i eth0 'tcp and dst host mail.example.org' 2>&1 | p0f-analyzer.pl 2345
diff --git a/p0f-analyzer.pl-old b/p0f-analyzer.pl-old
index 9c63638..c15cc9c 100755
--- a/p0f-analyzer.pl-old
+++ b/p0f-analyzer.pl-old
@@ -5,35 +5,38 @@
# utility, keep results in cache for a couple of minutes, and answer queries
# over UDP from some program (like amavisd-new) about collected data.
#
-# Author: Mark Martinec <mark.martinec at ijs.si>
-# Copyright (C) 2006 Mark Martinec, All Rights Reserved.
+# Author: Mark Martinec <Mark.Martinec at ijs.si>
#
-# Redistribution and use in source and binary forms, with or without
-# modification, are permitted provided that the following conditions are met:
+# Copyright (c) 2006, Mark Martinec
+# All rights reserved.
#
-# * Redistributions of source code must retain the above copyright notice,
-# this list of conditions and the following disclaimer.
-# * Redistributions in binary form must reproduce the above copyright notice,
-# this list of conditions and the following disclaimer in the documentation
-# and/or other materials provided with the distribution.
-# * Neither the name of the author, nor the name of the "Jozef Stefan"
-# Institute, nor the names of contributors may be used to endorse or
-# promote products derived from this software without specific prior
-# written permission.
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright notice,
+# this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright notice,
+# this list of conditions and the following disclaimer in the documentation
+# and/or other materials provided with the distribution.
#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
-# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
-# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
-# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER
-# OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
-# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
-# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
-# OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
-# WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
-# OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
-# ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS
+# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+# POSSIBILITY OF SUCH DAMAGE.
#
-#(the license above is the new BSD license, and pertains to this program only)
+# The views and conclusions contained in the software and documentation are
+# those of the authors and should not be interpreted as representing official
+# policies, either expressed or implied, of the Jozef Stefan Institute.
+
+# (the above license is the 2-clause BSD license, also known as
+# a "Simplified BSD License", and pertains to this program only)
#
# Patches and problem reports are welcome.
# The latest version of this program is available at:
@@ -45,7 +48,7 @@
use Errno qw(EAGAIN EINTR);
use Socket;
use vars qw($VERSION);
- $VERSION = '1.400';
+ $VERSION = '1.401';
# Example usage:
# p0f -i bge0 -l 'tcp dst port 25' 2>&1 | p0f-analyzer.pl 2345
--
Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/amavisd-new/pkg-amavisd-new.git
More information about the Amavisd-new-commits
mailing list