[Amavisd-new-commits] [SCM] Debian packaging for amavisd-new branch, master, updated. debian/1%2.7.0-1-17-gc4baab8
Alexander Wirt
formorer at debian.org
Sun Apr 29 06:52:42 UTC 2012
The following commit has been merged in the master branch:
commit a1fea9f3337e8137a263cec5123e04647e3f48da
Author: Alexander Wirt <formorer at debian.org>
Date: Sun Apr 29 08:26:52 2012 +0200
Imported Upstream version 2.7.1
diff --git a/README_FILES/README.customize b/README_FILES/README.customize
index 000b014..326f09a 100644
--- a/README_FILES/README.customize
+++ b/README_FILES/README.customize
@@ -1,7 +1,7 @@
Customization of notification messages and log entries
======================================================
Mark Martinec <Mark.Martinec at ijs.si>,
- 2002, 2004, 2006, 2007, 2008, 2010, 2011
+ 2002, 2004, 2006, 2007, 2008, 2010, 2011, 2012
Since March 2002 amavisd-new provides a way to customize e-mail notification
messages that are sent in response to a virus (and spam) detection,
@@ -68,9 +68,10 @@ The substitution text for the following simple macros is built-in:
is still in place);
h dns name of this host, or configurable name (variable $myhostname)
HOSTNAME same as h
- n amavis internal task id (also called message id, am_id) as shown
+ n amavis internal log id (also called task id, am_id) as shown
in the log and by amavisd-nanny, e.g. 58725-05-2
- b message digest of mail body (MD5, hex)
+ b message digest of a mail body: digest calculated by MD5 algorithm,
+ encoded as hex digits, high nybble first;
date_unix_utc timestamp of the message reception - Unix time
(seconds since 1970-01-01T00:00Z as a decimal integer)
@@ -80,7 +81,8 @@ The substitution text for the following simple macros is built-in:
date_rfc2822_local timestamp of the message reception - RFC 2822
local date-time format
week_iso8601 returns an ISO 8601 week number (between 1 and 53)
- corresponding to date_iso8601_local
+ corresponding to the current message reception time
+ (date_iso8601_local)
partition_tag give the current value of a partition_tag attribute which
will be stored in SQL records when SQL quarantining or
logging is enabled; the value is derived from a setting
@@ -103,21 +105,26 @@ The substitution text for the following simple macros is built-in:
through a TLS-encrypted session, otherwise result is empty
t first entry in the 'Received' trace of the mail header
- g original SMTP session client DNS name (empty if unknown, e.g. no XFORWARD)
- e best guess of the originator IP address collected from the Received trace
- client_helo client-supplied EHLO/HELO domain name from the original session
- as obtained through XFORWARD or from a 'helo_name' AM.PDP attribute;
- client_addr original SMTP session client source IP address, same as %a
- as obtained through XFORWARD or from a 'helo_name' AM.PDP attribute
- or by parsing the topmost Received header field with a valid IP address;
+ g original SMTP client DNS name as obtained from an XFORWARD NAME
+ field, or from a 'client_name' attribute in an AM.PDP request;
+ empty if unknown;
+ e best guess of the originator IP address: the bottom-most public IP
+ address as obtained by parsing Received trace fields;
+ client_helo client-supplied EHLO/HELO domain name from the original
+ SMTP session as obtained through XFORWARD HELO or from a 'helo_name'
+ attribute in an AM.PDP request;
+ client_addr original SMTP client source IP address, same as %a,
+ as obtained through XFORWARD ADDR or from a 'client_address' attribute
+ in an AM.PDP request, or by parsing the topmost Received header field
+ with a valid IP address as a last resort;
a is a synonym for client_addr
- client_port original SMTP session client source TCP port number
- as obtained through XFORWARD or from a 'helo_name' AM.PDP attribute;
+ client_port original SMTP client source TCP port number as obtained through
+ XFORWARD PORT or from a 'client_port' attribute in an AM.PDP request;
client_addr_port combines addr and port, similar to: \[%a\]:[:client_port]
l (letter ell, suggesting 'local') is true if a variable 'originating' is
true, and is an empty string otherwise; the boolean variable 'originating'
is under policy bank control, and usually corresponds to a sending host
- (SMTP client's IP address) matching @mynetworks_maps, or the client being
+ (SMTP client's IP address) matching @mynetworks_maps, or a client being
an authenticated roaming user;
o best attempt at determining true sender of the virus - normally same as %s
S address that will get sender notification;
@@ -157,8 +164,9 @@ The substitution text for the following simple macros is built-in:
which would have been added (or will be added later) by the local
delivery agent if mail would have been delivered to a mailbox.
z original mail size (in bytes)
- i long-term unique mail_id on this system, possibly used in log and in
- quarantine names (also in releasing from a quarantine), e.g. jaUETfyBMJHG
+ i long-term unique mail_id on this system, possibly used in log
+ and in quarantine names (also in releasing from a quarantine),
+ encoded in base64url (RFC 4648), e.g. jaUETfyBMJHG
q list of quarantine mailbox names, or empty if not quarantined
ccat_maj (deprecated, use [:ccat|major]) a major category number
diff --git a/README_FILES/README.sql-mysql b/README_FILES/README.sql-mysql
index 583fd37..53f37dc 100644
--- a/README_FILES/README.sql-mysql
+++ b/README_FILES/README.sql-mysql
@@ -106,7 +106,7 @@ CREATE TABLE policy (
virus_lover char(1) default NULL, -- Y/N
spam_lover char(1) default NULL, -- Y/N
- unchecked_lovers_maps char(1) default NULL, -- Y/N
+ unchecked_lover char(1) default NULL, -- Y/N
banned_files_lover char(1) default NULL, -- Y/N
bad_header_lover char(1) default NULL, -- Y/N
@@ -380,14 +380,14 @@ Some examples of a query:
-- mail from last two minutes:
SELECT
UNIX_TIMESTAMP()-msgs.time_num AS age, SUBSTRING(policy,1,2) as pb,
- content AS c, dsn_sent as dsn, ds, bspam_level AS level, size,
+ msgrcpt.content AS c, dsn_sent as dsn, ds, bspam_level AS level, size,
SUBSTRING(sender.email,1,18) AS s,
SUBSTRING(recip.email,1,18) AS r,
SUBSTRING(msgs.subject,1,10) AS subj
FROM msgs LEFT JOIN msgrcpt ON msgs.mail_id=msgrcpt.mail_id
LEFT JOIN maddr AS sender ON msgs.sid=sender.id
LEFT JOIN maddr AS recip ON msgrcpt.rid=recip.id
- WHERE content IS NOT NULL AND UNIX_TIMESTAMP()-msgs.time_num < 120
+ WHERE msgrcpt.content IS NOT NULL AND UNIX_TIMESTAMP()-msgs.time_num < 120
ORDER BY msgs.time_num DESC;
-- clean messages ordered by count, grouped by domain:
@@ -396,7 +396,7 @@ SELECT count(*) as cnt, avg(bspam_level), sender.domain
LEFT JOIN msgrcpt ON msgs.mail_id=msgrcpt.mail_id
LEFT JOIN maddr AS sender ON msgs.sid=sender.id
LEFT JOIN maddr AS recip ON msgrcpt.rid=recip.id
- WHERE content='C'
+ WHERE msgrcpt.content='C'
GROUP BY sender.domain ORDER BY cnt DESC LIMIT 50;
-- top spamy domains with >10 messages, sorted by spam average,
@@ -448,87 +448,89 @@ constraint may (or may not) facilitate deletion of expired records.
ADD FOREIGN KEY (rid) REFERENCES maddr(id) ON DELETE RESTRICT;
-BRIEF EXAMPLE of a log/report/quarantine database housekeeping
-==============================================================
+EXAMPLE of a log/report/quarantine database housekeeping
+========================================================
-DELETE FROM msgs WHERE time_num < UNIX_TIMESTAMP() - 14*24*60*60;
-DELETE FROM msgs WHERE time_num < UNIX_TIMESTAMP() - 60*60 AND content IS NULL;
-DELETE FROM maddr
- WHERE NOT EXISTS (SELECT 1 FROM msgs WHERE sid=id)
- AND NOT EXISTS (SELECT 1 FROM msgrcpt WHERE rid=id);
+Using a changing partition_tag, perhaps by using an ISO 8601 week number
+(value 1 to 53) as a partition_tag:
+ $partition_tag =
+ sub { my($msginfo)=@_; sprintf("%02d",iso8601_week($msginfo->rx_time)) };
-BRIEF EQUIVALENT EXAMPLE based on time_iso if its data type is TIMESTAMPS
-=========================================================================
-(don't forget to set: $timestamp_fmt_mysql=1 in amavisd.conf)
+allows for probably the fastest method of purging old records, e.g.:
-DELETE FROM msgs WHERE time_iso < UTC_TIMESTAMP() - INTERVAL 14 day;
-DELETE FROM msgs WHERE time_iso < UTC_TIMESTAMP() - INTERVAL 1 hour
- AND content IS NULL;
-DELETE FROM maddr
- WHERE NOT EXISTS (SELECT 1 FROM msgs WHERE sid=id)
- AND NOT EXISTS (SELECT 1 FROM msgrcpt WHERE rid=id);
+DELETE FROM msgs WHERE partition_tag >= 13 AND partition_tag <= 23;
+DELETE FROM msgrcpt WHERE partition_tag >= 13 AND partition_tag <= 23;
+DELETE FROM quarantine WHERE partition_tag >= 13 AND partition_tag <= 23;
+DELETE FROM maddr WHERE partition_tag >= 13 AND partition_tag <= 23;
+ Note: using native SQL table partitioning as offered by MySQL, the above
+ may be even faster by dropping entire partitions. Not documented here.
+
+
+Alternatively, purge records from table msgs by their creation time:
+
+DELETE FROM msgs WHERE time_num < UNIX_TIMESTAMP() - 21*24*3600;
+DELETE FROM msgs WHERE time_num < UNIX_TIMESTAMP() - 3600 AND content IS NULL;
+
+Optionally certain content types may be given shorter lifetime:
+
+DELETE FROM msgs WHERE time_num < UNIX_TIMESTAMP() - 7*24*3600
+ AND (content='V' OR (content='S' AND spam_level > 20));
+
+( or equivalently, if a data type of msgs.time_iso is TIMESTAMPS
+ and in amavisd.conf the $timestamp_fmt_mysql is set to true:
+ DELETE FROM msgs WHERE time_iso < UTC_TIMESTAMP() - INTERVAL 21 day;
+ DELETE FROM msgs WHERE time_iso < UTC_TIMESTAMP() - INTERVAL 1 hour
+ AND content IS NULL;
+ DELETE FROM msgs WHERE time_iso < UTC_TIMESTAMP() - INTERVAL 7 day
+ AND (content='V' OR (content='S' AND spam_level > 20));
+)
+
+
+Then delete unreferenced records from tables msgrcpt and quarantine,
+unless they were already automatically deleted while purging the msgs
+table and FOREIGN KEY ... ON DELETE CASCADE is in place:
-COMMENTED LONGER EXAMPLE of a log/report/quarantine database housekeeping
-=========================================================================
-
--- discarding indexes makes deletion faster; if we expect a large proportion
--- of records to be deleted it may be quicker to discard index, do deletions,
--- and re-create index; for daily maintenance this does not pay off
---DROP INDEX msgs_idx_sid ON msgs;
---DROP INDEX msgrcpt_idx_rid ON msgrcpt;
---DROP INDEX msgrcpt_idx_mail_id ON msgrcpt;
-
--- delete old msgs records based on timestamps only (for time_iso see next),
--- and delete leftover msgs records from aborted mail checking operations
-DELETE FROM msgs WHERE time_num < UNIX_TIMESTAMP()-14*24*60*60;
-DELETE FROM msgs WHERE time_num < UNIX_TIMESTAMP()-60*60 AND content IS NULL;
-
--- provided the time_iso field was created as type TIMESTAMP DEFAULT 0,
--- instead of purging based on numerical Unix timestamp as above, one may
--- select records based on ISO 8601 UTC timestamps.
---DELETE FROM msgs WHERE time_iso < now() - INTERVAL '14 days';
---DELETE FROM msgs WHERE time_iso < now() - INTERVAL '1 h' AND content IS NULL;
-and is also possible with MySQL, using slightly different format:
---DELETE FROM msgs
--- WHERE time_iso < UTC_TIMESTAMP() - INTERVAL 14 day;
---DELETE FROM msgs
--- WHERE time_iso < UTC_TIMESTAMP() - INTERVAL 1 hour AND content IS NULL;
-
--- optionally certain content types may be given shorter lifetime
---DELETE FROM msgs WHERE time_num < UNIX_TIMESTAMP()-7*24*60*60
--- AND (content='V' OR (content='S' AND spam_level>20));
-
--- when a FOREIGN KEY ... ON DELETE CASCADE is not used, tables msgrcpt
--- and quarantine need to be purged explicitly, e.g.:
-DELETE FROM quarantine
- WHERE NOT EXISTS (SELECT 1 FROM msgs WHERE mail_id=quarantine.mail_id);
DELETE FROM msgrcpt
WHERE NOT EXISTS (SELECT 1 FROM msgs WHERE mail_id=msgrcpt.mail_id);
--- or:
--- DELETE quarantine FROM quarantine LEFT JOIN msgs USING(mail_id)
--- WHERE msgs.mail_id IS NULL;
--- DELETE msgrcpt FROM msgrcpt LEFT JOIN msgs USING(mail_id)
--- WHERE msgs.mail_id IS NULL;
-
--- re-create indexes (if they were removed in the first step):
---CREATE INDEX msgs_idx_sid ON msgs (sid);
---CREATE INDEX msgrcpt_idx_rid ON msgrcpt (rid);
---CREATE INDEX msgrcpt_idx_mail_id ON msgrcpt (mail_id);
-
--- delete unreferenced e-mail addresses
+
+DELETE FROM quarantine
+ WHERE NOT EXISTS (SELECT 1 FROM msgs WHERE mail_id=quarantine.mail_id);
+
+( or equivalently:
+ DELETE msgrcpt FROM msgrcpt LEFT JOIN msgs USING(mail_id)
+ WHERE msgs.mail_id IS NULL;
+ DELETE quarantine FROM quarantine LEFT JOIN msgs USING(mail_id)
+ WHERE msgs.mail_id IS NULL;
+)
+
+
+Finally delete unreferenced records from table maddr:
+
DELETE FROM maddr
WHERE NOT EXISTS (SELECT 1 FROM msgs WHERE sid=id)
AND NOT EXISTS (SELECT 1 FROM msgrcpt WHERE rid=id);
--- alternatively, purging each table by partition_tag may be
--- the fastest method of purging old records, e.g.:
---
--- DELETE FROM msgs WHERE partition_tag >= 13 AND partition_tag <= 23;
--- DELETE FROM msgrcpt WHERE partition_tag >= 13 AND partition_tag <= 23;
--- DELETE FROM quarantine WHERE partition_tag >= 13 AND partition_tag <= 23;
--- DELETE FROM maddr WHERE partition_tag >= 13 AND partition_tag <= 23;
--- (optional) optimize tables once in a while
---OPTIMIZE TABLE msgs, msgrcpt, quarantine, maddr;
+
+SOME FURTHER THOUGHTS on a log/report/quarantine database housekeeping
+======================================================================
+
+Discarding indexes makes deletion faster; if we expect a large proportion
+of records to be deleted it may be quicker to discard index, do deletions,
+and re-create index; for daily maintenance this does not pay off
+
+DROP INDEX msgs_idx_sid ON msgs;
+DROP INDEX msgrcpt_idx_rid ON msgrcpt;
+DROP INDEX msgrcpt_idx_mail_id ON msgrcpt;
+
+Re-create indexes (if they were removed in the first step):
+
+CREATE INDEX msgs_idx_sid ON msgs (sid);
+CREATE INDEX msgrcpt_idx_rid ON msgrcpt (rid);
+CREATE INDEX msgrcpt_idx_mail_id ON msgrcpt (mail_id);
+
+Optionally, optimize tables once in a while:
+
+OPTIMIZE TABLE msgs, msgrcpt, quarantine, maddr;
diff --git a/README_FILES/README.sql-pg b/README_FILES/README.sql-pg
index 8efddf8..49f760a 100644
--- a/README_FILES/README.sql-pg
+++ b/README_FILES/README.sql-pg
@@ -1,8 +1,8 @@
USING SQL FOR LOOKUPS, LOG/REPORTING AND QUARANTINE
===================================================
-This text only describes SQL specifics for a PostgreSQL database
-and provides a schema. In most respects it applies to a SQLite database.
+This text describes SQL specifics for a PostgreSQL database, and
+provides a schema. In most respects it also applies to an SQLite database.
For general aspects of lookups, please see README.lookups.
For general SQL notes and further examples please see README.sql.
@@ -33,14 +33,28 @@ otherwise PostgreSQL will complain with:
when amavisd tries to execute SQL commands.
Starting with amavisd-new-2.7.0, three fields need to be added
-to table msgrcpt and one to table msgs:
- ALTER TABLE msgrcpt ADD rseqnum integer DEFAULT 0 NOT NULL;
- ALTER TABLE msgrcpt ADD content char(1) DEFAULT ' ' NOT NULL;
- ALTER TABLE msgrcpt ADD is_local char(1) DEFAULT ' ' NOT NULL;
- ALTER TABLE msgs ADD originating char(1) DEFAULT ' ' NOT NULL;
+to table 'msgrcpt', and one to table 'msgs':
+ ALTER TABLE msgrcpt ADD COLUMN rseqnum integer DEFAULT 0 NOT NULL;
+ ALTER TABLE msgrcpt ADD COLUMN content char(1) DEFAULT ' ' NOT NULL;
+ ALTER TABLE msgrcpt ADD COLUMN is_local char(1) DEFAULT ' ' NOT NULL;
+ ALTER TABLE msgs ADD COLUMN originating char(1) DEFAULT ' ' NOT NULL;
+
+Table 'policy' received a couple of new optional fields with 2.7.0, and
+dropped one field. As all fields in this table are optional and any extra
+field is just ignored by amavisd, it is not necessary to update this table
+unless one really needs these new fields. The following should adjust
+a pre-2.7.0 schema:
+ ALTER TABLE policy ADD COLUMN unchecked_lover char(1) default NULL;
+ ALTER TABLE policy ADD COLUMN spam_tag3_level real default NULL;
+ ALTER TABLE policy ADD COLUMN spam_subject_tag3 varchar(64) default NULL;
+ ALTER TABLE policy ADD COLUMN disclaimer_options varchar(64) default NULL;
+ ALTER TABLE policy ADD COLUMN forward_method varchar(64) default NULL;
+ ALTER TABLE policy ADD COLUMN sa_userconf varchar(64) default NULL;
+ ALTER TABLE policy ADD COLUMN sa_username varchar(64) default NULL;
+ ALTER TABLE policy DROP COLUMN spam_modifies_subj;
If you need to create a primary key on table msgrcpt for some reason
-(clustering?), try something like:
+(clustering perhaps?), try something like:
UPDATE msgrcpt SET rseqnum=1+floor(999999999*random()) WHERE rseqnum=0;
CREATE UNIQUE INDEX msgrcpt_idx_primary
ON msgrcpt (partition_tag,mail_id,rseqnum);
@@ -61,6 +75,8 @@ data type of affected fields from varchar or char to bytea:
ALTER TABLE quarantine
ALTER mail_id TYPE bytea USING decode(mail_id,'escape');
+
+
Version of Perl module DBD::Pg 1.48 or higher should be used;
Short installation notes for PostgreSQL 8.2 are available at:
@@ -92,7 +108,7 @@ host mail_awl vscan 127.0.0.1/32 md5
host mail_awl vscan ::1/128 md5
-Create a SQL username (role) for use by amavisd, e.g. vscan:
+Create an SQL username (role) for use by amavisd, e.g. vscan:
$ createuser -U pgsql -S -D -R -P -e vscan
Create databases for amavisd:
@@ -109,7 +125,7 @@ to create a database. The '--' introduces comments according to SQL specs.
Populate databases using the schema below:
$ psql -U vscan mail_prefs <...
$ psql -U vscan mail_log <...
-(for SA database schema see its documentation: sql/README*)
+(for SpamAssassin database schema see its documentation: sql/README*)
Something like the following can be placed into amavisd.conf
@@ -121,10 +137,10 @@ Something like the following can be placed into amavisd.conf
@storage_sql_dsn =
([ 'DBI:Pg:database=mail_log', 'vscan', 'LK40.gtklkKK' ]);
-Equivalent settings for AWL and Bayes databases belong to a
-SA configuration file local.cf, according to SpamAssassin documentation.
-Amavisd and SA need not use the same usernames or passwords, nor do they
-need to reside on the same SQL server.
+Equivalent settings for AWL and Bayes databases belong to a SpamAssassin's
+configuration file local.cf, according to SpamAssassin documentation.
+Amavisd and SpamAssassin need not use the same username or password,
+nor do they need to reside on the same SQL server.
SQLite notes:
@@ -132,42 +148,13 @@ SQLite notes:
- SQLite is well suited for lookups database, but is not appropriate
for @storage_sql_dsn due to coarse lock granularity;
-
--- local users
-CREATE TABLE users (
- id serial PRIMARY KEY, -- unique id
- priority integer NOT NULL DEFAULT '7', -- sort field, 0 is low prior.
- policy_id integer NOT NULL DEFAULT '1' CHECK (policy_id >= 0),
- -- JOINs with policy.id
- email bytea NOT NULL UNIQUE, -- email address, non-rfc2822-quoted
- fullname varchar(255) DEFAULT NULL -- not used by amavisd-new
- -- local char(1) -- Y/N (optional field, see note further down)
-);
-
--- any e-mail address (non- rfc2822-quoted), external or local,
--- used as senders in wblist
-CREATE TABLE mailaddr (
- id serial PRIMARY KEY,
- priority integer NOT NULL DEFAULT '7', -- 0 is low priority
- email bytea NOT NULL UNIQUE
-);
-
--- per-recipient whitelist and/or blacklist,
--- puts sender and recipient in relation wb (white or blacklisted sender)
-CREATE TABLE wblist (
- rid integer NOT NULL CHECK (rid >= 0), -- recipient: users.id
- sid integer NOT NULL CHECK (sid >= 0), -- sender: mailaddr.id
- wb varchar(10) NOT NULL, -- W or Y / B or N / space=neutral / score
- PRIMARY KEY (rid,sid)
-);
-
CREATE TABLE policy (
- id serial PRIMARY KEY, -- 'id' this is the _only_ required field
- policy_name varchar(32), -- not used by amavisd-new, a comment
+ id serial PRIMARY KEY, -- 'id' is the _only_ required field
+ policy_name varchar(32), -- not used by amavisd-new, a comment
virus_lover char(1) default NULL, -- Y/N
spam_lover char(1) default NULL, -- Y/N
- unchecked_lovers_maps char(1) default NULL, -- Y/N
+ unchecked_lover char(1) default NULL, -- Y/N
banned_files_lover char(1) default NULL, -- Y/N
bad_header_lover char(1) default NULL, -- Y/N
@@ -190,6 +177,7 @@ CREATE TABLE policy (
spam_kill_level real default NULL, -- higher score triggers evasive actions
-- e.g. reject/drop, quarantine, ...
-- (subject to final_spam_destiny setting)
+
spam_dsn_cutoff_level real default NULL,
spam_quarantine_cutoff_level real default NULL,
@@ -218,6 +206,33 @@ CREATE TABLE policy (
sa_username varchar(64) default NULL
);
+-- local users
+CREATE TABLE users (
+ id serial PRIMARY KEY, -- unique id
+ priority integer NOT NULL DEFAULT 7, -- sort field, 0 is low prior.
+ policy_id integer NOT NULL DEFAULT 1 CHECK (policy_id >= 0) REFERENCES policy(id),
+ email bytea NOT NULL UNIQUE, -- email address, non-rfc2822-quoted
+ fullname varchar(255) DEFAULT NULL -- not used by amavisd-new
+ -- local char(1) -- Y/N (optional, see SQL section in README.lookups)
+);
+
+-- any e-mail address (non- rfc2822-quoted), external or local,
+-- used as senders in wblist
+CREATE TABLE mailaddr (
+ id serial PRIMARY KEY,
+ priority integer NOT NULL DEFAULT 9, -- 0 is low priority
+ email bytea NOT NULL UNIQUE
+);
+
+-- per-recipient whitelist and/or blacklist,
+-- puts sender and recipient in relation wb (white or blacklisted sender)
+CREATE TABLE wblist (
+ rid integer NOT NULL CHECK (rid >= 0) REFERENCES users(id),
+ sid integer NOT NULL CHECK (sid >= 0) REFERENCES mailaddr(id),
+ wb varchar(10) NOT NULL, -- W or Y / B or N / space=neutral / score
+ PRIMARY KEY (rid,sid)
+);
+
-- R/W part of the dataset (optional)
-- May reside in the same or in a separate database as lookups database;
@@ -229,8 +244,8 @@ CREATE TABLE policy (
-- provide unique id for each e-mail address, avoids storing copies
CREATE TABLE maddr (
- partition_tag integer DEFAULT 0, -- see $partition_tag
id serial PRIMARY KEY,
+ partition_tag integer DEFAULT 0, -- see $partition_tag
email bytea NOT NULL, -- full e-mail address
domain varchar(255) NOT NULL, -- only domain part of the email address
-- with subdomain fields in reverse
@@ -238,8 +253,8 @@ CREATE TABLE maddr (
);
-- information pertaining to each processed message as a whole;
--- NOTE: records with NULL msgs.content should be ignored by utilities,
--- as such records correspond to messages just being processes, or were lost
+-- NOTE: records with a NULL msgs.content should be ignored by utilities,
+-- as such records correspond to messages just being processed, or were lost
CREATE TABLE msgs (
partition_tag integer DEFAULT 0, -- see $partition_tag
mail_id bytea NOT NULL, -- long-term unique mail id, dflt 12 ch
@@ -269,6 +284,7 @@ CREATE TABLE msgs (
from_addr varchar(255) DEFAULT '', -- mail From header field, UTF8
subject varchar(255) DEFAULT '', -- mail Subject header field, UTF8
host varchar(255) NOT NULL, -- hostname where amavisd is running
+ CONSTRAINT msgs_partition_mail UNIQUE (partition_tag,mail_id),
PRIMARY KEY (partition_tag,mail_id)
--FOREIGN KEY (sid) REFERENCES maddr(id) ON DELETE RESTRICT
);
@@ -294,6 +310,7 @@ CREATE TABLE msgrcpt (
wl char(1) DEFAULT ' ', -- sender whitelisted by this recip
bspam_level real, -- per-recipient (total) spam level
smtp_resp varchar(255) DEFAULT '', -- SMTP response given to MTA
+ CONSTRAINT msgrcpt_partition_mail_rseq UNIQUE (partition_tag,mail_id,rseqnum),
PRIMARY KEY (partition_tag,mail_id,rseqnum)
--FOREIGN KEY (rid) REFERENCES maddr(id) ON DELETE RESTRICT,
--FOREIGN KEY (mail_id) REFERENCES msgs(mail_id) ON DELETE CASCADE
@@ -328,14 +345,14 @@ Some examples of a query:
-- mail from last two minutes:
SELECT
now()-time_iso AS age, SUBSTRING(policy,1,2) as pb,
- content AS c, dsn_sent as dsn, ds, bspam_level AS level, size,
+ msgrcpt.content AS c, dsn_sent as dsn, ds, bspam_level AS level, size,
SUBSTRING(sender.email,1,18) AS s,
SUBSTRING(recip.email,1,18) AS r,
SUBSTRING(msgs.subject,1,10) AS subj
FROM msgs LEFT JOIN msgrcpt ON msgs.mail_id=msgrcpt.mail_id
LEFT JOIN maddr AS sender ON msgs.sid=sender.id
LEFT JOIN maddr AS recip ON msgrcpt.rid=recip.id
- WHERE content IS NOT NULL AND now() - time_iso < INTERVAL '2 minutes'
+ WHERE msgrcpt.content IS NOT NULL AND now() - time_iso < INTERVAL '2 minutes'
ORDER BY msgs.time_num DESC;
-- clean messages ordered by count, grouped by domain:
@@ -344,7 +361,7 @@ SELECT count(*) as cnt, avg(bspam_level), sender.domain
LEFT JOIN msgrcpt ON msgs.mail_id=msgrcpt.mail_id
LEFT JOIN maddr AS sender ON msgs.sid=sender.id
LEFT JOIN maddr AS recip ON msgrcpt.rid=recip.id
- WHERE content='C'
+ WHERE msgrcpt.content='C'
GROUP BY sender.domain ORDER BY cnt DESC LIMIT 50;
-- top spamy domains with >10 messages, sorted by spam average,
@@ -370,39 +387,44 @@ SELECT count(*) as cnt, avg(bspam_level) as spam_avg, sender.domain
-Upgrading from pre 2.4.0 amavisd-new SQL schema to the 2.4.0 schema
-requires adding column 'quar_loc' to table msgs.
-Creating a FOREIGN KEY ... ON DELETE CASCADE constraint may (or may not)
-facilitate deletion of expired records.
-
-The following clauses should be executed for upgrading pre-2.4.0 amavisd-new
-SQL schema to the 2.4.0 schema:
+EXAMPLE of a log/report/quarantine database housekeeping
+========================================================
--- mandatory change:
- ALTER TABLE msgs ADD quar_loc varchar(255) DEFAULT '';
+Using a changing partition_tag, perhaps by using an ISO 8601 week number
+(value 1 to 53) as a partition_tag:
--- optional, avoids need to purge tables msgrcpt and quarantine explicitly:
- ALTER TABLE quarantine
- ADD FOREIGN KEY (mail_id) REFERENCES msgs(mail_id) ON DELETE CASCADE;
- ALTER TABLE msgrcpt
- ADD FOREIGN KEY (mail_id) REFERENCES msgs(mail_id) ON DELETE CASCADE;
+ $partition_tag =
+ sub { my($msginfo)=@_; sprintf("%02d",iso8601_week($msginfo->rx_time)) };
--- the following two ALTERs are not essential; if data type of maddr.id is
--- incompatible with msgs.sid and msgs.rid (e.g. BIGINT vs. INT) and MySQL
--- complains, don't bother to apply the constraint:
- ALTER TABLE msgs
- ADD FOREIGN KEY (sid) REFERENCES maddr(id) ON DELETE RESTRICT;
- ALTER TABLE msgrcpt
- ADD FOREIGN KEY (rid) REFERENCES maddr(id) ON DELETE RESTRICT;
+allows for probably the fastest method of purging old records, e.g.:
+DELETE FROM msgs WHERE partition_tag >= 13 AND partition_tag <= 23;
+DELETE FROM msgrcpt WHERE partition_tag >= 13 AND partition_tag <= 23;
+DELETE FROM quarantine WHERE partition_tag >= 13 AND partition_tag <= 23;
+DELETE FROM maddr WHERE partition_tag >= 13 AND partition_tag <= 23;
-BRIEF EXAMPLE of a log/report/quarantine database housekeeping
-==============================================================
+Alternatively, purge records from table msgs by their creation time:
DELETE FROM msgs WHERE time_iso < now() - INTERVAL '3 weeks';
DELETE FROM msgs WHERE time_iso < now() - INTERVAL '1 h' AND content IS NULL;
+Optionally certain content types may be given shorter lifetime:
+
+DELETE FROM msgs WHERE time_iso < now() - INTERVAL '1 week'
+ AND (content='V' OR (content='S' AND spam_level > 20));
+
+and then delete unreferenced records from tables msgrcpt, quarantine,
+and maddr:
+
+DELETE FROM msgrcpt WHERE mail_id IN
+ (SELECT mail_id FROM msgrcpt LEFT JOIN msgs USING(mail_id)
+ WHERE msgs.mail_id IS NULL);
+
+DELETE FROM quarantine WHERE mail_id IN
+ (SELECT mail_id FROM quarantine LEFT JOIN msgs USING(mail_id)
+ WHERE msgs.mail_id IS NULL);
+
DELETE FROM maddr
WHERE NOT EXISTS (SELECT 1 FROM msgs WHERE sid=id)
AND NOT EXISTS (SELECT 1 FROM msgrcpt WHERE rid=id);
@@ -421,31 +443,12 @@ Check also a thread 'Faster purging of SQL logging database'
(2007-06) on the amavis-user mailing list, archived at:
http://marc.info/?t=118190428300003
--- optionally certain content types may be given shorter lifetime:
-DELETE FROM msgs WHERE time_iso < now() - INTERVAL '1 week'
- AND (content='V' OR (content='S' AND spam_level > 20));
--- TODO, experimental, may be disregarded:
---
--- when a FOREIGN KEY ... ON DELETE CASCADE is not used, tables msgrcpt
--- and quarantine need to be purged explicitly, e.g.:
---
--- ALTER TABLE quarantine DROP CONSTRAINT quarantine_mail_id_fkey;
--- ALTER TABLE msgrcpt DROP CONSTRAINT msgrcpt_mail_id_fkey;
---
--- DELETE FROM msgrcpt WHERE mail_id IN
--- (SELECT mail_id FROM msgrcpt LEFT JOIN msgs USING(mail_id)
--- WHERE msgs.mail_id IS NULL);
---
--- DELETE FROM quarantine WHERE mail_id IN
--- (SELECT mail_id FROM quarantine LEFT JOIN msgs USING(mail_id)
--- WHERE msgs.mail_id IS NULL);
---
--- alternatively, purging each table by partition_tag may be
--- the fastest method of purging old records, e.g.:
---
--- DELETE FROM msgs WHERE partition_tag >= 13 AND partition_tag <= 23;
--- DELETE FROM msgrcpt WHERE partition_tag >= 13 AND partition_tag <= 23;
--- DELETE FROM quarantine WHERE partition_tag >= 13 AND partition_tag <= 23;
--- DELETE FROM maddr WHERE partition_tag >= 13 AND partition_tag <= 23;
+The third option for purging old records is to use:
+
+ FOREIGN KEY ... ON DELETE CASCADE
+
+on tables msgrcpt and quarantine, in which case these records will
+be deleted automatically when a corresponding record in table msgs
+is deleted. This seems to be the slowest method.
diff --git a/RELEASE_NOTES b/RELEASE_NOTES
index ab87d1d..1f8c891 100644
--- a/RELEASE_NOTES
+++ b/RELEASE_NOTES
@@ -1,3 +1,133 @@
+ April 29, 2012
+amavisd-new-2.7.1 release notes
+
+BUG FIXES
+
+- prevent rmdir() from failing with 'Invalid argument' on Solaris 10 when
+ deleting a temporary directory: current working directory must not be
+ within a directory which is about to be deleted; reported and diagnosed
+ by Maciej Uhlig;
+
+- forwarding or quarantining through a 'pipe:' method failed with
+ "Insecure dependency in exec while running with -T switch" when a
+ sendmail command-line option -N was needed; reported by Andreas Schulze;
+
+- when multiple sockets are specified (e.g. in $forward_method) as a
+ redundancy/failover mechanism, and SMTP session caching is enabled,
+ a failed forwarding session does not clear a cached session, so all
+ further attempts are stuck with the failed server, instead of picking
+ a different server from the list; discovered by Michael Storz;
+
+- on establishing a SMTP session when multiple sockets are specified
+ (e.g. in $forward_method) as a redundancy/failover mechanism, the
+ random choice never picked the last socket in a list;
+ discovered by Michael Storz;
+
+- fix defanging by mimedefang, it was failing with perl 5.10 or later
+ due to an unhandled "Insecure dependency in sprintf" while logging the
+ result if the $log_level was 2 or higher, or when debugging was enabled;
+ thanks to Steve Scotter for a problem report;
+
+- fix defanging by Anomy::Sanitizer, it was failing with an error message:
+ "mangling by anomy failed: replacement size 0, mail will pass unmodified";
+
+- fix the 'xz' entry in a default @decoders list (in files amavisd.conf,
+ amavisd.conf-default and amavisd); the first two variants ('xzdec' and
+ 'xz') were glued together, so the xz decoder was only available if found
+ under names 'unxz' or 'xzcat';
+
+- provide a workaround for a bug [rt.cpan.org #64642] in a perl module
+ Encode, which gratuitously untaints a string when encoding or decoding it:
+ https://rt.cpan.org/Public/Bug/Display.html?id=64642
+ (still unfixed in Encode 2.44, perl 5.14.2);
+ A module Scalar::Util is now required, which should not be a compatibility
+ problem, as this module is a Perl core module since perl 5.8.0.
+
+- avoid the use of Encode::is_utf8 due to a bug in a perl module Encode
+ as bundled with versions of Perl 5.8.0 to 5.8.8 (fixed in March 2007):
+
+ Perl bug tracking: #32687:
+ Encode::is_utf8 on tainted UTF8 string returns false
+ https://rt.perl.org/rt3/Public/Bug/Display.html?id=32687
+ also referenced by #37170:
+ https://rt.perl.org/rt3/Public/Bug/Display.html?id=37170
+
+ This is a re-manifestation of the same problem we had back in 2004,
+ with a workaround provided by amavisd-new-2.2.1. Forgot that people
+ are still using Perl 5.8 :) Reported by Peter Dieth;
+
+- fix a warning: _WARN: Invalid conversion in sprintf: "%a"
+
+- write informational messages during a stop/start/restart to stdout,
+ instead of to stderr, avoiding unnecessary cron job messages;
+ thanks to Cristian Seres, Sandro Janke and John Griffiths;
+ also: https://bugzilla.redhat.com/show_bug.cgi?id=561389
+
+- fix a syntactically incorrect 'Avira SAVAPI' av entry (missing
+ closing bracket) in a sample configuration file amavisd.conf;
+
+- minor: get_body_digest incorrectly logged 8-bit body as 8-bit header;
+
+- no longer insist on a minimal version 2.22 of a module Digest::MD5,
+ the 'clone' method is no longer needed since amavisd-new-2.7.0;
+
+- do not call $parser->max_parts($MAXFILES) with some old versions
+ of MIME::Parser which did not yet provide this method;
+
+- pre-load a module File::Glob even with perl 5.8.0, otherwise
+ autowhitelisting in SpamAssasssin may fail with "Insecure dependency";
+
+- documentation: (files README.sql-mysql and README.sql-pg):
+ fixed a field name "policy.unchecked_lover", previously incorrectly
+ specified as "policy.unchecked_lovers_maps"; reported by TimH;
+
+- documentation: fixed the two SELECT examples in files README.sql-pg and
+ README.sql-mysql, the field 'select' needs to be qualified with a table
+ name: 'msgrcpt.content' to avoid ambiguity; reported by Gary V;
+
+- documentation bug in amavisd.conf-default: 'ESMTP' is not a valid
+ setting for $protocol, just use 'SMTP' instead; reported by Pascal Volk;
+
+
+COMPATIBILITY
+
+- commented out the LHA entry in the default @decoders list and in
+ do_executable(). The program seems to be unmaintained, was seen crashing
+ and as such it may pose a security risk; pointed out by Thomas Jarosch;
+
+- due to popular demand, bring the 'spam-tag:' log line back to log level 2
+ (version 2.7.0 dropped it to log level 3) to retain compatibility with
+ some log analyzers. Caveat: 'spam-tag' string is now entirely in lowercase.
+ Suggested by Stefan Jakobs;
+
+
+OTHER
+
+- if a message is quarantined to more than one location using different
+ quarantine methods, the SQL field msgs.quar_type indicates only the
+ type of the last one. When archival quarantining is enabled this choice
+ is unfortunate, as the primary quarantine type is more interesting
+ than the permanent archival quarantine type. This is now reversed,
+ the msgs.quar_type field now reflects the first quarantine type.
+ Suggested by Patrick Ben Koetter.
+
+- SMTP session caching now no longer re-uses old sessions which are
+ in use for more than a minute since their establishment; suggested
+ by Michael Storz;
+
+- having the archive quarantine enabled should not be a sufficient reason
+ to store information to SQL when $sql_store_info_for_all_msgs is off;
+ Suggested by Patrick Ben Koetter.
+
+- ClamAV-clamd and ClamAV-clamd-stream av scanners: changed socket name
+ in a sample configuration file amavisd.conf to /var/run/clamav/clamd.sock
+ (previously the socket name was /var/run/clamav/clamd); this makes it
+ compatible with a default socket name under several Linux distributions
+ and under FreeBSD; suggested by Oliver Schinagl;
+
+- documentation updates;
+
+
---------------------------------------------------------------------------
July 1, 2011
amavisd-new-2.7.0 release notes
@@ -56,8 +186,8 @@ NEW FEATURES SUMMARY
- @listen_sockets setting offers a unified configuration of listening
sockets; it may be configured directly, or the traditional way: the
- $inet_socket_port, $unix_socketname and $inet_socket_bind just add their
- entries to the @listen_sockets list;
+ $inet_socket_port, $unix_socketname and $inet_socket_bind just add
+ their entries to the @listen_sockets list;
- lists of lookup tables (the @*_maps variables) can now contain
explicit SQL and LDAP lookup objects as their elements, instead of
@@ -88,7 +218,7 @@ filtering setup became a sensible / better behaved solution:
Postfix 2.7.0 (20091101), improves decoupling between SMTP clients
and a content filter in a proxy setup, reducing the number of content
filtering processes needed for the same mail load. With this option
- turned on, a Postfix SMTP server receives the entire message before
+ turned on, a Postfix SMTP server receives entire message before
connecting to a before-queue content filter;
- a master_deadline option and its API equivalent, available in SpamAssassin
@@ -98,7 +228,7 @@ filtering setup became a sensible / better behaved solution:
- reworked sub-task time limiting in amavisd, along with its counterpart
solution in SpamAssassin, makes it better suited to a real-time nature
- of pre-queue filtering setups, where one has no control over how long
+ of pre-queue filtering setups where one has no control over how long
SMTP clients are willing to wait at the data-end stage;
- a re-purposed command line option 'reload' now does a warm restart,
@@ -123,7 +253,7 @@ three products only rise the thresholds where trouble starts, and make
the whole setup better behaved.
-COMPATIBILITY WITH 2.6.4 / 2.6.5 / 2.6.5
+COMPATIBILITY WITH 2.6.4 / 2.6.5 / 2.6.6
- due to popular demand to reduce undesired and unintentional backscatter,
defaults for the settings $final_spam_destiny and $final_banned_destiny
@@ -140,10 +270,11 @@ COMPATIBILITY WITH 2.6.4 / 2.6.5 / 2.6.5
you.
For a pre-queue content filtering setup (smtp proxy or milter) a suitable
- value is D_REJECT. For a post-queue filtering setup preferred choices
- are to tag-and-deliver (D_PASS), or to drop (D_DISCARD) and quarantine.
+ value for undesired content is D_REJECT. For a post-queue filtering setup
+ preferred choices are to tag-and-deliver (D_PASS), or to drop (D_DISCARD)
+ and quarantine.
- It is still possible to use a D_BOUNCE setting, but please restrict
+ It is still possible to use a D_BOUNCE setting, but please limit
and monitor your backscatter. Due to a default setting of
@viruses_that_fake_sender_maps the backscatter on viruses has been
fully suppressed since amavisd-new-20021116 even with a D_BOUNCE.
@@ -166,7 +297,7 @@ COMPATIBILITY WITH 2.6.4 / 2.6.5 / 2.6.5
operation, but flags a message with a CC_UNCHECKED contents category
(just like a failure of decoders/dearchivers), and allows the usual
controls (*_destiny, *_quarantine_*) to be used to choose behaviour.
- The $virus_scanners_failure_is_fatal=1 reverts to the previous behaviour,
+ The $virus_scanners_failure_is_fatal=1 reverts to previous behaviour,
see below;
- a default value of $hdr_encoding and $bdy_encoding has been changed
@@ -174,7 +305,7 @@ COMPATIBILITY WITH 2.6.4 / 2.6.5 / 2.6.5
- default encoding for reading text templates from the tail of a file
'amavisd' has been changed to 'utf8', which allows replacing a default
- text by non-ascii Unicode templates, encoded as UTF-8;
+ text by a non-ascii Unicode template, encoded as UTF-8;
- when using SQL for logging/penpals: three fields need to be added
to a table msgrcpt: msgrcpt.content, msgrcpt.rseqnum, msgrcpt.is_local,
@@ -220,7 +351,7 @@ COMPATIBILITY WITH 2.6.4 / 2.6.5 / 2.6.5
The following SQL directives can be used to add these new fields:
ALTER TABLE msgrcpt ADD rseqnum integer DEFAULT 0 NOT NULL;
- ALTER TABLE msgrcpt ADD content char(1) DEFAULT ' ' NOT NULL
+ ALTER TABLE msgrcpt ADD content char(1) DEFAULT ' ' NOT NULL;
ALTER TABLE msgrcpt ADD is_local char(1) DEFAULT ' ' NOT NULL;
ALTER TABLE msgs ADD originating char(1) DEFAULT ' ' NOT NULL;
@@ -235,21 +366,22 @@ COMPATIBILITY WITH 2.6.4 / 2.6.5 / 2.6.5
ON msgrcpt (partition_tag,mail_id,rseqnum);
If keeping a possibly customized copy of %sql_clause in a configuration
- file, entries 'ins_rcp' and 'upd_msg' will need to be adjusted accordingly.
+ file, entries 'ins_rcp' and 'upd_msg' will need to be updated accordingly.
- To facilitate transition from 2.6.4 to 2.7.0, it is possible to configure
+ To facilitate transition from 2.6.6 to 2.7.0, it is possible to configure
amavisd 2.7.0 to supply with SELECT and INSERT clauses a subset of
- parameters as used by 2.6.4. A configuration setting $sql_schema_version
- controls this backwards compatibility. Its default value is 2.007000 .
- By setting it to 2.006004 a subset of parameters as was used with a
- version 2.6.4 is selected. SQL clauses in $sql_clause{'upd_msg'}
- and $sql_clause{'ins_rcp'} need to be adjusted according to a chosen
- version of actual parameters. Below is an example of a required setting
- compatible with both amavisd-new 2.6.4 and 2.7.0, which lets amavisd 2.7.0
- use an SQL schema of 2.6.4, which lacks the four newly added fields:
+ parameters as used by 2.6.6. A configuration setting $sql_schema_version
+ controls this backward compatibility. Its default value is 2.007000 .
+ By setting it to a value below 2.007000 (such as 2.006006 or 2.006004) a
+ subset of parameters as was used with a version 2.6.6 or 2.6.4 is selected.
+ SQL clauses in $sql_clause{'upd_msg'} and $sql_clause{'ins_rcp'} need to
+ be adjusted according to a chosen version of actual parameters. Below is
+ an example of a required setting compatible with both amavisd-new 2.6.6
+ and 2.7.0, which lets amavisd 2.7.0 use an SQL schema of 2.6.6, which
+ lacks the four newly added fields:
our($sql_schema_version) if $myversion_id_numeric < 2.007000;
- $sql_schema_version = 2.006004;
+ $sql_schema_version = 2.006006;
$sql_clause{'upd_msg'} =
'UPDATE msgs SET content=?, quar_type=?, quar_loc=?, dsn_sent=?,'.
' spam_level=?, message_id=?, from_addr=?, subject=?, client_addr=?'.
@@ -264,7 +396,7 @@ COMPATIBILITY WITH 2.6.4 / 2.6.5 / 2.6.5
- SQL fields msgs.content and msgrcpt.content used to encode a content
type CC_SPAMMY as 's', and CC_MTA as 't'. With default case-insensitive
- queries on a data type char it was not possible to distinguish
+ queries on a data type CHAR it was not possible to distinguish
between lowercase 's' (= CC_SPAMMY) and uppercase 'S' (= CC_SPAM), so
the CC_SPAMMY is now encoded as 'Y', and CC_MTA as 'T' (just in case).
Please adjust your management tools if necessary.
@@ -277,15 +409,15 @@ COMPATIBILITY WITH 2.6.4 / 2.6.5 / 2.6.5
- SQL clause $sql_clause{'sel_quar'} no longer uses a coalesce() function
(introduced in amavisd-new-2.6.2) which attempted to deal with NULL
quarantine.partition_tag or with undefined $partition_tag, when releasing
- a message from a SQL quarantine - but payed the price of not using index.
- If releasing from a SQL quarantine is desired, either ensure there are
- no (old) records in a table 'quarantine' with a NULL partition_tag
+ a message from an SQL quarantine - but payed the price of not using an
+ index. If releasing from an SQL quarantine is desired, either ensure there
+ are no (old) records in a table 'quarantine' with a NULL partition_tag
(e.g. replace such fields with a 0, and don't leave $partition_tag
undefined in amavisd.conf - set it to 0 for example when partitioning
is not needed), or assign a former clause to $sql_clause{'sel_quar'}
in amavisd.conf :
- $sql_clause{sel_quar} =
+ $sql_clause{'sel_quar'} =
'SELECT mail_text FROM quarantine'.
' WHERE coalesce(partition_tag,0)=coalesce(?,0) AND mail_id=?'.
' ORDER BY chunk_ind';
@@ -302,7 +434,7 @@ COMPATIBILITY WITH 2.6.4 / 2.6.5 / 2.6.5
lots of aged or distracting information;
- old helper programs amavis.c and amavis-milter.c are no longer distributed
- with the package, along with the whole helper-progs subdirectory.
+ with the package, along with the entire helper-progs subdirectory.
As a milter client please use the more modern 'amavisd-milter' package by
Petr Rehor, available at http://sourceforge.net/projects/amavisd-milter/
@@ -315,11 +447,11 @@ COMPATIBILITY WITH 2.6.4 / 2.6.5 / 2.6.5
- a sample AM.PDP client program for mail submission to amavisd which was
previously distributed as 'helper-progs/amavis.pl' has been renamed to
'amavisd-submit' and slightly modernized. It provides partial functional
- compatibility with a very early AMaViS client program amavis.c, which
- takes a message on stdin, copies it to a temporary file, passes its
- name to amavisd daemon using AM.PDP protocol, and based on the response
- adjusts its exit status value so that an invoking script or program may
- decide whether to deliver the mail message or not;
+ compatibility with a very early AMaViS client program amavis.c . It takes
+ a message on stdin, copies it to a temporary file, passes its name to
+ amavisd daemon using AM.PDP protocol, and based on the response adjusts
+ its exit status value so that an invoking script or program may decide
+ whether to deliver the mail message or not;
- mail_id and secret_id are now composed of characters from a character set
[ A-Z, a-z, 0-9, -, _ ] instead of [ A-Z, a-z, 0-9, +, - ] (i.e. now uses
@@ -342,7 +474,7 @@ COMPATIBILITY WITH 2.6.4 / 2.6.5 / 2.6.5
- caching of virus and spam check results based on a mail body hash has been
removed. It was very beneficial years ago when virus storms were common
- and spam was not personalized. Nowadays (2010) the feature barely pays
+ and spam was not personalized. Nowadays (2011) the feature barely pays
for itself (savings are comparable to additional processing needed), and
is incompatible with per-recipient spam checks (as introduced with this
version), and incompatible with DKIM verification on locally originating
@@ -362,10 +494,11 @@ COMPATIBILITY WITH 2.6.4 / 2.6.5 / 2.6.5
- updated (rarely used) AV entries 'Sophos SAVI', 'Mail::ClamAV'
and 'av_smtp' in an incompatible way (they now use ask_daemon interface
- instead of a dedicated subroutine), please update your AV entries;
+ instead of a dedicated subroutine), please update your AV entries
+ according to the new sample file amavisd.conf;
- internal: spam_level() and spam_tests() are no longer properties of a
- message, but are now a property of each recipient, which makes possible
+ message but are now a property of each recipient, which makes possible
per-recipient spam checking settings (e.g. rules, bayes username, ...);
- internal: a delivery_method() is no longer a property of a message, but
@@ -382,7 +515,7 @@ BUG FIXES SINCE 2.6.6
- take a more cautious approach on keeping evidence on an SMTP session
transaction state when feeding a message back to MTA. Under certain
abnormal circumstances an MTA could respond to end-of-data with a temporary
- failure but retain an active transaction state, while amavisd would assume
+ failure but retain an active transaction state while amavisd would assume
the transaction was closed, leading to a 'MAIL transaction in progress'
failure on the next message using the same cached SMTP session.
Now amavisd considers a transaction state to be unknown when there is any
@@ -440,23 +573,27 @@ NEW FEATURES
A downside is that a HUP-ed daemon has already dropped root privileges
during its first start, so it must restart as a nonprivileged user
- (typically 'vscan' or 'amavis'), which rules out its capability to chroot,
+ (typically 'vscan' or 'amavis'), which rules out its ability to chroot,
and requires that configuration files, DKIM signing keys files, and
- perl modules must be readable by this GID or UID, otherwise restart
+ perl modules must be readable by this GID or UID, otherwise a restart
fails and a daemon process no longer exists. Depending on a version
of perl and operating system in use, it might be necessary to specify
an absolute path to amavisd on the initial start. To debug warm-restart
problems it may be useful to first try a warm restart on a non-daemonized
- process (# amavisd foreground), so that potential errors on stderr are
- visible.
+ process (started manually as: amavisd foreground, or: amavisd debug),
+ so that potential errors on stderr are visible.
+
+ A sensible protection of configuration files and files with DKIM keys is
+ to set their group ownership to vscan (amavis) and UID ownership to root,
+ and mode to 0640 (u=rw,g=r,o=).
A need for non-root accessibility of DKIM signing keys can be avoided
by using a new signing service daemon included with this release (see
further down: amavisd-signer).
One additional feature of a warm reload is that SNMP counters in a
- database (visible through amavisd-agent or amavisd-snmp-subagent) are
- not reset to zero.
+ database (visible through amavisd-agent or amavisd-snmp-subagent)
+ are not reset to zero, unlike the restart which clears them.
- on stop, restart or reload, currently busy child processes are left
to complete their current task instead of being abruptly stopped.
@@ -472,7 +609,7 @@ NEW FEATURES
log template. This information is also passed back to a re-entry MTA if
it announces a support for this attribute (enabled on a back-end smtpd
service with an option smtpd_authorized_xforward_hosts), so the log entries
- are now easier to correlate:
+ are now easier to correlate in a post-queue filtering setup:
back-end MTA:
postfix/smtpd[72995]: 553261D1CB0: client=localhost[::1],
@@ -487,9 +624,9 @@ NEW FEATURES
status=sent (250 2.0.0 from MTA(smtp:[::1]:10025):
250 2.0.0 Ok: queued as 553261D1CB0)
-- support Postfix 2.9 long queue IDs (enable_long_queue_ids=yes) as available
- since postfix-20110321 by adjusting default values for $log_short_templ
- and $log_verbose_templ templates;
+- support Postfix 2.9 long queue IDs (enable_long_queue_ids=yes)
+ as available since postfix-20110321 by adjusted default values
+ of $log_short_templ and $log_verbose_templ templates;
- improved support for pre-queue content filtering setups: reorganized time
limiting on processing to obey more strictly a deadline time, which is the
@@ -514,11 +651,11 @@ NEW FEATURES
this is potentially useful as a short-term safety net when testing
configuration changes on a low-traffic server;
-- added an AV entry and a supporting code for Sophos-SSSP, implementing
- the client side of the Sophos SSSP protocol, talking to a savdid daemon
+- added an AV entry and supporting code for Sophos-SSSP, implementing the
+ client side of the Sophos SSSP protocol, talking to a savdid daemon
(a replacement for Sophie) using its native protocol;
-- added an AV entry and a supporting code for AVIRA SAVAPI protocol,
+- added an AV entry and supporting code for AVIRA SAVAPI protocol,
implementing the client side of the protocol, talking to a savapi daemon;
- added an AV entry for clamdscan which can serve as a useful backup scanner,
@@ -545,21 +682,21 @@ NEW FEATURES
to list in @*_maps variables (when true), or not (when false).
The default value is 1 for compatibility with previous versions.
- Regardless of the $lookup_maps_imply_sql_and_ldap controls setting,
- the @*_maps lists of lookup tables/objects may now contain explicit
- lookup objects for arbitrarily named SQL fields and LDAP attributes.
- This provides more flexibility: the order of lookups is now configurable
- (previously SQL and LDAP lookup objects were prepended to lists and thus
- always executed first), and the names of SQL fields or LDAP attributes
- can now be specified as arguments to SQL and LDAP lookup objects
- (previously field and attribute names were hardwired into code).
+ Regardless of the $lookup_maps_imply_sql_and_ldap setting, the @*_maps
+ lists of lookup tables/objects may now contain explicit lookup objects
+ for arbitrarily named SQL fields and LDAP attributes. This provides more
+ flexibility: the order of lookups is now configurable (previously SQL and
+ LDAP lookup objects were prepended to lists and thus always looked up
+ first), and the names of SQL fields or LDAP attributes can now be
+ specified as arguments to SQL and LDAP lookup objects (previously field
+ and attribute names were hardwired into code).
Three shorthand functions are available for creating SQL lookup (query)
objects: q_sql_s, q_sql_n, q_sql_b, and three for creating LDAP lookup
(query) objects: q_ldap_s, q_ldap_n, q_ldap_b. The _s, _n and _b suffixes
- imply a data type of the expected result: a string, a numeric value, and
- a boolean. Due to Perl's forgiveness a string data type can in most cases
- be used as a number or a boolean and may be used when data type conversion
+ imply a data type of the expected result: a string, a numeric value, and a
+ boolean. Due to Perl's forgiveness a string data type can in most cases be
+ used as a number or as a boolean and may be used when data type conversion
and value normalization is not necessary or when a data type is not known.
Here are some examples:
@@ -578,15 +715,15 @@ NEW FEATURES
q_sql_s('subject_tag'),
);
- In addition to simple scalar arguments (field or attribute names),
- these six lookup object -creating functions can take as their argument
- a listref of field or attribute names, or a hashref where hash element
- values are field (or attribute) names and hash element keys are the
- result data names. Lookups resulting from such lookup objects will
- return a hashref of key/value pairs instead of a single scalar result.
- This is currently only useful in the @dkim_signature_options_bysender_maps
- list of lookups which expects such hash results (sets of data names
- and their values, i.e. entire records).
+ In addition to simple scalar arguments (a field or attribute name), these
+ six lookup object-creating functions can take as their argument a listref
+ of field or attribute names, or a hashref where hash entry values are
+ SQL field names (or LDAP attribute names), and hash entry keys are the
+ result data names. Lookups resulting from such lookup objects will return
+ a hashref of key/value pairs instead of a single scalar result. This is
+ currently only useful in the @dkim_signature_options_bysender_maps list
+ of lookups which expects such hash results (sets of data names and their
+ values, i.e. entire records).
The listref argument is just a shorthand notation which can be used
in place of a hashref when field names (or attribute names) are the
@@ -644,7 +781,7 @@ NEW FEATURES
not to sign at this stage, e.g. when a private key corresponding to the
requested signing domain and selector is not available. If a signing
service is not available or cannot sign, amavisd falls back to its own
- configured list of signing keys ( dkim_key() ) for backwards compatibility.
+ configured list of signing keys ( dkim_key() ) for backward compatibility.
The main reason for separating the signing act from the main amavisd daemon
is to make it possible to do the DKIM signing without letting amavisd
@@ -702,7 +839,7 @@ NEW FEATURES
(re-entry) MTA servers, as the selection from a list is random.
Session caching still works, so if a recently used SMTP/LMTP session
is still open, it will be reused, in which case no server randomization
- takes place.
+ takes place for as long as the established session remains open.
Typical configuration variables where this feature is available are:
$forward_method, $notify_method, $resend_method, $release_method, and
@@ -976,6 +1113,25 @@ NEW FEATURES
ACTION => sub { Amavis::Util::snmp_count('UserCounter2') },
};
+- a policy bank may now be loaded based on a path name of a Unix socket
+ receiving a connection.
+
+ Example use:
+
+ @listen_sockets = (
+ "$helpers_home/amavisd.sock1",
+ "$helpers_home/amavisd.sock2",
+ "$helpers_home/amavisd.sock3",
+ );
+
+ $interface_policy{"$helpers_home/amavisd.sock1"} = 'UX-S1';
+ $interface_policy{"$helpers_home/amavisd.sock2"} = 'UX-S2';
+ $interface_policy{"$helpers_home/amavisd.sock3"} = 'UX-S3';
+
+ $policy_bank{'UX-S1'} = { ... };
+ $policy_bank{'UX-S2'} = { ... };
+ $policy_bank{'UX-S3'} = { ... };
+
- transparent archival quarantine is a special case of archive quarantining
which retains all recipient addresses unmodified in an envelope of a message
directed to a quarantine. It makes sense when $archive_quarantine_method
@@ -988,9 +1144,9 @@ NEW FEATURES
recipients.
Think of the '%a' as a placeholder in a replacement string, being
- substituted by a full original address. There may be other substitution
- placeholders available in the future, equivalent to placeholders %l, %d,
- etc. in SQL query templates.
+ substituted by a full original recipient address. There may be other
+ substitution placeholders available in the future, equivalent to
+ placeholders %l, %d, etc. in SQL query templates.
Example:
$archive_quarantine_method = 'smtp:127.0.0.1:7777';
@@ -1023,8 +1179,9 @@ NEW FEATURES
to prevent potential backscatter. This is an additional safeguard to
prevent potential backscatter, therefore it is recommended that the
receiving quarantine server implements and announces the DSN capability.
- Specifying an empty string for the $mailfrom_to_quarantine achieves
- the same effect (implies NOTIFY=NEVER), preventing backscatter.
+ Specifying an empty string for the $mailfrom_to_quarantine achieves the
+ same effect (a null return path implies NOTIFY=NEVER) thus preventing
+ backscatter, but loses original sender address in the envelope.
Suggested by Patrick Ben Koetter.
@@ -1102,11 +1259,12 @@ NEW FEATURES
CC_CATCHALL, sub { ca('forward_method_maps') },
);
-- added a global configuration setting $allow_preserving_evidence,
- defaults to true; turning it off disables preserving temporary files
- (as evidence) in case of trouble; potentially useful for unattended
- and unmonitored operation; the setting has no influence on preserving
- evidence in case of @debug_sender_maps triggering;
+- added a global configuration setting $allow_preserving_evidence, defaults
+ to true. Turning it off disables preserving temporary files (as evidence)
+ in case of trouble, which is potentially useful for unattended and
+ unmonitored operation. The setting has no influence on preserving evidence
+ in case of @debug_sender_maps or @debug_recipient_maps triggering, which
+ always preserves evidence;
- an entry for CC_UNCHECKED was added to %admin_maps_by_ccat, defaulting
to @virus_admin_maps. Hence administrator notifications are also sent
@@ -1165,6 +1323,14 @@ NEW FEATURES
$partition_tag =
sub { my($msginfo)=@_; iso8601_yearweek($msginfo->rx_time) };
+ or based on a day of a week for short-term cycling (Mo=1, Tu=2,... Su=7):
+ $partition_tag =
+ sub { my($msginfo)=@_; ((localtime($msginfo->rx_time))[6]+6)%7+1 };
+ (a note from a future: starting with 2.8.0 the following is equivalent:
+ $partition_tag =
+ sub { my($msginfo)=@_; iso8601_weekday($msginfo->rx_time) };
+ )
+
Suggested by Michael Scheidell.
- the two placeholders %k and %a in templates for SQL lookup clauses
@@ -1299,7 +1465,7 @@ NEW FEATURES
per-domain Bayes databases when SpamAssassin is configured to keep
its Bayes database on an SQL server. It also makes it possible to load
per-recipient SpamAssassin preferences (configurations) from an SQL
- database (see previous section).
+ database (as described in a previous section).
Switching between Bayes usernames is cheap compared to switching between
SpamAssassin configuration files. A multi-recipient message whose
@@ -1320,11 +1486,11 @@ NEW FEATURES
to msa_networks;
- a new configuration variable $mail_id_size_bits allows setting the size
- of randomly generated mail_id and secret_id codes, which are used to
+ of randomly generated mail_id and secret_id codes which are used to
identify a message on releasing it from a quarantine, and are used as a
key when logging to SQL (penpals) or storing to quarantine. The variable
specifies a length of mail_id in bits, and must be an integral multiple
- of 24 (i.e. must be divisible by 6 and 8). The mail_id is represented
+ of 24 (i.e. must be divisible by 6 and by 8). The mail_id is represented
externally as a base64url-encoded string of $mail_id_size_bits / 6
characters, and internally as a string of $mail_id_size_bits / 8 octets.
The default value is 72 bits, as in previous versions.
@@ -1332,7 +1498,7 @@ NEW FEATURES
See entry "introduce a concept of 'mail_id'" in amavisd-new-2.3.0 release
notes for probability analysis of collisions. The default size should fit
- all practical needs.
+ all practical current needs.
The size of SQL fields msgs.mail_id, msgs.secret_id, msgrcpt.mail_id and
quarantine.mail_id may need increasing to accommodate $mail_id_size_bits/6
@@ -1378,7 +1544,7 @@ NEW FEATURES
like =?iso-8859-2?Q?...?=, =?koi8-r?B?...?=) and is converted to UTF-8,
optionally truncated to the specified size at clean UTF-8 boundaries,
and returned as a result. Suggested by Bastian.
- The macro can be useful in a logging template or in other templates
+ The macro is useful in a logging template or in notification templates
to decode Subject or From header fields, e.g.:
[? [:header_field|Subject]||,\
@@ -1401,7 +1567,7 @@ NEW FEATURES
- added a macro 'client_port', yields a TCP source port number of an
original SMTP session; the information is obtained through an XFORWARD
extension to an SMTP protocol as provided by Postfix, or through a
- 'client_port' attribute in an AM.PDP request;
+ 'client_port' attribute in an AM.PDP request. See RFC 6302;
- added a macro 'client_addr_port' which combines a client's IP address
and a TCP source port number (if available) of an original SMTP session;
@@ -1413,9 +1579,9 @@ NEW FEATURES
This macro is now included in a default main log template, so the TCP
source port number is logged along its IP address. This information is
useful in reporting abuse (e.g. client behind a NAT), troubleshooting,
- forensics and law enforcement. If this information is not needed, please
- assign a customized template to the $log_templ configuration variable.
- See also the draft-ietf-intarea-server-logging-recommendations document.
+ forensics and law enforcement. If this information is not desired, one
+ may assign a customized template to the $log_templ configuration variable.
+ See RFC 6302: Logging Recommendations for Internet-Facing Servers.
Suggested by Rok Potočnik.
- added a macro 'banned_parts_as_attr' and an associated per-recipient
@@ -1445,9 +1611,10 @@ NEW FEATURES
The single-character attribute names are unchanged from previous versions.
For documentation, here is a legend:
P: part's base name, i.e. a file name in a ./parts/ temporary directory
- L: part's location (path) in a mail tree
+ L: part's location (path) in a mail tree (branch enumeration, top-down)
M: MIME type as declared in MIME header fields of a message
- T: short part's content type according to a file(1) utility
+ T: short part's content type according to a file(1) utility and mapped
+ through @map_full_type_to_short_type_maps
N: declared part names (none, one or more), as declared in MIME header
fields or in an archive (tar, zip, ...)
A: part's attributes as follows:
@@ -1456,7 +1623,7 @@ NEW FEATURES
- macro 'header_field' and its alias 'HEADER' now has an optional third
parameter (index), which chooses the header field in case of multiple
header fields of the same name; the default (-1) is to return the last
- occurrence, as before; see README.customize for details;
+ (bottommost) occurrence, as before; see README.customize for details;
- added a macro 'actions_performed', which expands into a comma-separated
list of words: Accepted, Relayed(Untagged), RelayedTagged, Discarded,
@@ -1486,7 +1653,7 @@ NEW FEATURES
to the newly added SNMP variables 1.3.6.1.4.1.15312.2.1.1.19 - .26
(with the exception that 'RelayedUntagged' counter is abbreviated
in this macro as 'Relayed'). Please see their detailed description
- in AMAVIS-MIB.txt .
+ in a file AMAVIS-MIB.txt .
- added a macro 'rusage', which expands to a resource usage entry as provided
by a system service getrusage(2); the argument to the macro should be one
@@ -1521,9 +1688,11 @@ NEW FEATURES
generated by amavisd itself, primarily to provide true per-recipient
handling; header field names must still be listed in the associative
array %allowed_added_header_fields in order to be inserted;
- overrides are configurable through %prefer_our_added_header_fields;
+ overrides are configurable through %prefer_our_added_header_fields,
+ for example:
+ $prefer_our_added_header_fields{lc('X-Spam-Status')} = 0;
-- added attribute 'log_id' to the server responses in AM.PDP protocol,
+- added an attribute 'log_id' to server responses in an AM.PDP protocol,
allowing the client to match its request with the amavisd daemon logging;
- added LDAP attributes: amavisAddrExtensionVirus,
@@ -1541,13 +1710,12 @@ NEW FEATURES
tables @disclaimer_options_bysender_maps, so the replacement of the
_OPTIONS_ placeholder in @altermime_args_disclaimer could be made dynamic;
suggested by Quanah Gibson-Mount;
- (TODO: update LDAP schema)
- for consistencly, added LDAP attribute amavisUncheckedLover, along
with its corresponding SQL field 'unchecked_lover' and a statical
list of lookup tables @unchecked_lovers_maps, which appears in the
%lovers_maps_by_ccat. Previously the CC_UNCHECKED entry of the
- %lovers_maps_by_ccat was (ab)used and shared the @virus_lovers_maps
+ %lovers_maps_by_ccat was (ab)used, and shared the @virus_lovers_maps
value. Suggested by Patrick Ben Koetter;
- added a variable $myprogram_name, which defaults to a program name
@@ -1561,8 +1729,8 @@ NEW FEATURES
mechanism (such as DSPAM or CRM114), make header fields produced by them
visible to SpamAssassin too, so that its rules can benefit from additional
information. Note that in order for SpamAssassin to be able to see such
- header fields from other scanners, they must be listed in the @spam_scanners
- list *before* the 'SpamAssassin' entry. Suggested by Marco.
+ header fields from other scanners, such scanners must be listed in the
+ @spam_scanners list *before* the 'SpamAssassin' entry. Suggested by Marco.
OPTIMIZATIONS
@@ -1644,8 +1812,8 @@ OTHER
header field from a first pass.
- updated ARF notifications to RFC 5965 (An Extensible Format for Email
- Feedback Reports); the $report_format = 'arf' was based on ARF draft,
- now it complies to RFC 5965;
+ Feedback Reports); the $report_format = 'arf' implementation was based
+ on ARF draft, now it complies with RFC 5965;
- tightened some sanity limits on DKIM verification to better handle
mail messages with a huge number of signatures; problem reported
@@ -1667,24 +1835,25 @@ OTHER
"***UNCHECKED*** Fwd: ***UNCHECKED*** Re: foo bar" ; based on a
patch by Thomas Arendsen Hein;
-- p0f-analyzer.pl: treat an 'IPv4-mapped IPv6 addresses in alternative
- form' as an IPv4 address, otherwise the p0f-analyzer.pl would ignore such
- queries (knowing that the p0f daemon does not handle IPv6); some TCP/IP
- stacks prefer to present an IPv4 address in this form to applications;
- problem reported by Vytautas Kasparavicius;
+- p0f-analyzer.pl: convert an 'IPv4-mapped IPv6 addresses in alternative
+ form' to an IPv4 address, otherwise the p0f-analyzer.pl would ignore
+ such queries, as the p0f daemon did not handle IPv6 until version 3.
+ The 'IPv4-mapped IPv6 addresses' is returned for an IPv4 connection
+ when TCP/IP stack is configured to allow inet6 sockets to accept inet
+ sessions; problem reported by Vytautas Kasparavicius;
- suppress generating a non-delivery notification if a SpamAssassin test
DKIM_ADSP_DISCARD is hit, honouring RFC 5617;
- amavisd.conf: commented-out calls to do_ascii to match defaults in the
amavisd program; the uulib code (as invoked by Convert::UUlib) has a
- history of stability problems, seems it is causing more grief than are
- the benefits it brings;
+ history of stability problems, seems it is causing more grief compared
+ to the benefits it brings;
- new AV entry for 'Avira for UNIX 3.x', thanks to g0rbi, Thomas Mueller,
Steffen Ille, Klaus Fuerstberger and Andreas Schulze;
-- add three more exception cases to mercifully ignore a EBADF I/O error
+- add three more exception cases to mercifully ignore an EBADF I/O error
due to a Perl bug on line-by-line reading;
- entries 'SpamAssassin' and 'SpamdClient' in the @spam_scanners list
@@ -1723,7 +1892,7 @@ OTHER
it is now equivalent to %s, but should no longer be used and might be
retired or re-purposed with the next version; default notification
templates were adjusted accordingly - please adjust your customized
- if using them;
+ templates if using them;
- drop dependency on Digest::SHA1;
@@ -1774,7 +1943,7 @@ CLEANING
taking into consideration the actual time left till the deadline;
An attempt to set its value to a non-default value produces a warning.
-- retired a setting $sa_spam_report_header, it was obsoleted by
+- retired a setting $sa_spam_report_header, it was obsoleted in
amavisd-new-2.4.3 with the introduction of %allowed_added_header_fields.
To enable insertion of X-Spam-Report header field, please use instead:
$allowed_added_header_fields{lc('X-Spam-Report')} = 1;
@@ -1800,7 +1969,7 @@ CLEANING
but their value is ignored. An attempt to set the value of a variable
$sa_spam_modifies_subj to a non-default value produces a warning.
-- retired a setting $insert_received_line, it was obsoleted by
+- retired a setting $insert_received_line, it was obsoleted in
amavisd-new-2.4.3 with the introduction of %allowed_added_header_fields.
To disable insertion of a Received header field, please use instead:
$allowed_added_header_fields{lc('Received')} = 0;
@@ -1847,11 +2016,12 @@ CLEANING
Amavis::SpamControl::{ExtProg,SpamdClient,SpamAssassin}::check
As a compatibility measure, the do_quarantine() may still be called
- with or without the first argument $conn, its value is now ignored.
+ with or without the first argument $conn, its value is now ignored
+ if present.
Although the $conn argument is also redundant in calls to custom hooks
- (as this information is available as $msginfo->conn_obj), these calls
- are left unchanged for compatibility with existing custom hooks.
+ (as this information is available through $msginfo->conn_obj), these
+ calls are left unchanged for compatibility with existing custom hooks.
---------------------------------------------------------------------------
@@ -1932,20 +2102,20 @@ BUG FIXES
zip unpacking; reported by Tuomo Soini;
- when logging or quarantining to SQL, execute a clause: SET NAMES 'utf8'
- after connecting to a database, to ensure the decoded Subject and From
- header fields are correctly interpreted by a SQL server as UTF-8 encoded
+ after connecting to a database to ensure the decoded Subject and From
+ header fields are correctly interpreted by an SQL server as UTF-8 encoded
strings. It seems the module DBD::mysql does not observe a MySQL setting
for 'character_set_client' and needs an explicit SET NAMES. The problem
did not affect PostgreSQL. Reported by Zhang Huangbin;
-- avoid LDAP lookups aborting the scan when a %d placeholder is used in
- a $default_ldap{base} setting and the resulting base do not exist in
- an LDAP schema; reported by Zhang Huangbin;
+- avoid LDAP lookups aborting the scan when a %d placeholder is used
+ in a $default_ldap{base} setting and the resulting base does not exist
+ in an LDAP schema; reported by Zhang Huangbin;
- the amavisd-new 2.6.3 relaxed semantics of a number of hard links on a
directory in TempDir::prepare(_dir), but left out an equivalent change
necessary in TempDir::check, which is now fixed; the change only affects
- certain file system (like the one used on Mac OSX);
+ certain file system (like the one used on Mac OS X);
- treat an empty PID file or a junk one-liner file the same as a nonexistent
PID file; previously an empty PID file (e.g. after an unclean shutdown)
@@ -1978,17 +2148,13 @@ OTHER
now represented as native Perl character strings (Unicode), and as such
may also end up in reported names of banned parts. Regular expressions
in @banned_filename_maps, $banned_filename_re and $banned_namepath_re
- may also see these strings as native Perl characters, as well as in their
+ may also see these strings as native Perl characters, along with their
MIME-encoded form. The change also affects interpretation of names with
earlier versions of MIME-Tools, making them behave more like the 5.500.
- amavisd.conf: exclude names starting with 'cid:' from matching the
double extensions banning rule, avoiding false positives;
-- support Postfix 2.9 long queue IDs (enable_long_queue_ids=yes) as
- available since postfix-20110321 by adjusting a default value of
- a $log_templ template;
-
- a small update to a default @virus_name_to_spam_score_maps;
- the 'originating' flag is now passed on to SpamAssassin through its
@@ -2388,7 +2554,7 @@ NEW FEATURES
The list is traversed in order, the first matching networks list stops
the search and its associated policy name is used. Suggested by Jo Rhett.
- The default setting retains backwards compatibility:
+ The default setting retains backward compatibility:
@client_ipaddr_policy = map { $_ => 'MYNETS' } @mynetworks_maps;
@@ -2401,7 +2567,7 @@ NEW FEATURES
Example:
@client_ipaddr_policy = (
- [qw( 0.0.0.0/8 127.0.0.1/8 [::] [::1] )] => 'LOCALHOST',
+ [qw( 0.0.0.0/8 [::] 127.0.0.0/8 [::1] )] => 'LOCALHOST',
[qw( !172.16.1.0/24 172.16.0.0/12 192.168.0.0/16 )] => 'PRIVATENETS',
[qw( 192.0.2.0/25 192.0.2.129 192.0.2.130 )] => 'PARTNER',
\@some_other_networks => 'OTHER',
@@ -2452,7 +2618,7 @@ NEW FEATURES
Amavis::SpamControl::ExtProg (which is only loaded if needed).
This is similar in concept to @av_scanners list, and allows using
amavisd with different spam scanners, not just with SpamAssassin.
- The default setting is backwards compatible:
+ The default setting is backward compatible:
@spam_scanners = (
['SpamAssassin', 'Amavis::SpamControl::SpamAssassin'],
@@ -2466,7 +2632,7 @@ NEW FEATURES
Currently supported spam scanners are:
- - SpamAssassin: backwards compatible, uses the module Mail::SpamAssassin
+ - SpamAssassin: backward compatible, uses the module Mail::SpamAssassin
directly as before;
- SpamdClient: a client to spamd, equivalent to a spamc usage; the main
@@ -2531,7 +2697,7 @@ NEW FEATURES
pairs. Unrecognized options are ignored. Currently the only two options are:
mail_body_size_limit ... the ExtProg module only feeds up to about this
- number of bytes (or slightly over) of a message to a spam scanner;
+ number of bytes (or slightly more) of a message to a spam scanner;
if unspecified or undefined a default limit is $sa_mail_body_size_limit,
and if that is undefined, an entire message is passed regardless of
its size;
@@ -2604,7 +2770,7 @@ OTHER
- reduce log level of a test in TempDir::prepare for a number of links
left on a directory after purging it, seems like it does not play well
- with a file system on Mac OSX, producing an occasional warning:
+ with a file system on Mac OS X, producing an occasional warning:
TempDir::prepare:
directory /var/amavis/tmp/amavis-2009... has 2 subdirectories
reported by Matthias Schmidt;
@@ -2761,7 +2927,7 @@ BUG FIXES
- amavisd.conf-default: definition of %sql_clause default was out of date;
reported by Roland;
-- releasing a non-existent message from a SQL quarantine produced an
+- releasing a non-existent message from an SQL quarantine produced an
inappropriate error message about a subsequent failure, instead of
reporting a missing record; reported by Rick (rn). Also let SQL treat
a NULL in mail_text.partition_tag as 0 by using coalesce() - changed
@@ -3167,7 +3333,7 @@ BUG FIXES
was failing in some setups (even though it was in accordance with
a BerkeleyDB module documentation); reported by Leo Baltus;
-- README.sql-mysql: fixed a SQL data type mismatch between maddr.id (used as
+- README.sql-mysql: fixed an SQL data type mismatch between maddr.id (used as
a foreign key) and msgs.sid & msgrcpt.rid; they all should be of the same
type, either integer unsigned or bigint unsigned; a schema as published
in README.sql-mysql could not be built because of a conflict in a data
@@ -3245,7 +3411,7 @@ OTHER
particular effect (and is not really necessary) on existing 2.6.0 databases
where a primary key is mail_id (it is just a redundant extra condition),
but saves a day when a primary key is a composite: (partition_tag,mail_id),
- which may be a requirement of a SQL partitioning mechanism.
+ which may be a requirement of an SQL partitioning mechanism.
Thanks to Thomas Gelf for his testing of MySQL partitioning, reporting
deficiency in amavisd SQL schema (primary keys) which did not meet MySQL
requirements for partitioning, and suggestions;
@@ -3260,7 +3426,7 @@ OTHER
If a partition_tag information is readily available to a requester, it
is advised that the attribute is included in a request even if mail_id
is known to be unique. This may expedite a search and provide a double
- check to a validity of a request. For backwards compatibility amavisd
+ check to a validity of a request. For backward compatibility amavisd
performs a query on msgs.mail_id for a partition_tag value if it is
missing form a request, the query uses an SQL clause in a new entry
$sql_clause{'sel_msg'}. If exactly one record matches, then everything
@@ -3320,7 +3486,7 @@ make any difference, and one is free to choose between having mail_id
unique across the entire table or just within each partition_tag value.
Changing a primary key to (partition_tag,mail_id) brings consequences
-to quarantining, in particular to releasing from a SQL quarantine,
+to quarantining, in particular to releasing from an SQL quarantine,
where it no longer suffices to specify mail_id=xxx in AM.PDP request,
but may be necessary to specify also a partition_tag=xx to distinguish
between SQL-quarantined messages which happen to have the same mail_id.
@@ -3987,7 +4153,7 @@ NEW FEATURES
accepted as an AM.PDP protocol attribute 'client_port';
- updated p0f-analyzer.pl now supports a source port number information in
- queries while preserving backwards compatibility with previous versions
+ queries while preserving backward compatibility with previous versions
of amavisd-new. Version 2.6.0 of amavisd requires a new version of
p0f-analyzer.pl (supplied in the 2.6.0 distribution) if operating system
fingerprinting is enabled. A source port number information in a query
@@ -4007,8 +4173,8 @@ NEW FEATURES
There are currently no advantages (and some disadvantages) in choosing
direct queries to p0f, compared to sending queries to p0f-analyzer.pl,
so this new method is not currently recommended. Disadvantages are:
- * p0f uses a linear search over its list of recent sessions, whereas
- p0f-analyzer.pl uses a fast hash lookup method;
+ * p0f uses a linear search over its list of recent sessions (at least
+ as of version 2.0.8), whereas p0f-analyzer.pl uses a fast hash lookup;
* p0f keeps a relatively small list of recent sessions which is limited
by the number of slots (size can be specified on a command line, but
is limited by a linear search time), whereas p0f-analyzer.pl expires
@@ -4479,7 +4645,7 @@ OTHER
- a release request now takes its default recipients list from a header
field X-Envelope-To-Blocked, no longer from X-Envelope-To. This avoids
releasing a message to recipients which have already received it in
- the first place, e.g. spam lovers. For backwards compatibility, if
+ the first place, e.g. spam lovers. For backward compatibility, if
X-Envelope-To-Blocked header field is not found in a quarantined
message, the recipients list defaults to X-Envelope-To as before.
A release request can still provide its explicit list of recipients
@@ -4986,7 +5152,7 @@ amavisd-new-2.5.2 release notes
BUG FIXES
- in a milter setup log_id was left undefined, which resulted in log lines
- without id, and a SQL constraint violation "Column 'am_id' cannot be null"
+ without id, and an SQL constraint violation "Column 'am_id' cannot be null"
when logging to SQL was enabled. The bug was introduced in 2.5.1;
problem reported by Martin Svensson;
@@ -5013,7 +5179,7 @@ NEW FEATURES
examining rules which contributed significantly to the score may
indicate which rules need adjustment;
-- when preparing a SQL SELECT clause in lookup_sql, provide an additional
+- when preparing an SQL SELECT clause in lookup_sql, provide an additional
placeholder %a in a clause template, which is much like the existing %k,
but evaluates to an exact mail address (i.e. the same as the first entry
in the %k list), which makes it suitable for SQL pattern matching;
@@ -5099,7 +5265,7 @@ OTHER
Patrick Ben Koetter; retired file: README.postfix.old
- documentation: updated README.sql-pg to include a faster alternative to
- purging a SQL logging database: the alternative 'DELETE FROM maddr' on
+ purging an SQL logging database: the alternative 'DELETE FROM maddr' on
PostgreSQL runs faster by a factor of 1.5 to 2 from the one previously
suggested;
@@ -5181,7 +5347,7 @@ NEW FEATURES
BUG FIXES
-- fixed quarantining to a SQL database of messages with a null envelope
+- fixed quarantining to an SQL database of messages with a null envelope
sender address (broken in 2.5.0, causing such messages to tempfail);
reported by Markus Edholm, Vahur Jõesalu and Michael Scheidell;
@@ -5418,9 +5584,10 @@ BUG FIXES
quarantine id containing a body hash in a name (%b in template);
reported by Ron Rademaker;
-- skip a SQL-logging database operation if an associated clause in %sql_clause
- is disabled, e.g. set to undef or ''; this allows for example to selectively
- disable SQL logging based on a policy bank; thanks to Riaan Kok;
+- skip an SQL-logging database operation if an associated clause
+ in %sql_clause is disabled, e.g. set to undef or ''; this allows
+ for example to selectively disable SQL logging based on a policy bank;
+ thanks to Riaan Kok;
- let LHA decoder (do_lha) recognize also other listing formats, e.g. MS-DOS,
symlinks, not just plain Unix archives; problem reported by Ryuhei Funatsu;
@@ -5777,7 +5944,7 @@ NEW FEATURES
that outgoing mail is not being checked, so infected internal hosts are
able to pollute the world. Also the pen pals feature is no longer useful,
as it requires the information on previous outgoing mail to be present
- in a SQL database.
+ in an SQL database.
- a new command line option -i, it takes one argument which can be any
@@ -6249,8 +6416,8 @@ SECURITY
- p0f-analyzer.pl will no longer reply to queries coming from low-numbered
UDP ports below 1024 or from nfsd port 2049, and will ignore queries
with nonce longer than 1024 character or containing characters outside
- of \040-\177 range to limit its usefulness as a potential reflector
- for an attacker from internal networks.
+ of \040-\177 (octal) range to limit its usefulness as a potential
+ reflector for an attacker from internal networks.
INCOMPATIBLE CHANGE WITH 2.4.4
@@ -6500,7 +6667,7 @@ BUG FIXES:
USING decode(replace(mail_text,'\\','\\\\'),'escape');
If conversion of data type for 'quarantine.mail_text' is not done,
- the following error will be reported when storing a message to a SQL
+ the following error will be reported when storing a message to an SQL
quarantine is attempted:
TROUBLE in check_mail: quar+notif FAILED:
@@ -6982,7 +7149,7 @@ OTHER CHANGES AND SMALL FEATURES:
of header lines longer than 998 characters, and is a pre-condition for
$allow_fixing_improper_header_folding, controlling removal of all-whitespace
continuation lines. The $allow_fixing_improper_header defaults to true
- for backwards compatibility. Fixing header may protect poorly written
+ for backward compatibility. Fixing header may protect poorly written
mail readers, but may break DomainKeys/DKIM validation of messages
with illegal header if verification is done after content filtering,
so if this is of concern, one has a choice of turning it off;
@@ -7400,7 +7567,7 @@ NEW FEATURES:
their sender, recipients, delivery status, mail contents type (no changes
there, this feature was introduced with amavisd-new-2.3.0); for the
purpose of pen pals scheme only records with local-domain senders matter;
- * when a message is received, a SQL lookup against a SQL logging database
+ * when a message is received, an SQL lookup against an SQL logging database
is performed, looking for previous messages sent in reverse direction,
i.e. from a local user (which is now a recipient of the current mail)
to the address that is now the sender of the message being processed;
@@ -7475,8 +7642,8 @@ NEW FEATURES:
* there may be multiple MTA+amavisd servers, but all must use the same
logging SQL database;
* forwarding is compatible with the pen pals scheme;
- * broken forwarding scheme like SRS (with SPF), where envelope sender
- address is replaced by a forwarding mailbox address is counterproductive;
+ * a forwarding scheme like SRS (with SPF), where envelope sender address
+ is replaced by a forwarding mailbox address is counterproductive;
for example: a local user may also have an external mailbox at some remote
provider with poor spam protection; forwarding from the remote to a local
mailbox is set up and a forwarding MTA misguidedly substitutes original
@@ -7707,7 +7874,7 @@ MAJOR NEW ENHANCEMENTS:
%admin_maps_by_ccat %dsn_bcc_by_ccat
%warnrecip_maps_by_ccat %addr_extension_maps_by_ccat
gradually phasing out separate configuration variables for each category;
- the change is fully backwards compatible, existing variables are referenced
+ the change is fully backward compatible, existing variables are referenced
through default values of the new variables, and no longer used directly;
The chain of lookups adhere to the following evaluation sequence
@@ -8042,7 +8209,7 @@ OTHER CHANGES:
on multi-homed boxes one may need to specify interface and IP address
where MTA is listening, the filter syntax is the same as in tcpdump, e.g.:
- p0f -l -i bge0 'dst host 192.0.2.66 and tcp dst port 25' 2>&1 \
+ p0f -l -i em1 'dst host 192.0.2.66 and tcp dst port 25' 2>&1 \
| p0f-analyzer.pl 2345 &
* the program p0f-analyzer.pl reads p0f reports on stdin, keeps a cache
@@ -8152,7 +8319,7 @@ OTHER CHANGES:
- new configuration variable $allow_fixing_improper_header_folding (also
a member of policy banks) controls fixing improperly folded header fields
made up entirely of whitespace (prohibited by RFC 2822) by removing
- all-whitespace continuation lines; defaults to true for backwards
+ all-whitespace continuation lines; defaults to true for backward
compatibility; fixing such header fields is desirable and can protect
poorly written mail readers, but may break DomainKeys/DKIM validation of
messages with illegal header, so if this is of concern, one has a choice
@@ -8691,7 +8858,7 @@ BUG FIXES since 2.3.1:
Such messages are still let through with $final_bad_header_destiny set to
D_BOUNCE, as otherwise they will be lost because a bounce is suppressed
for null sender messages and for mail from mailing list. This behaviour
- is retained for backwards compatibility, but may need to be reconsidered.
+ is retained for backward compatibility, but may need to be reconsidered.
- fix regexp for extracting am_id from amavis-milter helper program requests;
@@ -8851,8 +9018,8 @@ QUICK OVERVIEW:
Provides more flexible configuration of decoders. Allows recipients to have
individual banning rules. Assigns a long-term unique id to each message,
reducing clashes and facilitating retrieval of information. The daemon can
-store information to a SQL database for logging, reporting and quarantine
-retrieval, optionally storing entire message to a SQL database. File-based
+store information to an SQL database for logging, reporting and quarantine
+retrieval, optionally storing entire message to an SQL database. File-based
quarantine can disperse files to 62 subdirectories. Provides a quarantine
release mechanism. Reconnects to SQL if connection is broken. Can skip
quarantining high-score spam. Compatibility with IPv6-enabled Postfix
@@ -9064,21 +9231,21 @@ NEW FEATURES:
granting user a right to release a quarantined message addressed to him.
- SQL: can store information about every processed mail to SQL; the information
- is similar to level 0 log entries, but more detailed; a SQL database can be
+ is similar to level 0 log entries, but more detailed; an SQL database can be
used as a basis for searching for a particular mail, for preparing reports
and to facilitate quarantine management (searching and releasing).
Enabled by configuring the @storage_sql_dsn list which contains information
- about a SQL server and dataset name, just like the @lookup_sql_dsn does
+ about an SQL server and dataset name, just like the @lookup_sql_dsn does
for the SQL lookups. If @storage_sql_dsn is the same as the @lookup_sql_dsn,
a single connection to SQL database will be used, otherwise separate and
independent connections are established, possibly to different SQL servers.
Loosely based on ideas from Maia Mailguard by Robert LeBlanc and a patch
by Brian Wong. Thanks to Brian Wong for testing and valuable feedback.
- See README.lookups for a SQL schema. See new file amavisd-sql-maintain
+ See README.lookups for an SQL schema. See new file amavisd-sql-maintain
(incorporated into README.sql in later versions) for ideas on database
housekeeping (expiring old entries).
-- SQL: can quarantine to a SQL database; selected by setting config variables
+- SQL: can quarantine to an SQL database; selected by setting config variables
$*_quarantine_method to 'sql:' The @storage_sql_dsn list of dataset names
is used to choose SQL server and dataset name, and must be nonempty when
$*_quarantine_method is 'sql:'; When $*_quarantine_method is set to 'sql:'
@@ -9090,7 +9257,7 @@ NEW FEATURES:
are invalidated on reconnect, and are dynamically 'prepared' as needed.
- SQL: thanks to a reorganization of SQL objects an automatic reconnect
- to a SQL server is done without temporary failing a processed message;
+ to an SQL server is done without temporary failing a processed message;
- SQL: new configuration variable (an associative array) %sql_clause, also
part of policy maps, allows SQL clauses to be switched with policy banks.
@@ -9115,7 +9282,7 @@ NEW FEATURES:
a new utility program 'amavisd-release', which currently mostly serves to
demonstrate how to request releasing of a quarantined file. Currently the
supported quarantine types are: plain file, gzipped plain file with a name
- ending in .gz, and a SQL-based quarantine. Currently not supported is a
+ ending in .gz, and an SQL-based quarantine. Currently not supported is a
release from a BSMTP-encoded plain file or from a mbox (Unix-style) mailbox
quarantine file.
@@ -9128,13 +9295,13 @@ NEW FEATURES:
The first argument is a (relative) quarantine file name, as reported in
the log. It must include a 12-character mail_id which is automatically
recognized. The second argument is a secret_id, which can be fetched from
- a SQL database if @storage_sql_dsn is enabled (see README.sql),
+ an SQL database if @storage_sql_dsn is enabled (see README.sql),
for example by the command:
$ mysql amavis -e 'SELECT secret_id FROM msgs WHERE mail_id="V5htXBh0y0Hr"'
or (preferably) by some other more advanced utility program.
Current simple-minded heuristics in the amavisd-release program is to assume
- a message is stored in a SQL database when the file name (first argument)
+ a message is stored in an SQL database when the file name (first argument)
consists only of a 12-character mail_id. Please adjust the program if this
assumption is not true, e.g. when $virus_quarantine_method='local:%m'
instead of a default $virus_quarantine_method='local:virus-%m';
@@ -9458,7 +9625,7 @@ CHANGES AND MINOR NEW FEATURES:
- added @av_scanners entry for File::Scan;
-- when preparing a SQL SELECT clause for white/blacklisting lookup,
+- when preparing an SQL SELECT clause for white/blacklisting lookup,
take into account a relative position of ? and %k in the
$sql_select_white_black_list template to improve flexibility
of specifying the clause; suggested by Matt Petteys;
@@ -11517,7 +11684,7 @@ Patch: amavisd-new-20030616-p7
Patch: amavisd-new-20030616-p6
- change SQL lookup code to better handle SQL database server restarts.
- After a SQL server restart amavisd-new would previously TEMPFAIL (4xx)
+ After an SQL server restart amavisd-new would previously TEMPFAIL (4xx)
all messages until amavisd child process would run down, then resume
normal operation with the new child birth. Now the SQL server reconnect
is done when the next mail arrives, so only one mail with each amavisd
@@ -12090,7 +12257,7 @@ NEW FEATURES
or an SMTP REJECT if desired) with the full explanation of the problem,
with offending header fields trimmed, sanitized and included in the text.
- New setting $final_bad_header_destiny, defaults to D_PASS for backwards
+ New setting $final_bad_header_destiny, defaults to D_PASS for backward
compatibility, but a value D_BOUNCE is suggested. Similarly to other
*lovers* settings, a hash and an ACL lookup %bad_header_lovers and
\@bad_header_lovers_acl are available, and a setting $warnbannedsender.
@@ -12329,7 +12496,7 @@ amavisd-new-20030314 release notes
MAIN NEW FEATURES AT A GLANCE
-- per-user white- and blacklisting, including a SQL lookup mechanism;
+- per-user white- and blacklisting, including an SQL lookup mechanism;
refined semantics for white/blacklists;
- easier to run in chroot jail, see README.chroot;
@@ -12785,7 +12952,7 @@ OTHER
recipient, and the result should be an inner lookup table (hash or ACL
or RE), where the key used will be the sender.
- Per-recipient white/blacklisting is also available as a SQL lookup.
+ Per-recipient white/blacklisting is also available as an SQL lookup.
See configuration variable $sql_select_white_black_list and README.lookups .
@@ -13910,7 +14077,7 @@ Changes since amavisd-new-20020418:
- avoid historical (misleading) parameter name $localhost_ip;
use $relayhost and $relayhost_port instead, but take old variables into
- account for backwards compatibility with existing amavisd.conf files
+ account for backward compatibility with existing amavisd.conf files
- Here is an overall picture (sequence of events)
of how pieces fit together:
diff --git a/amavisd b/amavisd
index cce1f05..1939023 100755
--- a/amavisd
+++ b/amavisd
@@ -10,7 +10,7 @@
# on amavisd-snapshot-20020300).
#
# All work since amavisd-snapshot-20020300:
-# Copyright (C) 2002-2011 Mark Martinec,
+# Copyright (C) 2002-2012 Mark Martinec,
# All Rights Reserved.
# with contributions from the amavis-user mailing list and individuals,
# as acknowledged in the release notes.
@@ -217,7 +217,7 @@ sub fetch_modules($$@) {
}
BEGIN {
- if ($] < 5.008000) { # deal with a perl 5.6.1 glob() taint bug
+ if ($] <= 5.008) { # deal with a glob() taint bug (perl 5.6.1, 5.8.0)
fetch_modules('REQUIRED BASIC MODULES', 1, qw(File::Glob));
File::Glob->import(':globally'); # use the same module as Perl 5.8 uses
}
@@ -277,7 +277,7 @@ use constant CC_VIRUS => 9;
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
%EXPORT_TAGS = (
'dynamic_confvars' => # per- policy bank settings
@@ -655,7 +655,7 @@ use vars qw($read_config_files_depth @actual_config_files);
BEGIN { # init_primary: version, $unicode_aware, base policy bank
$myprogram_name = $0; # typically 'amavisd'
$myproduct_name = 'amavisd-new';
- $myversion_id = '2.7.0'; $myversion_date = '20110701';
+ $myversion_id = '2.7.1'; $myversion_date = '20120429';
$myversion = "$myproduct_name-$myversion_id ($myversion_date)";
$myversion_id_numeric = # x.yyyzzz, allows numerical compare, like Perl $]
@@ -1513,7 +1513,7 @@ BEGIN {
['gz', \&Amavis::Unpackers::do_uncompress, \$gunzip],
['bz2', \&Amavis::Unpackers::do_uncompress, \$bunzip2],
['xz', \&Amavis::Unpackers::do_uncompress,
- ['xzdec'. 'xz -dc', 'unxz -c', 'xzcat'] ],
+ ['xzdec', 'xz -dc', 'unxz -c', 'xzcat'] ],
['lzma', \&Amavis::Unpackers::do_uncompress,
['lzmadec', 'xz -dc --format=lzma',
'lzma -dc', 'unlzma -c', 'lzcat', 'lzmadec'] ],
@@ -1533,7 +1533,7 @@ BEGIN {
['arj', \&Amavis::Unpackers::do_unarj, \$unarj],
['arc', \&Amavis::Unpackers::do_arc, \$arc],
['zoo', \&Amavis::Unpackers::do_zoo, \$zoo],
- ['lha', \&Amavis::Unpackers::do_lha, \$lha],
+# ['lha', \&Amavis::Unpackers::do_lha, \$lha], # not safe
['doc', \&Amavis::Unpackers::do_ole, \$ripole],
['cab', \&Amavis::Unpackers::do_cabextract, \$cabextract],
['tnef', \&Amavis::Unpackers::do_tnef_ext, \$tnef],
@@ -2071,7 +2071,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&init &collect_log_stats &log_to_stderr &log_fd
&write_log &open_log &close_log);
@@ -2250,7 +2250,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&init §ion_time &report &get_time_so_far);
}
@@ -2338,7 +2338,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&untaint &min &max &minmax &unique_list &unique_ref
&safe_encode &safe_decode &q_encode
@@ -2368,9 +2368,16 @@ use subs @EXPORT_OK;
use Errno qw(ENOENT EACCES EAGAIN ESRCH EBADF);
use IO::File ();
-use Digest::MD5 2.22; # need 'clone' method
+use Digest::MD5; # 2.22 provides 'clone' method, no longer needed since 2.7.0
use MIME::Base64;
-# use Encode; # Perl 5.8 UTF-8 support
+use Encode; # Perl 5.8 UTF-8 support
+use Scalar::Util qw(tainted);
+
+use vars qw($enc_tainted);
+BEGIN {
+ $enc_tainted = substr($ENV{PATH}.$ENV{HOME}, 0,0); # tainted empty string
+ tainted($enc_tainted) or warn "Amavis::Util: can't obtain a tainted string";
+}
# Return untainted copy of a string (argument can be a string or a string ref)
#
@@ -2433,23 +2440,32 @@ sub unique_ref(@) {
}
# A wrapper for Encode::encode, avoiding a bug in Perl 5.8.0 which causes
-# Encode::encode to loop and fill memory when given a tainted string
+# Encode::encode to loop and fill memory when given a tainted string.
+# Also works around a CPAN bug #64642 in module Encode:
+# Tainted values have the taint flag cleared when encoded (or decoded)
+# https://rt.cpan.org/Public/Bug/Display.html?id=64642
+# (still unresolved with Encode as bundled with Perl 5.14.2)
#
sub safe_encode($$;$) {
- my($encoding,$str,$check) = @_;
- $check = 0 if !defined $check;
- # obtain taintedness of the string, with UTF8 flag unconditionally off
- my($taint) = Encode::encode('ascii',substr($str,0,0));
- $taint . Encode::encode($encoding,untaint($str),$check); # preserve taint
+# my($encoding,$str,$check) = @_;
+ my $encoding = shift;
+ return undef if !defined $_[0]; # must return undef even in a list context!
+ my $enc = Encode::find_encoding($encoding);
+ $enc or warn "safe_encode: unknown encoding '$encoding'";
+ return $enc->encode(@_) if !tainted($_[0]);
+ # propagate taintedness across taint-related bugs in module Encode
+ $enc_tainted . $enc->encode(untaint($_[0]), $_[1]);
}
sub safe_decode($$;$) {
- my($encoding,$str,$check) = @_;
- $check = 0 if !defined $check;
+# my($encoding,$str,$check) = @_;
+ my $encoding = shift;
+ return undef if !defined $_[0]; # must return undef even in a list context!
my $enc = Encode::find_encoding($encoding);
- return $str if !$enc;
- my $taint = substr($str,0,0); # taintedness of the string
- $taint . $enc->decode(untaint($str),$check); # preserve taint
+ return $_[0] if !$enc;
+ return $enc->decode(@_) if !tainted($_[0]);
+ # propagate taintedness across taint-related bugs in module Encode
+ $enc_tainted . $enc->decode(untaint($_[0]), $_[1]);
}
# Do the Q-encoding manually, the MIME::Words::encode_mimeword does not
@@ -2479,7 +2495,8 @@ sub q_encode($$$) {
#
sub xtext_encode($) { # RFC 3461
my($str) = @_; local($1);
- $str = safe_encode('UTF-8',$str) if Encode::is_utf8($str);
+ # avoid Encode::is_utf8 check, always false on tainted, Perl bug #32687
+ $str = safe_encode('UTF-8',$str); # if Encode::is_utf8($str);
$str =~ s/([^\041-\052\054-\074\076-\176])/sprintf('+%02X',ord($1))/egs;
$str;
}
@@ -2864,7 +2881,8 @@ BEGIN {
}
sub sanitize_str {
my($str, $keep_eol) = @_;
- $str = safe_encode('UTF-8',$str) if Encode::is_utf8($str);
+ # avoid Encode::is_utf8 check, always false on tainted, Perl bug #32687
+ $str = safe_encode('UTF-8',$str); # if Encode::is_utf8($str);
local($1);
if ($keep_eol) {
$str =~ s/([^\012\040-\133\135-\176])/ # and \240-\376 ?
@@ -3334,7 +3352,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&exit_status_str &proc_status_ok &kill_proc &cloexec
&run_command &run_command_consumer &run_as_subprocess
@@ -3865,7 +3883,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
@EXPORT = qw(
&rfc2822_timestamp &iso8601_timestamp &iso8601_utc_timestamp
@@ -4779,7 +4797,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
import Amavis::Util qw(ll do_log fmt_struct);
}
@@ -4900,7 +4918,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION $have_patricia);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&lookup_ip_acl);
import Amavis::Util qw(ll do_log);
@@ -5311,7 +5329,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&lookup &lookup2 &lookup_hash &lookup_acl);
import Amavis::Util qw(ll do_log fmt_struct unique_list);
@@ -5596,7 +5614,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&expand &tokenize);
import Amavis::Util qw(ll do_log);
@@ -5888,7 +5906,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform :confvars c cr ca);
import Amavis::Timing qw(section_time);
@@ -5934,6 +5952,10 @@ sub DESTROY {
} else {
eval { do_log(5,"TempDir::DESTROY called") };
eval {
+ # must step out of the directory which is about to be deleted,
+ # otherwise rmdir can fail (e.g. on Solaris)
+ chdir($TEMPBASE)
+ or do_log(-1,"TempDir::DESTROY can't chdir to %s: %s", $TEMPBASE, $!);
$self->{fh_pers}->close
or do_log(-1,"Error closing temp file: %s",$!) if $self->{fh_pers};
undef $self->{fh_pers};
@@ -6115,6 +6137,9 @@ sub strip {
my $self = shift;
my($dname) = $self->{tempdir_path};
do_log(4, "TempDir::strip: %s", $dname);
+ # must step out of the directory which is about to be deleted,
+ # otherwise rmdir can fail (e.g. on Solaris)
+ chdir($TEMPBASE) or die "TempDir::strip: can't chdir to $TEMPBASE: $!";
my(@stat_list) = lstat($dname);
my($errn) = @stat_list ? 0 : 0+$!;
if ($errn == ENOENT) {
@@ -6413,7 +6438,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
}
use Errno qw(EIO);
@@ -6547,7 +6572,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform);
import Amavis::Util qw(ll do_log min max minmax);
@@ -6593,7 +6618,7 @@ sub new {
$n_candidates > 0 or die "Can't connect, no sockets specified!?"; # sanity
for (;;) {
if ($n_candidates > 1) { # pick one at random, put it to head of the list
- my($j) = int(rand($n_candidates-1));
+ my($j) = int(rand($n_candidates));
ll(5) && do_log(5, "picking candidate #%d (of %d) in %s",
$j+1, $n_candidates, join(', ',@$socket_specs));
@$socket_specs[0,$j] = @$socket_specs[$j,0] if $j != 0;
@@ -6738,12 +6763,20 @@ sub close {
} elsif (!defined(fileno($sock))) { # not really open
$sock->close; # ignoring errors
} else {
- $self->flush;
+ my($flush_status) = 1;
+ eval { # don't let errors during flush prevent us from closing a socket
+ $flush_status = $self->flush;
+ } or do {
+ undef $flush_status;
+ my $eval_stat = $@ ne '' ? $@ : "errno=$!"; chomp $eval_stat;
+ do_log(1, "Amavis::IO::RW: Error flushing on close: %s", $eval_stat);
+ };
$self->{last_event} = 'close';
$self->{last_event_time} = Time::HiRes::time;
- $status = $sock->close;
+ $! = 0; $status = $sock->close;
$status or do_log(1, "Amavis::IO::RW: Error closing socket: %s",
!$self->{ssl_active} ? $! : $sock->errstr.", $!" );
+ $status = $flush_status if $status && !$flush_status;
}
$status;
}
@@ -7011,7 +7044,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
}
@@ -7044,7 +7077,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform);
import Amavis::Util qw(setting_by_given_contents_category_all
@@ -7233,7 +7266,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform);
import Amavis::rfc2821_2822_Tools qw(rfc2822_timestamp quote_rfc2821_local
@@ -7535,7 +7568,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&hdr);
import Amavis::Conf qw(:platform c cr ca);
@@ -7686,7 +7719,7 @@ sub hdr($$$;$) {
$str = join("\n", at lines);
}
}
- $str .= "\n" if $str !~ /\n\z/; # append final NL
+ $str =~ s{\n*\z}{\n}s; # ensure a single final NL
do_log(5, "header: %s", $str);
$str;
}
@@ -7814,7 +7847,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
@EXPORT = qw(&mail_dispatch);
import Amavis::Conf qw(:platform :confvars c cr ca);
@@ -7968,7 +8001,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&parse_ip_address_from_received &first_received_from);
import Amavis::Conf qw(:platform c cr ca);
@@ -8041,7 +8074,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&consumed_bytes);
import Amavis::Conf qw(c cr ca
@@ -8126,7 +8159,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
import Amavis::Util qw(ll do_log);
}
@@ -8213,7 +8246,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter MIME::Parser::Filer); # subclass of MIME::Parser::Filer
}
# This package will be used by mime_decode().
@@ -8255,7 +8288,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&check_header_validity &check_for_banned_names);
import Amavis::Util qw(ll do_log min max minmax untaint sanitize_str);
@@ -8557,7 +8590,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&mime_decode);
import Amavis::Conf qw(:platform c cr ca $MAXFILES);
@@ -8687,7 +8720,7 @@ sub mime_traverse($$$$$) {
$part->name_declared(@rn==1 ? $rn[0] : \@rn) if @rn;
my $val = $head->mime_attr('content-type.report-type');
if (defined $val && $val ne '') {
- # $val = safe_encode('UTF-8',$val) if Encode::is_utf8($val);
+ # $val = safe_encode('UTF-8',$val);
$part->report_type($val);
}
}
@@ -8717,7 +8750,8 @@ sub mime_decode($$$) {
# "NEST" complains with "part did not end with expected boundary" when
# the outer message is message/partial and the inner message is chopped
$parser->extract_uuencode(1); # to enable or not to enable ???
- $parser->max_parts($MAXFILES) if defined($MAXFILES) && $MAXFILES > 0;
+ $parser->max_parts($MAXFILES) if defined $MAXFILES && $MAXFILES > 0 &&
+ $parser->UNIVERSAL::can('max_parts');
my($entity);
snmp_count('OpsDecByMimeParser');
if (ref($fileh)) { # assume open file handle
@@ -8760,7 +8794,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter MIME::Body); # subclass of MIME::Body
import Amavis::Util qw(ll do_log);
}
@@ -8825,7 +8859,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&delivery_status_notification &delivery_short_report
&build_mime_entity &defanged_mime_entity
@@ -9874,7 +9908,7 @@ sub msg_from_quarantine($$$) {
$recip_obj->recip_addr_modified($bcc);
$recip_obj->recip_destiny(D_PASS);
$recip_obj->dsn_notify(['NEVER']);
- $recip_obj->contents_category(CC_CLEAN);
+ $recip_obj->add_contents_category(CC_CLEAN,0);
$msginfo->per_recip_data([@{$msginfo->per_recip_data}, $recip_obj]);
do_log(2,"adding recipient - always_bcc: %s", $bcc);
}
@@ -9919,7 +9953,7 @@ use re 'taint';
BEGIN {
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
import Amavis::Conf qw(:platform :sa :confvars c cr ca);
import Amavis::Util qw(untaint min max minmax unique_list unique_ref
ll do_log update_current_log_level
@@ -11388,7 +11422,7 @@ sub post_process_request_hook {
do_log(3,$load_report) if defined $load_report;
}
# workaround: Net::Server 0.91 forgets to disconnect session
- if (Net::Server->VERSION eq '0.91') { close STDIN; close STDOUT }
+ if (Net::Server->VERSION == 0.91) { close STDIN; close STDOUT }
}
### Child is about to be terminated
@@ -12773,7 +12807,7 @@ sub check_mail($$) {
$recip_obj->recip_destiny(D_PASS);
$recip_obj->dsn_notify(['NEVER']);
$recip_obj->contents_category($msginfo->contents_category);
- # $recip_obj->contents_category(CC_CLEAN);
+ # $recip_obj->add_contents_category(CC_CLEAN,0);
$msginfo->per_recip_data([@{$msginfo->per_recip_data}, $recip_obj]);
do_log(2,"adding recipient - always_bcc: %s", $bcc);
}
@@ -13943,7 +13977,7 @@ sub add_forwarding_header_edits_per_recip($$$$$$$) {
if (defined $suppl_attr_val && $suppl_attr_val ne '') {
if (!$allowed_hdrs->{lc $hf_name}) {
do_log(5,'fwd: scanner provided %s, '.
- 'inhibited by %allowed_added_header_fields', $hf_name);
+ 'inhibited by %%allowed_added_header_fields', $hf_name);
} else {
do_log(5,'fwd: scanner provided %s, inserting', $hf_name);
$hdr_edits->add_header($hf_name,
@@ -14055,9 +14089,9 @@ sub add_forwarding_header_edits_per_recip($$$$$$$) {
scalar(@recip_cluster), $per_recip_data_len,
join(', ', map($_->recip_addr_smtp, @recip_cluster)));
}
- if (ll(3) && defined($cluster_full_spam_status) && @recip_cluster) {
+ if (ll(2) && defined($cluster_full_spam_status) && @recip_cluster) {
my($s) = $cluster_full_spam_status; $s =~ s/\n[ \t]/ /g;
- do_log(3, "Spam-tag, %s -> %s, %s", $msginfo->sender_smtp,
+ do_log(2, "spam-tag, %s -> %s, %s", $msginfo->sender_smtp,
join(',', map($_->recip_addr_smtp, @recip_cluster)), $s);
}
($hdr_edits, \@recip_cluster, $done_all);
@@ -14091,11 +14125,13 @@ sub prepare_modified_mail($$$$) {
$inp_fh->seek($msginfo->skip_bytes, 0)
or die "Can't rewind mail file: $!";
$out_fh = IO::File->new;
- $out_fh->open($repl_fn, O_CREAT|O_EXCL|O_RDWR, 0640)
+ $out_fh->open($repl_fn, O_CREAT|O_EXCL|O_WRONLY, 0640)
or die "Can't create file $repl_fn: $!";
binmode($out_fh,':bytes') or die "Can't cancel :utf8 mode: $!";
- if ($enable_anomy_sanitizer &&
- $mail_mangle !~ /^(?:altermime|disclaimer)\z/) {
+ if (lc $mail_mangle eq 'anomy' && !$enable_anomy_sanitizer) {
+ die 'Anomy requested, but $enable_anomy_sanitizer is false';
+ } elsif ($enable_anomy_sanitizer &&
+ $mail_mangle !~ /^(?:altermime|disclaimer)\z/) {
$actual_mail_mangle = 'anomy';
$enable_anomy_sanitizer or die "Anomy not available: $mail_mangle";
my(@scanner_conf); my($e); my($engine) = Anomy::Sanitizer->new;
@@ -14104,6 +14140,12 @@ sub prepare_modified_mail($$$$) {
if ($e = $engine->error) { die $e }
my($ret) = $engine->sanitize($msginfo->mail_text, $out_fh);
if ($e = $engine->error) { die $e }
+ # close flushes buffers, makes it possible to check file size below
+ $out_fh->close or die "Can't close file $repl_fn: $!";
+ # re-open as read-only
+ $out_fh = IO::File->new;
+ $out_fh->open($repl_fn,'<') or die "Can't open file $repl_fn: $!";
+ binmode($out_fh,':bytes') or die "Can't cancel :utf8 mode: $!";
} else { # use altermime for adding disclaimers or defanging
$actual_mail_mangle = 'altermime';
$altermime ne '' or die "altermime not available: $mail_mangle";
@@ -14176,7 +14218,8 @@ sub prepare_modified_mail($$$$) {
"--input=$repl_fn", @altermime_args);
my($r,$status) = collect_results($proc_fh,$pid,$altermime,16384,[0]);
undef $proc_fh; undef $pid;
- do_log(2,"program $altermime said: %s",$$r) if ref $r && $$r ne '';
+ do_log(2,"program %s said: %s",
+ $altermime, $$r) if ref $r && $$r ne '';
$status == 0 or die "Program $altermime failed: $status, $$r";
$out_fh = IO::File->new;
$out_fh->open($repl_fn,'<') or die "Can't open file $repl_fn: $!";
@@ -14436,7 +14479,8 @@ sub do_quarantine($$$$;@) {
: '?';
}
}
- $msginfo->quar_type($quar_type);
+ # remember only the first quarantine method
+ $msginfo->quar_type($quar_type) if !defined $msginfo->quar_type;
$msginfo->quarantined_to(!@qa ? undef : \@qa); # remember quar. location
do_log(5, "DO_QUARANTINE done");
}
@@ -14612,7 +14656,7 @@ sub prepare_header_edits_for_quarantine($) {
$hf_name);
} elsif (!$allowed_hdrs->{$hf_name_lc}) {
do_log(5,'quar: scanner provided %s, '.
- 'inhibited by %allowed_added_header_fields', $hf_name);
+ 'inhibited by %%allowed_added_header_fields', $hf_name);
} else {
do_log(5,'quar: scanner provided %s, inserting', $hf_name);
$hdr_edits->add_header($hf_name, $hf_body, 2);
@@ -14629,7 +14673,7 @@ sub prepare_header_edits_for_quarantine($) {
if (defined $suppl_attr_val && $suppl_attr_val ne '') {
if (!$allowed_hdrs->{lc $hf_name}) {
do_log(5,'quar: scanner provided %s, '.
- 'inhibited by %allowed_added_header_fields', $hf_name);
+ 'inhibited by %%allowed_added_header_fields', $hf_name);
} else {
do_log(5,'quar: scanner provided %s, inserting', $hf_name);
$hdr_edits->add_header($hf_name,
@@ -14811,7 +14855,7 @@ sub do_notify_and_quarantine($$) {
section_time('notif-quar');
if (@q_tuples || $archive_any) {
- if (!defined $msginfo->mail_id) {
+ if (!defined($msginfo->mail_id) && grep($_->[2] ne 'Arch', @q_tuples)) {
# delayed mail_id generation - now we really need it
$snmp_db->register_proc(2,0,'G',$msginfo->log_id) if defined $snmp_db;
# create a mail_id unique to a database and save preliminary info to SQL
@@ -15099,7 +15143,7 @@ sub get_body_digest($$) {
while (($len = $fh->read($_,16384)) > 0) {
$bctx->add($_);
$body_size += $len + tr/\n//; # count \n compensating for CRLF (RFC 1870)
- $h_8bit = 1 if !$h_8bit && tr/\000-\177//c;
+ $b_8bit = 1 if !$b_8bit && tr/\000-\177//c;
if ($feed_dkim) {
s{\n}{\015\012}gs;
eval {
@@ -15358,7 +15402,7 @@ sub fetch_modules_extra() {
Net::LDAP->VERSION(0.32) if $extra_code_ldap;
# needed a working last_insert_id in the past, no longer so but nevertheless:
DBI->VERSION(1.43) if $extra_code_sql_base;
- MIME::Entity->VERSION ne '5.419'
+ MIME::Entity->VERSION != 5.419
or die "MIME::Entity 5.419 breaks quoted-printable encoding, ".
"please upgrade to 5.420 or later (or use 5.418)";
# load optional modules SAVI and Mail::ClamAV if available and requested
@@ -16056,7 +16100,7 @@ eval {
kill('HUP',$amavisd_pid) or $! == ESRCH
or die "Can't SIGHUP amavisd[$amavisd_pid]: $!";
my($msg) = "Signalling a SIGHUP to a running daemon [$amavisd_pid]";
- do_log(2,"%s",$msg); print STDERR "$msg\n";
+ do_log(2,"%s",$msg); print STDOUT "$msg\n";
exit(0);
} elsif ($cmd =~ /^(?:restart|stop)\z/) { # stop or restart
defined $amavisd_pid or die "The amavisd daemon is not running\n";
@@ -16075,7 +16119,7 @@ eval {
}
if ($waited < 60 || $sigkill_sent) {
do_log(2,"Waiting for the process [%s] to terminate",$amavisd_pid);
- print STDERR
+ print STDOUT
"Waiting for the process [$amavisd_pid] to terminate\n";
} else { # use stronger hammer
do_log(2,"Sending SIGKILL to amavisd[%s]",$amavisd_pid);
@@ -16094,14 +16138,14 @@ eval {
my($msg) = !defined($killed_amavisd_pid) ? undef :
"Daemon [$killed_amavisd_pid] terminated by SIG$kill_sig_used";
if ($cmd eq 'stop') {
- if (defined $msg) { do_log(2,"%s",$msg); print STDERR "$msg\n" }
+ if (defined $msg) { do_log(2,"%s",$msg); print STDOUT "$msg\n" }
exit(0);
}
if (defined $killed_amavisd_pid) {
- print STDERR "$msg, waiting for dust to settle...\n";
+ print STDOUT "$msg, waiting for dust to settle...\n";
sleep 5; # wait for the TCP socket to be released
}
- print STDERR "becoming a new daemon...\n";
+ print STDOUT "becoming a new daemon...\n";
} else {
die "$myversion: Unknown command line parameter: $cmd\n\n" . usage();
}
@@ -16239,7 +16283,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform $myversion $myhostname
$snmp_contact $snmp_location $nanny_details_level);
@@ -16502,7 +16546,7 @@ use warnings FATAL => qw(utf8 void);
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
import Amavis::Conf qw($db_home $daemon_chroot_dir);
import Amavis::Util qw(untaint ll do_log);
@@ -16591,7 +16635,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
import Amavis::Util qw(ll do_log);
import Amavis::Conf qw($trim_trailing_space_in_lookup_result_fields);
@@ -16716,7 +16760,7 @@ use warnings FATAL => qw(utf8 void);
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform :confvars c cr ca);
import Amavis::Timing qw(section_time);
@@ -16935,7 +16979,7 @@ BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION
$have_sasl $ldap_sys_default $have_inet6);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
$have_sasl = eval { require Authen::SASL };
$have_inet6 = eval { require IO::Socket::INET6 };
@@ -17147,7 +17191,7 @@ use re 'taint';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
import Amavis::Util qw(ll do_log);
import Amavis::Conf qw($trim_trailing_space_in_lookup_result_fields);
@@ -17272,7 +17316,7 @@ BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION
$ldap_sys_default @ldap_attrs @mv_ldap_attrs);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform :confvars c cr ca);
import Amavis::Timing qw(section_time);
@@ -17483,7 +17527,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform :confvars c cr ca);
import Amavis::Util qw(ll do_log debug_oneshot untaint snmp_counters_init
@@ -17950,6 +17994,9 @@ sub check_ampdp_policy($$$$) {
do_log(4, "AM.PDP: tempdir and file being removed: %s, %s",
$tempdir,$fname);
unlink($fname) or die "Can't remove file $fname: $!" if $fname ne '';
+ # must step out of the directory which is about to be deleted,
+ # otherwise rmdir can fail (e.g. on Solaris)
+ chdir($TEMPBASE) or die "Can't chdir to $TEMPBASE: $!";
rmdir_recursively($tempdir);
}
}
@@ -18132,7 +18179,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform :confvars c cr ca);
import Amavis::Util qw(ll do_log untaint am_id new_am_id snmp_counters_init
@@ -19254,7 +19301,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform);
import Amavis::Util qw(ll do_log min max minmax);
@@ -19332,7 +19379,7 @@ sub datasend {
my $self = shift;
my($buff) = @_ == 1 ? $_[0] : join('', at _);
do_log(-1,"WARN: Unicode string passed to datasend")
- if Encode::is_utf8($buff);
+ if Encode::is_utf8($buff); # always false on tainted, Perl bug #32687
# do_log(5,"smtp print %d bytes>", length($buff));
$buff =~ tr/\r//d if $self->{strip_cr}; # sanitize bare CR if necessary
$buff =~ s{\n}{\015\012}gs; # CR/LF are never split across a buffer boundary
@@ -19454,7 +19501,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&rundown_stale_sessions);
import Amavis::Conf qw(:platform c cr ca $smtp_connection_cache_enable);
@@ -19488,16 +19535,21 @@ sub new {
$cache_key, $self->{transaction_count});
} else {
do_log(3, "smtp session: setting up a new session");
+ $cache_key = undef;
$self = bless {
- socket_specs => $socket_specs, socketname => undef, protocol => undef,
+ socket_specs => $socket_specs,
+ socketname => undef, protocol => undef, smtp_handle => undef,
deadline => $deadline, timeout => undef, in_xactn => 0,
- transaction_count => 0, state => 'down', smtp_handle => undef,
+ transaction_count => 0, state => 'down', established_at_time => undef,
wildcard_implied_host => $wildcard_implied_host,
wildcard_implied_port => $wildcard_implied_port,
}, $class;
- $sessions_cache{$cache_key} = $self;
}
$self->establish_or_refresh;
+ if (!defined $cache_key) { # newly established session
+ $cache_key = sprintf("%s:%s", $self->protocol, $self->socketname);
+ $sessions_cache{$cache_key} = $self;
+ }
$self;
}
@@ -19516,6 +19568,10 @@ sub session_state
sub in_smtp_transaction
{ my $self = shift; !@_ ? $self->{in_xactn} : ($self->{in_xactn}=shift) }
+sub established_at_time
+ { my $self = shift; !@_ ? $self->{established_at_time}
+ : ($self->{established_at_time}=shift) }
+
sub transaction_begins {
my $self = $_[0];
!$self->in_smtp_transaction
@@ -19586,7 +19642,7 @@ sub close {
if (defined $smtp_handle) {
$smtp_handle->close
or do_log(1, "Error closing Amavis::Out::SMTP::Protocol obj");
- $self->in_smtp_transaction(0);
+ $self->in_smtp_transaction(0); $self->established_at_time(undef);
$self->smtp_handle(undef); $self->session_state('down');
}
if (defined $self->socketname) {
@@ -19602,16 +19658,21 @@ sub rundown_stale_sessions($) {
for my $cache_key (keys %sessions_cache) {
my($smtp_session) = $sessions_cache{$cache_key};
my($smtp_handle) = $smtp_session->smtp_handle;
- my($last_event_time); my($now) = Time::HiRes::time;
+ my($established_at_time) = $smtp_session->established_at_time;
+ my($last_event_time);
$last_event_time = $smtp_handle->last_io_event_timestamp if $smtp_handle;
+ my($now) = Time::HiRes::time;
if ($close_all || !$smtp_connection_cache_enable ||
!defined($last_event_time) || $now - $last_event_time >= 30) {
- ll(3) && do_log(3,"smtp session rundown%s%s, %s, state %s",
+ ll(3) && do_log(3,"smtp session rundown%s%s%s, %s, state %s",
$close_all ? ' all sessions'
: $smtp_connection_cache_enable ? ' stale sessions'
: ', cache off',
!defined($last_event_time) ? ''
: sprintf(", idle %.1f s", $now - $last_event_time),
+ !defined($established_at_time) ? ''
+ : sprintf(", since %.1f s ago",
+ $now - $established_at_time),
$cache_key, $smtp_session->session_state);
if ($smtp_session->session_state ne 'down' &&
$smtp_session->session_state ne 'quitsent' &&
@@ -19703,6 +19764,7 @@ sub establish_or_refresh {
$self->socketname($smtp_handle->socketname);
$self->protocol($smtp_handle->protocol);
$self->session_state('connected');
+ $self->established_at_time(time);
$self->timeout($smtp_connect_timeout);
$smtp_resp = $self->smtp_response; # fetch greeting
do_log(3,"smtp greeting: %s", $smtp_resp);
@@ -19777,7 +19839,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
@EXPORT = qw(&mail_via_smtp);
import Amavis::Conf qw(:platform c cr ca $smtp_connection_cache_enable);
@@ -20248,6 +20310,7 @@ sub mail_via_smtp(@) {
}
}
if ($pipelining && !$smtp_connection_cache_enable) {
+ do_log(5,"smtp connection_cache disabled, sending QUIT");
$smtp_session->quit; #flush! QUIT
# can't be sure until we see a response, consider uncertain just in case
$smtp_session->transaction_ends_unconfirmed;
@@ -20352,15 +20415,21 @@ sub mail_via_smtp(@) {
do_log(-1, "mail_via_smtp: NOTICE: aborting SMTP session, %s", $err);
$smtp_session->close(0); # abruptly terminate SMTP session, ignore status
} else {
- $smtp_session->timeout(1); # don't wait for too long
- $smtp_session->quit; #flush! # send a QUIT regardless of success so far
- $smtp_session->transaction_ends_unconfirmed;
- for (my($cnt)=0; ; $cnt++) { # curious if there are any pending responses
- my($smtp_resp) = $smtp_session->smtp_response;
- last if !defined $smtp_resp;
- do_log(0,"discarding unprocessed reply: %s", $smtp_resp);
- if ($cnt > 20) { do_log(-1,"aborting, discarding many replies"); last }
- }
+ do_log(5,"smtp session done, sending QUIT");
+ eval {
+ $smtp_session->timeout(1); # don't wait for too long
+ $smtp_session->quit; #flush! # send a QUIT regardless of success so far
+ $smtp_session->transaction_ends_unconfirmed;
+ for (my($cnt)=0; ; $cnt++) { # curious if there are any pending responses
+ my($smtp_resp) = $smtp_session->smtp_response;
+ last if !defined $smtp_resp;
+ do_log(0,"discarding unprocessed reply: %s", $smtp_resp);
+ if ($cnt > 20) { do_log(-1,"aborting, discarding many replies"); last }
+ }
+ } or do {
+ my $eval_stat = $@ ne '' ? $@ : "errno=$!"; chomp $eval_stat;
+ do_log(-1, "mail_via_smtp: error during QUIT: %s", $eval_stat);
+ };
$smtp_session->close(0); # terminate SMTP session, ignore status
}
undef $smtp_handle; undef $smtp_session;
@@ -20439,7 +20508,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
@EXPORT = qw(&mail_via_pipe);
import Amavis::Conf qw(:platform c cr ca);
@@ -20489,14 +20558,20 @@ sub mail_via_pipe(@) {
$pipe_args =~ s/^argv=//i;
my(@pipe_args) = split(' ',$pipe_args); my(@command) = shift(@pipe_args);
my($dsn_capable) = c('propagate_dsn_if_possible'); # assume, unless disabled
+ $dsn_capable = 0 if $command[0] !~ /sendmail/; # a hack, don't use -N or -V
if ($dsn_capable) { # DSN is supported since Postfix 2.3
# notify options are per-recipient, yet a command option -N applies to all
my($common_list); my($not_all_the_same) = 0;
for my $r (@{$msginfo->per_recip_data}) {
my($dsn_notify) = $r->dsn_notify;
- my($d) = uc(join(',', $msginfo->sender eq '' ? ('NEVER')
- : !$dsn_notify ? ('DELAY','FAILURE') # sorted
- : sort @$dsn_notify)); # normalize order
+ my $d;
+ if ($msginfo->sender eq '') {
+ $d = 'NEVER';
+ } elsif (!$dsn_notify) {
+ $d = 'DELAY,FAILURE'; # sorted
+ } else {
+ $d = uc(join(',', sort @$dsn_notify)); # normalize order
+ }
if (!defined($common_list)) { $common_list = $d }
elsif ($d ne $common_list) { $not_all_the_same = 1 }
}
@@ -20625,7 +20700,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
@EXPORT = qw(&mail_via_bsmtp);
import Amavis::Conf qw(:platform $QUARANTINEDIR c cr ca);
@@ -20817,7 +20892,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&mail_to_local_mailbox);
import Amavis::Conf qw(:platform c cr ca
@@ -21158,7 +21233,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
import Amavis::Util qw(ll do_log);
}
@@ -21312,7 +21387,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform c cr ca);
import Amavis::Util qw(ll do_log);
@@ -21584,7 +21659,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform :confvars c cr ca);
import Amavis::rfc2821_2822_Tools;
@@ -22032,7 +22107,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
import Amavis::Util qw(ll do_log untaint min max minmax);
}
@@ -22328,7 +22403,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
@EXPORT = qw(&mail_via_sql);
import Amavis::Conf qw(:platform c cr ca $sql_quarantine_chunksize_max);
@@ -22461,7 +22536,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform :confvars c cr ca);
import Amavis::Util qw(ll untaint min max minmax unique_list do_log
@@ -23577,7 +23652,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform c cr ca);
import Amavis::Util qw(ll do_log untaint unique_list);
@@ -23885,7 +23960,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform :confvars :sa c cr ca);
import Amavis::Util qw(ll do_log sanitize_str min max minmax
@@ -24261,7 +24336,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform :confvars :sa c cr ca);
import Amavis::Util qw(ll do_log sanitize_str min max minmax);
@@ -24425,7 +24500,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
# let a 'require' understand that this module is already loaded:
$INC{'Mail/SpamAssassin/Logger/Amavislog.pm'} = 'amavisd';
@@ -24462,7 +24537,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
import Amavis::Conf qw(:platform :confvars :sa $daemon_user c cr ca);
import Amavis::Util qw(ll do_log sanitize_str prolong_timer add_entropy
@@ -24701,7 +24776,7 @@ sub new_SpamAssassin_instance {
do_log(2, "SpamAssassin loaded plugins: %s", join(', ', sort
map { my($n) = ref $_; $n =~ s/^Mail::SpamAssassin::Plugin:://; $n }
@plugins));
-# printf STDERR ("%s\n", join(", ", at plugins));
+# printf STDOUT ("%s\n", join(", ", at plugins));
# not in use: AccessDB AntiVirus TextCat; ASN BodyRuleBaseExtractor
# OneLineBodyRuleType Rule2XSBody Shortcircuit
}
@@ -25342,7 +25417,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&init &decompose_part &determine_file_types);
import Amavis::Util qw(untaint min max minmax ll do_log snmp_count
@@ -25640,8 +25715,10 @@ sub decompose_part($$) {
do_log($ll,"Decoding of %s (%s) failed, leaving it unpacked: %s",
$part->base_name, $part->type_long, $eval_stat);
$sts = 2; # keep the original, along with possible decoded files
- chdir($tempdir) or die "Can't chdir to $tempdir: $!"; # just in case
};
+ if ($any_called) {
+ chdir($tempdir) or die "Can't chdir to $tempdir: $!"; # just in case
+ }
if ($sts == 1 && lookup2(0,$part->type_long,\@keep_decoded_original_maps)) {
# don't trust this file type or unpacker,
# keep both the original and the unpacked file
@@ -26885,10 +26962,10 @@ sub do_executable($$@) {
chomp $@;
do_log(3, "do_executable: not a RAR sfx, ignoring: %s", $@) if $@ ne '';
- # LHA?
- return 2 if defined $lha && eval { do_lha($part,$tempdir,$lha,1) };
- chomp $@;
- do_log(3, "do_executable: not a LHA sfx, ignoring: %s", $@) if $@ ne '';
+# # LHA? not safe, tends to crash
+# return 2 if defined $lha && eval { do_lha($part,$tempdir,$lha,1) };
+# chomp $@;
+# do_log(3, "do_executable: not a LHA sfx, ignoring: %s", $@) if $@ ne '';
# ARJ?
return 2 if defined $unarj && eval { do_unarj($part,$tempdir,$unarj,1) };
@@ -26901,7 +26978,7 @@ sub do_executable($$@) {
# my($k,$v,$fn);
# while (($k,$v) = each(%::)) {
# local(*e)=$v; $fn=fileno(\*e);
-# printf STDERR ("%-10s %-10s %s\n",$k,$v,$fn) if defined $fn;
+# printf STDOUT ("%-10s %-10s %s\n",$k,$v,$fn) if defined $fn;
# }
# Given a file handle (typically opened pipe to a subprocess, as returned
@@ -27027,7 +27104,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&dkim_key_postprocess &generate_authentication_results
&dkim_make_signatures &adjust_score_by_signer_reputation
@@ -28108,7 +28185,7 @@ no warnings 'uninitialized';
BEGIN {
require Exporter;
use vars qw(@ISA @EXPORT @EXPORT_OK %EXPORT_TAGS $VERSION);
- $VERSION = '2.301';
+ $VERSION = '2.303';
@ISA = qw(Exporter);
@EXPORT_OK = qw(&show_or_test_dkim_public_keys &generate_dkim_private_key
&convert_dkim_keys_file);
diff --git a/amavisd-new-courier.patch b/amavisd-new-courier.patch
index 7ae23c2..3f02059 100644
--- a/amavisd-new-courier.patch
+++ b/amavisd-new-courier.patch
@@ -1,5 +1,5 @@
---- amavisd.ori 2011-07-01 18:18:08.952738010 +0200
-+++ amavisd 2011-07-01 18:18:17.410664981 +0200
+--- amavisd.ori 2012-04-29 02:31:15.655242548 +0200
++++ amavisd 2012-04-29 02:31:46.672242261 +0200
@@ -101,5 +101,5 @@
# Amavis::In::AMPDP
# Amavis::In::SMTP
@@ -14,14 +14,14 @@
+ IO::Handle IO::File IO::Select IO::Socket IO::Socket::UNIX IO::Socket::INET
IO::Stringy Digest::MD5 Unix::Syslog File::Basename
Compress::Zlib MIME::Base64 MIME::QuotedPrint MIME::Words
-@@ -10795,5 +10795,5 @@
+@@ -10829,5 +10829,5 @@
#
sub post_configure_hook {
-# umask(0007); # affect protection of Unix sockets created by Net::Server
+ umask(0007); # affect protection of Unix sockets created by Net::Server
}
-@@ -10822,4 +10822,34 @@
+@@ -10856,4 +10856,34 @@
### Net::Server hook
+### This hook takes place immediately after the "->run()" method is called.
@@ -56,7 +56,7 @@
+### Net::Server hook
### This hook occurs in the parent (master) process after chroot,
### after change of user, and change of group has occured.
-@@ -10874,4 +10904,15 @@
+@@ -10908,4 +10938,15 @@
}
$spamcontrol_obj->init_pre_fork if $spamcontrol_obj;
+ if ($courierfilter_shutdown) {
@@ -72,7 +72,7 @@
+ }
my(@modules_extra) = grep(!exists $modules_basic{$_}, keys %INC);
if (@modules_extra) {
-@@ -11313,5 +11354,7 @@
+@@ -11347,5 +11388,7 @@
$ampdp_in_obj->process_policy_request($sock, $conn, \&check_mail, 0);
} elsif ($suggested_protocol eq 'COURIER') {
- die "unavailable support for protocol: $suggested_protocol";
@@ -81,7 +81,7 @@
+ $courier_in_obj->process_courier_request($sock, $conn, \&check_mail);
} elsif ($suggested_protocol eq 'QMQPqq') {
die "unavailable support for protocol: $suggested_protocol";
-@@ -11392,4 +11435,24 @@
+@@ -11426,4 +11469,24 @@
}
+### Net::Server hook
@@ -106,7 +106,7 @@
+
### Child is about to be terminated
### user customizable Net::Server hook
-@@ -15625,4 +15688,9 @@
+@@ -15669,4 +15732,9 @@
undef $Amavis::Conf::log_verbose_templ;
+# courierfilter shutdown needs can_read_hook, added in Net::Server 0.90
@@ -116,14 +116,14 @@
+
if (defined $desired_user && $daemon_user ne '') {
local($1);
-@@ -16191,4 +16259,6 @@
+@@ -16235,4 +16303,6 @@
host => $bind_to[0], # default bind, redundant, merged to @listen_sockets
listen => $listen_queue_size, # undef for a default
+ # need to set multi_port for can_read_hook
+ multi_port => $courierfilter_shutdown ? 1 : undef,
max_servers => $max_servers, # number of pre-forked children
!defined($min_servers) ? ()
-@@ -19239,5 +19309,424 @@
+@@ -19286,5 +19356,424 @@
no warnings 'uninitialized';
-BEGIN { die "Code not available for module Amavis::In::Courier" }
diff --git a/amavisd-new-qmqpqq.patch b/amavisd-new-qmqpqq.patch
index 7305748..277af2d 100644
--- a/amavisd-new-qmqpqq.patch
+++ b/amavisd-new-qmqpqq.patch
@@ -1,36 +1,36 @@
---- amavisd.ori 2011-07-01 18:18:08.952738010 +0200
-+++ amavisd 2011-07-01 18:20:01.302769163 +0200
+--- amavisd.ori 2012-04-29 02:31:15.655242548 +0200
++++ amavisd 2012-04-29 02:33:01.036242047 +0200
@@ -102,4 +102,5 @@
# Amavis::In::SMTP
#( Amavis::In::Courier )
+# Amavis::In::QMQPqq
# Amavis::Out::SMTP::Protocol
# Amavis::Out::SMTP::Session
-@@ -4023,4 +4024,5 @@
+@@ -4041,4 +4042,5 @@
$myproduct_name,
$conn->socket_port eq '' ? 'unix socket' : "port ".$conn->socket_port);
+ # must not use proto name QMQPqq in 'with'
$s .= "\n with $smtp_proto" if $smtp_proto=~/^(ES|S|L)MTPS?A?\z/i; #RFC 3848
$s .= "\n id $id" if defined $id && $id ne '';
-@@ -9970,4 +9972,5 @@
+@@ -10004,4 +10006,5 @@
$extra_code_sql_lookup $extra_code_ldap
$extra_code_in_ampdp $extra_code_in_smtp $extra_code_in_courier
+ $extra_code_in_qmqpqq
$extra_code_out_smtp $extra_code_out_pipe
$extra_code_out_bsmtp $extra_code_out_local $extra_code_p0f
-@@ -9997,4 +10000,5 @@
+@@ -10031,4 +10034,5 @@
# Amavis::In::AMPDP, Amavis::In::SMTP and In::Courier objects
use vars qw($ampdp_in_obj $smtp_in_obj $courier_in_obj);
+use vars qw($qmqpqq_in_obj); # Amavis::In::QMQPqq object
use vars qw($sql_dataset_conn_lookups); # Amavis::Out::SQL::Connection object
-@@ -10701,4 +10705,5 @@
+@@ -10735,4 +10739,5 @@
do_log(0,"SMTP-in proto code %s loaded", $extra_code_in_smtp ?'':" NOT");
do_log(0,"Courier proto code %s loaded", $extra_code_in_courier ?'':" NOT");
+ do_log(0,"QMQPqq-in proto code %s loaded", $extra_code_in_qmqpqq ?'':" NOT");
do_log(0,"SMTP-out proto code %s loaded", $extra_code_out_smtp ?'':" NOT");
do_log(0,"Pipe-out proto code %s loaded", $extra_code_out_pipe ?'':" NOT");
-@@ -11315,5 +11320,9 @@
+@@ -11349,5 +11354,9 @@
die "unavailable support for protocol: $suggested_protocol";
} elsif ($suggested_protocol eq 'QMQPqq') {
- die "unavailable support for protocol: $suggested_protocol";
@@ -41,25 +41,25 @@
+ $qmqpqq_in_obj->process_qmqpqq_request($sock,$conn,\&check_mail);
} elsif ($suggested_protocol eq 'TCP-LOOKUP') { #postfix maps, experimental
process_tcp_lookup_request($sock, $conn);
-@@ -11411,4 +11420,5 @@
+@@ -11445,4 +11454,5 @@
do_log(5,"child_finish_hook: invoking DESTROY methods");
undef $smtp_in_obj; undef $ampdp_in_obj; undef $courier_in_obj;
+ undef $qmqpqq_in_obj;
undef $sql_storage; undef $sql_wblist; undef $sql_lookups;
undef $sql_dataset_conn_lookups; undef $sql_dataset_conn_storage;
-@@ -11421,4 +11431,5 @@
+@@ -11455,4 +11465,5 @@
# do_log(5,"at the END handler: invoking DESTROY methods");
undef $smtp_in_obj; undef $ampdp_in_obj; undef $courier_in_obj;
+ undef $qmqpqq_in_obj;
undef $sql_storage; undef $sql_wblist; undef $sql_lookups;
undef $sql_dataset_conn_lookups; undef $sql_dataset_conn_storage;
-@@ -15443,4 +15454,5 @@
+@@ -15487,4 +15498,5 @@
$extra_code_sql_lookup, $extra_code_ldap,
$extra_code_in_ampdp, $extra_code_in_smtp, $extra_code_in_courier,
+ $extra_code_in_qmqpqq,
$extra_code_out_smtp, $extra_code_out_pipe,
$extra_code_out_bsmtp, $extra_code_out_local, $extra_code_p0f,
-@@ -15763,5 +15775,11 @@
+@@ -15807,5 +15819,11 @@
undef $extra_code_in_courier;
}
- if ($needed_protocols_in{'QMQPqq'}) { die "In::QMQPqq code not available" }
@@ -72,7 +72,7 @@
+ }
}
-@@ -19245,4 +19263,276 @@
+@@ -19292,4 +19310,276 @@
__DATA__
#
+package Amavis::In::QMQPqq;
@@ -349,8 +349,8 @@
+#
package Amavis::Out::SMTP::Protocol;
use strict;
---- amavisd.conf.ori 2011-07-01 18:17:55.440742604 +0200
-+++ amavisd.conf 2011-07-01 18:20:01.302769163 +0200
+--- amavisd.conf.ori 2012-04-29 02:31:32.418239154 +0200
++++ amavisd.conf 2012-04-29 02:33:01.037242947 +0200
@@ -55,6 +55,6 @@
# option(s) -p overrides $inet_socket_port and $unix_socketname
diff --git a/amavisd.conf b/amavisd.conf
index 6184f3b..7edddde 100644
--- a/amavisd.conf
+++ b/amavisd.conf
@@ -321,7 +321,7 @@ $banned_filename_re = new_RE(
['gz', \&do_gunzip],
['bz2', \&do_uncompress, 'bzip2 -d'],
['xz', \&Amavis::Unpackers::do_uncompress,
- ['xzdec'. 'xz -dc', 'unxz -c', 'xzcat'] ],
+ ['xzdec', 'xz -dc', 'unxz -c', 'xzcat'] ],
['lzma', \&Amavis::Unpackers::do_uncompress,
['lzmadec', 'xz -dc --format=lzma',
'lzma -dc', 'unlzma -c', 'lzcat', 'lzmadec'] ],
@@ -337,7 +337,7 @@ $banned_filename_re = new_RE(
['arj', \&do_unarj, ['arj','unarj'] ],
['arc', \&do_arc, ['nomarch','arc'] ],
['zoo', \&do_zoo, ['zoo','unzoo'] ],
- ['lha', \&do_lha, 'lha'],
+# ['lha', \&do_lha, 'lha'], # unmaintained - security risk
# ['doc', \&do_ole, 'ripole'],
['cab', \&do_cabextract, 'cabextract'],
['tnef', \&do_tnef_ext, 'tnef'],
@@ -367,19 +367,19 @@ $banned_filename_re = new_RE(
# ['Avira SAVAPI',
# \&ask_daemon, ["*", 'savapi:/var/tmp/.savapi3', 'product-id'],
# qr/^(200|210)/m, qr/^(310|420|319)/m,
-# qr/^(?:310|420)[,\s]*(?:.* <<< )?(.+?)(?: ; |$)/m
+# qr/^(?:310|420)[,\s]*(?:.* <<< )?(.+?)(?: ; |$)/m ],
# settings for the SAVAPI3.conf: ArchiveScan=1, HeurLevel=2, MailboxScan=1
# ### http://www.clamav.net/
# ['ClamAV-clamd',
-# \&ask_daemon, ["CONTSCAN {}\n", "/var/run/clamav/clamd"],
+# \&ask_daemon, ["CONTSCAN {}\n", "/var/run/clamav/clamd.sock"],
# qr/\bOK$/m, qr/\bFOUND$/m,
# qr/^.*?: (?!Infected Archive)(.*) FOUND$/m ],
-# # NOTE: run clamd under the same user as amavisd, or run it under its own
+# # NOTE: run clamd under the same user as amavisd - or run it under its own
# # uid such as clamav, add user clamav to the amavis group, and then add
# # AllowSupplementaryGroups to clamd.conf;
# # NOTE: match socket name (LocalSocket) in clamav.conf to the socket name in
-# # this entry; when running chrooted one may prefer socket "$MYHOME/clamd".
+# # this entry; when running chrooted one may prefer a socket under $MYHOME.
# ### http://www.clamav.net/ and CPAN (memory-hungry! clamd is preferred)
# # note that Mail::ClamAV requires perl to be build with threading!
@@ -746,7 +746,7 @@ $banned_filename_re = new_RE(
# [0], qr/:.*\sFOUND$/m, qr/^.*?: (?!Infected Archive)(.*) FOUND$/m ],
# ['ClamAV-clamd-stream',
-# \&ask_daemon, ["*", 'clamd:/var/run/clamav/clamd'],
+# \&ask_daemon, ["*", 'clamd:/var/run/clamav/clamd.sock'],
# qr/\bOK$/m, qr/\bFOUND$/m,
# qr/^.*?: (?!Infected Archive)(.*) FOUND$/m ],
diff --git a/amavisd.conf-default b/amavisd.conf-default
index cce6ee3..b72988f 100644
--- a/amavisd.conf-default
+++ b/amavisd.conf-default
@@ -92,9 +92,9 @@ use strict;
# @inet_acl = qw( 127.0.0.1 [::1] );
# $listen_queue_size = undef;
-# $protocol = ... defaults to 'SMTP' or 'LMTP' (autodetect) on inet socket;
-# must be configured explicitly for Unix sockets.
-# Possible values: 'ESMTP', 'SMTP', 'LMTP', 'AM.PDP',
+# $protocol = ... defaults to 'SMTP' or 'LMTP' (autodetected) on inet and inet6
+# sockets; must be configured explicitly for Unix sockets.
+# Possible values: 'SMTP', 'LMTP', 'AM.PDP',
# and with appropriate patches applied also: 'COURIER' or 'QMQPqq'
# $soft_bounce = undef;
@@ -339,7 +339,7 @@ use strict;
# ['gz', \&do_uncompress, \$gunzip],
# ['bz2', \&do_uncompress, \$bunzip2],
# ['xz', \&do_uncompress,
-# ['xzdec'. 'xz -dc', 'unxz -c', 'xzcat'] ],
+# ['xzdec', 'xz -dc', 'unxz -c', 'xzcat'] ],
# ['lzma', \&do_uncompress,
# ['lzmadec', 'xz -dc --format=lzma',
# 'lzma -dc', 'unlzma -c', 'lzcat', 'lzmadec'] ],
@@ -359,7 +359,7 @@ use strict;
# ['arj', \&do_unarj, \$unarj],
# ['arc', \&do_arc, \$arc],
# ['zoo', \&do_zoo, \$zoo],
-# ['lha', \&do_lha, \$lha],
+### ['lha', \&do_lha, \$lha], # not safe
# ['doc', \&do_ole, \$ripole],
# ['cab', \&do_cabextract, \$cabextract],
# ['tnef', \&do_tnef_ext, \$tnef],
--
Debian packaging for amavisd-new
More information about the Amavisd-new-commits
mailing list