Linux4Dummies: Postfix Installation on Ubuntu 10.10/11.10

Introduction

Installing and mantaining a modern mail server with anti-spam, imap, filters, webmail and web-admin panels it's an hard task for a beginner. The advice is to test the installation and usage on a test machine (maybe a VM). After an initial testing phase you can try to put the server in "production" with a subdomain or a domain you can buy just for this test. The first issue is to get to know all the various components of a mail server and understand how they talk to each other, this phase is needed to be able to do some troubleshooting later on. This guide is based on the MTA (Mail Transport Agent) Postfix so it's important to understand it's various components; like other Unix MTA Postfix divides the job of handling the mail messages to various processes that works in concert to achieve a complete mail server.
https://help.ubuntu.com/10.04/serverguide/postfix.html

Software Versions

This guide will assume the following software versions:

Postfix (2.8.1) for receiving incoming emails from the internet and doing basic checks
Dovecot (1.2.12) to store emails on hard disk and allow users to access their emails using POP3 and IMAP
MySQL (5.1.49) as the database backend storing information about domains, user accounts and email forwardings
SpamAssassin (3.2.5) for spam checking
Awstats (7.0) for statistics
Roundcube (0.5.1) as webmail

and is tested on a server running Ubuntu 10.10

Hostname check

You should check that your server has a valid fqdn hostname (complete with the domain part):

hostname --fqd

If you get something like "localhost" or "mail" it's not ok and you have to do some work on /etc/hosts and /etc/hostname

Time zone

Check your time with the command:

date

and eventually adjust the time zone as needed (this is important for logging and mail):

dpkg-reconfigure tzdata

Software installation

To simplify the initial installation we decided in this guide to get most of the stuff already compiled from debian packages and not from source. Since Postfix 2.8 is recently the new stable version and Ubuntu 11.04 is just around the corner we just got the new package from Ubuntu 10.10 backports (we first addedd "deb http://gb.archive.ubuntu.com/ubuntu maverick-backports main" to /etc/apt/sources.list). If you're on later version of Ubuntu don't worry about this, Postfix 2.8 is already on their repositories.
After adding the new repositories and issuing an apt-get update command we simply start by installing most of the software we need trought apt-get, just follow the wizards and write the passwords you choose in a safe place:

apt-get update
 
# simple text editor
apt-get install nano
 
# the mta, choose "Internet Site" as Postfix configuration
apt-get install postfix postfix-mysql
 
# the backend for user database
apt-get install mysql-server
 
# the pop3 and imap server
apt-get install dovecot-pop3d dovecot-imapd
 
# the bayes anti spam filter
apt-get install spamassassin pyzor razor
 
# spf filtering
apt-get install tumgreyspf
 
# the server web for administration, webmail and statistics
apt-get install apache2
 
# php for the webserver and phpmyadmin
apt-get install php5 php5-mysql php5-mcrypt php5-intl phpmyadmin
 
# basic utilities
apt-get install wget tcpflow dnsutils nmap unzip whois

Preliminary Apache and PHP configuration

At this point you'll have the basic software installed and you should configure dns for a first domain to be able to easily address the server. A good basic apache configuration should be as follow:

a2enmod rewrite
a2enmod ssl
# remove all default files from /var/www/
rm -R /var/www/*
mkdir /var/www/webmail
mkdir /var/www/mysql

now remove all default virtual host and create a new one that directs all the connection to the ssl one

rm /etc/apache2/sites-available/*
rm /etc/apache2/sites-enabled/*
nano /etc/apache2/sites-available/default

a good example is the following configuration (you'll need to customize hosts and ssl certificates):

<VirtualHost *:80>
    ServerAdmin info@example.com
    DocumentRoot /var/www/webmail/
    ErrorLog /var/log/apache2/mail.example.com-error.log
    CustomLog /var/log/apache2/mail.example.com-access.log combined
    RewriteEngine On
    RewriteCond %{HTTPS} off
    RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
</VirtualHost>
 
<IfModule mod_ssl.c>
<VirtualHost *:443>
    DocumentRoot /var/www/webmail/
    Alias /mysql /var/www/mysql/
    <Directory />
       Options FollowSymLinks
       AllowOverride None
    </Directory>
    <Directory /var/www/>
       Options FollowSymLinks MultiViews
       AllowOverride All
       Order allow,deny
       allow from all
    </Directory>
 
    ErrorLog /var/log/apache2/mail.example.com-error.log
    CustomLog /var/log/apache2/mail.example.com-access.log combined
 
    SSLEngine on
 
    SSLCertificateFile       /etc/apache2/ssl/example-startssl/ssl.pem
    SSLCertificateKeyFile    /etc/apache2/ssl/example-startssl/ssl.key
    SSLCertificateChainFile  /etc/apache2/ssl/startssl/sub.class1.server.ca.crt
    SSLCACertificateFile     /etc/apache2/ssl/startssl/ca.crt
    <FilesMatch "\.(cgi|shtml|phtml|php)$">
       SSLOptions +StdEnvVars
    </FilesMatch>
    <Directory /usr/lib/cgi-bin>
       SSLOptions +StdEnvVars
    </Directory>
 
    BrowserMatch ".*MSIE.*" \
                nokeepalive ssl-unclean-shutdown \
                downgrade-1.0 force-response-1.0
</VirtualHost>
</IfModule>

At this point you should also give a basic security configuration of Apache editing the appropriate conf file:

nano /etc/apache2/conf.d/security

and setting some of the options, for instance:

<Directory />
        AllowOverride None
        Order Deny,Allow
        Deny from all
</Directory>
 
ServerTokens Prod
 
ServerSignature Off

Finaly restart Apache:

/etc/init.d/apache2 restart

and check that you can connect to the web address.
When you are satisfied with your Apache configuration remember to have a look at the PHP configuration file:

nano /etc/php5/apache2/php.ini

change the parameters you feel needed, give a special attention to:

display_errors = off
upload_max_filesize = 20M
date.timezone = Europe/Rome

The restart apache again:

a2ensite default
/etc/init.d/apache2 restart

MySQL Database

Postfix does not come with a pre-built MySQL database, this give the admin the freedom to build the database he wants (or even use one that already exists in his organization) to define the users and domains the mail server should manage.

So, how postfix knows how to access and use this db? In the configuration file postfix expects the admin to fill in some SQL querys that give the MTA different users view (domain, aliases, etc.); so the DB can be large and have lots of stuff regarding the user (for example for accounting purpose, invoices, etc.) and you can make postfix see just a subset of this.

You will find a .sql file attached to this guide with a predefined database structure that works well with an already existing web-management interface (GRSoft Mail Manager).

If you want to know a little more on how Postfix access the DB, read along this citation from http://workaround.org/ispmail/lenny/virtual-domains-in-db:
Instead of defining your email domains in a text file and creating many system accounts there is a better choice for non-trivial Postfix-based mail servers. Domain names and user accounts can be stored in a directory like LDAP or a database like MySQL or PostgreSQL. Postfix will just need to know how to access the database to get this control information. In such a configuration email domains that Postfix will be responsible for are called virtual domains. Similarly the user accounts are called virtual users. They just live in a database. There are two basic types of virtual domains that Postfix knows. "Virtual alias domains" can be used for forwarding ("aliasing") email from an email address to another email address (or multiple addresses). Virtual alias domains do not receive email for any users. They only forward mail somewhere else. The virtual_alias_maps mapping contains forwardings (source, destination) of users or domains to other email addresses or whole domains. Incidentally virtual_alias_maps also works for local email addresses, too. So you do not really need virtual alias domains as you can declare all domains as virtual mailbox domains and use virtual alias maps for aliases. The other type of virtual domains are "virtual mailbox domains" which are more important here. They define domains which are used to actually receive emails.
Ok, everything fine? well, don't worry, it will get better in the next chapter, open up phpmyadmin and download the .sql attached to this guide:

import the .sql file in phpmyadmin in a new db "mailserver_db"
give the privilege for the "mailserver\_%" to a new "mailserver" user with a good strong password and localhost access only

Postfix to Database Mappings

This part of the guide is heavily dependent on the SQL structure you are using, so be carefull with any modification you may have done. Now that the database is ready to be filled with information on user accounts we need to tell Postfix how to get to the database-stored information, we'll do this in a step-by-step fashion checking every time that everything works as it should.
More extensive and detailed information (with database examples) could be read on-line at http://workaround.org/ispmail/lenny/postfix-database-mappings, a good time to have a look at the page is just after completing this section. Be carefull that the DB we are using is diferent from the one at workaround.org since we are using a different web based administration panel.

virtual_mailbox_domains

Let's start by telling postfix which virtual domains you have.

nano /etc/postfix/mysql-virtual-mailbox-domains.cf

and fill it with something like this:

user = mailserver
password = secretpassword
hosts = 127.0.0.1
dbname = mailserver_db
query = SELECT 1 FROM virtual_domains WHERE name='%s'

let postfix know about this file:

postconf -e virtual_mailbox_domains=mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf

and now let's configure (by hand) our first domain (you should have already decided and configured the DNS for your testing host):

mysql -p mailserver_db -u mailserver
mysql> INSERT INTO virtual_domains (id, name) VALUES (1, 'example.com');
exit

we now check that postfix execute the query as it should:

postmap -q example.com mysql:/etc/postfix/mysql-virtual-mailbox-domains.cf

Remember to change the hostname from "example.com" to the one you're using, if you get '1' as a result then your first mapping is working; most of the problem you could get at this stage concern the authentication with MySQL.

virtual_mailbox_maps

Let's now tell postfix about your users.

nano /etc/postfix/mysql-virtual-mailbox-maps.cf

and fill it with something like this:

user = mailserver
password = secretpassword
hosts = 127.0.0.1
dbname = mailserver_db
query = SELECT 1 FROM virtual_users WHERE email='%s'

let postfix know about this file:

postconf -e virtual_mailbox_maps=mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf

and now let's configure (by hand) our first user:

mysql -p mailserver_db -u mailserver
mysql> INSERT INTO virtual_users (id, domain_id, email, password) VALUES (1, 1, 'info@example.com', MD5('secretpsw'));
exit

we now check that postfix execute the query as it should:

postmap -q info@example.com mysql:/etc/postfix/mysql-virtual-mailbox-maps.cf

Remember to change the hostname from "example.com" to the one you're using, if you get '1' as a result then go straight to the next mapping.

virtual_alias_maps

The virtual_alias_maps mapping is used for forwarding emails from one email address to another. This can get tricky: for each user you'll have an implicit alias saying that the mail to "user@domain" should be sent to "user@domain", you can add additional ones for forwarding mail also to other addresses. You can even have a catch-all account "@domain" to "user@domain" but this is highly advised only if you'd love to receive a whole lot of spam!

nano /etc/postfix/mysql-virtual-alias-maps.cf

and fill it with something like this:

user = mailserver
password = secretpassword
hosts = 127.0.0.1
dbname = mailserver_db
query = SELECT destination FROM virtual_aliases WHERE source='%s'

and now let's configure (by hand) our first alias:

mysql -p mailserver_db -u mailserver
mysql> INSERT INTO virtual_aliases (id, domain_id, source, destination) VALUES (1, 1, 'postmaster@example.com', 'john@example.com');
exit

before telling postfix about our "mysql-virtual-alias-maps.cf" we need to take care of the catch-all account (you may sometime need them after all), we want postfix to first check the more specific e-mail address and just then send everything else to an eventual catch-all account:

nano /etc/postfix/mysql-email2email.cf

user = mailserver
password = secretpassword
hosts = 127.0.0.1
dbname = mailserver_db
query = SELECT email FROM virtual_users WHERE email='%s'

let postfix know about these two mapping files:

postconf -e virtual_alias_maps=mysql:/etc/postfix/mysql-virtual-alias-maps.cf,mysql:/etc/postfix/mysql-email2email.cf

and we now check that postfix execute the query as it should:

postmap -q postmaster@example.com mysql:/etc/postfix/mysql-virtual-alias-maps.cf

Remember to change the hostname from "example.com" to the one you're using, you should get the forwarding address "info@example.com"

postmap -q info@example.com mysql:/etc/postfix/mysql-email2email.cf

you should get the e-mail address "info@example.com", and you should be good to go with the mappings! Make sure the permission of these configuration file are sound:

chgrp postfix /etc/postfix/mysql-*.cf
chmod u=rw,g=r,o= /etc/postfix/mysql-*.cf

Take your time to wrap up your head around this section before diving on the next part, if you're tired and have everything working up to now this should be a good time to have a break and grab some snack.

Sending mail to Dovecot

When Postfix receive an e-mail it will send it to another software: "Dovecot", this one will then manage the IMAP and POP3 services for the users.
Let's tell postfix to do this, adding the following line at the end of the file /etc/postfix/master.cf

dovecot   unix  -       n       n       -       -       pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -d ${recipient}

and specifing a few more options in the config file:

postconf -e virtual_transport=dovecot
postconf -e dovecot_destination_recipient_limit=1

let's now restart postfix:

postfix reload

now brace yourself for the configuration of Dovecot!

Dovecot configuration

Let us now configure Dovecot which will do several things for us: # get emails from Postfix and save them to disk # watch quotas (how much space a user may use on your disk) # execute user-based "sieve" filter rules (can be used to put away emails to different folders) # allow the user to fetch emails using POP3 or IMAP Let's create a user and a group just for storing e-mails (choose a free uid/gid):

groupadd -g 5000 vmail
useradd -g vmail -u 5000 vmail -d /var/vmail -m

We now start to custom the main dovecot configuration file:

nano /etc/dovecot/dovecot.conf

browse trough the beginning file and edit the option as you feel things should be, be carefull to at least match the configuration below:

# define which protocols to enable
protocols = imap imaps pop3 pop3s managesieve
 
# permit authentication even with password in plain sight
disable_plaintext_auth = no
 
# fake greeting for client
login_greeting = Microsoft Exchange 2011
 
# tell dovecot where to store the mail
mail_location = maildir:/var/vmail/%d/%n/Maildir

You also need to go inside the section auth default and change:

mechanisms = plain login

you should also comment the "pam" method for authentication and enable the SQL one:

passdb sql {
    args = /etc/dovecot/dovecot-sql.conf
}

in the same way you should comment the "passwd" userdb and enable the static one:

userdb static {
    args = uid=5000 gid=5000 home=/var/vmail/%d/%n allow_all_users=yes
}

then uncomment the "socket listen" part and configure it like this:

socket listen {
  master {
      path = /var/run/dovecot/auth-master
      mode = 0600
      user = vmail
  }
 
  client {
      path = /var/spool/postfix/private/auth
      mode = 0660
      user = postfix
      group = postfix
  }
}

we then enable the local delivery agent (LDA) to allow for things like sieve and quota:

protocol lda {
    postmaster_address = postmaster@example.com
    mail_plugins = sieve
    auth_socket_path = /var/run/dovecot/auth-master
    log_path = /var/vmail/dovecot-deliver.log
}

as a final step we enable the logging (this could be very usefull with troubleshooting):

log_path = /var/vmail/dovecot-deliver.log

let's close the configuration file and since the dovecot log may become pretty-big pretty-fast we should rotate it:

nano /etc/logrotate.d/dovecot-deliver

/var/vmail/dovecot-deliver.log {
        weekly
        rotate 14
        compress
}

You now need to tell Dovecot how to access MySQL (remember the path we entered a short time ago inside the configuration file?):

nano /etc/dovecot/dovecot-sql.conf

and fill it in with something like this (change passwords as needed):

driver = mysql
connect = host=127.0.0.1 dbname=mailserver_db user=mailserver password=secretpassword
default_pass_scheme = PLAIN-MD5
password_query = SELECT email as user, password FROM virtual_users WHERE email='%u';

Let's, at last, give everything good permission and restart dovecot:

chgrp vmail /etc/dovecot/dovecot.conf
chmod g+r /etc/dovecot/dovecot.conf
/etc/init.d/dovecot restart
chown vmail.vmail /var/vmail/dovecot-deliver.log

in your /var/vmail/dovecot-deliver.log you should have something like this:

dovecot: Info: Dovecot v1.2.12 starting up (core dumps disabled)
auth-worker(default): Info: mysql: Connected to 127.0.0.1 (mailserver_db)

SMTP test with telnet

Your mail server should now be able to handling it's first e-mail, let's test it from telnet:

telnet localhost smtp

and have a conversation like this

Trying 127.0.0.1...
Connected to localhost.localdomain.
Escape character is '^]'.
220 localhost.localdomain ESMTP Postfix (Ubuntu)
ehlo example.com
250-localhost.localdomain
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
mail from:<steve@example.com>
250 2.1.0 Ok
rcpt to:<info@example.com>
250 2.1.5 Ok
data
354 End data with <CR><LF>.<CR><LF>
test
.
250 2.0.0 Ok: queued as 3A4EF8C116
quit
221 2.0.0 Bye
Connection closed by foreign host.

If everything went right you should find in /var/log/syslog or /var/log/mail.log something like:

postfix/qmgr[29550]: 3A4EF8C116: from=<steve@example.com>, size=341, nrcpt=1 (queue active)
postfix/pipe[29725]: 3A4EF8C116: to=<info@example.com>, relay=dovecot, delay=387, delays=387/0.01/0/0.02, dsn=2.0.0, status=sent (delivered via dovecot service)
postfix/qmgr[29550]: 3A4EF8C116: removed

and in /var/vmail/dovecot-deliver.log something like:

deliver(info@example.com): Info: msgid=<20110408161104.3A4EF8C116@localhost.localdomain>: saved mail to INBOX

You should then check that the mail is correctly saved to disk and readable:

cd /var/vmail/example.com/info/Maildir
find .
less ./new/1302279438.M125478P.......

If everything works you're on the right track to a basic mail server.

POP3 test with telnet

Let's test the POP3 service:

telnet localhost pop3

and have this sort of conversation:

Trying 127.0.0.1...
Connected to localhost.localdomain.
Escape character is '^]'.
+OK Microsoft Exchange Ready.
 
user info@example.com
+OK
 
pass secretpassword
+OK Logged in.
 
list
+OK 1 messages:
1 417
.
 
retr 1
+OK 417 octets
Return-Path: <steve@example.com>
Delivered-To: info@example.com
Received: from example.com (localhost.localdomain [127.0.0.1])
 by localhost.localdomain (Postfix) with ESMTP id 3A4EF8C116
 for <info@example.com>; Fri,  8 Apr 2011 17:10:50 +0100 (BST)
Message-Id: <20110408161104.3A4EF8C116@localhost.localdomain>
Date: Fri,  8 Apr 2011 17:10:50 +0100 (BST)
From: steve@example.com
 
test
.
 
quit
+OK Logging out.
Connection closed by foreign host.

IMAP test with telnet

Let's test the IMAP service:

telnet localhost imap2

and have this sort of (complicated) conversation:

Trying 127.0.0.1...
Connected to localhost.localdomain.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE STARTTLS AUTH=PLAIN AUTH=LOGIN] Microsoft Exchange Ready.
 
1 login info@example.com secretpassword
1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT IDLE CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS] Logged in
 
2 list "" "*"
* LIST (\HasNoChildren) "." "INBOX"
2 OK List completed.
 
3 select "INBOX"
* FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
* OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted.
* 1 EXISTS
* 0 RECENT
* OK [UIDVALIDITY 1302279438] UIDs valid
* OK [UIDNEXT 2] Predicted next UID
* OK [HIGHESTMODSEQ 1] Highest
3 OK [READ-WRITE] Select completed.
 
4 fetch 1 all
* 1 FETCH (FLAGS (\Seen) INTERNALDATE "08-Apr-2011 18:17:18 +0200" RFC822.SIZE 417 ENVELOPE ("Fri, 8 Apr 2011 17:10:50 +0100 (BST)" NIL ((NIL NIL "steve" "example.com")) ((NIL NIL "steve" "example.com")) ((NIL NIL "steve" "example.com")) NIL NIL NIL NIL "<20110408161104.3A4EF8C116@localhost.localdomain>"))
4 OK Fetch completed.
 
5 fetch 1 body[]
* 1 FETCH (BODY[] {417}
Return-Path: <steve@example.com>
Delivered-To: info@example.com
Received: from example.com (localhost.localdomain [127.0.0.1])
 by localhost.localdomain (Postfix) with ESMTP id 3A4EF8C116
 for <info@example.com>; Fri,  8 Apr 2011 17:10:50 +0100 (BST)
Message-Id: <20110408161104.3A4EF8C116@localhost.localdomain>
Date: Fri,  8 Apr 2011 17:10:50 +0100 (BST)
From: steve@example.com
 
test
)
5 OK Fetch completed.
 
6 logout
* BYE Logging out
6 OK Logout completed.
Connection closed by foreign host.

Testing Dovecot's SSL

You should also test the SSL service, more comfortably from your desktop e-mail client. Dovecot during the installation already generated a self-signed certificate but you should put the one from your CA on these locations: * certificate in /etc/ssl/certs/dovecot.pem* private key in /etc/ssl/private/dovecot.pemand be carefull with the private key permissions:

chmod o= /etc/ssl/private/dovecot.pem

To make dovecot use the new certificate remember to issue a restart command:

/etc/init.d/dovecot restart

Authenticated SMTP with SSL

Now that you have dovecot configured for receiving e-mail it's time to configure the SMTP for sending them.
If the mail server should act as a smtp server for an internal network without any authentication you can enable that trough this command:

postconf -e mynetworks=192.168.50.0/24

otherwise just make it always require authentication except for the local host (this is the preferred choice):

postconf -e mynetworks=127.0.0.0/8

We now need to tell postfix to get the username and password directly from the sql database, instead of querying directly the DB we will do this trough the help of dovecot sasl authentication:

postconf -e smtpd_sasl_type=dovecot
postconf -e smtpd_sasl_path=private/auth
postconf -e smtpd_sasl_auth_enable=yes

We should now enable the SSL also for the SMTP service, the best choice is to use the same certificate and key we already got for dovecot, we just need to tell postfix where those are located:

postconf -e smtpd_tls_cert_file=/etc/ssl/certs/dovecot.pem
postconf -e smtpd_tls_key_file=/etc/ssl/private/dovecot.pem
postconf -e smtpd_use_tls=yes
postconf -e smtpd_tls_auth_only=no

You'll also have to uncomment and edit the smtps line on /etc/postfix/master.cf:

nano /etc/postfix/master.cf

this line should look like this:

smtps     inet  n       -       n       -       -       smtpd -o smtpd_tls_wrappermode=yes -o smtpd_sasl_auth_enable=yes -o smtpd_client_restrictions=permit_sasl_authenticated,reject

now some editing on the main.cf:

nano /etc/postfix/main.cf

and be sure to reload postfix:

postfix stop
postfix start

At this point you should test every mode of authentication (especially the encrypted ones) from your mail client and if everything looks good you have a working mail server.
This is a good time to call it a day or make a snapshot (if you're lucky enough to be on a VM) before starting to address all the anti-spam issues.

Filtering Spam with SpamAssassin

A good way to filter both spam and viruses is to use AMaViS which integrates very easily with SpamAssassin and ClamAV. AMaVis is VERY heavy on RAM usage so we will skip it and just use SpamAssassin directly with postfix.
We will configure SpamAssassin with various distributed filtering plugins: Pyzor, Razor and Dcc (look for more info on google). Since Dcc is not included in the ubuntu repositories we have to compile it manually. You can find more information on this on http://ailoo.net/2009/11/integrate-spamassassin-into-postfix-dovecot/

groupadd dcc
useradd -g dcc -s /bin/false -d /var/dcc dcc
apt-get install build-essential
mkdir ~/build
cd ~/build
wget http://www.dcc-servers.net/dcc/source/dcc-dccproc.tar.Z
tar xzvf dcc-dccproc.tar.Z
cd dcc-dccproc-1.3.XXX

Now we will configure and compile the dcc client, just ignore the complains about sendmail!

./configure --with-uid=dcc
make
make install
chown -R dcc.dcc /var/dcc
ln -s /var/dcc/libexec/dccifd /usr/local/bin/dccifd

Now we'll have to configure SpamAssassin to use this plugins:

groupadd spamd
useradd -g spamd -s /bin/false -d /var/lib/spamassassin spamd

and change some settings:

nano /etc/default/spamassassin

[...]
 
# Spamassassin home
SAHOME="/var/lib/spamassassin"
 
# Change to one to enable spamd
ENABLED=1
 
# Options
# See man spamd for possible options. The -d option is automatically added.
 
# SpamAssassin uses a preforking model, so be careful! You need to
# make sure --max-children is not set to anything higher than 5,
# unless you know what you're doing.
 
OPTIONS="--create-prefs -x --max-children 3 --username spamd --helper-home-dir ${SAHOME} -s ${SAHOME}/spamd.log --virtual-config-dir=${SAHOME}/users/%d/%l"
 
[...]

We then create the home directory:

mkdir -p /var/lib/spamassassin/users
chown spamd.spamd /var/lib/spamassassin -R

Now, let’s do some configuring. You can find all relevant config files in /etc/spamassassin. First of all, local.cf:

nano /etc/spamassassin/local.cf

[...]
 
#   Save spam messages as a message/rfc822 MIME attachment instead of
#   modifying the original message (0: off, 2: use text/plain instead)
#
report_safe 0
 
[...]
 
use_dcc 1
dcc_path /usr/local/bin/dccproc
 
use_pyzor 1
pyzor_path /usr/bin/pyzor
 
use_razor2 1
razor_config /etc/razor/razor-agent.conf

Afterwards, edit v310.pre

nano /etc/spamassassin/v310.pre

you'll have to check that the DCC, Razor and Pyzor plugins are enabled (DCC is disabled by default).

You can check your Spamassassin configuration with lint (use the -D flag for output):

spamassassin --lint

And you can update SA’s rules with sa-update:

sa-update --no-gpg

Now we are ready to start the SA daemon (will be automatically started at boot time):

/etc/init.d/spamassassin start

Now let's configure Postfix to talk with SpamAssassin.
First, edit /etc/postfix/master.cf, duplicate the existing dovecot transport and edit the new one to look as follows:

dovecot-spamass   unix  -       n       n       -       -       pipe flags=DRhu user=vmail:vmail argv=/usr/bin/spamc -u ${recipient} -e /usr/lib/dovecot/deliver -d ${recipient}

This transport will pass the mail to spamc which will pass it to deliver after checking.
Now just change the transport in /etc/postfix/main.cf.
We are using a second transport to be flexible. In case anything should go wrong with Spamassassin we will just need to change the transport in main.cf back to just "dovecot" and we'll get the mail withouth SpamAssassin the loop.
So:

nano /etc/postfix/main.cf

and change:

virtual_transport = dovecot-spamass
dovecot-spamass_destination_recipient_limit = 1

The second line is essential if you want to be able to receive e-mails for multiple recipients!
That’s all you need on the Postfix side. Just restart it to use the new transport.

/etc/init.d/postfix restart

Basic checks to see if SpamAssassin filtering is working

At this point you should have already forwarded a domain DNS to the mail server (MX and all of that) so you should just check from your thunderbird client that all e-mail received have the X-Spam headers.
To verify the spam filtering just wait for some spam or user the GTUBE string inside the body of an e-mail:

XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X

Inserting this string should make SpamAssasin filter the mail as Spam.

Install the web administration panel

We'll be configuring the GrSoft Mail Manager. The sql file you imported earlier already created the basic tables for the 2.0 lenny version, different versions will use different DB and you'll need to change also the queries for postfix! The installation script will be creating more table for the mailmanager users and for the translations.

cd /var/www/
mkdir mail-manager
cd mail-manager
wget http://www.grs-service.ch/pub/grs_mailmgr_v2_0-Lenny.tgz
tar xvzf grs_mailmgr_v2_0-Lenny.tgz
chown -Rf www-data:www-data ../mail-manager

You should then add an appropriate Alias rule on the Apache virtualhost.
Now just connect to the web server, for example: https://mail.example.com/mail-manager/install.php and follow the wizard specifying the same db and username as used previously with Postfix. If you interrupt the configuration the script will remember at what step you left it trough a cookie in your browser, keep that in mind if you experience any problem.
You should enter a valid e-mail address of an user of the system as the main MailManager (this user will be able to do anything to the mail server) and be sure to follow all the steps and when you finish succesfully: * Remove install.php * Remove conf/cnf_main_template.php * Remove folder install

rm -R install* conf/cnf_main_template.php

To get a nicer graphic interface just substitute the original files ("mgr_main.php" and "login.php") with the one provided with this guide (this is valid just for version 2.0 lenny!) and the connect to the manager at the address https://mail.example.com/mail-manager/login.php
The manager default languages are English and Deutsch, an Italian translation is a work in progress, it can all be done from the Mysql DB so it's very easy to add new languages.

More Spam control: adding SPF filtering

This part of the guide is already very well written on "http://workaround.org/ispmail/lenny/spf" so this is almost a copy-paste.

If you are tired of faked emails from popular domains like eBay or Paypal then there is hope. And you can even protect your own domain from spammers sending in your name. It is about SPF which is short for sender policy framework. Basically the owner of a domain defines which IP addresses are allowed to send email. Let's take an example.

dig +short workaround.org txt

and you should see something like

"v=spf1 ip4:85.214.93.191 ip4:85.214.149.150 -all"

This entry means that if someone sends you an email with a sender email address of …@workaround.org then you should only accept it if it was sent from one of these two IP addresses. The "-all" at the end tells you that no other IP addresses should be accepted (FAIL). Another definition like "~all" means a SOFTFAIL - you should at least be suspicious and perhaps increase the spam score. (Actually "~all" is widely used but pretty useless. Either you know what you are doing and use "-all" or leave it off entirely. If you want to test if SPF workd then you could use "?all".) So if someone sent you an email from …@workaround.org from another IP address then you should drop it - it's spam.

Many organisations already have SPF records that you can use to reduce the amount of spam you receive. We could set an SPF entry also for our domains but that means we then have to force our user to send e-mail only from our smtp (or a few defined ones), we'll be skipping that for now and just checking SPF entries of sending email servers and reject email coming from unauthorized IP addresses, this will limit a lot of spam from spammers faking popular domains! (If you ever want to set up SPF for our own domain is as easy as inserting a new TXT record on the DNS, see www.openspf.org for a wizard!)

To check SPF entries we will use a Python daemon named tumgreyspf that you should have already installed at the beginning of the guide.
tumgreyspf is a policy daemon that does both greylisting and SPF checking of incoming emails. Using it is detailed in the /usr/share/doc/tumgreyspf/README.Debian file. Basically it boils down to adding one line to your smtpd_sender_restrictions (or smtpd_recipient_restrictions if you put everything in there) making use of it.
Just add the line check_policy_service unix:private/tumgreyspf on the recipient rescritions:

nano /etc/postfix/main.cf

smtpd_recipient_restrictions =
               permit_sasl_authenticated,
               reject_non_fqdn_sender,
               reject_non_fqdn_recipient,
               reject_unknown_sender_domain,
               reject_unknown_recipient_domain,
               reject_unauth_pipelining,
               permit_mynetworks,
               reject_unauth_destination,
               reject_rbl_client zen.spamhaus.org,
               check_policy_service unix:private/tumgreyspf,
               permit

And to define the program that is called when using this policy service you need to add one line to your /etc/postfix/master.cf:

nano /etc/postfix/master.cf

The line to add:

tumgreyspf unix  -      n       n       -       -       spawn user=tumgreyspf argv=/usr/bin/tumgreyspf

To disable graylisting you need to edit the tumgreyspf configuration:

nano /etc/tumgreyspf/default.conf

CHECKERS = spf

Now reload your Postfix and that's it:

postfix reload

Checking that SPF is working as expected

When you receive an e-mail you should see the SPF check result on your /var/log/mail.log logfile.

Case: SPF okay

Every incoming email should now log an additional line from tumgreyspf. If the SPF check was positive then you will get:

tumgreyspf[24672]: sender SPF authorized: QUEUE_ID=""; identity=mailfrom;
   client-ip=26.21.244.31; helo=squedge2.squ.edu.om;
   envelope-from=…@squ.edu.om;
   receiver=…@workaround.org;

This means that the sender …@squ.edu.om was allowed in after checking the SPF entry.

Case: SPF fail

If the SPF check fails then you will see something like:

tumgreyspf[24672]: SPF fail - not authorized: QUEUE_ID=""; identity=mailfrom;
   client-ip=41.234.18.141; helo=gmx.de;
   envelope-from=…gmx.de;
   receiver=…christoph-haas.de;

The email got rejected. One spam mail less in the world. :)

Case: SPF softfail

The third case is when the SPF entry does not enforce a FAIL (-all) but just uses SOFTFAIL (~all). In your log file it will look like:

tumgreyspf[20408]: domain owner discourages use of this host: QUEUE_ID="";
   identity=mailfrom; client-ip=220.245.2.67; helo=220-245-2-67.static.tpgi.com.au;
   envelope-from=…@rollouts.com; receiver=…@workaround.org

Unfortunately the SOFTFAIL does not actually reject the email. But the sender still believes that this IP address should not be sending email in their name. Luckily tumgreyspf adds this information to a "Received-SPF" header into the email. Just like:

Received-SPF: Softfail (domain owner discourages use of this host) identity=mailfrom;
   client-ip=61.146.93.243; helo=mail.163gd.com;
   envelope-from=…@cantv.net; receiver=…@christoph-haas.de;

So you still have a chance to mark such emails in spam in your email client by filtering out emails having a "Received-SPF: Softfail" header. (Unfortunately tumgreyspf does not have a configuration option to treat SOFTFAIL as FAIL.)

Case: No SPF information

If the remote domain is ignorant and stupid and does not have any SPF entries yet then your log file will read:

Received-SPF: Neutral (access neither permitted nor denied) identity=mailfrom;
   client-ip=80.65.65.222; helo=mail.unze.ba;
   envelope-from=…@gmail.com; receiver=…@christoph-haas.de;

Server-side filters: Sieve

You already have a fully functioning mail server that even tags spam. But so far you left the actual task of sorting out the tagged spam to the user. We can do better by setting up server-side filters. The task ahead is to move all spam emails to a user's "Spam" folder. We are using the Sieve feature of Dovecot which is a mail filter like procmail (which does not work for virtual mailboxes). Sieve has a simple scripting language that allows us to forward all emails that are tagged by SpamAssassin to a "Spam" folder automatically. At the end of the /etc/dovecot/dovecot.conf file you will find a "plugin" section. Within there you can define a "sieve_global_path" which points to a default filter file that takes effect if the recipient user hasn't defined any per-user filtering rules.

By the way the term "server-side filtering" means that the rules are executed on the server automatically. As opposed to "client-side filtering" which is configured in the mail program on the user's computer. Apparently server-side filtering is preferred as it happens even if the user is offline. And it gives you interesting options such as a vacation autoresponder or filtering based on header fields.

Sieve filtering can be tricky because the dovecot module has been changed a number of times for different versions of dovecot, so be carefull on what you're doing, assuming you have the same version of dovecot as in this guide you should open the dovecot configuration file and add two lines on the "plugin" section:

nano /etc/dovecot/dovecot.conf

and edit:

[...]
plugin {
  [...]
  sieve=~/.dovecot.sieve
  sieve_before = /var/vmail/spam.sieve

Now let's add a basic sieve script that will be executed before the personal script of each user:

nano /var/vmail/spam.sieve

this sieve filter will assume the users want spam on a directory named "Spam":

require ["fileinto"];
# Move spam to spam folder
if header :contains "X-Spam-Flag" ["YES"] {
  fileinto "Spam";
  stop;
}

Restart dovecot:

chown vmail.vmail /var/vmail/spam.sieve
/etc/init.d/dovecot restart

At this point you should check that a SPAM email gets moved in the Spam folder for each users, if the "Spam" folder does not exist for a new user it will get automatically created.. so be sure that the every user subscribes to it from their mail client.
More advanced users will the be able to add personal rules on their sieve file trough a special plugin on the webmail interface.

Statistics: Awstats

Analysing the mail log gives a better understanding on how the mail server is being used, we can use the Awstats tool (the same that is widely used for basic web log analysis).

You should download it from http://awstats.sourceforge.net/ and then install it:

unzip awstats-7.0.zip
mv awstats-7.0 /usr/local/awstats
cd /usr/local/awstats/tools/
perl awstats_configure.pl

follow the wizard: * give the path of apache conf "/etc/apache2/httpd.conf" * define the first configuration file as just "mailserver" * say yes to all the questions At this point you'll have to configure awstats to analyse properly your mail server:

nano /etc/awstats/awstats.mailserver.conf

To analyse Postfix logs you'll have to edit this line:

LogFile="perl /usr/local/awstats/tools/maillogconvert.pl standard < /var/log/mail.log |"

and then be sure to change this options:

LogType=M
LogFormat="%time2 %email %email_r %host %host_r %method %url %code %bytesd"
LevelForBrowsersDetection=0
LevelForOSDetection=0
LevelForRefererAnalyze=0
LevelForRobotsDetection=0
LevelForWormsDetection=0
LevelForSearchEnginesDetection=0
LevelForKeywordsDetection=0 
LevelForFileTypesDetection=0
ShowMenu=1
ShowSummary=HB
ShowMonthStats=HB
ShowDaysOfMonthStats=HB
ShowDaysOfWeekStats=HB
ShowHoursStats=HB
ShowDomainsStats=0
ShowHostsStats=HBL
ShowAuthenticatedUsers=0
ShowRobotsStats=0
ShowEMailSenders=HBML
ShowEMailReceivers=HBML
ShowSessionsStats=0
ShowPagesStats=0
ShowFileTypesStats=0
ShowFileSizesStats=0
ShowDownloadsStats=0
ShowBrowsersStats=0
ShowOSStats=0
ShowOriginStats=0
ShowKeyphrasesStats=0
ShowKeywordsStats=0
ShowMiscStats=0
ShowHTTPErrorsStats=0
ShowSMTPErrorsStats=1

In the same file you can also customize the "Logo" directive to brand the page with a nice logo.
Now let's test the configuration:

mkdir /var/lib/awstats
perl /usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=mailserver
/etc/init.d/apache2 restart

We should already see the statistics at the page: https://mail.example.com/awstats/awstats.pl?config=mailserver
Let's configure a line on cron:

crontab -e

as follow to update statistics every 55 minutes:

*/55 * * * * /usr/local/awstats/tools/awstats_updateall.pl now > /var/log/awstats.log

Webmail: Roundcube with Sieve plugin

Roundcube is a webmail interface that after a slow-start a few years ago is now gaining huge popularity thanks to an outstanding graphic interface.
You can download it from www.roundcube.net and then uncompress the content in /var/www/webmail/. Let's install it, from a terminal customize the following query with a good random password (write it down, you will need it later on):

mysql -u root -p
> CREATE DATABASE roundcubemail /*!40101 CHARACTER SET utf8 COLLATE utf8_general_ci */;
> GRANT ALL PRIVILEGES ON roundcubemail.* TO roundcube@localhost
    IDENTIFIED BY 'password';
> quit

then let's import the empty database:

mysql -u root -p roundcubemail < SQL/mysql.initial.sql

At this point you can take advantage of the automatic configuration, just make sure you have the correct permissions:

chown -R www-data:www-data /var/www/webmail

and point your browser at https://mail.example.com/installer/ and follow the wizard checking that everything is sound and giving the appropriate information, take special care with: * Select the "ip_check" functionality and for identities_level choose "one identity with possibility to edit all params but not email address. * Insert the mysql password you wrote down earlier. * For "default_host" just enter "localhost" * On "junk_mbox" insert "Spam" as defined earlier in the Sieve script * For "smtp_server" just enter "localhost" * Select "Use the current IMAP username and password for SMTP authentication" * Enable "preview_pane", the users usually want it Now copy the two configuration file in the /var/www/webmail/config/ directory.

On the final step Roundcube allows you to test both smtp and imap: do it!
Remember to delete the installar folder for security purpose:

rm -R /var/www/webmail/installer

At this point the basic roundcube should be working fine, let's configure the Sieve and Password plugin editing manually the file config/main.inc.php:

nano /var/www/webmail/config/main.inc.php

and editing the line:

$rcmail_config['plugins'] = array('managesieve','password');

The Sieve filter plugin should be already be working out-of-the-box (remember that on dovecot the "managesieve" protocol must be enabled!) and filtering everything after the spam filter, we just have to configure the password changing routine:

cp /var/www/webmail/plugins/password/config.inc.php.dist /var/www/webmail/plugins/password/config.inc.php
nano  /var/www/webmail/plugins/password/config.inc.php

and change these two options:

$rcmail_config['password_driver'] = 'sql';
$rcmail_config['password_db_dsn'] = 'mysql://mailserver:password@127.0.0.1/mailserver_db';
$rcmail_config['password_query'] = 'UPDATE virtual_users SET password=MD5(%p) WHERE email = %u AND password=MD5(%o) LIMIT 1';

Now test it, should be working!

Remember to change the skin with the correct branding logo.

Testing procedure

Now you should have everything that it takes for a good mail server, let's test it out:

Connect to http://mail.example.com and verify that it redirect you to https://mail.example.com, check that the SSL certificate is ok
Connect to SMTP without SSL (port 25) and without password and verify that you CAN'T send e-mail to external domains (i.e. Gmail)
Connect to SMTP without SSL (port 25) and WITH password and verify that you CAN send e-mail to external domains (i.e. Gmail)
Connect to SMTP with SSL (port 465) and without password and verify that you CAN'T send e-mail to external domains (i.e. Gmail) and that the SSL certificate is ok
Connect to SMTP with SSL (port 465) and WITH password and verify that you CAN send e-mail to external domains (i.e. Gmail)
Connect to POP3 without SSL (port 110) and verify that you can download your messages
Connect to POP3 with SSL (port 995) and verify that you can download your messages and that the SSL certificate is ok
Connect to IMAP4 without SSL (port 143) and verify that you can download your messages
Connect to IMAP4 with SSL (port 993) and verify that you can download your messages and that the SSL certificate is ok
Send an ordinary e-mail from a few ordinary domains and verify it get's trough fine, play with attachment sizes, html, etc.
Send a spam e-mail from an ordinary e-mail address (i.e. GTUBE) and verify it gets moved on the "Spam" folder (this checks both SpamAssassin and the sieve part)
Check the RBL blocker by sending an email (any email) to: nelson-sbl-test@crynwr.com. The Crynwr system robot should answer you to tell you if your server is correctly blocking SBL-listed IPs or not, this could take even an hour, a good way to tell if everything works fast is to monitor the connection on the server ("tcpflow -c port 25") while sendint the e-mail to nelson-sbl-test@crynwr.com
Check the SPF filtering as described in the guide on the logs and/or by sending a fake email from a domain with SPF enabled (if you have access to it)
Try to add users/domain from the administration panel and see if the server works for those
Try to use the webmail (both receiving and sending)
Test the sieve filter by defining a new filter on the webmail
Test the webmail, the folders, etc.

Linux4Dummies

Pages

Sunday, July 15, 2012

Postfix Installation on Ubuntu 10.10/11.10

Introduction

Software Versions

Hostname check

Time zone

Software installation

Preliminary Apache and PHP configuration

MySQL Database

Postfix to Database Mappings

virtual_mailbox_domains

virtual_mailbox_maps

virtual_alias_maps

Sending mail to Dovecot

Dovecot configuration

More editing on Postfix configuration

SMTP test with telnet

POP3 test with telnet

IMAP test with telnet

Testing Dovecot's SSL

Authenticated SMTP with SSL

Filtering Spam with SpamAssassin

Basic checks to see if SpamAssassin filtering is working

Install the web administration panel

More Spam control: adding SPF filtering

Checking that SPF is working as expected

Case: SPF okay

Case: SPF fail

Case: SPF softfail

Case: No SPF information

Server-side filters: Sieve

Statistics: Awstats

Webmail: Roundcube with Sieve plugin

Testing procedure

No comments:

Post a Comment