Lab 08 - Email Services

In this lab, you will install and configure basic email MTA, MDA, and MUA, and email web app functionality with postfix, dovecot, and roundcube software.
We will be setting up an Internet site email server for our domain. We are not adding any advanced functions, but simply setting up the basic MTA/MDA and then adding a web application MUA to access the mail on our server.
To do this, we will be taking the following actions:

Add an MX record for our email domain name to our DNS
Create a custom SSL certificate to use on our MTA
Allow email service through the firewall
Install the MTA/MDA/MUA software
Please replace lastname45678.mytld with your actual last name and student ID for your domain as per your DNS configuration from Lab 03.

Configure DNS

Add an mx record for your domain, with a target of mail.lastname45678.mytld and a priority of 10.
- In your zone file (/etc/bind/db.lastname45678.mytld), the record will look like: @ IN MX 10 mail.
Reload your DNS server and verify that you can ping mail.lastname45678.mytld and that you can retrieve the MX record for your domain and get mail.lastname45678.mytld as the response.
Verify that you can nslookup mail.lastname45678.mytld and get 172.16.5.2 as the address.
Do not proceed until these nslookups work correctly.

Commands to accomplish these tasks:

# Add MX Record (increment Serial)
nano /etc/bind/db.lastname45678.mytld

# Reload Service
rndc reload

# Test your DNS modifications
ping mail

# For MX lookup, should get answers such that: 
#   lastname45678.mytld	mail exchanger = 10 mail.lastname45678.mytld.
#   mail.lastname45678.mytld	internet address = 172.16.5.2
nslookup -querytype=MX lastname45678.mytld

# Should get Address: 172.16.5.2
nslookup mail.lastname45678.mytld

# Add MX Record (increment Serial)
nano /etc/bind/db.lastname45678.mytld

# Reload Service
rndc reload

# Test your DNS modifications
ping mail

# For MX lookup, should get answers such that: 
#   lastname45678.mytld	mail exchanger = 10 mail.lastname45678.mytld.
#   mail.lastname45678.mytld	internet address = 172.16.5.2
nslookup -querytype=MX lastname45678.mytld

# Should get Address: 172.16.5.2
nslookup mail.lastname45678.mytld

Example zone file after adding MX record:

;
$TTL	86400
@	IN	SOA	ns1.lastname45678.mytld. hostmaster.lastname45678.mytld. (
        2025121500		; Serial
             604800		; Refresh
              86400		; Retry
            2419200		; Expire
              86400 )	; Negative Cache TTL
;
@	IN	NS	ns1
@	IN	MX	10	mail
ns1	IN	A	172.16.3.2
www	IN	A	172.16.4.2
mail	IN	A	172.16.5.2
router3	IN	A	172.16.3.1
pop	IN	CNAME	mail
secure	IN	CNAME	ns1

;
$TTL	86400
@	IN	SOA	ns1.lastname45678.mytld. hostmaster.lastname45678.mytld. (
        2025121500		; Serial
             604800		; Refresh
              86400		; Retry
            2419200		; Expire
              86400 )	; Negative Cache TTL
;
@	IN	NS	ns1
@	IN	MX	10	mail
ns1	IN	A	172.16.3.2
www	IN	A	172.16.4.2
mail	IN	A	172.16.5.2
router3	IN	A	172.16.3.1
pop	IN	CNAME	mail
secure	IN	CNAME	ns1

Generate Key and Certificate

Since we are only really concerned with having encryption for mail transport, we don't usually create real certificates for mail servers. But it is good practice, and this is a course lab. Generate an SSL certificate and key for use with your mail server:

# Must be root! 
cd /etc/openvpn/easy-rsa

# The "nopass" is to skip adding a passphrase to the key
# Make note of the path to your key
./easyrsa gen-req mail.lastname45678.mytld nopass

# Sign the request to generate the certificate
# You will need to know the passphrase for your ca.key!
./easyrsa sign-req server mail.lastname45678.mytld

# Must be root! 
cd /etc/openvpn/easy-rsa

# The "nopass" is to skip adding a passphrase to the key
# Make note of the path to your key
./easyrsa gen-req mail.lastname45678.mytld nopass

# Sign the request to generate the certificate
# You will need to know the passphrase for your ca.key!
./easyrsa sign-req server mail.lastname45678.mytld

Copy the crt and key files for the mail server to /etc/ssl. Put the certificate file in the certs subdirectory, and the key file in the private subdirectory.

cp pki/issued/mail.lastname45678.mytld.crt /etc/ssl/certs/
cp pki/private/mail.lastname45678.mytld.key /etc/ssl/private/

cp pki/issued/mail.lastname45678.mytld.crt /etc/ssl/certs/
cp pki/private/mail.lastname45678.mytld.key /etc/ssl/private/

Add Firewall Rules to Allow Email

Even though we have no other machines to really communicate with our mail server, add UFW rules to allow email service through your firewall:

# SMTP
ufw allow 25/tcp

# SMTP with STARTTLS
ufw allow 587/tcp

# SMTPS (Implicit TLS) 
ufw allow 465/tcp

# POP3
ufw allow 110/tcp

# POP3S
ufw allow 995/tcp

# IMAP
ufw allow 143/tcp

# IMAPS
ufw allow 993/tcp

# SMTP
ufw allow 25/tcp

# SMTP with STARTTLS
ufw allow 587/tcp

# SMTPS (Implicit TLS) 
ufw allow 465/tcp

# POP3
ufw allow 110/tcp

# POP3S
ufw allow 995/tcp

# IMAP
ufw allow 143/tcp

# IMAPS
ufw allow 993/tcp

Note: The above is the list of every default port and supported protocol. If you are configuring a production server, you may want to NOT allow any unencrypted service through the network by configuring your dovecot and postfix servers so the non-encrypted protocols are rejected and not used. The roundcube web mail client will require port 143 (IMAP) for communication (does not support IMAPS), but unencrypted traffic can be strictly limited to the localhost when running on the same machine.

Install The Email Software

Install the postfix, dovecot, and roundcube software packages:

sudo apt update
sudo apt install postfix dovecot-common dovecot-core dovecot-imapd dovecot-pop3d roundcube

sudo apt update
sudo apt install postfix dovecot-common dovecot-core dovecot-imapd dovecot-pop3d roundcube

During the installation of the packages, you may be asked some questions:

Specify your site type to be Internet Site
Specify lastname45678.mytldname as your mail system name
Specify Yes to use dbconfig-common for roundcube
Leave the roundcube database password blank to use a randomly generated password

You will need to manually edit the dovecot SSL configuration file to specify the correct key and certificate file to use:

# Add the path to your key and cert files
nano /etc/dovecot/conf.d/10-ssl.conf

#### Example: Modify your own file ####
#
## SSL settings
##

# SSL/TLS support: yes, no, required. <doc/wiki/SSL.txt>
#ssl = yes

# PEM encoded X.509 SSL/TLS certificate and private key. They're opened before
# dropping root privileges, so keep the key file unreadable by anyone but
# root. Included doc/mkcert.sh can be used to easily generate self-signed
# certificate, just make sure to update the domains in dovecot-openssl.cnf
#ssl_cert = </etc/ssl/certs/mail.lastname45678.mytld.crt
#ssl_key = </etc/ssl/private/mail.lastname45678.mytld.key

# Restart and check the service:
service dovecot restart
service dovecot status

# Verify the current configuration using doveadm utility:
doveadm config
doveconf -n

# Add the path to your key and cert files
nano /etc/dovecot/conf.d/10-ssl.conf

#### Example: Modify your own file ####
#
## SSL settings
##

# SSL/TLS support: yes, no, required. <doc/wiki/SSL.txt>
#ssl = yes

# PEM encoded X.509 SSL/TLS certificate and private key. They're opened before
# dropping root privileges, so keep the key file unreadable by anyone but
# root. Included doc/mkcert.sh can be used to easily generate self-signed
# certificate, just make sure to update the domains in dovecot-openssl.cnf
#ssl_cert = </etc/ssl/certs/mail.lastname45678.mytld.crt
#ssl_key = </etc/ssl/private/mail.lastname45678.mytld.key

# Restart and check the service:
service dovecot restart
service dovecot status

# Verify the current configuration using doveadm utility:
doveadm config
doveconf -n

The `postfix` Installation Review and Modification

Verify that the email services are running:

service postfix status
service dovecot status

service postfix status
service dovecot status

Verify that the ports are being listened to, and see what the running program names are. Look for the port numbers associated with the email service from the presentation.

sudo ss -tlpn

sudo ss -tlpn

Run the postconf command to view all possible settings for your postfix service. Compare that to postconf -n which only shows the settings actually specified in your /etc/postfix/main.cf file.

postconf
postconf -n

postconf
postconf -n

Configure your mailbox, hostname, SASL authentication to use dovecot, and TLS/SSL settings using the following postconf commands.
The settings that will tell postfix to use dovecot for SASL authentication, and correct the mail server hostname have been provided to you here. Please modify with your own names:

postconf -e "home_mailbox ="
postconf -e "smtpd_sasl_type = dovecot"
postconf -e "smtpd_sasl_path = private/auth-client"
postconf -e "smtpd_sasl_local_domain ="
postconf -e "smtpd_sasl_security_options = noanonymous"
postconf -e "broken_sasl_auth_clients = yes"
postconf -e "smtpd_sasl_auth_enable = yes"
postconf -e "smtp_tls_note_starttls_offer = yes"
postconf -e "smtpd_tls_loglevel = 1"
postconf -e "smtpd_tls_received_header = yes"
postconf -e "myhostname = mail.lastname45678.mytld"
postconf -e 'smtpd_tls_key_file = /etc/ssl/private/mail.lastname45678.mytld.key'
postconf -e 'smtpd_tls_cert_file = /etc/ssl/certs/mail.lastname45678.mytld.crt'

postconf -e "home_mailbox ="
postconf -e "smtpd_sasl_type = dovecot"
postconf -e "smtpd_sasl_path = private/auth-client"
postconf -e "smtpd_sasl_local_domain ="
postconf -e "smtpd_sasl_security_options = noanonymous"
postconf -e "broken_sasl_auth_clients = yes"
postconf -e "smtpd_sasl_auth_enable = yes"
postconf -e "smtp_tls_note_starttls_offer = yes"
postconf -e "smtpd_tls_loglevel = 1"
postconf -e "smtpd_tls_received_header = yes"
postconf -e "myhostname = mail.lastname45678.mytld"
postconf -e 'smtpd_tls_key_file = /etc/ssl/private/mail.lastname45678.mytld.key'
postconf -e 'smtpd_tls_cert_file = /etc/ssl/certs/mail.lastname45678.mytld.crt'

Reload your postfix service and check the config to see that your changes went in as you expected and that the service is still running properly.

service postfix reload
postconf -n
service postfix status

service postfix reload
postconf -n
service postfix status

Check the log to ensure there were no errors.

tail /var/log/mail.err

tail /var/log/mail.err

Command Line Email

Send an email to your Linux user email account from root using the mail command to verify you can send and deliver email. Review the mail log to see what gets put there when an email is handled by the service.

# Run as root:
apt install mailutils
mail username
# Type your email and use Ctrl+D when done typing!

# Check the logs:
tail /var/log/mail.log

# Run as root:
apt install mailutils
mail username
# Type your email and use Ctrl+D when done typing!

# Check the logs:
tail /var/log/mail.log

Protocol testing

These types of protocol-based tests may or may not complete depending on the configuration of your server. The default settings for the service daemons change over time, so from semester to semester, these tests may generate differing results.

POP3

Use telnet localhost 110 to connect to the POP service (port 110), and issue the user (USER yourfirstname) and password (PASS yourpassword) handshake. It will either let you log in or refuse it
- Notice that telnet is a plain-text-only protocol!
If you successfully logged in with POP3, use the list (LIST) protocol commands to see how many email messages you have waiting.
You can exit your pop connection using QUIT.

Here is an example output:

telnet localhost 110
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
+OK Dovecot (Ubuntu) ready.
USER a*****
+OK
PASS <*oh-no-my-pass-can-leak*>
+OK Logged in.
LIST
+OK 2 messages:
1 506
2 487
.
QUIT
+OK Logging out.
Connection closed by foreign host.

telnet localhost 110
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
+OK Dovecot (Ubuntu) ready.
USER a*****
+OK
PASS <*oh-no-my-pass-can-leak*>
+OK Logged in.
LIST
+OK 2 messages:
1 506
2 487
.
QUIT
+OK Logging out.
Connection closed by foreign host.

IMAP4

Use telnet localhost 143 to connect to the IMAP service (port 143).
Issue the login (a LOGIN yourfirstname yourpassword) protocol command.
If the login succeeded, try status (a STATUS INBOX (MESSAGES UNSEEN)), and examine (a EXAMINE INBOX) IMAP protocol commands to see how many messages are in your INBOX.
You can use the logout command to exit your IMAP (a LOGOUT)session.

Example Output:

telnet localhost 143
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE LITERAL+ STARTTLS AUTH=PLAIN] Dovecot (Ubuntu) ready.
a LOGIN a***** <*oh-no-my-pass-can-leak-again*>
a OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE SNIPPET=FUZZY PREVIEW=FUZZY PREVIEW STATUS=SIZE SAVEDATE LITERAL+ NOTIFY SPECIAL-USE] Logged in
a STATUS INBOX (MESSAGES UNSEEN)
* STATUS INBOX (MESSAGES 2 UNSEEN 2)
a OK Status completed (0.001 + 0.000 secs).
a EXAMINE INBOX
* FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
* OK [PERMANENTFLAGS ()] Read-only mailbox.
* 2 EXISTS
* 0 RECENT
* OK [UNSEEN 1] First unseen.
* OK [UIDVALIDITY 1765835986] UIDs valid
* OK [UIDNEXT 3] Predicted next UID
a OK [READ-ONLY] Examine completed (0.001 + 0.000 secs).
a LOGOUT
* BYE Logging out
a OK Logout completed (0.001 + 0.000 secs).
Connection closed by foreign host.

telnet localhost 143
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE LITERAL+ STARTTLS AUTH=PLAIN] Dovecot (Ubuntu) ready.
a LOGIN a***** <*oh-no-my-pass-can-leak-again*>
a OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE SNIPPET=FUZZY PREVIEW=FUZZY PREVIEW STATUS=SIZE SAVEDATE LITERAL+ NOTIFY SPECIAL-USE] Logged in
a STATUS INBOX (MESSAGES UNSEEN)
* STATUS INBOX (MESSAGES 2 UNSEEN 2)
a OK Status completed (0.001 + 0.000 secs).
a EXAMINE INBOX
* FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
* OK [PERMANENTFLAGS ()] Read-only mailbox.
* 2 EXISTS
* 0 RECENT
* OK [UNSEEN 1] First unseen.
* OK [UIDVALIDITY 1765835986] UIDs valid
* OK [UIDNEXT 3] Predicted next UID
a OK [READ-ONLY] Examine completed (0.001 + 0.000 secs).
a LOGOUT
* BYE Logging out
a OK Logout completed (0.001 + 0.000 secs).
Connection closed by foreign host.

The `roundcube` Webmail Application

You should have already installed roundcube in the previous step.

Since we are not concerned about the implications of running roundcube on all of our sites, uncomment the Alias line near the start of the /etc/apache2/conf-available/roundcube.conf file. Reload your apache2 service and verify it is still running properly.

nano /etc/apache2/conf-available/roundcube.conf

apachectl graceful
service apache2 status

nano /etc/apache2/conf-available/roundcube.conf

apachectl graceful
service apache2 status

Verify you can access the webmail interface with your browser by accessing http://your-ip/roundcube and logging in using your Linux account.

Review log files

Examine the content of mail.log and mail.err in /var/log to see what is being logged for your activity on your mail server. Examine the content of errors in /var/log/roundcube/ to see if you are having any problems with the roundcube webapp.

Evaluate your server

Run sudo /root/server-check.sh -l 8 firstname lastname studentnumber
Review any problems detected and correct as necessary.
Do not move on to the next lab until you have this lab at 100%
If you are having trouble getting Lab 8 to 100%, try running sudo /root/server-check.sh -l 1234567 firstname lastname studentnumber to make sure you haven't accidentally broken what was completed in previous labs.

Extra Exercise (Optional, Requires an External Server)

This extra exercise does not count for marks and is not required. But it is a useful practice for setting up email relaying through an ISP which blocks the SMTP port (most modern ISPs block this and force you to send email through their servers). If you have your email server set up like shown in this lab, you can use the following commands to add relaying via an external server. Use the actual name of the mail server you are relaying through instead of yourispmailserverdomainname and your ISP-required email login and password instead of myemailaddress and myemailpassword.

externalrelayhost=yourispmailserverdomainname
emailaddr='myemailaddress'
emailpass='myemailpassword'
sudo postconf -e "smtp_sasl_auth_enable = yes"
sudo postconf -e "smtp_tls_security_level = encrypt"
sudo postconf -e "smtp_sasl_tls_security_options = noanonymous"
sudo postconf -e "relayhost = [$externalrelayhost]:submission"
sudo postconf -e "smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd"
echo "[$externalrelayhost]:submission $emailaddr:$emailpass" | sudo tee /etc/postfix/sasl_passwd >/dev/null
sudo postmap hash:sasl_passwd
sudo systemctl restart postfix

externalrelayhost=yourispmailserverdomainname
emailaddr='myemailaddress'
emailpass='myemailpassword'
sudo postconf -e "smtp_sasl_auth_enable = yes"
sudo postconf -e "smtp_tls_security_level = encrypt"
sudo postconf -e "smtp_sasl_tls_security_options = noanonymous"
sudo postconf -e "relayhost = [$externalrelayhost]:submission"
sudo postconf -e "smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd"
echo "[$externalrelayhost]:submission $emailaddr:$emailpass" | sudo tee /etc/postfix/sasl_passwd >/dev/null
sudo postmap hash:sasl_passwd
sudo systemctl restart postfix

Your Linux mail server should now be able to send email to any legitimate Internet email address, although it cannot send email to your email server without further work on your part. It is still useful for sending yourself an email from your Linux system instead of having to log onto the Linux system to read mail sent to root or other users. You can even create an alias for root to forward root mail to your normal email account. You can test this using a command like:

sudo nano /etc/aliases
sudo newaliases
mail -s "Test message from COMP1071 server" root <<< "testing...testing"

sudo nano /etc/aliases
sudo newaliases
mail -s "Test message from COMP1071 server" root <<< "testing...testing"

Further Resources and Reading

Review [http://www.postfix.org/SASL_README.html] for a detailed how-to on setting up other scenarios for postfix using either dovecot or cyrus for Simple Authentication and Security Layer.
Review [https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto] for an overview of how to configure multiple email domains on a single postfix/dovecot server.
Review Postfix SOHO Hints and Tips for settings and commands for configuring email servers connected to ISPs that require you to relay mail through servers.