2014-10-15

OS X Server - Postfix

Postfix handles the mail sending and receiving. Email servers and email clients connect to postfix and either give it email to send somewhere else (either to another email server, or send it to dovecot for local users). It does SMTP, not IMAP or POP. Dovecot does IMAP and POP.

Here are my notes on managing postfix on OS X Server (3.1.2) on Mavericks 10.9.4. This all assumes you know the basics of starting and configuring the Mail service using Server.app. A lot of this information also applies to the default postfix installation in OS X.

Note, this post is part of a series. Here are the other posts.

Working with the Postfix using non-postfix tools.

Make sure MX record is set correctly

dig -t MX example.com

Check the status

sudo serveradmin fullstatus mail

Stop the mail service

sudo serveradmin stop mail

Start the mail service

sudo serveradmin start mail

A full list of settings

sudo serveradmin settings mail

A lot of information on the different settings that can be configured is detailed at krypted.com.

OS X Client vs Server

OS X has postfix installed by default and it has a default set of config files. The default config files are configured to send email only, it will not act as an email server.

The launchdaemon at /System/Library/LaunchDaemons/org.postfix.master.plist controls how the system launches postfix. By default, that launchdaemon instructs launchd to watch the directory /var/spool/postfix/maildrop, and if it changes, then it executes /usr/libexec/postfix/master, the master postfix daemon. The launchdaemon also gives the master daemon the arguments "-e 60", which tells master to quit after 60 seconds.

The Server.app changes a few things. It installs a new set of config files at /Library/Server/Mail/Config/postfix. It changes the launchdaemon shown above to launch postfix using the new config files. It doesn't quit the master daemon after 60 seconds. And it watches a different directory for email to send, /Library/Server/Mail/Data/spool/maildrop.

If you want to see the differences between OS X client and server, compare /etc/postfix with /Library/Server/Mail/Config/postfix.

Specify the correct config files with -c config_dir

When running postfix commands you need to specify which config files located at /etc/postfix you want to read or else it will read the default config files that are installed with the OS.

I'm not going to show the -c argument in this document for every command I run. If I show the output of a command, I'll probably show the output of the default /etc/postfix directory, and maybe I'll show what is different for /Library/Server/Mail/Config/postfix. Either way, you kind of have to read the text to figure out if I'm talking about /etc/postfix or /Library/Server/Mail/Config/postfix because I don't want to list "-c /Library/Server/Mail/Config/postfix" all the time.

View non-default settings on OS X client (/etc/postfix).

postconf -n

View non-default settings on OS X Server.

postconf -n -c /Library/Server/Mail/Config/postfix

If you get tired of typing that -c argument, you could create an alias.

For sh (using a hypothetical postsomething command)

alias postsomething2="postsomething -c /Library/Server/Mail/Config/postfix"

For tcsh (using a hypothetical postsomething command)

alias postsomething2 "postsomething -c /Library/Server/Mail/Config/postfix"

Or you could assign it to a shell variable.

For sh (dc is short for "dash c")

dc="-c /Library/Server/Mail/Config/postfix"
postconf -n $dc

For tcsh

setenv dc "-c /Library/Server/Mail/Config/postfix"
postconf -n $dc

You might want to setup your shell to always alias these commands or assign the variables if you use the command line often, because I can imagine typing that path is going to get lame fast.

Or if you're lazy like me, you could just create a symlink so you don't have to type the full path.

ln -s /Library/Server/Mail/Config/postfix /p
postconf -n -c /p

Learning Postfix

Official docs can be found at postfix.org. Note, OS X Server 3.1 includes Postfix 2.9.4 (I found that by looking at /var/log/mail.log while starting the mail service, and by running postconf mail_version).

The Postfix version matters because the documentation is for all versions of Postfix and when different versions behave differently, the documentation specifies which version does what. As of July 2014, the lastest version of Postfix is 2.11. I have heard that many institutions are about 2 dot versions behind, so I don't think it's too alarming that OS X Server is 2 dot versions behind.

Read man pages:

Postfix Basics

Start and Stop Postfix

postfix start
postfix stop

To re-read configuration files (after you make a change), enter:

postfix reload

The log is located at: /var/log/mail.log. This is set by /etc/asl.conf.

If you tail -f /var/log/mail.log you'll see the mail server sending and receiving mail as it runs. It's very informative.

Postfix Settings

Displaying settings

This will display a long list (over 800) of settings (including default settings).

postconf

Display settings that are not default (changed in main.cf).

postconf -n

On OS X it produces this. OS X Server has a much longer list.

biff = no
command_directory = /usr/sbin
config_directory = /Library/Server/Mail/Config/postfix
daemon_directory = /usr/libexec/postfix
data_directory = /Library/Server/Mail/Data/mta
debug_peer_level = 2
debugger_command = PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin xxgdb $daemon_directory/$process_name $process_id & sleep 5
dovecot_destination_recipient_limit = 1
html_directory = /usr/share/doc/postfix/html
imap_submit_cred_file = /Library/Server/Mail/Config/postfix/submit.cred
inet_interfaces = loopback-only
inet_protocols = all
mail_owner = _postfix
mailbox_size_limit = 0
mailq_path = /usr/bin/mailq
manpage_directory = /usr/share/man
message_size_limit = 10485760
mydomain_fallback = localhost
mynetworks = 127.0.0.0/8, [::1]/128
newaliases_path = /usr/bin/newaliases
queue_directory = /Library/Server/Mail/Data/spool
readme_directory = /usr/share/doc/postfix
recipient_delimiter = +
sample_directory = /usr/share/doc/postfix/examples
sendmail_path = /usr/sbin/sendmail
setgid_group = _postdrop
smtpd_client_restrictions = permit_mynetworks permit_sasl_authenticated permit
smtpd_tls_ciphers = medium
smtpd_tls_exclude_ciphers = SSLv2, aNULL, ADH, eNULL
tls_random_source = dev:/dev/urandom
unknown_local_recipient_reject_code = 550
use_sacl_cache = yes

To see how Mac OS X configures Postfix differently than the default, run these commands.

postconf > /tmp/postconf_all
postconf -d > /tmp/postconf_default
diff /tmp/postconf_all /tmp/postconf_default

Display a particular setting:

postconf -h config-value

Change a setting

postconf -e param=value

Or edit your mail.cf file.

Some settings you should probably look up

What the machine name is (this is not shown by postconf -n)

postconf myhostname

What Postfix version you have

postconf mail_version

Postfix Mappings

Mappings are similar but different then settings. Note, postmap uses the -c to specify the config directory just like postconf. Mappings are stored in plain text but then they are converted into a database so that postfix and read them quickly. They have different formats.

Find out what lookup table types your Postfix supports.

postconf -m

OS X and OS X Server support these lookup tables (that means this version of postfix was compiled with support for these, there are others that could be added but Apple didn't choose to include them).

btree
cidr
environ
fail
hash
internal
ldap
memcache
pcre
proxy
regexp
static
tcp
texthash
unix

Mostly you will work with hash.

Lookup tables

There are many different lookup tables. They are located in the config directory, /etc/postfix for OS X client (the send only version), or /Library/Server/Mail/Config/postfix for OS X Server. These are the tables in /etc/postfix.

access
aliases
canonical
custom_header_checks
generic
header_checks
relocated
transport
virtual

All of these files have commented out instructions and are essentially empty (except for customheaderchecks, which has one entry). If you ever change one of these files you need to run postmap and point it to the file so that it can update the indexed database that postfix uses whenever you change the canonical table.

Update a mapping

postmap /etc/postfix/canonical

The exception is aliases, you need to run newaliases. There is also the postalias command that allows you to maintain the alias database. Also, OS X Server appears to be configured to us /etc/aliases instead of /Library/Server/Mail/Config/postfix/aliases (that file exists, but isn't used). Run postconf -c /Library/Server/Mail/Config/postfix alias_maps to make sure.

OS X Server has a few more lookup tables. Server.app edits these extra files and they basically have comments telling you not to edit them.

rbl_whitelist
system_user_maps
virtual_domains
virtual_users

This is what you see in rbl_whitelist.

### DO NOT MANUALLY EDIT THIS FILE ###
# This file is automatically generated
# any manual additions to this file will be lost

Display a mapping.

postmap -q key lookup_table_type:lookup_table

Example:

postmap -q "subject: " regexp:/etc/postfix/custom_header_checks

Regular Expressions:

postmap -q "string" regexp:/etc/postfix/filename
postmap -q - regexp:/etc/postfix/filename <inputfile

Perl Compatible Regular Expressions:

postmap -q "string" pcre:/etc/postfix/filename
postmap -q - pcre:/etc/postfix/filename <inputfile

Name rewriting

To find out the location for your system, execute the command "postconf alias_maps".

aliases
alias_maps = hash:/etc/aliases

Queues

View the queue:

mailq

It prints a list of emails that look like this.

D62105E96CBD!    5805 Tue Jan 19 17:34:29  sender@mail1.example.com
(lost connection with mail1.example.com[10.0.0.2] while receiving the initial server greeting)
                                         to@mail2.example.com

You can do a few things with these messages. You can requeue them ("-r"), you can put them on permanent hold ("-h"), you can release them from hold ("-H"), or you can delete them ("-d"). The ways you can do this are with a single message ID like this.

postsuper -h D62105E96CBD

Or you can use a "-" and then enter many message ID's.

postsuper -d -
1F2E25E965AA
1FAE75E965AB
201DD5E965AC

Or you can use the "ALL" keyword to affect all of them.

postsuper -r ALL

You can also use the ALL keyword followed by a queue name (hold, incoming, active and deferred)

postsuper -r ALL deferred

I found it interesting that GoDaddy decided to blacklist me once. I think GoDaddy probably blocks with little provocation. It's been awhile, but I think I had to call them and tell them my server was legit, or submit an email or something.

host mailstore1.secureserver.net refused to talk to me: 554 ...

Authentication

You should read the SASL_README, and in particular, the section Enabling SASL authentication and authorization in the Postfix SMTP server.

To see all of the settings related to authentication, use this command

postconf | grep sasl

If "smtpdsaslauth_enable" is "yes", then it is turned on. On OS X in /etc/postfix it is off. The server config (/Library/Server/Mail/Config/postfix) has it turned on.

Find out the SASL compiled into postfix:

postconf -a

OS X and OS X Server include these:

cyrus dovecot

Broken Outlook

Outlook up to and including version 2003 and Outlook Express up to version 6 is broken. The brokensaslauth_clients configuration option lets Postfix talk to these broken clients using non-standard communications. This option does not hurt other clients.

To view the setting.

postconf -h broken_sasl_auth_clients

To change it.

postconf -e broken_sasl_auth_clients=yes

Disable non TLS plain text password

By default Postfix allows plain text authentication over a non-TLS connection. OS X Server uses the default setting as well. You can change it so that non-TLS will not allow plain text but a TLS connection will allow plaintext like this.

postfix -e smtpd_sasl_tls_security_options=noanonymous
postfix -e smtpd_sasl_security_options=noanonymous,noplaintext

TLS/SSL and certificates.

You should read the TLS_README.

Learn what TLS, SSL, and STARTTLS really means. Summary: it means something different to each email client. STARTTLS is suppose to mean start talking unencrypted and switch to encryption for SMTP, IMAP, and POP over ports 25/587, 143, and 110 respecively. TLS/SSL is suppose to mean start talking encrypted from the beginning, SMTP, IMAP, and POP over ports 465, 995, and 993 respectively.

Unencrypted at first (STARTTLS switches it to encrypted):

  • SMTP 25/587 - STARTTLS
  • IMAP 143 - STARTTLS
  • POP 110 - STARTTLS

Encrypted the whole time:

  • SMTP 465 - SSL/TLS
  • IMAP 993 - SSL/TLS
  • POP 995 - SSL/TLS

If you use TLS/SSL you must get a valid certificate for your server or else your clients will get constant warnings that your server certificate is not valid (because OS X self-signs it). The University of Utah has a contract with a certificate provider.

It took me several attempts to get my certificate to work with OS X. First you have to generate a certificate signing request (CSR) using the Server app. You send that to the vendor for a certificate. When you get the certificate back, you are suppose to drop it in the Server app but I kept getting errors that there was no corresponding private key. I really hate how OS X does private keys because I haven't been able to find out which one is for what.

Anyway, I did get my certificate installed but I'm not entirely sure what I did. I believe I dragged it into KeyChain Access.

Spam/Quarantine

My server gets so much quarantined email that it is impossible to keep up with it. The server can email you when it gets an alert for one, but I had to filter all of the to the trash because I got so many. They are stored in /Library/Server/Mail/Data/scanner/quarantine. They are eventually deleted.

You can read quarantined email with these commands.

zless /Library/Server/Mail/Data/scanner/quarantine/spam-jSrs79ZVXT8I.gz
gzcat /Library/Server/Mail/Data/scanner/quarantine/spam-jSrs79ZVXT8I.gz
zgrep "not spam" /Library/Server/Mail/Data/scanner/quarantine/

If you are using Time Machine, you can search all the backups like this.

This shows you the part of each file that matches.

zgrep -r "text" /Volumes/TimeMachine/Backups.backupdb/Your_Server/*/Macintosh\ HD/Library/Server/Mail/Data/scanner/quarantine

This shows the list of files.

zgrep -rl --null "text" /Volumes/TimeMachine/Backups.backupdb/biomail/*/Macintosh\ HD/Library/Server/Mail/Data/scanner/quarantine | xargs -0 ls -l

This shows the first 15 lines of each message.

zgrep -rl --null "text" /Volumes/TimeMachine/Backups.backupdb/biomail/*/Macintosh\ HD/Library/Server/Mail/Data/scanner/quarantine | xargs -0 head -n15

Avoiding getting blacklisted

There are free services you can sign up that will monitor the real-time blacklist servers (RBL).

You can also view your mail server score

Other things you should do. Make sure your DNS is correct. Use an MX record. Use an SPF record. This article on email server DNS records is pretty helpful.

Outlook problem #2

In /Library/Server/Mail/Config/postfix/main.cf, locate the smtpdhelorestrictions setting. Remove "rejectnonfqdnhelohostname" from the list of settings. Restart the Mail service. Apple KB article.

Debugging

You should read the DEBUG_README.

You can do bottleneck analysis.

To debug a particular site: debugpeerlist = example.com debugpeerlevel = 2

There are several other links about debugging postfix.

3rd party utils:

[postfinger](http://ftp.wl0.org/SOURCES/postfinger)
[saslfinger](http://postfix.state-of-mind.de/patrick.koetter/saslfinger/)

Postfix version 2.1 and later can produce debugging reports, a "what-if" report and a "what-happened" report.

What-if:

$ /usr/sbin/sendmail -bv address...

What happened:

$ /usr/sbin/sendmail -v address...

The reports are emailed to you and the format of these reports is practically identical to that of ordinary non-delivery notifications.

To see what ports are listening:

netstat -t -a | grep LISTEN
lsof -i tcp:25

Stress test (don't run this on a production server):

smtp-source
smtp-sink

Example:

smtp-sink -c localhost:25 1000

time smtp-source -s 20 -l 5120 -m 100 -c -f sender@example.com -t recipient@example.com localhost:25

Meaning of flags.

  • "-s 20" - 20 parallel sessions
  • "-l 5120" - 5KB message size
  • "-m 100" - 100 total messages
  • "-c" - display a counter
  • "-f sender@example.com" - From
  • "-t recipient@example.com" - To
  • "localhost:25" - target

Sending email from the command line:

echo bla | /usr/sbin/sendmail -f root root && tail -f /var/log/maillog

Copyright 2016 James Reynolds