July 14th, 2011 by John Reeve

Many web-based applications, including online project management software, feature the ability to accept and process emails. For example, an incoming email may be tweeted, posted to flickr, or appended to a task. The online abundance of productivity tools has helped us get out of our inboxes, but they haven’t replaced email entirely. In fact, many web-based apps out there integrate email quite nicely.

We recently began overhauling the Intervals email integration feature, transitioning from polling our customers POP/IMAP servers toward enabling customers to email their Intervals accounts directly. To accomplish this, we needed to configure our mail servers to accept mail on wildcard subdomains using Postfix and Linux. It took a bit of tinkering and reading through a myriad of online forums and documentation to get it working. Thought I would blog how we did it, in the hopes that someone else may find this useful if faced with the same challenge.

Step 1: Configure DNS

DNS takes a while to propogate, so the first thing you will want to do is set up your DNS so it will serve up the proper MX records for subdomain requests. You will need to add an A record for the IP of your mail server, and a wildcard MX record for the domain. It should look like this:
mx.example.com. 86400 IN A 1.2.3.4
* 86400 IN MX 10 mx.example.com.
Note: You may want to set the TTL lower until you get things working smoothly.

To test if the DNS updates for the MX record have propagated to your servers, run the following command:
dig MX subdomain.example.com
In the response you should see:
;; ANSWER SECTION:
subdomain.example.com. 86400 IN MX 10 mx.example.com.
Once you see this you know DNS has updated. But don’t wait around for DNS to resolve before starting in on the next steps. There is plenty you need to be doing in the meanwhile to configure the mail server for wildcard virtual subdomains.

Step 2: Create a user and group on the Linux server for reading and writing to the mailbox file

Once we have Postfix completely configured, emails will be written to a text-based mailbox file on the server. Postfix is going to need the UID and GID of a user that can access this mailbox file. In this step we create the user and group that Postfix, and your web-based application, will use to receive, update, and remove emails on the server.

First, we create a group:
groupadd -g 5000 vmail
Second, we create a user with no login access and the mail directory as their home directory:
useradd -d /var/spool/mail -s /sbin/nologin -u 5000 -g 5000 -G mail -M vmail
Note: The -G option adds this user to the ‘mail’ group so that it can gain write access to /var/spool/mail. On your system, this should match whatever group is specified for /var/spool/mail directory.

Third, we change the group on the mail file that was created so that our vmail user owns the file outright:
chgrp vmail /var/spool/mail/vmail
Finally, we add apache to the vmail group so that our web scripts can access the mailbox file. To do this, we edit /etc/group and add apache to the vmail line, so it looks like this:
vmail:x:5000:apache
Note: ‘apache’ should be whatever group your web server is running under. To find this out, look at the httpd.conf file.

When you are finished, you should see something like this in the /var/spool/mail directory:
-rw-rw---- 1 imail imail 65404 Jul 14 10:09 vmail

By creating a vmail user and group and associating it with the apache user and group, we avoid having to step on the toes of the postfix and mail users. This is important! We want to avoid a scenario where the apache or vmail user might accidentally, or maliciously, be compromised to gain access to the other mailbox files on the server.

Step 3: Configure Postfix to handle multiple virtual subdomains

The final step is to configure Postfix to accept email being sent to wildcard virtual subdomains. To accomplish this we use the virtual mailbox features of postfix and a few regular expressions to enable wildcards for our subdomains.

First, use your favorite text editor to create the virtual mailbox domain file in /etc/postfix/vdomains. This file contains the regular expression for matching the recipient’s email subdomain to a valid virtual domain.
/((\w[\w\-]*)\.)+example\.com/ OK

Second, create the virtual mailbox maps file in /etc/postfix/vmailbox. This file contains the regular expression for matching the recipient’s email subdomain to a valid virtual mailbox.
/@((\w[\w\-]*)\.)+example\.com/ vmail

Third, add the following configuration values to the /etc/postfix/main.cf file. These configuration values tell Postfix where and how to store email sent to these virtual subdomains, and who has the permission to access the virtual mailbox file.
virtual_mailbox_domains = pcre:/etc/postfix/vdomains
virtual_mailbox_base = /var/mail
virtual_mailbox_maps = pcre:/etc/postfix/vmailbox
virtual_minimum_uid = 5000
virtual_uid_maps = static:5000
virtual_gid_maps = static:5000
virtual_mailbox_limit = 0
Note: The virtual_mailbox_limit is set to zero here to denote that the mailbox filesize should not be restricted, since the default is only 52mb. You can set this value to whatever you would like.

Finally, create a lookup table file for the virtual mailbox maps and then reload postfix. Type these commands at the Linux prompt:
postmap /etc/postfix/vmailbox
postfix reload

That’s it! Emails should start flowing into the server now, assuming DNS has propagated. Now it’s up to you to do something with the emails. If you happen to be using PHP, take a look at the imap functions or the Mail_Mbox PEAR library for working with emails in a mailbox file. While there are ways to further configure Postfix to stream emails directly to PHP, we’ve opted to keep them in their native mbox format. This way we preserve the original emails and can use other tools, as can you, in any programming language you choose to parse emails from the mailbox file. Now that you’ve set up your mail server to receive email on wildcard virtual subdomains, it’s up to you what you do with the emails. That’s the fun part. Good luck!

Resources

I could not have figured any of this out without others having already done most of the work. Here is a list of articles, forum discussions, and general documentation that helped me out.

1. Postfix Virtual Domain Hosting Howto
2. pcre_table – format of Postfix PCRE tables
3. Postfix Virtual Wildcard Subdomains
4. catchall alias for wildcard subdomains?
5. Postfix message and mailbox size limits
6. DNS and sendmail: 21.3.4 Wildcard MX Records
7. Wildcard Records
8. Postfix – Hosting Multiple Domains with Virtual Accounts

Notes

1. If the mail server is behind a firewall, you will need to open up port 25 so that it can communicate with other SMTP servers.
2. This post assumes you already have Postfix installed and sending emails.
3. To find out what types of lookup tables your Postfix system supports use the “postconf -m” command.
4. We are using PCRE for the regular expression matching, which requires a replacment string. Since the vdomains file is only looking for a match, we can use any arbitrary replacement string. In our example, we simply use “OK.” An empty string works as well. If the replacement string is not specified, the pattern matching will still work, however, warnings will show up in the mail logs.
5. Although the documentation for Postfix reads “separate domains, non-UNIX accounts,” you still need at least one user account devoted to handling these emails. This is because you’ll need a user for writing mail to the mailbox file, so might as well have the mailbox be for the user we created in step 2.
6. If you’d like to do some filtering on incoming emails, the header_checks configuration parameter is a great place to start.
7. If you’ve done all of the above and are still running into problems, post a comment with questions. I’ll do the best I can to help out.

Photo credit: Horia Varlan

Tags: linux, postfix, system administration, web development, web-based application