In Part 1 of this post I explained that I had four different e-mail accounts, which were collectively attracting increasing amounts of spam and how I desperately needed to find a solution to this problem.
My solution centred around the use of a Linux server that, up until this point, I was using primarily as a web development system. Part 1 of this post details how I configured this system to automatically download e-mails from my four mail accounts and how I set-up a local POP3 mail server on the Linux machine that my Microsoft Outlook e-mail client could access to subsequently download these collected e-mails.
The following steps in this post explain how I configured the Linux server to perform a two stage filtering process on all incoming mail to block spam.
To this point I had configured my Linux server to automatically download e-mail from my four mail accounts using fetchmail at regular intervals. The downloaded e-mails were deposited into a single e-mail account on this server via Postfix. Using Dovecot as a local POP3 mail server, my Microsoft Outlook e-mail client could now download these e-mails from the single e-mail account onto my Windows machine.
The next stage in the process was to begin filtering the downloaded e-mails and the little program that I was pining all my anti-spam hopes on was Procmail.
Filtering Mail with Procmail
Procmail works by sequentially going through a list of rules called “recipes”, with each recipe having a test that is performed on the supplied e-mail. If the incoming mail message meets the test’s requirements, than the conditional action is performed. For example:
This rule tests to see if the e-mail was sent to an e-mail address at mydomain.com with the e-mail email@example.com (a typical random made up address by spammers, but to a valid domain). If this condition is met, the e-mail is passed to the “bit bucket”, which was predefined as /dev/null. /dev/null is a Unix system file with a difference; anything written to this file is discarded, effectively deleting the file and therefore the e-mail in this instance.
To get the procmail program to run as the final mail delivery agent (MDA) from within the Postfix mail server, I had to add an additional line to the main Postfix configuration file (/etc/postfix/main.cf):
mailbox_command = /usr/bin/procmail -y -a
With this line in place, all mail received by Postfix would be passed on to the procmail program for conditional processing.
Configuring procmail via Webmin (my usual choice for administration functions)didn’t really seem to offer any advantages over directly editing its configuration file (/etc/procmailrc), so this is the route that I took. The following is the edited configuration file that I used for procmail. The key line (11) told procmail where all of the e-mails were that it had to process:
1: # Note: Anything after a hash character (#) is a comment & is ignored by procmail.
3: #### Begin Variables Section ####
5: # Next line is needed if programs are invoked from your procmailrc, e.g. spamassassin
6: SHELL = "/bin/sh"
8: PATH = "$HOME/bin:/usr/bin:/usr/local/bin:/usr/include:/usr/local/sbin:/bin:/sbin:/usr/sbin"
10: MAILDIR = "/var/mail"
11: DEFAULT = "/var/mail/mymail"
12: JUNKMAIL = "/var/mail/junkmail"
14: BITBUCKET = "/dev/null"
16: # Directory for storing procmail configuration files
19: # LOGFILE should be specified ASAP so everything below it is logged
20: # Put ## before the next line if you want no logging (not recommended)
21: LOGFILE = "/var/log/mail.procmail"
23: # To insert a blank line between each message's log entry in $LOGFILE,
24: # uncomment the next two lines (this is helpful for debugging)
28: #### End Variables Section; Begin Processing Section ####
34: # Messages that fall through all your procmail recipes are delivered
35: # to the default INBOX. To find it, run 'procmail -v'
37: #### End Processing Section ####
The three included files (lines 30 – 32) contained the actual filtering rules (recipes) that procmail had to run on every incoming e-mail. All of the rules could have been incorporated directly in the main procmail configuration file, but including them as separate files helped to modularise the system and make it easier to maintain. Having said this, when I was initially configuring the system I did have all of the rules in the single file, but as the number of rules grew it started getting a little unwieldy, hence the decision to split them up.
The included files all have filenames beginning with the prefix rc, which is a Unix type convention standing for runtime configuration.
In the file rc.badaddresses I put a list of rules to delete all e-mails sent to known bad addresses on my domains that the spammers had used in the past. Overnight this change cut the amount of spam that I received by 90%, exactly the result that I had hoped for. Moreover, if any spammer now makes up a new random address on one of my domains, it’s now only a two minute job to add the new spam address to my home made blacklist.
I also had a few mail addresses that spamers sometimes used, but which I didn’t want to completely delete the e-mail for, for example e-mail addresses with a misspelling of my name. More than likely to be spam, but occasionally a genuine mistake if I had given the e-mail address out over the phone. These e-mail addresses were listed in my rc.suspectaddresses file, which passed them to a junk mail folder where they could be reviewed before being deleted manually.
The final procmail rule file that I included (rc.spamassassin) was used to invoke the SpamAssasin anti-spam program as a final attack on the spam onslaught:
1: # Scan for spam and process
3: # The condition line ensures that only messages smaller than 250 kB
4: # (250 * 1024 = 256000 bytes) are processed by SpamAssassin. Most spam
5: # isn't bigger than a few k and working with big messages can bring
6: # SpamAssassin to its knees
8: # The lock file ensures that only 1 spamassassin invocation happens
9: # at 1 time, to keep the load down
12: :0fw: spamassassin.lock
13: * < 256000
14: | /usr/bin/spamassassin
16: # All mail tagged as spam (eg. with a score higher than the set threshold)
17: # is moved to junkmail
19: * ^X-Spam-Flag: YES
This file forwarded any e-mails less than than 250kB (spam e-mails aren’t usually any bigger than this) to the SpamAssassin program for checking. If SpamAssassin found an e-mail that it thought was spam, it tagged it with an additional header in the e-mail (X-Spam-Flag), which the following procmail rule (lines 18 – 20) checked for. If it was found to be spam, procmail forwarded it to the junk mail folder for subsequent review.
I now have an easily expandable system that automatically downloads e-mails from multiple e-mail accounts at regular intervals into a single e-mail account on my local server. These e-mails can be subsequently downloaded by my e-mail client anytime I wish. This streamlines my system and reduces the load on my desktop e-mail client from having to check multiple accounts every few minutes.
Furthermore, the advanced rule based filtering system can be quickly modified at a moment’s notice, thanks in part to its modularisation, to selectively block or let through single e-mail addresses, or groups of addresses.
Finally, the spam filter filters any remaining e-mails to known, good addresses for spam keywords and phrases.
Room for Improvement
If I require improved spam filtering in the future, I could implement the learning feature of the SpamAssasin program to improve its effectiveness, but for the moment it seems to be doing a perfectly adequate job. It’s nice to have a reserve plan up my sleeve in the anti-spam war though.
Leave a Reply