Spam filtering with qpsmtpd

We all know e-mail is broken. It's become an arms race between spammers and the world to keep the stuff out of our inboxes. Well, my spam-prevention process got a big shot in the arm yesterday, by installing qpsmtpd, a drop-in replacement for qmail-smtpd.

qpsmtpd is another fine product by Ask Bjørn Hansen. It's written in Perl. Quite modular and extensible through a plugin architecture, so it's naturally a good fit with me.

At this point, if I get a spam e-mail, it's pretty close to a miracle. Here's what an inbound e-mail has to go through:

  • First line of defense is qpsmtpd itself, which runs a variety of checks as it receives each e-mail (all of these are provided by qpsmtpd out of the box, unless otherwise indicated):
    • "Early talker" check -- if the smtp client starts dictating the e-mail message before qpsmtpd even responds to the HELO (or EHLO) command, the conversation is cut off with a "don't be rude" error, since the client is talking out of turn and is most likely a automated spamming process that isn't waiting on anything to do it's business.
    • A check of unrecognized SMTP commands. This is good for rejecting spam sent through HTTP proxies (or at least so says the documentation). More than 4 of these and the connection is cut.
    • Check for a resolvable "from" host. Very effective for anonymous spamming, as it requires that the from host resolve. Yes, if Verisign reimplements Sitefinder, they will break this step.
    • rhsbl - Domain-based blacklist provided by rfc-ignorant.org.
    • dnsbl - Another domain-based blacklist, powered by Spamhaus, Mail-abuse.org and others.
    • From domain check. Simple user-defined list of unacceptable domain names.
    • Recipient check. Simple user-defined list of unacceptable recipient names.
    • Check for spam "HELO". Rejects messages where the HELO command is from a user-defined list of domains (handles cases where spams announce themselves as coming from yahoo.com or aol.com, both of which never handshake like that).
    • Greylisting. Brilliant technique where e-mails are initially, but temporarily rejected. If the originating server retries the message after the greylisting timeout has passed, the mail will pass through. You know all those hijacked computers out there that are spamming? They don't handle soft-bounces like this. Read more about greylisting here. (This is a 3rd party plugin for qpsmtpd, available here.)
    • Relay check. Prevents relaying.
    • ClamAV scan virus scan (signatures are updated daily).
    • SpamAssassin scoring.
  • After all this, e-mails are delivered to their inboxes. But wait-- there's more!
  • Whitelist/blacklist/challenge-response. I use Knowspam as my POP server, so it picks up the e-mail from my mail server, passing through any valid e-mails and sending challenge e-mails to anyone not already recognized. (I think there are qpsmtpd solutions for this too, but I like the way Knowspam does the challenge/response process, so I'm sticking with them for now.)
  • My Apple Mail junk filters are the last line of defense if all else fails. But so far, by this point, there's nothing left to filter.

I'm considering enabling ClamAV as well which will scan for any virus content. I'll wait until I get a viral message first though (not a concern either way since I don't do Windows anymore, but would be helpful to others that use my server for e-mail). (ClamAV is now in the mix.)

If a spam message gets through after all the above, it probably means it's being sent from an individual, using their own e-mail client and with pretty benign content that isn't recognized as spam for whatever reason. They would have also had to go through the Knowspam challenge process which means they have a valid e-mail address. At that point I can just block the e-mail address and/or domain if I want.

I should also point out that since signing up with Knowspam back in August, it has reportedly blocked 22,401 spam messages for me. Thats over 100 a day. Since enabling qpsmtpd (and the greylisting plugin for it) yesterday, that number hasn't gone up. Impressive.

My e-mail inflow has slowed to a trickle of it's former volume. A bittersweet victory. I feel relief, but also a little lonely all of a sudden.

TrackBack

TrackBack URL for this entry:
http://bradchoate.com/mt/feedback/tb/842

Listed below are links to weblogs that reference Spam filtering with qpsmtpd:

» http://feralboy.com/log/links/archives/2004_02_18.html#000875 from FeralBlog Link Sideblog
Brad Choate: Spam filtering with qpsmtpd... [Read More]

» Advanced Spam Filtering from Full Speed
Brad Choate has a fairly complicated spam filtering setup that's based on qpsmtpd. After reading his story, I'm very tempted to give qmail another shot.... [Read More]

6 Comments

Schmo said:

Thanks for switching your theme back. :) My head was starting to hurt.

Schmo said:

Doh! I just switch to the new Mozilla FireBird, err... Firefox, whatever they are calling it now. That caused my theme to go back to the default.

Thanks... Excellent Site!!!

Adam Kalsey said:

Check for a resolvable “from” host. Very effective for anonymous spamming, as it requires that the from host resolve. Yes, if Verisign reimplements Sitefinder, they will break this step.

Not if qsmtpd is looking for MX records instead of A records. Siteminder works with A records and doesn't care about email, so it wasn't doing MX. Since any host sending mail should have an MX record, resolving the MX should work fine.

Adam, it's looking for both MX and A records. Of course being a tiny bit of Perl code it's trivially easy to change it to ignore the Sitefinder IP if they enable it again.


- ask

Mark Jaquith said:

What is your rate of "overfiltering?" As I added additional layers to my mail (Brightmail, SpamAssassin, SpamPal, Regular Expression filtering, Bayesian filtering), I found that it was filtering out a good deal of wanted mail. I'm hoping that over time I can rely more on a combo of Bayesian/Regex and a whitelist, as those automated spam lists, such as SPEWS, sometimes get overzealous.

Hi Brad,

I bookmarked this article and now that I've set up my new mailserver I also implemented qpsmtpd and it really rocks! I have no account at Knowspam - so I searched for other solutions and stumbled accross TMDA (Tagged Message Delivery Agent). It's quite impressive how it can handle the challenge/response thing and it also offers "tagged" addresses. And the best: It is Open-Source and was initially developed to run under qmail.

So maybe I'll implement this if my Spam-rate is bad with the wonderful greylisting ;-) Check this out - maybe it's also a solution for you (while Knowspam reduces the money in your pocket).

have fun

About

This article was published on February 18, 2004 1:16 PM.

The article previously posted was MT-Textile 2.0.2.

The next article is MT-Textile 2.0.2 retouch.

Many more can be found on the home page or by looking through the archives.

Powered by Movable Type