Introducing SpamLookup

Yes, it's another anti-spam plugin. I would have called it MT-DSBL 2.0, but the feature set grew beyond the name, so a more generic name was chosen. This is a 2.0 release however... the plugin deserves more seniority than a 1.0 release, since it was built of the carcass that was MT-DSBL 1.1.

The first beta release is now available. Please visit the project page for SpamLookup to download it.

So, what does this thing do? Well, in a nutshell:

  • Filters inbound comments for all installed weblogs.
  • Filters inbound TrackBack pings for all installed weblogs.
  • Checks IP address of sender against DNSBL services.
  • Checks mentioned domain names against SURBL services.
  • IP match test for TrackBack pings.
  • URL count limit.
  • Dynamic proxy checking.
  • Moderation word list.
  • Block word list.
  • Passphrase requirement for comments and/or TrackBack pings.

All of these are fully configurable. You can also choose whether to block or to moderate on any of these conditions. Also, you get the choice of moderating TrackBack pings if you have also installed the MT-Moderate (version 1.1 or later) plugin by Chad Everett.

The plugin also has some sensible bypasses, if you choose to enable them.

  • Accept comments from authenticated commenters.
  • Accept comments that have no hyperlinks.
  • Accept comments that use a URL that was previously approved.

SpamLookup is more than just a collection of anti-spam filters. It also lets you manage your comments and TrackBack pings a little better. The comment and TrackBack ping management screens lets you view all records for all weblogs you have installed. From there you can do mass approval, deletion and moderation.

And finally, a despam option is available. Despam will scan your existing comments or TrackBack pings using all of the SpamLookup options you have enabled and will give you the chance to act on anything fishy that turn up. If you know you have existing comment spam, this is a handy way to easily dispatch it.

I've been running this plugin on my site for testing purposes, with it being the only antispam plugin installed and it has been performing very well. However, your mileage may vary.

SpamLookup does a great job combatting spam flooders since they predominately use open proxies to appear as if they are coming from different sources. The DNSBL lookups (quick, DNS based lookup based on IP against a database of known open proxies) put a stop to that. Furthermore, they domains they use to spam with are checked against SURBL lists (quick, DNS based lookup based on the domain against a database of known spam domains). URL limits are also helpful since a lot of weblog spam tries to push through dozens or more hyperlinks at a time.

No substitute for AUTHENTICATION!

Please, please. Use TypeKey. It's free, and it's pretty easy to set up. Authentication doesn't prevent spam, but being able to identify a source of spam goes a long way toward blocking it.

Installation, Support, Etc.

For installation instructions, please visit the project page for SpamLookup. This is a beta release, so there may be some remaining bugs. Please report them if you find any at the project site.

TrackBack

TrackBack URL for this entry:
http://bradchoate.com/mt/feedback/tb/990

Listed below are links to weblogs that reference Introducing SpamLookup:

» Brad Choate rocks! from Neil's World
Brad Choate has just released a beta of SpamLookup, a Movable Type plugin that builds on MT-DBSL to turn it into an all-round spam fighting beast. [Read More]

» Introducing from Fembat.Org : Miniblog
Spamlookup by Brad Choate. In beta, but a whole range of nifty little features potentially makes this an awesome spam fighting machine.... [Read More]

» MovableType plugin - SpamLookup from The Maelström
Brad Choate nous refait ça et publie un nouveau plugin fort utile pour Movable Type. SpamLookup permet de filtrer les commentaires laissés sur votre blog MT et éliminer (ou forcer l’approbation) dès le départ des commentaires douteux. Les cr... [Read More]

» Introducing SpamLookup from Groovy Links
http://daringfireball.net/linked/2005/april#fri-08-spamlookup [Read More]

» la réclame et les cris from dead finks don't talk
On chompy.net's fifth anti-spam plugin for MT, and what Jules Laforgue thinks of all this mess. [Read More]

» Captcha Spammers! Fugggedaboddit from cogdogblog
It's a new spam free day for CogDogBlog and our other affiliated MovableType 2.661 blogs here. I've successfully integrated James Seng's captcha plugin, so that all comment posts require a human to type in a randomly generated security code that... [Read More]

» links for 2005-04-09 from Dan Dickinson: The Primary Vivid Weblog
"Oh my God! There's an axe in my head." Translation fun. [via Joi] (tags: language funny useful) Brad Choate: Introducing SpamLookup Yet another good tool in the war on comment/trackback spam. (tags: movabletype blog spam useful)... [Read More]

» Introducing SpamLookup from JayAllen - The Daily Journey
In case you didn't see it on the Comment Spam Clearinhouse: Many times, MT-Blacklist has been characterized as the "Swiss... [Read More]

» links for 2005-04-10 from Ravensky's Blog
Yahoo! News - First Convicted Spammer Gets 9 Years (tags: spam jail idiots) Easiest way to post del.icio.us to MT Powered blog | Kevin Wen’s Web (tags: del.icio.us links movabletype blogs) Brad Choate: Introducing SpamLookup (tags: movabletype s... [Read More]

» Wake Up to a New Spam Fighter from the leftorium.com
Brad Choate, the Six Apart developer guru, has released an unbelievable new spam fighter for Movable Type blogs. This is exciting news, as I still keep getting spammed like crazy. And, this blog, The Leftorium, has not gotten spammed at... [Read More]

» Wird MT-Plugin SpamLookup der MT-Blacklist-Killer? from einfach-persoenlich
Brad Choate hat ein neues Movable Type-Plugin entwickelt, dass auf den Namen »SpamLoopup« hört. SpamLookup sagt bereits alles - Anti-Spam-Vorsorge ist seine Aufgabe. Der Beitrag informiert über Aufgaben und Funktionsumfang des MT-... [Read More]

» Introducing SpamLookup from MT-Blacklist/Comment Spam Clearinghouse
If MT-Blacklist is the "Swiss Army knife of weblog spam defenses", SpamLookup is the whole damn Swiss Army. [Read More]

» Best Trackback Spam Fighter Ever from majordojo
I have the pleasure of working with Brad at Six Apart and thus had the fortune of being able to preview Spam Lookup. Ever since I installed it, my trackback spam has all but ceased. And coupled with MT-Blacklist, this... [Read More]

» http://www.ericalba.org/elsewhere/archives/2005/04/index.php#002996 from elsewhere
Brad Choate: Welcome... [Read More]

» http://www.ericalba.org/elsewhere/archives/2005/04/index.php#002996 from elsewhere
Brad Choate: Welcome... [Read More]

» The engine room from Acid
Pretty much everyone with a blog faces the curse of comment spam. Brad Choate has just released SpamLookup, an uber-anti-spam plugin which plays nice with MT-Blacklist and which seems to just about kill comment and trackback spam. I'm very impressed... [Read More]

» Spamalot from The Pod Bay Door
:: Not, not the play currently on Broadway, as much as I'd like to see it. Maybe it will still be playing in the fall when I return to NYC (dates yet to be determined, trip yet to be confirmed.)... [Read More]

» Spamalot from The Pod Bay Door
:: Not, not the play currently on Broadway, as much as I'd like to see it. Maybe it will still be playing in the fall when I return to NYC (dates yet to be determined, trip yet to be confirmed.)... [Read More]

» Movable Type Comment and Trackback Spam Tools from Geek News Central
I have two tools I currently use to battle trackback and comment spam on this site, they get about 85%... [Read More]

» Spam ponderings from ***Dave Does the Blog
Comment and trackback spam continue to be a problem here, accumulating at a manageable but annoying pace. Now comes a new approach (hat tip Les) from Brad Choate, SpamLookup. And... [Read More]

» Iron curtain time, yo from Population: One
I'm trying out Brad Choate's cool new anti-spam plugin. If you have trouble commenting or sending a trackback, please drop... [Read More]

» Choate launches Spamlookup for MT from The Blog Herald: more blog news more often
Brad Choate has launched a new spam plugin for MovableType blogs "SpamLookup" which is winning high praise from MT bloggers, including the author of MT Blacklist, and now SixApart employee Jay Allen, who describes the plugin as "the whole damn Swiss Ar... [Read More]

» SpamLookup - Fantastic from dIon Gillard
I installed SpamLookup for a test drive instead of MT-Blacklist and SpamLookup's predecessor MT-DSBL, and after a couple of Windows only quirks, I have to say it's a fantastic utility for stopping trackback and comment spam dead in it's tracks.... [Read More]

» Spam, spam, spam, spam... from LilacRose
The past couple of days, I've been inundated with trackback spam attempts. MT-Blacklist has been blocking most of them (and... [Read More]

» The Comment Registration Act of 2005 from ***Dave Does the Blog
Dammit. Another burst of comment spam today. Nothing that showed up on the front page, but because the URL was innocuous (which is a bad sign of spammers getting smarter),... [Read More]

» testar filter för trackback-spam from Nästa: Bandhagen
Jag har installerat Brad Choates SpamLookup för att förhindra trackbackspam (som den senaste tiden anlänt i parti och minut här). Tyvärr funkar det inte riktigt som jag tänkt mig, jag fattar helt enkelt inte hur man lägger till en kontrollfråga... [Read More]

» Software Upgrades Complete from Wizbang
Several portions of the site software have been upgraded. I haven't seen anything that's broken as a result of the upgrades, but let me know if there are any issues. Aside from the latest updates to Movable Type and Blacklist,... [Read More]

» No spam for me from Chris Bevan
Over the weekend I finally got around to installing SpamLookup, a new anti-spam plugin for MovableType. I've been using MT-Blacklist for quite some time for the same purpose, however I've never trusted it enough to switch off comment moderation. As... [Read More]

» No spam for me from Chris Bevan
Over the weekend I finally got around to installing SpamLookup, a new anti-spam plugin for MovableType. I've been using MT-Blacklist for quite some time for the same purpose, however I've never trusted it enough to switch off comment moderation. As... [Read More]

» Comment/Trackback Policy from Wizbang
Comment and trackbacks at Wizbang are open and do not require registration. As a general rule comments and trackbacks will not be deleted. As with any general rule there are exceptions. Comments that will ALWAYS be deleted or modified include:... [Read More]

» Movable Type 3.16 and SpamLookup from Josh's Weblog
I've upgraded to Movable Type 3.16. So far I don't see much that's changed, and I did pull out the “nofollow” plugin immediately. The two changes that interest me the most areMade significant improvements to the posting interfaces including... [Read More]

» Movable Type 3.16 from Jacques Marneweck's Blog
Movable Type 3.16 is out! I keep telling myself that I should be writing my own blog software which would do the things that I want. I think that the spamlookup plugin from the sounds of it looks great!... [Read More]

» Movable Type 3.16 from Jacques Marneweck's Blog
Movable Type 3.16 is out! I keep telling myself that I should be writing my own blog software which would do the things that I want. I think that the spamlookup plugin from the sounds of it looks great!... [Read More]

» Movable Type 3.16 from Jacques Marneweck's Blog
Movable Type 3.16 is out! I keep telling myself that I should be writing my own blog software which would do the things that I want. I think that the spamlookup plugin from the sounds of it looks great!... [Read More]

» Spam Lookup from Blogula Rasa's Quick Links
I'd like to try this on the new MT3.16 beta... even Jay is enthusiastic about it. Some are using both MTBlacklist, some are now just using SpamLookup - more efficient handling of multiple blogs and different kinds of spam. [Read More]

» Testing Spam Lookup for Movable Type from Jacques Marneweck's Blog
Busy testing SpamLookup over the next couple of days. Apparently it is a big improvement over MT Blacklist.... [Read More]

» SpamLookUp from Johnnie Moore's Weblog
For the technically minded: I've installed Brad Choate's SpamLookup plug in for Movable Type. I'm running it alongside MTBlacklist to try to keep comment and trackback spam at bay. I've just tweaked Blacklist as per this forum comment to get... [Read More]

» SpamLookUp from Johnnie Moore's Weblog
For the technically minded: I've installed Brad Choate's SpamLookup plug in for Movable Type. I'm running it alongside MTBlacklist to try to keep comment and trackback spam at bay. I've just tweaked Blacklist as per this forum comment to get... [Read More]

» SpamLookUp from Johnnie Moore's Weblog
For the technically minded: I've installed Brad Choate's SpamLookup plug in for Movable Type. I'm running it alongside MTBlacklist to try to keep comment and trackback spam at bay. I've just tweaked Blacklist as per this forum comment to get... [Read More]

» SpamLookUp from Johnnie Moore's Weblog
For the technically minded: I've installed Brad Choate's SpamLookup plug in for Movable Type. I'm running it alongside MTBlacklist to try to keep comment and trackback spam at bay. I've just tweaked Blacklist as per this forum comment to get... [Read More]

» A little plugin before bed from Glimpse of a Grrl
I installed Brad Choate's SpamLookup tonight. It was an easy install and the interface is great. Now I just need some comments to test it. ;) I'll be installing it on the Homeport blogs tomorrow.... [Read More]

» ARGH! from Ego, Ego, Ego!
Today I deleted a legit comment by accident. It got mixed in with the spam I was batch-deleting during lunch. That's it. I will have some free time here before I leave to go home but after I'm off the... [Read More]

» Murmurs poetic on MT Plugins: MT-Blacklist 2.04b and SpamLookup. from GeminiGirl
Ah, Plugins, how do I love thee? Let me count the ways... I have waxed poetic about plugins before. I installed MT-Blacklist v. 2.01b last October, and Jay Allen was kind enough to respond in the comments of my post... [Read More]

» Spam, spam, spam, spam... from LilacRose
The past couple of days, I've been inundated with trackback spam attempts. MT-Blacklist has been blocking most of them (and... [Read More]

» SpamLookup - a review from Laura's Blog
Well, not a full review, but an update. Time of use: 3-4 weeks Rating: 4.8/5 stars It's been 3-4 weeks since install and today I got my first "moderated" comment. Yup. SpamLookup has blocked every spam, plus allowed me to blacklist a few IP's at the ap... [Read More]

» SpamLookup - a review from Laura's Blog
Well, not a full review, but an update. Time of use: 3-4 weeks Rating: 4.8/5 stars It's been 3-4 weeks since install and today I got my first "moderated" comment. Yup. SpamLookup has blocked every spam, plus allowed me to blacklist a few IP's at the ap... [Read More]

» SpamLookup - a review from Laura's Blog
Well, not a full review, but an update. Time of use: 3-4 weeks Rating: 4.8/5 stars It's been 3-4 weeks since install and today I got my first "moderated" comment. Yup. SpamLookup has blocked every spam, plus allowed me to blacklist a few IP's at the ap... [Read More]

» Upgraded to MT 3.16 from world in progress...
Upgraded to Movable Type 3.16. [Read More]

» Upgraded to MT 3.16 from world in progress...
Upgraded to Movable Type 3.16. [Read More]

» Installed SpamLookup from world in progress...
Well, it didn't take long for the next spam attack so installed SpamLookup. We'll see how it goes...... [Read More]

» SpamLookup for Movable Type from Neurotic Fishbowl
A few weeks ago I installed Brad Choate's SpamLookup plugin for and I now get virtually no comment or TrackBack... [Read More]

» SpamLookup for Movable Type from Neurotic Fishbowl
A few weeks ago I installed Brad Choate's SpamLookup plugin for Movable Type and I now get virtually no comment... [Read More]

» Another MT comment spam-fighting tool from Tech-Knowledge
Following the suggestion of Anil Dash from Movable Type and others, I've installed the Spam Lookup plug-in for this blog. It has been getting lots of good buzz so we'll see how it does in blocking junk posts and trackbacks.... [Read More]

» SpamLookup progress report from Tech-Knowledge
I'm impressed. I haven't had one comment or trackback spam get through since I installed SpamLookup on the Movable Type blogging platform I use. I don't want to say the problems are over but a look at the logs shows more than 200 comment and trackback ... [Read More]

» SpamLookup progress report from Tech-Knowledge
I'm impressed. I haven't had one comment or trackback spam get through since I installed SpamLookup on the Movable Type blogging platform I use. I don't want to say the problems are over but a look at the logs shows more than 200 comment and trackback ... [Read More]

» MT-Blacklist switch to mod_perl from Inchoate
MT-Blacklist is dog slow even on a fast machine running as a standard CGI script with (in our case), 3,500+ entries. The time spent is parsing the YAML blacklist; here about 10s. This 10s pause is passed onto commentators who... [Read More]

» goodbye ugly spam from Lynne d Johnson :: A Day In A Life
as most of us bloggers know, and especially we using MT, comment and trackback spam can be a crucial beyotch well i am proud to say no more - and i'm one who keeps all of my entries open indefinitely... [Read More]

» Stopping TrackBack Spam from gavinshearer.com
Spammers have discovered TrackBack. My TrackBack feature first got hit with spam a few weeks ago, and the trickle has turned in to a torrent - I awoke this morning to find eight (!) new junk entries, ranging from "online casino" to "phobia" to "tattoo ... [Read More]

» The engine room from Acid
Pretty much everyone with a blog faces the curse of comment spam. Brad Choate has just released SpamLookup, an uber-anti-spam plugin which plays nice with MT-Blacklist and which seems to just about kill comment and trackback spam. I'm very impressed... [Read More]

26 Comments

Basil Crow said:

Wow, Brad Choate is still alive!

Actually, I'd love to use TypeKey, but I don't even see a way to use it on your own site!

David said:

Just as a quick question, will the effectiveness of SpamLookup increase if it is run in conjunction with MT-Blacklist?

And, in your opinion, d'you think that SpamLookup would be more effective at removing spam in real-world conditions, as compared to MT-Blacklist?

Mark J said:

The last link to the project page is broken.

Neil T. said:

Brad, this is awesome. Thank you very much for it.

I echo David's question about compatibility with MT-Blacklist. This looks like a great plugin, and I'd love to install it, but I want to make sure first that it won't conflict with that other great plugin. Thanks!

Neil T. said:

Karl & Dave: Initial impressions from my end would suggest that this complements MT-Blacklist nicely.

Jay Allen said:

I think my review states it nicely... :-)

Simon Cox said:

As Mr Allen wanted - "Serious love for you Brad". This is an awesome tool - one we have been wishing for a long time! Many thanks from a weary community.

derwood said:

Dude.. This totally rocks.. Now I dont need to run my own DNSBL.. This is greatly appreciated.

Jamison said:

Jay, I don't think you could give much higher praise than you did. And certainly not to a person more deserving, you two are the bomb! (the bombs?!)

Hm. Since installing this plugin, I have not been able to receive incoming pings:

SpamLookup returned error: Moderating TrackBack ping for blog 10 since domain IP does not match ping IP for source URL http://virtuelvis.com/archives/2005/04/start-dot-com-newsreader; domain IP: 205.196.216.216; ping IP: 205.196.216.206 (TrackBack Ping I

The problem is that the trackback sent to moderation never ends up in the list of incoming pings. Inspecting the raw database shows that it never shows up there either.

Brad: I loaded SpamLookup this morning, and it totally rocks! I have had no comment or TB spam since its installation. You are the best. - Randy

randy said:

Brad: Could you perhaps expand on how we are to use the Passphrase Filters? For example, I have no idea to what "Form Field" is referring. I don't know what I am supposed to enter in that box. Also, you mention using Automatic Enkoderform for the passphrase value, but that requires creating a URL. In what context are we creating a URL, and where do we put it?

SL is working well, thanks for that. But could you expand on the Configuration Setup for those of us who are less than tech savvy with this? Maybe, please? :-)

Thanks, regardless _ Randy

Brad Author Profile Page said:

Randy, you should take a look at the wiki for the project. I've gone into more detail about the passphrase options there.

Brad Author Profile Page said:

And as for the Enkoder-- the "advanced" form at the bottom of the page lets you encode anything. Text, HTML, whatever. That's what I was referring to when I said "use the 'Advanced Form' at the bottom for freeform text encoding." :)

steffanie said:

Do the passphrase for trackbacks require that you have passphrase for comments? I only wish to use it on trackbacks and I tried reading the wiki but I still can't get it to work. look for example here:
http://www.steffanie.net/archives/2005/04/eftertanke.html

roland said:

Brad, any thoughts on this error after I installed the plugin:
Undefined subroutine &MT::Plugin::SpamLookup::extract_urls called at /kunden/homepages/34/d94705598/htdocs/MT/plugins/spamlookup/mt-spamlookup.cgi line 449.

ryan said:

each time i go to the ticket window and try to create a ticket i end up here. and my ticket doesn't get created....

Tim Bahula said:

Awesome. SpamLookup has stopped the trackback spam cold. MT-Blacklist was working... but not fast enough.

Mary said:

Damn that thing looks cool. Keep up the good work, Brad!

Neptunebaby said:

I am curious about how the "despam" works. I currently have close to 2000 spam comments pending that i would like to remove. I realize they aren't approved or showing up on my site, but I would still like them to be gone.

Also, do I have to have Blacklist installed for this to work or is this sufficient as a stand alone plugin?

Thanks much

The last link to the project page is broken.

Nermin said:

Dear Brad,

sorry for misusing your blog for contacting you on SpamLookup. I a using your plugin with my MT3.15 for a quite a long time and it was very successful. Until few days a go my readers start reporting difficulties with posting comments. They just want be accepted.

I try to open my log to read what happened, but after a long long time of opening it just appears a blank page!

I tried to look after some forum about SpamLookup, but all I found was the project site at spamlookup_dot_com and unfortunately I couldn't find any help there. I would have posted a new question with trace, but this "new question" button wasn't on the page.

Can I found the log file somewhere on my server also? Or could it be, that I have disabled logs during the installation or afterwards? Is this possible?

Thank for any remarks or help suggestions in advance,
Nermin

John said:

Spam is boring!!!

vertrag said:

Thanks for this document i`m search many days in the world wide web,but now i found this information on your site.

Thanks for help,thats the answer of all my questions i`v had


Thanks again

Timo

Spam Links webmaster said:

Brad,

Is SpamLookup still alive? The project page at projects/spamlookup is dead, so I'm not sure if the MT plugin or the DNSBL you were running are still active.

Regards,

Spam Links webmaster

About

This article was published on April 7, 2005 7:44 PM.

The article previously posted was MT-DSBL - Open proxy comment filter.

The next article is for:bschoate.

Many more can be found on the home page or by looking through the archives.

Powered by Movable Type