« Science | Main | The Web »

March 27, 2004

WiFi Spamcasting

Mark my words. Get ready for something I’ll call “WiFi spamcasting”.

Imagine a mobile WiFi hotspot that use a technique like this to redirect all client web traffic requests to advertising. Just ads.

So imagine turning on your laptop and browsing to Google, only to see an eyeful of obnoxious, perhaps even pornographic ads? They might be coming from a laptop bag left behind some counter. Or perhaps it’s carried around by some spam-financed drone who has nothing better to do.

It will be a pollution of the WiFi hotspot pool. A cheap technique to reach the most tech-savvy folks out there. Expect these to turn up at convention centers, hotels, airports, airplanes — anywhere you might want a WiFi signal.

Of course, you can switch to any available WiFi service by picking the SSID you want from a list of available sources. But what if they also intentionally jam the legitimate WiFi signal(s) so that theirs is the only one that is reachable? I don’t put anything past these low-lifes. And what can you do about that? WiFi is unregulated, you know.

Just imagine the scope of this problem once WiFi broadcasts can reach the 10-15 mile range.

Spam finds a way, believe me. Just a matter of time before they ruin this too. I’m not suggesting we start regulating WiFi, but this is a serious concern for me.

March 19, 2004

A simple plan

PIF icon Can we all agree that no one is legitimately going to e-mail a “.pif” file as an attachment to anyone? Every mail server/client in the world should be deleting these things automatically, upon arrival.

While we’re at it, lets add “.exe” files too. The only legitimate EXE files that get mailed around are those stupid little games people are always forwarding to one another. They’re probably carrying spyware too.

And “.scr” files.

And anything else that is a binary executable.

Oh, let’s not forget those Word and Excel documents that carry virii viruses.

Shoot.

February 24, 2004

Where's that remote?

Where is that remote? A valid and common question most of us have probably posed at one time or another.

And yet, no good solution. Well, there probably is one out there for $19.95 or something, but I haven’t seen it. And chances are anything out there would involve me replacing my beloved TiVo remote with some substandard chunk of plastic.

So what to do? Well, I understand the TiVo folk are hard at work engineering Series3, so if it’s not too late, may I make a wee suggestion?

WiFi-enable the TiVo Series3 remote. Why? Let me count the ways:

  1. Two-way communication. You can press a button on the TiVo console to have the remote beep or make some kind of audible/visible response as to it’s location.
  2. Enough with the infrared already. That’s sooo ’80s technology. I’d like to be able to fire my remote without having to point it at the TiVo directly.
  3. You could send TiVo remote signals from your computer (including one to locate the remote). Change channels or turn things on and off.
  4. Program the remote to control other devices that aren’t supported from factory defaults.

I hope there’s more at work in the Series3 than HDTV. Thanks to the TiVo Hacks book, there’s already a nice wishlist of what people want to do with their TiVo’s. With HDTV’s resolution, it wouldn’t be out of the question to enable web browsing on the thing. Or reading e-mail. A small-footprint, wireless keyboard with builtin trackpad/trackball/thumbpoint device would be ideal for this. As if I don’t spend enough time on my couch as it is…

And here are some other thoughts I have related to the nextgen TiVo.

February 18, 2004

Spam filtering with qpsmtpd

We all know e-mail is broken. It’s become an arms race between spammers and the world to keep the stuff out of our inboxes. Well, my spam-prevention process got a big shot in the arm yesterday, by installing qpsmtpd, a drop-in replacement for qmail-smtpd.

qpsmtpd is another fine product by Ask Bjørn Hansen. It’s written in Perl. Quite modular and extensible through a plugin architecture, so it’s naturally a good fit with me.

At this point, if I get a spam e-mail, it’s pretty close to a miracle. Here’s what an inbound e-mail has to go through:

  • First line of defense is qpsmtpd itself, which runs a variety of checks as it receives each e-mail (all of these are provided by qpsmtpd out of the box, unless otherwise indicated):
    • “Early talker” check — if the smtp client starts dictating the e-mail message before qpsmtpd even responds to the HELO (or EHLO) command, the conversation is cut off with a “don’t be rude” error, since the client is talking out of turn and is most likely a automated spamming process that isn’t waiting on anything to do it’s business.
    • A check of unrecognized SMTP commands. This is good for rejecting spam sent through HTTP proxies (or at least so says the documentation). More than 4 of these and the connection is cut.
    • Check for a resolvable “from” host. Very effective for anonymous spamming, as it requires that the from host resolve. Yes, if Verisign reimplements Sitefinder, they will break this step.
    • rhsbl - Domain-based blacklist provided by rfc-ignorant.org.
    • dnsbl - Another domain-based blacklist, powered by Spamhaus, Mail-abuse.org and others.
    • From domain check. Simple user-defined list of unacceptable domain names.
    • Recipient check. Simple user-defined list of unacceptable recipient names.
    • Check for spam “HELO”. Rejects messages where the HELO command is from a user-defined list of domains (handles cases where spams announce themselves as coming from yahoo.com or aol.com, both of which never handshake like that).
    • Greylisting. Brilliant technique where e-mails are initially, but temporarily rejected. If the originating server retries the message after the greylisting timeout has passed, the mail will pass through. You know all those hijacked computers out there that are spamming? They don’t handle soft-bounces like this. Read more about greylisting here. (This is a 3rd party plugin for qpsmtpd, available here.)
    • Relay check. Prevents relaying.
    • ClamAV scan virus scan (signatures are updated daily).
    • SpamAssassin scoring.
  • After all this, e-mails are delivered to their inboxes. But wait— there’s more!
  • Whitelist/blacklist/challenge-response. I use Knowspam as my POP server, so it picks up the e-mail from my mail server, passing through any valid e-mails and sending challenge e-mails to anyone not already recognized. (I think there are qpsmtpd solutions for this too, but I like the way Knowspam does the challenge/response process, so I’m sticking with them for now.)
  • My Apple Mail junk filters are the last line of defense if all else fails. But so far, by this point, there’s nothing left to filter.

I’m considering enabling ClamAV as well which will scan for any virus content. I’ll wait until I get a viral message first though (not a concern either way since I don’t do Windows anymore, but would be helpful to others that use my server for e-mail). (ClamAV is now in the mix.)

If a spam message gets through after all the above, it probably means it’s being sent from an individual, using their own e-mail client and with pretty benign content that isn’t recognized as spam for whatever reason. They would have also had to go through the Knowspam challenge process which means they have a valid e-mail address. At that point I can just block the e-mail address and/or domain if I want.

I should also point out that since signing up with Knowspam back in August, it has reportedly blocked 22,401 spam messages for me. Thats over 100 a day. Since enabling qpsmtpd (and the greylisting plugin for it) yesterday, that number hasn’t gone up. Impressive.

My e-mail inflow has slowed to a trickle of it’s former volume. A bittersweet victory. I feel relief, but also a little lonely all of a sudden.

January 25, 2004

An Apple IIc

Apple IIc If you read my Newly Digital post, you would know that I wanted an Apple IIc 20 years ago. (This was introduced in April ‘84 — a few months after the original Mac.) So here we are in 2004, and eBay just so happened to have one available. So, I bought it.

$50.51 for a functional Apple IIc with color monitor, printer, a set of nice games (Flight Simulator, Infocom’s H2G2), Print Shop, AppleWorks. Even a genuine Apple mouse to go with it. Even EA’s Music Construction Set — the predecessor to GarageBand (ok, that’s a stretch). Quite a deal. Not bad for a computer that initially sold for $1,295 retail.

I don’t know what I’ll do with it. I’m sure someone’s ported Linux/Mozilla/MAME to it, but no… I’ll stick with good ol’ ProDOS. I just gotta find a copy of Lode Runner for it.

June 21, 2003

RSS 2 Dates and Such

Mark Pilgrim: History of RSS date formats.

Go read Mark’s points regarding RSS 2 and the use of the Dublin Core elements to get some context for what follows. If you’re really new to RSS and the issues around it, Mark has another page with the goods.

As a developer, I totally agree that ISO 8601 is easier to parse. It’s just a better way to represent a timestamp for computers to read. That’s a technical reason for using the Dublin Core date element, but not a terribly compelling reason to abandon native RSS 2 elements. And, honestly, “either way works, but we like doing it this way” is not a compelling reason either.

I don’t think that

<pubDate>

should be omitted in preference to the Dublin Core counterpart. Why? Because like it or not, as things are,

<pubDate>

is the defined tag for expressing publication date in the RSS 2.0 spec.

Having said that, I fully realize that

<pubDate>

is defined as an optional element. And the use of externally defined elements via namespace is allowed. But, I think it’s odd to rely upon external elements that offer identical functionality (albeit in a different form) as the declared specification.

A Developer’s Perspective

Let’s consider the task of RSS 2 implementation from a developer’s perspective.

If I were to write a new RSS parser (and believe me, I am not that stupid), I would first and foremost take the specification for RSS 2 and implement that to the letter. Every element that is defined there would be supported, regardless of whether it were optional or not. (Maybe it’s just me, but as a developer, the spec is the Bible. It has to be followed or things don’t work.)

If I were to stop right there without support for any of the existing namespaces, my parser would work, but it would be poor indeed. It should be complete by implementing the RSS 2 spec alone (for RSS 2 feeds that is). But a growing number of RSS 2 feeds today rely on the Dublin Core namespace to express common tags defined by the RSS 2 spec, so my parser would ignore those elements since it just implements the RSS 2 spec. For example, with the new RSS 2 template for Movable Type, any feeds it produces would be missing dates and category assignment in the eyes of my RSS 2 reader. Pretty significant, considering the date is the when of blogging.

I think Dave’s concern about all this is that the specification of RSS 2 is weakened by using alternative formats to replace elements it already defines. Let’s take this to the extreme: where every single element of RSS 2 is replaced by externally defined elements. Silly, right? And destructive to the goal of RSS — using the acronym Dave acknowledges: “Really Simple Syndication”.

The Publisher’s Perspective

Let’s look at this from the other angle. If I were completely new to RSS and wanted to implement a feed for my site, I would first and foremost take the specification for RSS 2 and implement that to the letter. I would use

<pubDate>

since dc:date isn’t part of the specification and not having any information on the optional namespace extensions, I would be ignorant of it.

Having followed the specification, I would have an expectation that anyone using a RSS 2 reader should have no problem processing my RSS 2 feed fully. Dates and all. But if there are some readers that don’t process

<pubDate>

and expect Dublin Core date tags instead, that would be a little frustrating to me.

Common elements like date and category should be part of the basic elements defined by RSS 2. And they are. And the publisher should look no further than the simple spec that Dave provides.

So What Then?

<pubDate>

should be used as the preferred date tag for RSS 2 feeds. If you want to use ISO 8601 dates, feel free — but

<pubDate>

tags should also be present for the benefit of all RSS 2 readers (whether they provide support for the Dublin Core extension or not).

Here’s another interesting tidbit. RSS 2 defines both

<pubDate>

and

<lastBuildDate>

elements. Some are using the Dublin Core to replace

<pubDate>

— but that tag looks like this:

<dc:date>2003-06-21T10:00:00-05:00</dc:date>

. This tag doesn’t describe what it applies to — one might wonder if it is meant as the publication date or the last build date or maybe it defines the author’s birthday? Who knows? The RSS 2 spec doesn’t define what the Dublin Core “date” tag is used for. The Dublin Core spec doesn’t define what the “date” tag means in the context of an RSS 2.0 document. So one can only infer. Hardly a specification.

And no, I’m not taking Dave’s side just because he linked to me yesterday.

June 20, 2003

4,558,302

So long…

4,558,302

…and good riddance. United States Patent No. 4,558,302 expires today. Better known as the LZW patent. This is the technology behind the common GIF file.

As of the 21st, U.S. companies and organizations that produce software and hardware products that use the GIF format (and other LZW-related technologies) will no longer be required to pay license fees to Unisys for doing so. However, LZW patents are still in effect for Canada (until July 7, 2004), the UK, France, Germany (expiring June 18, 2004) and Japan (expires June 20, 2004).

Will the expiration of this patent be the death knell for PNG? I don’t think so. PNG is still the graphic format we should be using (although it currently doesn’t support animation like GIF does — MNG is the answer for that). But the quality is superior and offers transparency. Sadly, it’s not as widely supported as GIF (talking about you, Internet Explorer).

See Also

May 31, 2003

Newly Digital

The following is my entry for Newly Digital — a distributed writing experiment (thanks to Adam Kalsey for bringing the project together). (Other entries on this theme.)

Continue reading "Newly Digital" »

Microsoft and AOL kiss and make up

So the other day, AOL and Microsoft reach an agreement and settle their browser suit. MSNBC declared “Browser War Over” on their front page at the time and led with this article. CNET covered the story too, asking “Is this the end of Netscape?” A very good question, but I’m not so interested in the answer. I don’t use Netscape (I use Mozilla). Do you? Does anyone?

Continue reading "Microsoft and AOL kiss and make up" »

May 2, 2003

Collaborative Filtering

I’m on the hunt for information regarding collaborative filtering. You know, systems like Amazon.com uses for recommending what you might like based on your activity and the likes and dislikes of others.

I’m especially interested in any open source implementations. A constant-time algorithm similar to Eigentaste would be a plus.

If you have any leads, information or insights to share, please do either by comment or e-mail.

Continue reading "Collaborative Filtering" »

Sideblog Feed

    Error: gmmktime() expects parameter 3 to be long, string given