In Part 1 I took a “from first principles” look at the spam problem, and concluded that the only way to actually solve the problem was to make people pay to send e-mail.
Now, it’s time to look at what I mean by that—because there are almost as many ways to implement “pay to send” as there are ways to implement filtering.
This is going to be a bit more technical than part 1. I’m going to assume you know basically how SMTP e-mail works. If not, there are tutorials available.
A great many words have been written on the subject of e-mail spam. Effort has been poured into all kinds of technological measures against it. In my view, many of these efforts have been a waste of time, because they have failed to address the fundamental problem of spam.
To explain my thinking, I’ll start with some basic statements:
Your attention is a valuable resource. If you doubt this, you need only look at the amount of money spent on advertising in an attempt to acquire your attention.
Therefore, your inbox is a valuable resource. Many people, perhaps most people, now check e-mail multiple times a day. In fact, according to some surveys college students spend more time on the Internet than watching TV. They check their e-mail inbox more than they look at ad breaks.
SMTP e-mail allows anyone to send mail. There’s no centralized registration required in SMTP; there’s no control over the growth of the SMTP e-mail network. While some servers restrict which SMTP clients may connect to them, there’s essentially no control over who sends mail, as it’s always possible to open a new web e-mail account, buy a new ISP dial-up account, or whatever.
SMTP e-mail is free for the sender. Sure, many people pay for their Internet access; but once you have an Internet connection, sending e-mail basically doesn’t cost you anything—it has marginal cost.
Now, let me re-cast those four statements:
We have unrestricted access for anyone in the world to use arbitrary amounts of a valuable resource.
Can you think of any case where there has been a system like that, and it has worked? I can’t. The canonical example is the tragedy of the commons, but there are plenty of others, including the Cambridge ‘Green Bike’ scheme and the overfishing of cod.
In order to avoid a “tragedy of the commons” situation, we need to alter the situation so that one of the statements above is no longer true. Let’s go through them again and consider our options.
[2004-03-02] Well, pobox.com’s new spam filtering system picked up 2,982 spams in the last week, and 1 false positive. And that wasn’t really a false positive—it bounced a newsletter from sudhian.com because they’re apparently too incompetent to set up their MTA to provide a proper HELO hostname, so their SMTP request was invalid (as per the RFCs). I sent them e-mail to warn them, and it bounced because their newsletter reply address was invalid too. I’ve forwarded the bounce back to postmaster, what’s the betting they’re violating that RFC as well?
I have no problem with bouncing mail from anyone that incompetent, and 99.99% accuracy is plenty good enough, so I’ve switched the filters over to full automatic, and now they’ll reject the spam e-mail during the SMTP attempt. It won’t even reach my second-line adaptive bayesian filters.
[2006-03-09] About two years on, and the spam rate remains more or less constant: 2,840 spams in the last 7 days.
Contrast this with the claim from the FTC that the CAN-SPAM Act has been effective, and that consumers are receiving less spam than they used to.
It’s quite possible that consumers are receiving less spam, but from my numbers it seems clear that the amount of spam being sent hasn’t gone down. Instead, filtering for the average person is getting more effective.