Nov 19

Just got a robocall that said "is sending you a hey cosmo blast, press 1 to hear the message". That’s right, no indication of who, it just started with the word "is".

For obvious reasons, I didn’t push 1.

Google searches suggest that the company responsible is http://www.heycosmo.com/

On the off chance someone I know tried to use that site to send me a message: it failed.

My money’s on someone trying to use it to spam, though.

May 28

Once upon a time, back in the ancient history of the Internet–before the 1990s–domain names were carefully controlled and regulated. A single organization controlled each top level domain. If you wanted a domain name, you had to meet their requirements.

Often the policies enforced were quite picky. If you wanted a .uk domain name, you were required to actually be in the UK, for example. If you wanted a .org domain, you were required to be a non-profit organization. To be in .net, you were expected to be a network access provider or ISP.

A lot of people disliked the bureaucracy involved in domain registration, and objected to the fees charged. And so it was decided that the domain name system would be opened up. There would be many domain registrars for each major top level domain, all competing to give the best price and service. Anyone would be able to register a domain, with minimal bureaucracy. Domains would be bought, sold and transferred in a perfect Free Market.

At first, things looked good. The cost of registering a domain dropped rapidly. Rather than having to fax paperwork around and get signed documents from company directors, you could just register online with a credit card for whatever domain you wanted.

However, it quickly became clear that domains could have value. A small proportion of Internet users (around 5-10%) don’t seem to understand search engines or bookmarks. They find things by guessing domain names and typing them in. As a result, people found that domain names an idiot would guess first ended up with traffic, purely by existing. Suddenly instead of having to advertise your web site, you could buy a domain name that people would randomly visit anyway, and get instant traffic with no work required.

Domains like “sex.com”, “computers.com” and “cars.com” suddenly became very valuable, changing hands for large amounts of money. Some people weren’t very happy about it, but still, there was nothing wrong with it really.

Unfortunately, there were headline stories of domain names changing hands for millions of dollars. And suddenly, there was a gold rush. Everyone with a modem hurriedly registered every domain name they could think about.

This was a major pain. If you wanted to set up a web site, it became almost impossible to find a simple domain name that hadn’t been registered already. Almost all of them were unused, just a whois entry and nothing more, but if you approached the owner their eyes would light up with dollar signs and they’d demand extortionate rates for their “valuable property”.

Still, the situation was somewhat self-correcting. It did still cost $50 or so to hold a domain for a year, so eventually when nobody turned up to offer $100,000 for it, the holder would let the registration lapse and you’d be able to pick it up for $50.

Then someone invented banner ads. Suddenly, those unused domains could be used to make money. Domain registrations were still dropping in price, and there were ad companies who would pay you $0.01 each time you served up an ad to someone. $10 a year for a domain, and all you needed to do was show ads to at least 1,000 idiots who typed your domain in at random, and you’d break even.

And so suddenly, the Internet filled with junk web pages filled with ads and nothing else. There are now multi-million-dollar companies whose primary business is hoarding domains and filling them with content-free crap. Domain spam is now so mainstream that companies like Google actively encourage it.

The next step was obvious. Sure, you could think of a domain name that other people would be likely to guess at random, but most of those were already registered. So the domain spammers began watching the lists of domains that people failed to renew. So now, if a widely used open source project fails to renew its domain name, the page will suddenly be replaced with a spam site full of affiliate ads.

Not everyone appreciates ending up on a domain spam page, however. Plus, if your page doesn’t look like total spam, you might get search engine traffic, and boost your profits further. Hence, the new trend is automatic content generation.

Some domain speculators take the unsubtle approach, and simply rip off content wholesale. If you have a web site with significant readership (as measured by, say, technorati), someone will likely set up a spam site which copies the text of each post you make, covers it with ads, and re-posts it to one of their hoarded domains. Sure, it’s copyright violation, but the chances of getting caught are slim, and so long as you pick on personal web sites the chances of anyone going after you with a lawsuit are slim too.

(I don’t think it has happened to me yet, but if I include a made-up word that doesn’t appear on the web, like spozquak, I should be able to do a Google search in a month or two and see if anyone’s copied it.)

However, again thanks to the free market, there’s now a market for software that can generate moderately convincing looking content. You’ve seen it in spam e-mails, and now it’s being used to fill the web too. The first generation used random text generation, but now more sophisticated “auto content generator” software uses web feeds to pull in text, chops the text into individual sentences, and then recombines them based on keywords.

(So I guess I should clarify that spozquak is a great alternative to viagra, cures mesothelioma from asbestosis, and helps you make money at home.)

While the web was filling with crap, the domain name registrars kept competing in their free market. As the supply of new unregistered .com domains dried up, they had to think of new ways to pull in customers. The solution: trial periods. You can now register a domain name for a 5 day trial, see if it pulls in any suckers, and if not you don’t have to pay for it.

You can probably guess what happened next. Someone wrote software to repeatedly register domains for trial periods, automatically.

And so we arrive at today’s web, the ultimate result of applying unconstrained free market economics to the problem of naming web sites. It’s a world where every name you can think of is already registered and filled with spam, often by someone who isn’t even paying for the domain. A world where if you’re away on holiday when your domain name expires, it’s immediately filled with spam. A world where web searches return hundreds of pages filled with spam designed to look like content, ripped off from other people’s web sites.

Of course, there are a couple of things we could do that might help ameliorate the problem. They’re just utterly unacceptable to the free market faithful who make up the Internet’s core audience.

The first is this: Do not allow domain transfers between third parties.

You bought a domain? Great. You want to sell it? Can’t. I mean, you can’t sell your home address, your postal code or your telephone number, so why should you be able to sell a domain name?  Your friend wants the domain? Fine, you cancel it, he registers it for the standard price.

If you could sell telephone numbers, you’d see rampant speculation there as well. If you moved to Austin and wanted a 512 phone number so friends could call you without paying long distance fees, you’d probably have to buy one at auction for a few hundred dollars. Or if you were in Massachusetts and wanted one of the old 617 numbers so you’d look like a long-established business, you could end up paying thousands of dollars. But the phone company doesn’t allow reselling of phone numbers, so the problem doesn’t occur.

(It’s worth noting that you can sell toll-free numbers. And sure enough, you get rampant speculation in that chunk of the phone number namespace, with most of the good ones already taken.)

The second way to help reduce the damage caused by the free market in domains is to resurrect an idea from the 80s: that your domain registration is voided if you don’t actively use the domain. And by “use”, I mean more than simply putting up a blank page of ads.

I can tell that people are already sharpening their pitchforks and lighting their torches, but which is worse: a domain name system that doesn’t support your religious belief that a free market is the best solution to everything, or a free market domain name system where you can’t actually buy any domains you want and everything is full of spam?

May 21

InfoUSA is a list broker, a company that aggregates personal data and sells it to telemarketers and catalog sales companies. The New York Times reports:

InfoUSA advertised lists of “Elderly Opportunity Seekers,” 3.3 million older people “looking for ways to make money,” and “Suffering Seniors,” 4.7 million people with cancer or Alzheimer’s disease. “Oldies but Goodies” contained 500,000 gamblers over 55 years old, for 8.5 cents apiece. One list said: “These people are gullible. They want to believe that their luck can change.”

So InfoUSA actually sells lists of suckers deliberately selected for their gullibility. You might be wondering who buys these lists. Well, the NYT investigated.

InfoUSA sold [one list] dozens of times, to companies including HMS Direct, which Canadian authorities had sued the previous year for deceptive mailings; Westport Enterprises, the subject of consumer complaints in Kansas, Connecticut and Missouri; and Arlimbow, a European company that Swiss authorities were prosecuting at the time for a lottery scam.

[...]

Records also indicate that infoUSA sold thousands of other elderly Americans’ names to Windfall Investments after the F.B.I. had accused the company in 2002 of stealing $600,000 from a California woman.

Between 2001 and 2004, infoUSA also sold lists to World Marketing Service, a company that a judge shut down in 2003 for running a lottery scam; to Atlas Marketing, which a court closed in 2006 for selling $86 million of bogus business opportunities; and to Emerald Marketing Enterprises, a Canadian firm that was investigated multiple times but never charged with wrongdoing.

The story goes on to reveal that according to internal e-mails, InfoUSA knew some of their customers were scammy, but continued to sell them lists of sick and/or gullible elderly people to exploit.

Update: InfoUSA have put out a press release giving us their side of the story. My summary: “We’re not selling lists of suckers any more, we sold that part of the business. Plus, the authorities didn’t find us criminally liable, and anyway it was a long time ago.”

Oct 19

There’s a new service out there called PayPerPost. Basically, you get paid for posting ads in your online journal.

So far, so ho-hum. One thing that makes this one a bit different is that the ads aren’t separated into their own section alongside your postings, like Google AdWords; rather, the postings themselves are the ads. Furthermore, buyers get to dictate the wording of the links.

In addition, the question of disclosure is left entirely open. Maybe all your postings are ads; maybe some of them are. Maybe you tell people, maybe you don’t. It’s up to you.

I decided to take a look at what the result was like. In the forums I found some people who were pimping their web sites; here are some URLs.

Reading the above is an interesting experience. Sometimes it’s blatantly apparent where the ad is. (In these quotes, underlining shows where the links were in the originals.)

I really need a Caribbean vacation. It’s time to stop dreaming of going somewhere like this and just start to plan and save for it. [...] Warm Islands.com is a perfect place to read about all the things I can do when I get there.

I just know a friend of mine needs to reduce cholesterol in his diet. I think this is the reason he gets sick so often, and feels winded so easily. I’m going to have him try Vasacor an all natural cholesterol supplement.

Another thing that I used to be big into a couple of years ago, but kind of let go by the wayside is taking women’s vitamins. [...] Osteo Essentials is clinically shown to support bone protection - which to me means will help strengthen them. I want to promote and develop strong bones now before it’s too late.

I don’t think any human being ever uses the phrase “…is clinically shown to…” in conversation. Sometimes it’s not so clear, though:

After lugging my laptop bag around all day for three days and seeing other people with their wheeled laptop cases, I’m starting to think I need one. My new laptop is lighter than my previous one, but it’s still damn heavy, especially when walking through the enormous hotel here from my room to the business areas.

Check out Tumi at Luggage Online. Isn’t that bag sweet? It’s got room for everything: laptop plus all my paperwork in really nice organized compartments. I want it!

On the one hand, the author had already said she was attending a show in Las Vegas. On the other hand…

Here’s a quote from a posting which, to me, demonstrates the problem with the whole thing:

This time last year…we were caribbean bound! I had already been on one cruise, and was about to embark on another. In September 2005, we went on an adults online cruise with a few other couples, my sisters, a brother-in-law, and an adult nephew. [...]

During our day in Jamaica, we visited a beatiful garden at the top of a hill overlooking the city of Ocho Rios. After that, we headed to Dunn’s River Falls - a must see! You can literally climb up the side of the mountain by walking up the falls. [...]

I think the best way to visit the Caribbean is by cruise ship - and if you need help choosing the right cruise for you, look no further than these cruise ship reviews. Our 7 night cruise was with Carnival, on one of their newest ships, the Carnival Victory.

I started out reading it as reminiscence, triggered by her noticing that it was a year since her last cruise vacation. It starts to sound kinda interesting, approaching a travelogue. But then suddenly, you hit what looks like blatant paid linkage. Does she really think cruise ships are the best way to visit the Carribean, or is she just being paid to say so? Maybe the whole September 2005 cruise is fictitious, planted at the request of the advertiser in order to seed the idea of taking larger family groups on cruises. Are the sisters, brother-in-law and nephew real? If so, why don’t they have names?

The thing about trust is that once you lose it, it’s hard to get back. Once you realize someone has lied to you, you tend to view everything else they say with suspicion. I have a hard time understanding why anyone would want to read a personal web site where you had good reason to believe the author was lying a large proportion of the time. Then again, even Jason Fortuny has fans.

Jan 22

In part 1, I enumerated the approaches to spam eradication I was aware of, and explained my conclusion that the only approach which will work is an economic approach. In part 2 I discussed various options for tackling spam economically, ending with the one I think would actually be acceptable and useful: attention bonds.

Now I’ll run through (and shoot down) a few of the objections commonly brought up when the possibility of involving actual cash in e-mail sending is raised.

Continue reading »

Jan 22

In Part 1 I took a “from first principles” look at the spam problem, and concluded that the only way to actually solve the problem was to make people pay to send e-mail.

Now, it’s time to look at what I mean by that—because there are almost as many ways to implement “pay to send” as there are ways to implement filtering.

This is going to be a bit more technical than part 1. I’m going to assume you know basically how SMTP e-mail works. If not, there are tutorials available.

Continue reading »

Jan 22

A great many words have been written on the subject of e-mail spam. Effort has been poured into all kinds of technological measures against it. In my view, many of these efforts have been a waste of time, because they have failed to address the fundamental problem of spam.

To explain my thinking, I’ll start with some basic statements:

  1. Your attention is a valuable resource. If you doubt this, you need only look at the amount of money spent on advertising in an attempt to acquire your attention.

  2. Therefore, your inbox is a valuable resource. Many people, perhaps most people, now check e-mail multiple times a day. In fact, according to some surveys college students spend more time on the Internet than watching TV. They check their e-mail inbox more than they look at ad breaks.

  3. SMTP e-mail allows anyone to send mail. There’s no centralized registration required in SMTP; there’s no control over the growth of the SMTP e-mail network. While some servers restrict which SMTP clients may connect to them, there’s essentially no control over who sends mail, as it’s always possible to open a new web e-mail account, buy a new ISP dial-up account, or whatever.

  4. SMTP e-mail is free for the sender. Sure, many people pay for their Internet access; but once you have an Internet connection, sending e-mail basically doesn’t cost you anything—it has marginal cost.

Now, let me re-cast those four statements:

We have unrestricted access for anyone in the world to use arbitrary amounts of a valuable resource.

Can you think of any case where there has been a system like that, and it has worked? I can’t. The canonical example is the tragedy of the commons, but there are plenty of others, including the Cambridge ‘Green Bike’ scheme and the overfishing of cod.

In order to avoid a “tragedy of the commons” situation, we need to alter the situation so that one of the statements above is no longer true. Let’s go through them again and consider our options.

Continue reading »

Dec 16

Spam now accounts for 93% of all e-mail by volume, and 6% of it complies with US Federal laws.

Mar 02

[2004-03-02] Well, pobox.com’s new spam filtering system picked up 2,982 spams in the last week, and 1 false positive. And that wasn’t really a false positive—it bounced a newsletter from sudhian.com because they’re apparently too incompetent to set up their MTA to provide a proper HELO hostname, so their SMTP request was invalid (as per the RFCs). I sent them e-mail to warn them, and it bounced because their newsletter reply address was invalid too. I’ve forwarded the bounce back to postmaster, what’s the betting they’re violating that RFC as well?

I have no problem with bouncing mail from anyone that incompetent, and 99.99% accuracy is plenty good enough, so I’ve switched the filters over to full automatic, and now they’ll reject the spam e-mail during the SMTP attempt. It won’t even reach my second-line adaptive bayesian filters.

[2006-03-09] About two years on, and the spam rate remains more or less constant: 2,840 spams in the last 7 days.

Contrast this with the claim from the FTC that the CAN-SPAM Act has been effective, and that consumers are receiving less spam than they used to.

It’s quite possible that consumers are receiving less spam, but from my numbers it seems clear that the amount of spam being sent hasn’t gone down. Instead, filtering for the average person is getting more effective.

Jan 04

I’d always wanted to see a dam, and as dams go, they don’t come much bigger than the Hoover Dam. Named after Herbert, rather than the more infamous J. Edgar, it’s possibly the most impressive piece of structural engineering of the 20th Century. Hoover gets the credit because as well as being a politician, he was a former engineer, and arranged the contracts between 7 different US states which would allow the project to go ahead.

We arrange a dam tour for under $20 apiece. The bus picks us up from the hotel, and transports us through sprawl and desert to the dam site. Once there we are given a couple of hours to do the official tour and wander around.

The dam itself is one of those objects that’s so big that you can’t perceive how big it is. It’s “won’t fit in a photograph” big, for starters—if you’re close enough to see detail, you’ll never find a wide angle lens wide enough; if you’re far enough away to get it all in the shot, it’ll just look like a small brick wall. Nevertheless, I tried to take a few photos of the outside which I could stitch together into a bigger image. On the left you can see the dam wall, with people and vehicles just about visible along the top. On the right you’re looking down at the turbine rooms 221 meters below; you can just about see a maintenance van parked down there.

Water enters the dam via the inlet towers, two in Nevada and two in Arizona. The dam itself is on the border between the two states, which follows the Colorado river.

The turbine rooms are pretty impressive, and much quieter than I expected. The foreground object is one of the turbine wheels. In spite of the dam’s massive scale, the power output is a modest 1430 MW, less than a quarter of that output by the Grand Coulee Dam in Washington State.

The generated electricity ends up inconveniently at the bottom of a gorge, and has to be transported back up the hillsides. Unfortunately, the local rock is rich in iron ore, so cables can’t be run through or across it. Hence the walls of the canyon are clustered with strange angular pylons which lean over and carry the cables through the air to the generators.

The dam took three years to construct, and was built under budget. To achieve that, men worked in horrendously unsafe conditions, braving 60 degree temperatures in summer, working in tunnels filled with clouds of carbon monoxide, and rappelling the canyon walls to spend entire days working dangling from the end of a single rope tether. Though the official death toll was somewhere between 96 and 112, they only counted people who died on site; anyone whose death was slow enough that they could be moved to a hospital first got excluded from the statistics. Memorials on site commemorate the deceased.

The dam as a whole is beautifully decorated in art deco style. The marble floors inside the buildings have iconic circular murals based on Native American symbols meaning “power” or “energy”. Even the public lavatories (men in Nevada, women in Arizona) have polished brass doors and inscriptions. Finally, two winged figures sit above Oskar Hansen’s star map, which provides astronomical measurements that should enable any future civilization or extraterrestrial archaeological dig to date when the dam was constructed.

One final detail is off to the sides of the dam: the spillway tunnels, two emergency flood control channels. In the event that Lake Mead begins to rise to a dangerous level, the excess water can be drained through these massive pipes. The spillway doesn’t look like much in the photograph, but it’s vaguely menacing in person—after all, it’s big enough that you could shove a Boeing 747 down it if you snapped the wings off, and there’s a delightful 200m drop once you get inside. In water flow terms, each spillway could easily carry the entire flow of Niagara Falls, or 5.6 million liters of water per second, traveling at 190 km/h at the bottom!

Of course, dams are controversial things. Some view them as ecologically damaging. Yet in the case of the Hoover Dam, it’s not so clear. Initial attempts to irrigate southern California had led to widespread flooding, and the accidental creation of a vast inland lake now known as the Salton Sea. The Colorado River began cutting through the California desert at over a kilometer a day, and threatened to form a new giant canyon. The system of dams now in place make the river controllable, and have stopped the massive erosion—yet at the same time, the dams mean that the Salton Sea has turned into an ecological disaster area.

Returning from the dam to Las Vegas, we get a chance to see the city in all its glory. It’s really quite amazing, mile after mile of urban sprawl in the middle of a desert, with mountains or tall hills on all sides. Construction seems to be pretty much continuous; if legalized gambling in other states has hurt Nevada’s economy, it doesn’t seem to show.

The bus drops us off at the Hilton, which was Elvis’s home-from-home up until his death. We head for the hotel diner and have an Elvis-style meal, as we hadn’t had time for lunch. I decide on a tuna melt, which turns up in fried white bread with fried onion rings. We eat at the counter. It’s possible to play KENO at the counter too; illuminated boards behind us animate the results of each game. KENO is basically like the lottery, except it has more options for what you can bet on, and you can lose every couple of minutes instead of only once a week.

Next we head for “Star Trek: The Experience”, our real reason for visiting the Hilton. It’s really expensive, but if you’re a Star Trek fan (or the spouse of a Star Trek fan) it’s just one of those things you have to do.

The first section is a kind of Star Trek museum, a complete Trek timeline from the present day onwards. Every few meters there are glass cases with artifacts from the appropriate shows or movies—a model of the Phoenix, a type II phaser, the Nomad space probe, and so on. As sara puts it, “This is the geekiest thing I have ever seen.”

After that you find yourself herded into the “experience”. I won’t say anything about it, because I don’t want to spoil the surprises, but I will say that if I have any complaint to make it’s that it all happened so rapidly… still, the admission fee lets you go around as often as you want, or until you give yourself a neck injury.

The gift shop is a big disappointment. I’ve always wanted a TNG-era science tricorder, but there’s nothing that cool or tasteful. The best toys are the burbling tribbles, and I resist the temptation—after all, it wouldn’t make sense to just have one. You can also stop at Quark’s Bar to have something to eat or drink. We’re still full of grease, so we take look around and leave.

It’s dark outside as we walk to the Stratosphere Casino and Hotel. It’s the tallest building in Las Vegas, and at 350m it’s close to the height of the Empire State Building.

As sara waits, I queue and get tickets so we can go up to the observation deck. From there we are treated to a marvelous view of the entire strip. If you follow the trail of bright lights, you can make out all the big casino hotels. The green one that looks as if it’s at the end is the MGM Grand; Mandalay Bay is just to the right of it in this photo.

For those who enjoy gut-wrenching terror, there’s a rollercoaster on the top of the Stratosphere which lets you get thrown around above a 300m drop, and a new ride called the “X-Scream” which dangles the rollercoaster car over the edge of the track as if there’s been some kind of horrible engineering failure. Personally, I have no real fear of heights, but I also have no real desire to develop one, so we give the rides a miss.

Back down at ground level in the casino, I had noticed a set of SPAM slot machines! We decide I have to sneak a photo of sara playing the SPAM machine, so we can send it back to her relatives near Austin, Minnesota, home of the SPAM factory. Hopefully hilarity will ensue.