« Friends Make Search Better | Main | Burning Spam! »

Introducing the Spam Clock


www.spamclock.com

I consider myself a glass half full kind of guy, but it's hard to remain optimistic about the future of the World Wide Web. I think it's fantastic that my kids have access in real time to almost every piece of information and knowledge in the world. But ever since we started working on Blekko, I've become exposed to the dark side of the Internet.

Scratch below the surface of all this great information, or in our case dig deep below the surface, and it is shocking what is happening to the Internet. Millions upon millions of pages of junk are being unleashed on the web, a virtual torrent of pages designed solely to generate a few pennies in ad revenue for its creator. I fear that we are approaching a tipping point, where the volume of garbage soars beyond and overwhelms the valuable of what is on the web. Look at what has happened to email: Microsoft estimates that 90 percent of the mail that passes through its hotmail servers is spam.

What happened to email was the result of very powerful economics. Spammers and con artists discovered they could reach a massive audience for pennies. And this scale of audience essentially guaranteed a very small but profitable return. Today the economic incentives for web spammers are even more lucrative than email spam and almost guarantee a continued blizzard of trash on the web.

Web spammers simply have to create pages on the web and sit back and let search engines send them money. Current search engines have abandoned any attempt to enforce even the slightest modicum of quality control. Revenue is guaranteed if a page can draw a click.

The result is a global sweatshop workforce cranking out millions of pages of web trash. I fear we are looking at the very scary future of the web in the job postings at Mechanical Turk. Researchers recently reviewed job postings there and found that 41 percent of all jobs offerer over a two month period were aimed at recruiting workers to create spam. Most of these jobs offered folks a measly dollar a page. Some paid as little as 5 cents. But all these jobs are being filled and the spam gets spewed out.


("The most infamous girl in the history of the Internet")

Consider that in 2000 there were about 7 million hosts on the internet offering essentially all the content on the web. In 2010, the number of web hosts has soared to 250 million. How many of these 200 million plus hosts offer legitimate content? A small fraction. The rest is spam.

Which brings me to my larger point. This spam on the web is creating REAL problems that are affecting much more than our ability just to find information.

The energy and other costs for crawling, storing and serving this trash is soaring. I saw a recent estimate that 15% of the world's energy consumption in 10 years could go to support Internet usage. A fair amount of that energy is being burned by the thousands upon thousands of servers at incumbent search engines. Making search greener by weeding out spam could have a significant impact on energy consumption.

The problems and challenges of spam to the entire world are going to get worse. As the online economy continues to grow at double digits compared to stalled growth for the offline economy, the incentives for spammers get even more lucrative.

That's why we've created the world's first Spam Clock. This clock is going to record in real time the amount of web spam that is being spewed out. The clock is designed to bring greater attention to this growing problem. While it is illustrative more than scientifically accurate, it is truly indicative of the soaring spam problem.

Finally, what can we do about this? Honestly, we think our search engine can be an important solution but we need your help. If we can together create a search engine that is a curated resource of the best trusted sources on the web, we can do a great deal to reduce the economic incentive for creating spam. Spam operators won't even offer that nickel on Mechanical Turk if the chances are pretty good that a human editor will never include that page in the search database.

So we'd like to invite web searchers everywhere to help us clean up the web. It can be done. If we can just organize the best sources of information for the top 1000 search verticals we will drastically improve the web experience. And we will immediately create the first ever disincentive for polluting the web.

Please join us.

Read more:

Blekko launches Spam Slock to keep pressure on Google (Danny Sullivan)

The Spam Clock is live (Marksonland)

Comments (8)

So... Is the Spam Clock a clever link bait and thus a kind of spam - or is it meant for more?

Just kidding folks! Great article :-)

Great post Rich. Hearing a lot of chatter about this issue but so far this article is the most succinct and on-message piece I've come across.

It seems that while people in the industry are fully down with this, more needs to be done to make the case to average internet users, many of whom simply don't see that there's a problem (just like most people didn't see a problem with Yahoo's portal homepage in pre-Google 1998!)

Damion

Eugenia May-Montt:

Phew! Thanks God someone is thinking clearly about this situation and finding a solution to it. I like this article because laid out the facts and offer an alternative on how to solve the problem.

Nick:

The income from spam is often from advertising, say for example google adsense.

If the advertisers (collectively) made it clear that the ad views, and clicks from spam sites were not worth paying for, then the income will dry up and this problem will end.

(yeah, that doesn't kill trying to rig search rankings to make some sites rank above others, but it seems to kill the idea of promoting a site with little but advertising)

Yeah, but collective action like that is hard. The best way I can think to encourage it is a financial one on google's side: allow bidding different amounts for what are likely spam sites, and ones that are not....

[I think the advertiser would have an easier time deciding "I don't want my ad appearing on xyz site" than google would for them though]

Hey Rich,

As the administrator of a Startup, I cannot even begin to tell you of the kind of spam that we face on a daily basis. I think the Spam Clock is a fabulous idea, someone needs to clean up the system, and others online need to support that someone.

We had posted a blogpost on spammers earlier(out of sheer frustration & inability to tackle them), please check the story at the link below http://whatsnewonthenet.tumblr.com/

Also, today, we have given adequate coverage on our website to your Spam Clock in order to spread awareness about it. The link is http://www.whatsnewonthenet.com/Internet-News.php

We would be happy to help in any other way, too.


Thanks

NuNet
Website Administrator

Thanks for this. I have been thinking and speaking much about this for quite some time. Being a Technology Professional from Silicon Valley now with 5 years in the Philippines I see so many people trying to just churn out utter crap. I am against 'fake plastic trees', spam sites and spam blogs. Quality content is king, relationships are the foundation, offer compelling value, not rehashed crap, ride the cluetrain anyone?! thanks for this keep it coming.
-Kevin Leversee

Nice post Rich,
I think with the increase of SEO and online 'marketing' companies the amount of spam is going to increase exponetially now. I think it's going to be tough, but it's good to see that someone is putting out the call for non-spammers. You just have to make sure that there is no easy way of exploiting the system, as any site/engine/directory with a potential SEO benefit will be spammed.

Spamclock is an excellent ideea but I am interested in finding out how did you reached these metrics? I mean, where do you take your numbers of new spam sites from? Is that number just randomnly updates itself to match 1 million per hour or are there some more rigorous mathematical calculations behind it?

Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)

About

This page contains a single entry from the blog posted on January 6, 2011 12:00 PM.

The previous post in this blog was Friends Make Search Better.

The next post in this blog is Burning Spam!.

Many more can be found on the main index page or by looking through the archives.

Powered by
Movable Type 3.33