« Hank's Career Advice | Main | Getting Stuff Done: Activation and Method »

Random AOL user search tool

I had been using the dontdelete site to search the AOL user search data which was released last August. AOL's release of this data generated a storm of controversy, which lead to a bunch of staff resignations for the folks involved over there. Still, their idea that this data is a really valuable research tool for the world was correct, and folks at the big shops like AOL, MSN, ASK, and Google all have quality data like this to work with. The rest of us didn't, but now we do. Thanks AOL guys. You took one on the chin for us and we are grateful!

Unfortunately the dontdelete tool doesn't seem to work anymore. Smells like someone loaded it into mysql and the database isn't running anymore or something. So I hacked up a quick little replacement for the purpose I was using it it for, browsing a random user session. I based this on my joke code, so it would be fast. It's fast! 2-3ms to return a random session from the 577,663 available.

Why is this useful... When you get past the voyeuristic fun, I've found that it's actually really hard to think up representative random searches to try out search engines to see how they do. I've never been very good at this; someone sends me to a new search engine, and I type 'skrenta', and then I go blank. Mike typed 'britney spears' when I showed him AskX. The problem is that 'britney spears' has been hand-optimized at Yahoo, Google, MSN and ASK, because there are guys just like us working at all of those companies. It's supposedly a popular query category, it's obviously monetizable, and it's easy to license the AMG or Muze data and make them better. But I have this nagging suspicion that 'skrenta' and 'britney spears' aren't serving me very well to take effective soundings of a new engine's quality.

Hence my random search tool. Real users type such gonzo stuff into the search box. You can't make this stuff up, which is the point. I included fresh-window links to a basket of other SE's, so you can see how the query does on different engines.

My all-time favorite so far: [will anastasia hurt my pregnancy]

Easy for a human to correct! You know what she means ("anesthesia", i.e. what are the risks of pain meds during pregnancy, getting an epidural, etc.) But no search engine can do that phonetic correction yet based on the greater context of the sentence. Maybe Powerset is working on stuff like this.

Give it a try here:

Skrenta's random AOL user search tool

TrackBack

Listed below are links to weblogs that reference Random AOL user search tool:

» SearchCap: The Day In Search, March 6, 2007 from Search Engine Land: News About Search Engines & Search Marketing
Below is what happened in search today, as reported on Search Engine Land and from other places across the web:... [Read More]

» What do people search for online? from Nouveau by Zaw Thet - Interesting New Stuff
Rick Skrenta wrote a great little tool that pulls back what people were searching for online using AOL. As he pointed out, when AOL first accidentally released this data there was a huge fuss. There are some really disturbing searches that people do...... [Read More]

» Blake's Blackberry Boredom tools from Skrentablog
Blake told me he was using my aol random user query from his Blackberry while he was out somewhere waiting for his wife or something, since he found the queries interesting/amusing. But the page was too heavy to really work... [Read More]

About

This page contains a single entry from the blog posted on March 4, 2007 10:10 AM.

The previous post in this blog was Hank's Career Advice.

The next post in this blog is Getting Stuff Done: Activation and Method.

Many more can be found on the main index page or by looking through the archives.

Powered by
Movable Type 3.33