« September 2010 | Main | November 2010 »

October 2010 Archives

October 31, 2010

Crowdsourcing search relevance

How on earth do you try to disrupt the search space?

Search requires not only a big software system, but also a massive set of relevance data to help the algorithms make sense of the billions of pages on the web.

Bing and Google have hundreds of contractors that use web tools to refine this relevance data - classifying porn, spam, domain parks, ecommerce sites, fake 404's, markov-spam, official sites, and so on.

As a 20-person startup, we asked ourselves how blekko could assemble this essential data. Hire contractors? Use Mechanical Turk? Elance?

But - of course! - we know a much better way.... A way you can get orders of magnitude greater participation, while at the same time being very open about the process.

Let the public in.

We realized we could make web tools that let users sign up and help make the search engine better. If we opened up the process, we could not only get orders of magnitude more people involved than we could ever hope to employ, we could also create an open, accountable process around the search engine relevance data.

Not everyone has to participate for the model to work - most people don't edit Wikipedia, yet we have a vast encyclopedia which long ago dwarfed the closed Britannica.

But a small fraction of the web audience that does get involved can help make the search experience better for everyone else.

We're starting by letting users define their own vertical search experiences, using a feature we call slashtags. Slashtags let all of the vertical engines that people define on blekko live within the same search box. They also let you do a search and quickly pivot from one vertical to another.

I was looking for 2% cash back credit card. 1% cards are pretty common, but 2% are harder to track down. [cash back credit card] is a trainwreck of a spammed results any engine. So I made a blekko /money tag with the top 100 personal finance bloggers that I got from Kiplingers. Bingo, [cash back credit card /money] and I have great results.

Being able to go into a spammy category like health, personal finance, hotels or even lyrics and search just the best sites immediately uplevels the results. Trusted sources with no spam.

And our users are making tags we would never have thought of. One of our users created a /glutenfree tag, so you can search [chicken soup /glutenfree]. Another created a slashtag for user experience design sites, so now we have a great /ux tag.

We have a vision of curated algorithmic search that brings quality back to the web at scale, and involves the public to get there.

We're just getting started though, so stay tuned.

Cool searches which show off features:

cure for headaches
pregnancy tips
big island resorts
industrial design colleges
pan fried noodles
obama /date
global warming /conservative
bioinformatics /people
blekko /links
blekko /links /date
blekko /links /date /rss
techcrunch /seo
om malik /rank
techcrunch /seo

Read more:

A New Search Engine, Where Less Is More - New York Times

Start-Up Aims at Google: Blekko.com Taps Users to Narrow Results, Avoid Spam Sites - Wall Street Journal

Blekko, The “Slashtag” Search Engine, Goes Live - Search Engine Land

With Help from You, New Search Engine Slashes Through Spam - Wired

Alternative Search Engine Blekko Launches to Eliminate Spam in Search - Mashable

Blekko launches the biased search engine - CNET

Search engine Blekko to rely on the human touch - Reuters

Google And Blekko Head-To-Head: Blekko Lives To Fight Another Day - Search Engine Land

Power to the People: A new search engine, Blekko, uses human editors to promote quality pages and block spam content from its results. - MIT Technology Review

Interview of Rich Skrenta, of Blekko + Topix + DMOZ Fame - SEO Book

One Reason To Take New Search Engine Blekko Seriously - Business Insider

Blekko: The Newest Search Engine - PC Mag

"I recommend Blekko. It's the best out-of-the-chute new engine I've seen in the last 10 years, seriously." -- John C. Dvorak

About October 2010

This page contains all entries posted to Skrentablog in October 2010. They are listed from oldest to newest.

September 2010 is the previous archive.

November 2010 is the next archive.

Many more can be found on the main index page or by looking through the archives.

Powered by
Movable Type 3.33