« blekko coverage and twitter glow | Main | Get a cool blekko slashtag man (or woman) tee shirt »

Crowdsourcing search relevancy

How on earth do you try to disrupt the search space?

Search requires not only a big software system, but also a massive set of relevance data to help the algorithms make sense of the billions of pages on the web.

Bing and Google have hundreds of contractors that use web tools to refine this relevance data - classifying porn, spam, domain parks, ecommerce sites, fake 404's, markov-spam, official sites, and so on.

As a 20-person startup, we asked ourselves how blekko could assemble this essential data. Hire contractors? Use Mechanical Turk? Elance?

But - of course! - we know a much better way.... A way you can get orders of magnitude greater participation, while at the same time being very open about the process.

Let the public in.

We realized we could make web tools that let users sign up and help make the search engine better. If we opened up the process, we could not only get orders of magnitude more people involved than we could ever hope to employ, we could also create an open, accountable process around the search engine relevance data.

Not everyone has to participate for the model to work - most people don't edit Wikipedia, yet we have a vast encyclopedia which long ago dwarfed the closed Britannica.

But a small fraction of the web audience that does get involved can help make the search experience better for everyone else.

We're starting by letting users define their own vertical search experiences, using a feature we call slashtags. Slashtags let all of the vertical engines that people define on blekko live within the same search box. They also let you do a search and quickly pivot from one vertical to another.

I was looking for 2% cash back credit card. 1% cards are pretty common, but 2% are harder to track down. [cash back credit card] is a trainwreck of a spammed results any engine. So I made a blekko /money tag with the top 100 personal finance bloggers that I got from Kiplingers. Bingo, [cash back credit card /money] and I have great results.

Being able to go into a spammy category like health, personal finance, hotels or even lyrics and search just the best sites immediately uplevels the results. Trusted sources with no spam.

And our users are making tags we would never have thought of. One of our users created a /glutenfree tag, so you can search [chicken soup /glutenfree]. Another created a slashtag for user experience design sites, so now we have a great /ux tag.

We have a vision of curated algorithmic search that brings quality back to the web at scale, and involves the public to get there.

We're just getting started though, so stay tuned.

Cool searches which show off features:

cure for headaches
pregnancy tips
big island resorts
industrial design colleges
pan fried noodles
obama /date
global warming /conservative
bioinformatics /people
blekko /links
blekko /links /date
blekko /links /date /rss
techcrunch /seo
om malik /rank
techcrunch /seo


New York Times: A New Search Engine, Where Less Is More

Wall Street Journal: Start-Up Aims at Google: Blekko.com Taps Users to Narrow Results, Avoid Spam Sites

Search Engine Land: Blekko, The “Slashtag” Search Engine, Goes Live


This page contains a single entry from the blog posted on September 14, 2010 5:48 PM.

The previous post in this blog was blekko coverage and twitter glow.

The next post in this blog is Get a cool blekko slashtag man (or woman) tee shirt.

Many more can be found on the main index page or by looking through the archives.

Powered by
Movable Type 3.33