Crowdsourcing search relevance
How on earth do you try to disrupt the search space?
Search requires not only a big software system, but also a massive set of relevance data to help the algorithms make sense of the billions of pages on the web.
Bing and Google have hundreds of contractors that use web tools to refine this relevance data - classifying porn, spam, domain parks, ecommerce sites, fake 404's, markov-spam, official sites, and so on.
As a 20-person startup, we asked ourselves how blekko could assemble this essential data. Hire contractors? Use Mechanical Turk? Elance?
But - of course! - we know a much better way.... A way you can get orders of magnitude greater participation, while at the same time being very open about the process.
Let the public in.
We realized we could make web tools that let users sign up and help make the search engine better. If we opened up the process, we could not only get orders of magnitude more people involved than we could ever hope to employ, we could also create an open, accountable process around the search engine relevance data.
Not everyone has to participate for the model to work - most people don't edit Wikipedia, yet we have a vast encyclopedia which long ago dwarfed the closed Britannica.
But a small fraction of the web audience that does get involved can help make the search experience better for everyone else.
We're starting by letting users define their own vertical search experiences, using a feature we call slashtags. Slashtags let all of the vertical engines that people define on blekko live within the same search box. They also let you do a search and quickly pivot from one vertical to another.
I was looking for 2% cash back credit card. 1% cards are pretty common, but 2% are harder to track down. [cash back credit card] is a trainwreck of a spammed results any engine. So I made a blekko /money tag with the top 100 personal finance bloggers that I got from Kiplingers. Bingo, [cash back credit card /money] and I have great results.
Being able to go into a spammy category like health, personal finance, hotels or even lyrics and search just the best sites immediately uplevels the results. Trusted sources with no spam.
And our users are making tags we would never have thought of. One of our users created a /glutenfree tag, so you can search [chicken soup /glutenfree]. Another created a slashtag for user experience design sites, so now we have a great /ux tag.
We have a vision of curated algorithmic search that brings quality back to the web at scale, and involves the public to get there.
We're just getting started though, so stay tuned.
Cool searches which show off features:
cure for headaches
big island resorts
industrial design colleges
pan fried noodles
global warming /conservative
blekko /links /date
blekko /links /date /rss
om malik /rank
A New Search Engine, Where Less Is More - New York Times
Start-Up Aims at Google: Blekko.com Taps Users to Narrow Results, Avoid Spam Sites - Wall Street Journal
Blekko, The “Slashtag” Search Engine, Goes Live - Search Engine Land
Google And Blekko Head-To-Head: Blekko Lives To Fight Another Day - Search Engine Land
One Reason To Take New Search Engine Blekko Seriously - Business Insider
Blekko: The Newest Search Engine - PC Mag
"I recommend Blekko. It's the best out-of-the-chute new engine I've seen in the last 10 years, seriously." -- John C. Dvorak