Skrentablog

This is a great post with some great ideas. Well done.

I do have to ask, though: wouldn't you say number five, "forget about interface innovation," files in the face of number six, seven, eight, and nine -- which seem to be all about interface innovation? :)

Other than that seeming inconsistency, I love this post. Great job.

I admire your chutzpah. You've got moxie!

Easier said than done, of course. But imagine the glory for the company that would topple Google.

"And gentlemen in England, now a-bed
Shall think themselves accursed they were not here,
And hold their manhoods cheap whiles any speaks
That fought with us upon Saint Crispin's day."

@Jeff: I disagree. Point 6,7,8,9 actually confirm point 5. They say: don't try to innovate, learn why google beat the shit out of altavista, but make sure you do LOOK different. The last part is all about recognition which is not that much usefull for a google competitor as for any site.

Well, I suppose that's one way to look at it. I saw several things in six, seven, eight, and nine that seemed like suggestions for better interfaces:

> Forget about asking users to do anything besides typing two words into a box.

Google requires me to do more than this. It requires me to push a button, and specifically to choose between showing the results and "feeling lucky."

> Extra work from users is going the wrong way. You want to figure out how the user can do even less work.

Less work than Google, right? That would be an interface improvement. :)

> You need to do something they can't copy, not because they're not technically capable of doing so, but because of the constraints of their legacy interface on Google.com.

In other words, you can't do the exact same interface as Google, or you'll be forcing upon yourself the same legacy constraints that Google has.

To be clear again -- I thought this was a great post. My personal opinion is that Google has quite a good interface overall, but there is room for improvement. The only think I didn't like here was the suggestion that a potential competitor shouldn't try to improve upon Google's already-good UI.

"Users do not click on clusters, or tags, or categories, or directory tabs, or pulldowns. Ever."

What's your source for this? Sure, it kind of sounds right -- doesn't mean it's true.

Your point 9 seems contradictory to some of your other points: "Your product must look different than Google in some way that is deliberately incompatible with their UI" vs. "Your results need to be in a single column".

I think about the list as a set of design constraints. Some of the items appear to be in conflict, but I believe you have to satisfy them all to have a viable attack approach. I've thought of a some potential solutions, but it's more interesting to hear what others think. I also see a lot of projects out there that don't conform to these rules, and I think those efforts are going to be ineffective.

Rick, I totally agree with what you're saying and love the new ideas. Goliath is not insurmountable.

All I know is this.. just because they're Google.. and enormous.. and control -to a large degree- the online advertising world now and forseeable future, ContextWeb will not quit. It's a start (and a shameless plug - bear with me, Rick) but come sign up for the beta of our new ad network.

http://www.contextweb.com/publishers/

We're putting the web publishers in charge - guranteed CPMs AND you can still work with Google if you want to. No joke.

Rich, that was a fun post. I know I enjoy playing the thought experiment of "What's the best way to compete with Company X?". I'd be curious to hear your thoughts on how to beat other companies in the search/internet industry, or even other companies in general. It could make for an interesting series of posts.

Nice post, great direction. There are several weak points in their armor. Let's start with one - crawling. It has hardly changed since the early days of the Internet and it actually does not scale. To see why, you can crawl about 1 million pages a day with 1Mbps dedicated bandwidth with a good custom crawler or Nutch (bandwidth is all you need, crawling is not CPU intensive). So with 1 Gbps you could crawl 1billion pages/day i.e. about three weeks for the entire Internet. You can buy 1Gbps crawling bandwidth wholesale for as little as $5-6K/mo, Google probably spends more on coffee than that. So why aren't they crawling more often then? Because of robots.txt and other limitations (deep web dynamic stuff) in other words that's why search engine crawling does not scale today no matter how much money/bandwidth you throw at it. There are ways of resolving this (hint: people are working on it :) Beside this weakness, there are others ...

Rich,

Believe it or not, I can tell you how to rock (and possibly even cripple) Google in just two words (just don't tell Matt; he'd never believe it anyway ;-)

Paid Match

And I can prove it in just 15 minutes:

First, visit our own US Dept of Labor's GovBenefits.gov site and take a run through their automated benefits matching service/system...

Second, with the list of programs on your screen, imagine now that; instead of some subset of the 400+ gov programs this service draws from displayed to you; there are 50-100 perfectly matched product and service results...out of 100's of thousands of products and services being offered by many thousands of advertisers for you to choose from...

Finally, imagine now that these match engine results are arranged in monetary order, similar to what we see with the paid search ads...

Remember the old GoTo.com all-commercial results pages...and how popular they were with the advertisers and public (though the pundits and much of the media and many others--at least initially--fried Bill Gross and his team for daring to offer a "commercial" search service)?

Well; paid match is like paid search, only much better...because it strips out problematic search and replaces it with structured match...

So what do you think, Rich--does it make more sense to target us directly...than it does to target the words we type into little search boxes?

MatchTo.com's got the overview; pending patent #11/250,908's got the details...

I've never understood how Google can insist that their infrastructure costs are actually an impediment to any startup. Sure, it costs a lot to serve 200M queries a day, but 200M queries usually come with a lot of money attached.

Like Borislav mentions, it doesn't cost much to create a full-surfaced-web search index. Most costs for today's search engines scale with traffic, not index size. You can build an index very cheaply (at least, the kind we know about today).

Even weirder is when Microsoft says that their datacenter skills are somehow an advantage. My understanding is that a) MS is probably not in 1st place when it comes to infrastructure management, and b) they don't have any stupendously popular applications to run on that infrastructure. Maybe I'm missing something subtle...

> Users do not click on clusters, or tags,
> or categories, or directory tabs, or
> pulldowns. Ever.

Basically agreed, though I think some portion of users will be happy to use tabs or pulldowns to choose the type of search engine they want, for the time being. Even if you had the perfect natural text processing search engine, some users would still like to know its possibilities, which you can communicate using an explorative interface. "Oh, I didn't know I can find *music files*..."

Good point Phillip. Google Images is 7% of total Google traffic, so that is a case of a tab being used. But I think of trying to find a song or some stock photo as a completely separate task than the general search/refinement process (e.g. a context where a refinement like 'rich skrenta dmoz', 'rich skrenta topix' might be offered by something like Clusty).

Also, the Ries theory would argue that if you're going to offer music search, that's got to be your one thing, and you're going to muddle the branding if you try to do a bunch of different stuff. So tabs aren't only this usuability issue where you're not going to get sufficient user trial on any of the non-defaults for takeoff, but they're also compromising the branding.

I've never understood how Google can insist that their infrastructure costs are actually an impediment to any startup. Sure, it costs a lot to serve 200M queries a day, but 200M queries usually come with a lot of money attached.

Indeed. I'd go a step further and say that (as a startup) you don't need to be able to compete with Google on scale. All you need to do is be able to demonstrate a markedly better search experience than Google offers. If you can pull that off, you will have no problem getting the funding you will need to build the infrastructure for scaling. There will be people lined up around the block to get a chunk of "the next Google" at the ground floor.

AS to the tab thing, I've been thinking about this for a long time. While I couldn't agree more about the basic laizness of the searching public, I also question the way the whole thing works and often think to myself, "it doesn't have to be this way".

As personalization and user tracker come more and more into play, I think just grabbing your IP and assuming you would prefer data that I can identify as being in close physical proximity to you is presumptuous. BUT, if I don't assume and I can't expect you to type in more than two words or click a drop down box, then what other way could I offer to re-sort results based on your personal preference at the moment?

I think carving out a new niche is a very good idea. I don't know that I agree a niche is the best chance of building a Google killer but I certainly agree that focusing on a niche is your best chance of minimizing risk and maximizing return. But if let's say we build a fishing portal for example. A guy comes to Youfish.com and does a search for vacation. Without some system for refining the search, how do I determine relevancy for this guy? Put local stuff on top by zip code? Put the cheapest on top? The site that the most people before you stayed on the longest?

Without me somehow engaging you into letting me know something about what YOU think is relevant, and I know I'm competeing with Google which has already won that public perception of relevancy, how do I keep the UI simple, one column AND increase my odds of impressing you with how relevant I am? I'm kind of hoping you at least a little wrong about the whole no drop down, categories, or tabs thing.

I see how to build the engine, gather the data and make it fast. The problem has always been what to put on top and why? AND is my reason the same as yours for wanting to see something on top? If I can figure out how to make you comfortable, (I've decided to call that engage you), enough to tell me you want fishing poles sorted by local, cheap or something else, my chances of impressing you by giving you that perception of being more relevant than google goes WAAAYYYYY up.

One more thing I felt was missing from that list. I think #1 should have beenbuild your revenue model first THEN launch the service.

Agree with some of the thoughts about how to topple Google from its position on Search. I think that is the wrong leg of the table to tackle though.

The most effective way to take Google down is to take out their monopoly on online advertising. That's where all the cash is.

When I was at Microsoft in 96 the big competitor to worry about wasn't Netscape it was Corel because they were threatening to lower the pricing power of the cash cow Office.

You know what's missing?

13. Focus. Do not get distracted by idea du jour, or you wind up with some half assed jobs.
14. Perseverance. If you fail, try try again.

As I commented on another blog recently, I think the fundamental premise of battling Google on their search turf is wrong. Search is yesterday's war, personalization (which can apply to search, but also many other things) is tomorrow's war. Google didn't get to their current position by taking on dominant incumbants such as Microsoft head on . . .

So thinking more broadly than search, here are at least four key Google vulnerabilities that can be exploited individually or collectively:

1. They are optimized for the Plain Old Web (POW). In other words, sophisticated, large scale algorithms that address "dumb" content. They are not optimized for smart content that contains embedded meta-information, usage information, etc.

2. Everything they do must scale massively immediately for it to matter to them. Google loves to think this is a big advantage for them but it will also significantly inhibit innovation over time.

3. Their architecture and predisposition is fundamentally antithetical to P2P. Joost is already exploiting this chink in Google's armor, and many other models could as well.

4.Google is nowhere in the enterprise -- they can't handle enterprise security issues, they can't leverage social information because it is not yet created in most enterprises, etc.

On the one hand I'm quite sympathetic to this idea. On the other, how many consumer tech products have successfully targetted a niche a la beer and ketchup?

I guess this has happened in PC hardware (Apple and Voodoo appeal to different niches) and in videogame publishing.

But a lot of areas have either one dominant player (auctions, mp3 players, online video) or a number of very similar competitors (web email, job search sites, printers, monitors). Niche-ification doesn't seem as prevalent as you might think it should be.

Any counterexamples?

Mike: here are some examples off the top of my head. I'll try to think of others.

JDate vs. Match.com
StubHub vs. eBay
Zillow vs. Realtor.com

The only site out there that has consistently replaced Google for me in my day-to-day web usage is Wikipedia. For factual information, I no longer bother with Google (even though Wikipedia results come up first most of the time). They aren't a search play, but there are some commonalities in the mode of usage (looking for factual information, for example). Still they've come closest for me (and many others) to replacing Google.

Greg:
That's a good point. Same for me.

Rich:
You're right, these are good examples. The niche is usually smaller than the original, but still worthwhile.

OK, so here's one (relatively dull) shot at it:

Google has a basic interaction cycle of type-query, hit-enter, click-one-result-of-ten. I almost never click on a link lower than 3. This is a pretty laborious process considering that my initial text almost completely dictates the result.

Meanwhile, my Mac has a launcher utility called Quicksilver that shows a "launch nominee" as I type each character. When I type enough characters to determine the program I want, I hit enter and the current nominee is launched. It's a lot faster than the Google interaction cycle.

You can appeal to power users by making a client-side search engine that does Quicksilver-style type-completion for the web index. No need for two page fetches and a mouse click. Just type characters until your target comes up and hit enter.

For freely-distributable Wikipedia data, no need to even bother going to another site. Just show the text directly from the cached copy.

Yeah! That's cool. That would get rid of user work (the main page -> results & back cycle) and just autocompletes the entire web as you type... sweet. :-)

Rich, how does this address the points in your post?

http://livesearch.alltheweb.com/

@Mike Cafarella, Rich Skrenta:

You just described Google Suggest. That's also available in the Google Toolbar.

Shhh, Google will hear you talk and actually improve!

Other than that, great ideas :)

Neil -- Google Suggest, but with the site listings themselves instantly appearing, not just the probably search phrases.

Anon -- that livesearch thing is sicko cool!

A number of commenters have pointed out that some of the items on my list seemed to conflict with some of the others. Here's what lead to me coming up with that list.

Sometimes whole categories of startups are just done. There was a time to do a browser startup, you could get market share or acquired or something, but it's over now. You can't do a browser startup now, for instance. You can't do a music player like winamp. You can't do an OS startup. You can't do a PC desktop app. There were lots of PC app startups once, you could have done something like PowerPoint and gotten acquired by Microsoft, but that era is over.

Is search that way too? Is it so completely covered by Google that it's curtains for anyone trying to get a foothold in that category? Many people think so. If so, we should move on and find some new software to write, and VCs shouldn't fund any more search engines. Maybe we really do have to work on clean-tech or enterprise 2.0.

But if it is possible to take on Google, what would the attack look like? How could you recognize potential success if it showed up at your door?

So I set about making a list of requirements for what I thought an effective attack would look like. Those are the requirements I came up with; I think you have to satisfy them all to have a shot. Or if you break one of the rules, you'll have a really good reason. A reason so disruptive it actually invalidates our entire frame of reference for thinking about the problem.

#6: Forget about asking users to do anything besides typing two words into a box.

yea, like "hotels.com"

;D nmw

Google made search useful for the masses.

Yahoo = directory

Google = search

You = whatever is better than directories and searching

And concerning the OS, well, wait for a truly distributed OS.

@Mike Cafarella

Sounds like you're describing Inquisitor, which is like Spotlight/Quicksilver built into the search window in Safari. And it's free!

http://www.inquisitorx.com/safari/

Addressing the comments about personalization, and smart search that is can presort the results with more information about the user:

what if the google killer started as an add-on to a social neetworking site like linkedin or myspace?

These have pre-built profiles and content(blogs/profile/forums/
photos/etc.) that can be indexed.

When you search the web, your search results can be optimized based on the things you've already shown are important to you.

dave

I would like to agree if your gonna go after someone go all out.Often imitated but never duplicated is a thought that comes to mind when it comes to google.Microsoft and those big companies need to think on how to be different,marketable,and easier to use.If they can succed in these three areas presto they got a chance.I also would like to agree with Dave I think you're onto something there.

Big Arnie

Part 2: it doesn't need to be one company/startup, it can be a whole bunch of them, collaborating together, openly. See my blog post, I've been working on a new distributed protocol to do just this :)

What do you think about a search engine for ads? IF i need to buy something cheap, remotely, or etc..... I mean I have not seen an small ad indexing search engine( personal adds, ex. realestate, electronics...). What do you think? should that be possible technical wise and legal wise?

Riding on other brands, would be a good way to enter the market.

You build a search solution for social networks, including web search and site-wide search, maybe some crafty personalization from all the data they have available. Then get a few social networks to use it. Then you have a huge market share.

Hi Rich.

Do I think G can be unseated? Yes, absolutely. I say that having been at the keyboard and watching intently while The Almighty AV lost its dominance in a matter of months.

>VCs parrot
>get a stick

Remember MSN Champs? I think the answer lies in resorting to something like that --except it wouldn't be the namebrand corporate seo nicks (who spend most of their time posturing for clients). No, it'd take a team that knows something of the dark alleyways of search marketing to get things rolling ...sort of the 'Dirty Dozen'(if you're into old Lee Marvin movies).

Rick,

Hold on! Did Matt Cutts really say, "I'd be curious to hear your thoughts on how to beat other companies in the search/internet industry...?" Is this the same Matt Cutts who, when interviewed recently by Richard MacManus of Read/WriteWeb stated that it was Google's corporate policy to NOT discuss ANY alternative search engines BY NAME?

Here's the key part of the interview; Richard is speaking, "One of the most popular posts this year on R/WW was one called The Top 100 Alternative Search Engines. What are some of the "alternative" search engines that have most impressed you lately? Or if you can't mention names..."

Why couldn't you mention names, Matt??

BTW, that's my list of The Top 100 Alternative Search Engines (Alternatives to Google) Richard is talking about (but Matt won't talk about it).

Google can't mention names? Why not? Anyone that wants to can go to Read/WriteWeb (www.readwriteweb.com) and see me discuss Google AND the alternative search engines BY NAME - all 100 of them!.

Or you can go to the comments section and see readers comment on other search engines - and Google - By NAME. There are even some search engine CEOs talking to each other, BY NAME, about the features of their unique projects.

Why won't Google participate in the debate? Why was Google only willing to talk about itself? Don't you find it odd that the largest search engine in the world should be afraid of discussing these tiny alernative search engines?

Here's the link to the whole interview: http://www.readwriteweb.com/archives/interview_with_matt_cutts_next_generation_search.php

Very interesting reading...

I have just developed a search service called fisssh! (http://www.fisssh.com), and from the beginning our mission was to contribute to the 'search experience' rather than getting all consumed with 'beating' Google.

Whatever way, you look at it - at some point Google has helped everyone of us and has earned it's number one position.

No new search product is going to 'beat' Google or earn the trust of a mass audience over night... Fancy bells, whistles and algorithms that try to be too clever are not going to bring success any quicker.

Anyone trying to take on Google is in for the long haul...

Just my 2 Cents!

Thanks for bringing many ideas together.

Google has got an (quite|very) interesting start so far. So too is the Internet.

But if we look far down the tube we may see history in the very form of the future.

Hence it's equally valid to see Google Search as a very crude tool in the infant days of the Internet. The world has helped Google to be a fast growing giant,just because at that point in the space-time continuum we seemingly didn't have anything better. Q-DOS was another example.

The point is far into the future we will be using thought assistant tools much more advance than today's search engines. Some of us can apply reverse engineering from a preferred point of such a period.

Why fight Google? I agree with you Skrenta.

Sure, search results need to be made more relevant. However, rather than fighting with budgets which can't compete with those the giants have- either you:

1. Innovate and they'll buy you out for nice sum, or
2. Different approach. By this I mean not new algorithm.

Think: Google or Yahoo or MSN. they all have a market due to a service of use they provide for free. Their email (or other service) users will always see that SE and no other when they return to use their preferred service.

The answer lies in service.
Google innovated the best PPC model and this is their core income. This IS their business.
Search is merely the service to attract the market for their advertisers.

To go beating them you'll need to build your market:
A. get a huge publisher network showing your search engine not the giant's one. especially get large portals to join. if they're not contracted to a SE... they'll want good compensation NOW, not in 10 years. so you'll need simultaneously -
B. a huge advertiser network. They'll want to see a good reason why they should use your system. they'll want:
C. access to loads of targeted visitors. (publisher network & your own URL visitors).

In my humble opinion, if you want to gain substantial market share from any of the SE, the approach is to provide a better and innovative advertising monetization model.
A better PPC with incentive to click, with 100% fraud free, with non-aggressive and a non-intrusive visitor (end-user) experience and where everyone gets something out of it. Where we grow a huge & accurate detailed DB of end-user records (they have good reason to register- you can be sure), where it un-clutters websites and yet they can monetize nicely with cleaner look and useful offer. especially it's HOT and attractive for community type sites and huge portals. and much more...

Important is to remember, that behind search there are services attracting the searcher or visitor.
Web2.0 trend is to go free-services.
Communities are exploding in growth.
The new kind of search evolves, looking like personal recommendations and bookmarks.

Better search (and better bookmarking filtering out spam/ ads) will evolve from here. But these communities are the new visitor-glue, the new free service potential.
But they need good monetization model. otherwise they turn into crap services and junk posting sites.

My model is targeted for $1B-2B annual PPC gross rev. with 36% net.

(I have innovated a financial model on these lines and a full bizplan. If you'd like to partner & sponsor beta creation you may reach me via the contact you'll find on one of my sites: www.HaircuttingSecrets.com/products.html - only serious investors please, with credentials)

But, sure, the above is not pure search. Search will evolve for ever. Always someone will improve and have his algorithm purchased.
If your interest is pure public service then do an algorithm. Sell to Google. or donate it.... tomorrow spammers will develop a bypass and you can write another algorithm for cleaner results. and so on....

Search - whichever best JV - can be integrated into my model. Mine into theirs.

SE giants want new models to better grab online-ad-market share from Google. From eachother. Mobile, video, services of each kind. the key lies in services. FREE SERVICES. With strong glue factor and incentives. and in monetising with ads. in a better way.
That's where the money is.

######################

Hi to Jeremie Miller.
I have been working on this very idea in Brisbane, Australia and have even designed flash mapping which helps the user find the discount products in there area. Some how don't think it will be as big as google but one can dream.
regaeds john.

Very thoughtful analysis and suggestions. I don't think we have any company like this yet, either they're still in stealth mode, or Google already acquired them! It's indeed a challenging task. Yahoo was once a good match but now no longer. Microsoft already got a bad rep that's hard to fix. Google eventually will also go down this path. That, maybe the right timing for their killer to appear, not now.

Also, a strong social networking that buzzes your product far and fast. Anything that is titled "Google killer" is exciting and yet strongly criticized.

Better search (and better bookmarking filtering out spam/ ads) will evolve from here. But these communities are the new visitor-glue, the new free service potential.
But they need good monetization model. otherwise they turn into crap services and junk posting sites.

Your statement in point 8 - "Your results need to be in a single column...." doesn't make any sense. Now, that Google has a three column layout! Haha!!

Hello,

I believe that bing.com will be the next SE. They rule when you search for video. No ads too.. It is the best.

Thanx
Jason
http://www.searchsocialsecuritynumber.us/