To understand Wikia Search you have to go back to the launch of the Nutch project in 2003:
Meet Nutch, the open-source search engine. Open-source applications are unusual in that the code upon which the software runs is not owned by a private, commercial company but rather bound by a simple license that allows anyone to use, modify, and even profit from it free of charge, as long as they pledge to contribute their own innovations back into the code base. Because of this, anyone will be able to access Nutch's code and use it to their own ends, without paying licensing fees or hewing to a particular company's set of rules.
Perhaps more important, Google takes a "trust us" approach to search; they say they don't skew their PageRank formula to favor certain sites, but we have no way of knowing for sure. With Nutch, the indexing and page-ranking technologies are all open and visible; you can check them yourself if you have a problem with your page's ranking. Just as Linux has taken on Windows, revolutionizing the rules of search-engine development and distribution, Nutch could pose a threat to Google and other search giants. Interestingly, early Nutch development was supported in part by Overture's R&D division, and an Overture official sits on the Nutch board.
"Search is interesting again," says Doug Cutting, a founder and core project manager at Nutch. Cutting, whose development chops were honed at Xerox (XRX) PARC, Excite and Apple (AAPL), is building Nutch (that's his toddler's all-purpose word for "meal") with a small team of engineers based around the country. But Cutting says they hope that once Nutch is loosed on the world, tinkerers from Romania to China to Palo Alto will help build it into a robust platform, in the spirit of Linux or Apache (which has garnered more than 60 percent of the Web-server software market in just the last couple of years).
The thought I had at the time was, the open source model is great, but the problem with search is that without a sponsor to pay for racks full of machines and gigabits of bandwidth, eager would-be developers are stuck. You can't develop a search engine on a laptop sitting in the university cafe.
Thus there is no web-scale version today on Nutch.org, of course. But Nutch has succeeded in smaller scale deployments, such as indexing university intranets. Basically competing in the enterprise search space, against commercial products such as Thunderstone and the Google search appliance. Universities are more open to tinkering with the open source Nutch / Lucene alternative and so have been early adopters there.
Enter Jimmy Wales. Wikia is the web-scale sponsor that Nutch didn't have when it launched in 2003. Wikia has 1,000 servers now and can afford the multi-gigabit bandwidth bill. They're providing the hosting platform which Nutch has been starved for to let contributors show up and advance Nutch to industry-level.
Yes, the site looks like someone was thrilled to get it to compile for the first time the night before launch. The appalled reactions are understandable given the expectations and high-profile PR.
Look past that.
Early open source projects often look grim. If you go onto sourceforge and find some promising 0.1 project, you know what to expect. I agree with Markson that the mistake here was in Wikia's positioning of the launch. But I don't think that's necessarily going to have long-term effects. Ultimately they just need a small handful of developers and contributors to help move the rock uphill. And then iterate.
And don't count out the power of the open source model. Giving all of the academic researchers who only get to test their experimental ranking algos on little clusters a functioning web-scale search platform could enable real progress. Check back in 2 years and I'll bet that Wikia Search is going to be a valid competitive alternative search site. Certainly a long shot to unseat Google, but at least a worthy alternative.