Skrentablog

Thus began a six-month self-examination of why, exactly, our product sucked, and what we could do to un-suckify it.

As CEO I immediately rejected suggestions to reinvent the whole site as a myspace or digg clone, or any of the other fads du jour. I don't believe that you can win by making a clone of something else. That violates one of my rules of branded web products, which is basically that there can be only one of everything.

That would also be throwing the baby out with the bathwater. We had drawn a hyperlocal audience of millions of visitors spread over thousands of local city pages on our site. No one else had ever achieved this. We knew these visitors had shown up because they wanted to connect in some way with their town, online. But we weren't delivering the goods. We were leaving them unsatisfied.

We had many assets to draw on -- aggregation and AI technology, our recently launched local forums, content agreements with the AP, Reuters, Tribune, and 50 other top news organizations. Plenty of funding and engineers and seed traffic. If we couldn't somehow use these assets to build a great site, well the board should scrap our butts out of there.

Brand Therapy

This was a painful process. We crammed the entire company into a room, but no rah-rah speech this time. Instead we treated ourselves to a brainstorming session about why the site was lame. It was not fun.

We did the full marketing playbook. Focus groups with the mirrored glass and video cameras. On-site surveys. Telephone surveys. Accosting people on Caltrain and doing A/B surveys with paper mock-ups (no kidding). Therapy sessions with brandologists.

Finally we started to get somewhere.

Two key insights had emerged. The first was that users arriving at our site had no idea who we were or what the site was about. "Who the fuck are you guys?" was the question our site needed to answer for vistors, according to the brandologists. In person, and even on our corporate blog, we apparently came across as passionate about what we were doing. But none of this showed through on the site itself. "News untouched by human hands" was what we were actually delivering, and it wasn't working.

The second problem was sort of a structural flaw with our news pages. They didn't conform to any standard web page metaphor. Let me explain what I mean by that.

Back in 1995, when the web was new, visitors to a new site would lean forward, squint at the page, and try to figure out how it worked. The Southwest Airlines page was a picture of a check-in booth at the airport. You had to click on the picture of the phone to get the phone list, and so on.

That metaphor didn't last. People don't lean forward and squint at web pages to figure out how they work anymore. They instantly recognize -- within 100 milliseconds -- which class of site a page belong to -- search result, retail browse, blog, newspaper, spam site, message board, etc. And if they don't recognize what kind of page they're on, they generally give up and hit the back button.

Our news pages didn't conform to any standard metaphor. Some people thought they were search results. But they weren't, our pure news search was a separate section of the site. Some people thought we were a newspaper, with human editors. Some visitors thought we were a blog. But our news items didn't behave in very bloggy ways. Most people just didn't know who we were or what the page was trying to do. Further confusing matters was our front page, which really didn't have anything to do with the local news pages within the site. From the front we either looked like Google News or a national newspaper, depending on who you asked.

This all seems blindingly obvious in hindsight, but it was quite a bit to unravel. We were also left with the question -- ok, now we know what's wrong. But how do we fix it?

Reinventing Topix

So here is the plan we came up with.

• Ride the winners on our existing site. The part of our site that was growing like a weed were the locally-oriented forums. We'd had over a million people post in these forums over the past year and it's just under 50% of our traffic now. Clearly this part of our site was working. Our new product would emphasize people over the machine.

• Fix the local pages by making them work like community-edited blogs. Strictly obey the blog metaphor, with chronological posts, and all of the associated visual cues which tell you that you're on a blog, and not on, say, a google news search result.

• We would run the show just like DMOZ, although borrowing some subsequent innovations from Wikipedia. This was a reliable model, we had done this before with 75,000 volunteers, but no one had done it for news yet. We needed to build an editorial system that could provide an umbrella quality filter around thousands of daily contributors.

  This would also close the quality gap we had between our mechanical aggregation of the news, and the judgment that humans can apply.

  

• Anthropomorphize our existing technology into the roboblogger. This was a brilliant idea from one of our lead engineers. It simultaneously solves three problems: 1) Booting up a new city -- you need posting activity to draw the first editors. The roboblogger would give us that. But he is shy and gets out of the way if humans show up and take over a page. 2) If the community editors go on vacation, the roboblogger can step back in and take over while they're gone. 3) People know when a robot is editing the page vs. a human. His profile icon is a picture of a little tin-can robot. His handle is 'roboblogger'. No more confusion.

• Kill the home page. It should be an "enter your ZIP code" box. Putting national news on this page created too much confusion with our main mission, which has always been local.

• Streamline the experience. We'd joked that we have the old AOL audience on our site, when they email us feedback or bug reports they still have the CAPS LOCK key held down. We have sheriffs, teachers, doctors, airline pilots, bankers, real estate agents, lots of regular folk. And not just clumped on the coasts, but spread pretty evenly across all the states. Most our users are not bloggers, they're not fans of some Silicon Valley Web 2.0 startup. They just want to talk to people in their town. We had to make the experience simple for them.

The cool thing about this plan is that it leveraged a lot of the good stuff we had already done. But the aggregation technology we had built would be redirected at assisting human editors, providing dashboards of candidate stories for them, and taking care of the boot-up and vacation problem. The new site would put people front-and-center, and people in our local forums had been driving all of our growth over the past year. And we were better positioned than anyone else to do this, we had millions of users and the seed content to boot it up.

And the potential success case looks very, very interesting. When we launched NewHoo (dmoz's original name) in 1998, we figured it would be pretty cool if we signed up 1,000 editors. We signed up 1000 editors in the first 3 weeks, without any existing traffic or promotion. Ultimately 75,000 editors signed up to help. Topix is starting with a far broader base of seed traffic, and a pretty slick local news CMS for every city in the country.

We'll see! :-)

Read more:

• Reinventing Topix: Topix.Com(munity) (blog.topix.net)
• Your News. Our Site. My Take. (Marksonland)
• Topix.com homes in on citizen journalists (USA Today)
• Topix taps Web readers to bolster local news (Reuters)
• News aggregation site employs human eyes (AP)
• Topix Transforms Into Community Generated Local News Site (Search Engine Land)
• News site puts readers in editor's seat (SF Gate)
• Topix Reboots As a CitJ-News Search Hybird (Paid Content)
• Topix Aims For Citizen Journalists (TechCrunch)
• Topix reinvents itself as citizen journalist site (Cnet)
• Topix Lets Community Edit the News (Somewhat Frank)
• Topix.net Rebrands & Turns to Human Editors (Chip Griffin)
• Topix.com Does the Obvious Thing (Mashable)
• Topix Becomes a Socially Editable Online NewsPaper (Startup Meme)
• Topix.com relaunches with stronger citizen journalism focus (Susan Mernit)
• Think Locally, Surf Regionally (Social Media Club)
• Topix Portends Media Acquisitions (Ross Mayfield)
• Does Google SEO success 'suck'? (ZDNet)
• Topix takes citizen journalism local (Mathew Ingram)

A brand is a differentiator, a promise, a license to charge a premium. A brand is a mental shortcut that discourages rational thought, an infusing with the spirit of the maker, a naming that invites this essence to inhabit this body. A brand is a performance, a gathering, an inspiration. A brand is a semiotic enterprise of the firm, the companion spirit of the firm, a hologram of the firm. A brand is a contract, a relationship, a guarantee; an elastic covenant with loose rules of engagement; a non-zero-sum game; improvisational theater at best, guerrilla theater at worst. As perceived vessels of exploitation, brands provide the impetus for generics and voluntary simplicity, as well as targets for demonstrations of cultural nationalism. McDonaldization, Coco-Colonization, and Disneyfication are simultaneously courted and countered, imported and deported. The swooshstika becomes a badge of infamy, Ronald McDonald is toppled and graffitoed, and iPod adverts are morphed with images from the infamous Abu Ghraib prison to protest the war in i-Raq. The brand demands an antiphonal, overlapping call-and-response patterned singing among communicants. It requires collusion, collaboration, and the willing suspension of disbelief.

...

Imagine the brand as a Thai spirit house. A ubiquitous structure in residential and commercial neighborhoods, often mistaken by tourists as a bird house, this tiny building resembles a temple, and acts as a dwelling for spirits of the land and household, who are plied with offertory gifts by petitioners in search of favors or assuring pledges. The spirit house is often piled high with gifts of flowers, food and currency, left by suppliants in hope of intercession by the residents. As will be evident in the following pages, I view branding as the creation of household gods, the mythic charter of our consumer culture. The brand is also a habitat in which consumers can be induced to dwell. In that dwelling, consumers domesticate the space, transforming it, and themselves, to essence. The resulting glow emanating from the dwelling is the brand's aura.

    -- John F. Sherry, Jr., in Kellogg on Branding

I think this is great stuff, whatever it means. But if you're a product person and you find yourself in a marketing meeting, and some marketing dude starts throwing around the B-word, and it seems like a pretty wicked tool for them to wield since it basically can mean anything they want it to -- well, you'll know why.

In 1841, William Fox Talbot patented the calotype process, the first negative-positive process, making possible multiple copies [of photographs]. This invention permitted an almost limitless number of prints to be produced from a glass negative. Also, the reduction in exposure time made a true mass market for pornographic pictures possible. The technology was immediately employed to reproduce nude portraits. Paris soon became the centre of this trade. In 1848 only thirteen photography studios existed in Paris; by 1860, there were over 400. Most of them profited by selling illicit pornography to the masses who could now afford it. The pictures were also sold near train stations, by traveling salesmen and women in the streets who hid them under their dresses.
    -- wikipedia

But in search and traffic I think it's not pornographers, but the SEO industry that deserves the credit.

On one hand, the relevance issues introduced by spam sort of define the entire environment that both search and social media exist within. It's not a matter of just bolting a spam filter onto your product once it's done, how to be crap-resistant and be able to greyscale score content across the entire spectrum needs to be core to your software.

Beyond that, however, SEO's often pay more detailed and critical attention to the web industry than most of the industry analysts, who simply eat press releases and comment on them. SEOs are continually probing for weakness and insight into the evolving global online traffic market.

Some SEOs I follow:

SEO by the Sea has been methodically going through the patent filings for Google, Yahoo, Flickr, Ask, Technorati, etc. looking for insight into their ranking and anti-spam methologies. Cool. :-)

SEOmoz has a nice how-to guide for getting links into WikiPedia.

Another gem from SEOmoz: "Every so often, one of our employees will roll into the office and announce, 'I'm going to get on Digg today.'" How do they do that? sock puppets, amazon's mechanical turk, or just plain old linkbait?

I find this stuff hilarious but also insightful. If you're designing social media systems, you should be keeping an eye on the $2B industry that sells links from your site to their clients.

:-)

Universities are extraordinary institutions. They are in fact, the last bastions of mediaevalism left in modern society outside, perhaps, the church. Like churches they attracted a certain type of person who did not share the values of the commercial world. The oldest universities date from the eleventh and twelfth centuries - hundreds of years before the invention of the printing press. In an age where books were scarce, communication was difficult and people who could read and write were almost as rare as the books, it made sense to centralise the acquisition and dissemination of knowledge. If you wanted to learn you headed towards where the books were and the people who could read them and that meant the great universities like Paris and Oxford. Poor communication, expensive reading materials and illiteracy were the foundation blocks for the universities. If today we have excellent communications, free online information and general literacy, we also have an environment in which the universities are struggling to maintain their position. That, of course, is not an accident.

...

This is where this student begins by recognising that university, like school, is also fairly phony in many ways. What saves university is generally the beauty of the subject as built by great minds. But if you just look at the professors and don't see past their narrow obsession with their pointless and largely unread (and unreadable) publications to the great invisible university of the mind, you will probably conclude its as phony as anything else. Which it is.

    -- Mark Tarver, Why I am not a professor

Actually I think that universities are far more useful as a social accreditation filter than for academic enrichment. In other words, they prove you can do work you're told to, on time, and give you an opportunity to develop and prove your skills at getting along with peers and superiors. If you're going to study tech, or literature, or languages because you love the subjects, you're going to do that anyway. But the hoop-jumping is the real point of the test.

I knew a number of bright folks who, for whatever reason, couldn't quite get it together in school and left. Some just a credit or two shy of graduation. It all comes down to: "Do well."

Is Skrenta going to get fired or is the damn thing working? :-)

So far it looks GREAT. We've approved over 500 editors in the first week. Not all are active, and some of them signed up for non-local channels. But at this point we've got about 100 daily active local cities being edited.

You can see the list of editors here and the list of most recent local editor posting actions here.

Just after we launched this stuff I realized something which hadn't occurred to me before, about the DMOZ/wikipedia model and how we're trying to apply it to local news. There's an advantage to community-edited news which actually makes it a much easier problem to tackle than either a web directory or encyclopedia.

At DMOZ we signed up 75,000 editors, who ultimately created 400,000 categories and filled them with links. The problem was that, even with 400k categories, we hadn't even made a dent in the problem of organizing the web's information. 400k categories is less than 1% of what you need for that problem.

But local news is a finite domain. We have 32,500 local news channels. Once we approve an editor for a town, if they become active then that town's page is basically 'fixed'. We aim to sign up multiple editors, and of course the character and style of editing varies, but pretty much any human can do a better job than our roboblogging technology.

So if we signed up an equivalent number of editors to DMOZ, we'd have an average of 2 editors per locality. It wouldn't work out that evenly, we'd have clumps with multiple editors in bigger cities, and some small towns would still only be roboblogged. But I'm guessing we'd have coverage over about 1/3rd of the US map, or 10,000 towns.

Another neat thing about this model that I hadn't thought of before is that any kind of commercial spam we might receive is going to get "washed away" in the daily flow of articles on the local pages. Unlike the wikipedia or DMOZ, where a spammed link can hide for years, nothing really "sticks" to topix, since it's all new each day. This does mean we need continuously active editors though, or at least a steady re-supply of new editors if some of the old ones drop out.

Something about the redesign has also lead to a big jump in our local forum activity. We set a new forum record yesterday, with 47k posts. That's a 25% increase from two weeks ago.

So early results look good. I'll post more detailed stats on our active coverage and posting volume in about a month.

Freebase: one to watch:

nyet! no! eggads! do you know why google is popular?! the world's data does not want to be structured.

there are only three ways to do this:

1. treat all bits as potentially noisy and use probabilistic methods to try to fish it out. see: google.

2. given the impossibility of structuring all the world's data into domain-specific schemas of value, semi structure your data into one humongous associative array. okay, now what?

3. ignore (2) and actually try to create a bazillion community authored and maintained schemas. the only problem is that schema design isn't much fun, and amateur schemas will break easily.

How to beat Google, part 1:

I've never understood how Google can insist that their infrastructure costs are actually an impediment to any startup. Sure, it costs a lot to serve 200M queries a day, but 200M queries usually come with a lot of money attached.

Early adopter pilotfish: pornographers vs. SEOs:

Will it be porn that finally bootstraps IPv6?: http://www.ipv6experiment.com/

Yahoo Singing News:

The site is now up at http://underground.yahoo.com/ . Changed your mind yet?

Speculative Fiction

it is okay to blame terry, he has been paid very well, by any standard in corporate america. including his stock grants, semel has certainly extracted hundreds of millions of dollars in compensation. for that much money it is fair to expect results. while he doesn't strike one as the type of person to grasp the viability of search, he has surrounded himself with advisors who certainly should have been able to make this assessment.

one can also turn some blame on jerry yang, who was instrumental in attracting terry. jerry should have known that such a technophobe would have problems dealing with the inevitable semi-technical issues that a yahoo ceo would have to grasp on some meaningful level. it was those senior yahoos who were engaged in the ceo search who incorrectly assumed that yahoo was simply "another media company", and their search was predicated on this. had they understood that this generalization was meaningless, they would have directed their search elsewhere.

but to be fair, the winds of the internet have shifted. no one cares about "integrated networks" like yahoo and aol anymore because they have failed to deliver more utility than the rest of the web. google is a rest-of-the-web company...its search and advertizing products leverage the entire web instead of trying to fight it. i'm not sure anyone at yahoo saw this coming.

p.s. i have a double-digit employee number at yahoo. that doesn't make me right, but some of what i cite is based on observation, not speculation.

i'm on vacation this week so no polish, sorry. :-|

13. Both personalization and natural language approaches to search seem to mainly be about disambiguation. I've written a big disambiguation engine, one of the better commercial ones on the net. Disambiguation doesn't seem as interesting to me anymore.

Grouping terms and ranking compounds is more useful, IMO. Hence Ask having unfortunate results for stuff like lady diana car accident. Is this Edison yet?

Full blown question answering, apart from being something that nobody actually wants, is a matter of first structuring the web, and then converting english into some SQL-like stuff to run against it. If you could structure the web though you could skip the SQL business because you'd already have 98% of the win.

Sentence tagging doesn't seem that interesting. Parts of speech are this chompskian red herring where a set of artificial categories have been imposed on english. So you have an n% error tagger mapping these basically useless categories to web text. If you actually were able to put together some kind of probabilistic parse map, you could predict completions like "I played fetch with my <x>". Classic taggers don't do much for typical queries either.

Check out the great Ask patent screenshots from seo by the sea. So which rule(s) do they violate? We don't always really want what we think we want. Or maybe I'm wrong, it is a cool looking mock. :)

Vacation too short. 9am panel tomorrow morning at Moscone. zoinks.

Topix has come a long way in the past few years. We used to have the worst booth at these conferences - a shabby sign with a typo in it, and that's all. Now we have the full blown booth with street teams outside promoting the site. Pretty cool.

    -- marksonland

We really did have a typo in our first booth sign. I think that thing was proofed by like 5 different people, and everyone missed it. Our sales guy walked up when we had finally set up the booth at its first show and instantly pointed out the typo. We ended up getting some lame sticker thing to "fix" it from the printing company. We used that horrible booth for another year or so before finally replacing it.

Well our booth looks better now, plus we have a cool street team survey going on in front of Moscone:

More on flickr. :-)

(click for non awfully scaled image)

At first I thought our captcha had been broken and this had to be spam. But after rooting around we decided it wasn't spam, it's real activity. Then I looked at seo but I didn't see any change there. Our best bet now is that it's due to the redesign plus the new site being faster, which has lead to more on-site activity.

An odp editor is picking a bone with my "75k" number of dmoz editors. That's the number on the dmoz homepage. Yeah of course not all of them are active, the site is 9 years old and the number never goes down. Plus they stopped letting anyone new in about 4 years ago. A much more interesting number is the number of daily edits. But in the end it's apples-to-oranges, since dmoz in theory benefits from the previous edits of now-inactive editors, whereas Topix will only benefit from sustained daily editing activity.

Thus it's more interesting for me to say that Topix had 804 editor posting events yesterday, which is a 24% increase from the 645 posting events it had the previous monday. It's hard to predict success with two weeks of data, but of course we want to answer the question -- will Topix become big in its domain like dmoz and wikipedia, or will it moulder with no use like a Backfence. Well at this point it looks like it will become very big. No one will realize it for 1-2 years while it grows though. :)

Our launch party came off quite well last night. We had about 200 people at the St. Regis and I saw a lot of old pals I hadn't connected with in a while. The street teams worked out better than I expected too. I got some good quotage too from the morning panel. My talk tomorrow in front of the whole audience (wow big room) should be fun.

Afterwards I showed it to some guys from Snap who were chatting with some Japanese VCs but they all looked at me like I was insane.

Over the past couple of months I did press tours in NY and SF, with 25+ interviews, four panels, a high order bit at Web 2.0, had our launch party for about 200 people, and dealt with some stuff in our org.

I've scored myself on tests like the 16PF and I've generally come out as 50/50 introvert-extrovert. Questions like "Are you energized by meeting lots of new people at a party, or are you drained by the experience?" are the sort that help score you as an introvert or extrovert. The sterotypical engineer is an introvert. Remote employees should be extroverts. This may seem counter-intuitive at first; if you're working by yourself at home all the time, isn't that a better job for an introvert? But no, it turns out you want someone who will cross the extra barriers (telephone, IM, email, plane flights) to reach out to the group and over-communicate.

It's a big part of my job to run around and talk to people, but this particular media tour kinda left me fried and I got strange and grouchy in parts toward the end. So I apologize if you ran into me and I was brusque.

On a panel at Web 2.0 I made some comments about media fragmentation and advertising. Part of my comment got quoted, and then echo-sphere style some folks responded to the fragment instead of to what I had actually said. Forget going to email-only interviews, I should only communicate via my blog, best way to preserve the message delivery... :) In any event, here is something closer to what I said on the panel.

In 1960 an advertiser could spend $5M a year and reach 160M television viewers. With the right message, sustained over a few years, 85% message penetration into the audience was achievable.

That world was dying before the Internet came along. When three TV channels exploded into 300, the audience spread out across the new terrain. Putting the audience back together became difficult and took more money.

This was happening in magazines as well. More and more titles, you can't hit everyone with the right ad in Reader's Digest anymore.

The audience isn't huddled every night in front of three TV channels anymore. And you can't reach them with $5M of 1960 dollars. The audience is divided across 300 channels,dvds, tivo, itunes, youtube, bittorrent, flickr, mmorpgs, millions of other options. Saturation marketing costs something like $30+M for a few week blitz to launch a new movie. What if you wanted to saturate like they could in 1960, and drill your damn jingle into every consumer's head until there was no way they couldn't hear it when they saw the box in the drugstore? Costs to launch a new top-tier brand from scratch start at $150M now.

So it's more expensive to reach the same audience of people, because they spread out into a zillion different places. But it isn't the net, per se, that did this. It was scarcity that caused people to huddle around the same few media outputs in the first place.

Printing presses are expensive, and it's expensive to move paper around. So the number of newspapers and magazines was originally limited, and we all read the same ones. Radio and TV spectrum are limited and licensed, and with few channels we used to all tune into the same ones.

But with more efficient printing, distribution, spectrum use, choices were already multiplying. And when the Internet showed up -- the ultimate mass many-to-many zero-incremental-cost media distribution network -- well there goes "mass" media. If you can manage to put back together a few tens of millions of users on the Net -- a tiny fraction of the 160M 1960 TV audience -- you have a huge web business.

But, while the turnkey mass media channel that let you annoy everyone in the country for only $5M a year is toast, we're not back to the pre-print age. Messages do get takeoff, but they have to be self-propagating, in order to get voted up and linked and shared.

If the web follows what happened in magazines and television, audience domination by the biggest sites will eventually wane. Maybe we're all on Yahoo or Facebook or Youtube today, but in 10 years these sites will command a smaller fraction of the total audience, because there will be a steady proliferation of high quality, niche-targeted alternatives.

Linkbait (ahem, "Social Media Optimization") may be all we have left of mass media when this shift is over.