« October 2007 | Main | December 2007 »

November 2007 Archives

November 3, 2007

Tapping on keys

Apparently I can't code and blog at the same time. When I work on something it generally takes over my life and I think about it 24/7. If I'm writing code, even when I'm not sitting in front of the keyboard there is usually some part of my brain working through a problem or just turning the structure of the system around in my head to see what needs to be done.

On the other hand it's not like I'm holed up in a fire tower working on a novel or something. I'm still walking around and seeing stuff and talking to people, and could have stuff to post. Maybe if I could bang out the words faster I could keep the blog going concurrently. My posts have generally taken at least an hour to write in the past. That's too much time each day when that hour could be put to other use. I'm going to experiment over the next week and see if I can't optimize post-writing time down so I can keep stuff going here without compromising the other stuff I'm doing.

Rotting Pumpkins

November 4, 2007

Ranking Web 2.0 sites by server latency

Server latency is the start of the battle for site performance. There are great tutorials on how to optimise your html, but if your server takes too long sending the bytes out in the first place, there's nothing the browser can do but wait.

It gets even worse. Server latency directly affects your site's hardware requirements. Slow html, in some sense, is the user's problem; their browser will spin trying to render your spaghetti css. But the longer it takes your server to put out a page, the fewer pages it can serve per second. Which means buying more servers for the same load.

For example, if your server returns a page in 50ms, it can pump out 20 of those a second. If it takes 250ms instead, it can put out only four per second. That's a 5X difference in the number of machines needed to serve the same number of users.

Just for review here recall that 1 millisecond = 1/1000th of a second. So:

    50ms = 1/20th of a second
    250ms = 1/4 of a second
    500ms = 1/2 of a second

The human eye's threshold for perceiving latency is about 50ms. Crudely simplified, this means that if you render a screen within 50ms or less, it's going to be perceived as instantaneous. If it takes longer, people can sense the latency. Lots of subliminal processing occurs on an image in the first 250ms, with conscious processing happening after 270ms.

(Making a leap here... This means that people's well-trained subliminal neural hardware is deciding whether to click Back even before they've consciously realized what they're looking at...cool. :-)

I'd recommend the following performance yardstick for server latency:

    50ms = pretty good
    250ms = ave/sluggish, but still OK
    >500ms = your site is slow as molasses

Faster is always better, but if you're in the 50-100ms range you can feel pretty good about your platform. Over that, and there's probably some easy wins to be had, which will payoff in user satisfaction and a lower hardware ramp in the co-lo.

So how does the rest of the net stack up?

The following list is the result of running apachebench on 530 Web 2.0 sites pulled from CrunchBase. I also added some of the major sites such as Google, Yahoo, and so forth for comparision.

Caveat: I don't recommend trying this at home. I was deliberately trying to avoid overloading anyone's servers, so this was not a stress test, just a touch test done off-peak with a mean result coming from 10 non-concurrent fetches. Running apachebench against Google (or anyone else for that matter) is a great way to get IP-banned.

There is a huge range -- 100X! -- in performance figures, from <10ms for the fastest sites to slow sites which took seconds to squirt out their html.

Yahoo trumps google on this test, returning their homepage in a mean time of 13.620ms, vs. Google's 30.716ms. This despite Yahoo's page having nearly twice as many bytes (although still quite lean at just under 10k of html).

I included information about the site's webservers, but there doesn't seem to be any consistent trend. Apache appears on both the slowest and the fastest sites, with many other servers spread through the range too. Apparently it doesn't matter what brand of webserver you have, it's how you use it.

Anyway, on to the list.

Continue reading "Ranking Web 2.0 sites by server latency" »

November 7, 2007

Network Effect Entrepreneurs

"Today Silicon Valley is full of 'network-effect entrepreneurs' "
      -- Steve Perlman

It's 1998... Bob and I are writing code for Sun. C++, kernel, networking, heavy QA process. 18 month release trains to get into Solaris. No one uses our product. Sun's channel strategy for desktop software is fsck'd. Everyone in the IETF hates us. Typical 90's bigco software job.

But then the net comes along. Microsoft buys Hotmail for $400M.

Hotmail wasn't Netscape. A browser is a big honking piece of client code. You spend 30 minutes compiling against some gargantuan event-driven windowing framework only to crash your windows box when the thing runs. Hard work.

Hotmail was some web forms on top of sendmail. You use printf to make web forms. I could have written Hotmail. You could have written Hotmail.

Hotmail was so successful that the founders and VC's were arguing over who invented the "P.S. Get your free email at Hotmail" viral advert appended to every outgoing message. Success has many fathers.

Think about that. Not the server mail delivery/connection model. Not their anti-spam. Their big I.P. story was this one-line message appended to the end of the email.

Bob and I coded NewHoo in two months. HTML forms on top of a database. We got Wired, Red Herring, Netly News in the first month after launch. Bizdev from Looksmart, Infoseek, Lycos. printf and html forms were working great for us. This was a lot easier than debugging locks in the kernel.

Competing projects sprung up to chase us -- Freedir, Infoseek's Go Guides, Zeal, Wherewithal -- but we didn't think they had much chance. Something about being the first put us at the head of the pack. No matter how many users the followers signed up, we always stayed way ahead.

That's a network effect barrier to entry.

The barrier certainly wasn't our code. Our I.P. wasn't our C or Perl. It was directory data and users.

Ebay was like this too. You could write a clone of ebay in a weekend. It's printf's and a database. But there's no point, because the trick would be how you would get everyone from over there onto your site. ebay's barrier to entry isn't their code.

There are still products with technology I.P. Oracle, RenderMan, Google... those shadowy funds arbitraging adsense to yahoo in Europe. RenderMan awes me. Every year they make a better movie with it. All that ray tracing math to make hair and mist and fire and faces look more realistic. 10 years of hard work by a big team are in that package. That's cool.

But connect-the-dots has the day, thanks in no small part to the takedown of MSM and the pillaging of its ad dollars.

Fun fun!

Meeting today

"We wanted to check you guys out before coming over here. But your company webpage just had picture of a paper bag puppet on it. Then we googled you, but it said you were writing a virus for the iphone. We were wondering, who are these guys?"


November 8, 2007

Editorial Selection in Retail

I recently read Into the Wild, the story of Christopher McCandless's journey to Alaska. (If I had known that it was about to become a Sean Penn directed movie, I probably wouldn't have bought it ;)

I had picked up the book because it was on one of the "recommended" tables at Borders. It wasn't a new release, but I grabbed it because it looked interesting.

Louis Borders (he of the bookstore bearing his name) once told me that Borders had slipped after he had left, and that their "editorial selection" wasn't as good anymore.

This was a new idea to me... the idea that a bookstore could have an "editorial voice", based on what they feature on the the end caps, the "staff picks" tables, the books turned cover-facing-out on the bookshelves, and overall in the selection of books that are stocked.

It totally made sense to me. The old Printer's Inc in mountain view used to be a great bookstore. The ownership changed, they remodeled the store, and it feels to me like half the books are gone now. The half I wanted to buy.

It's not just bookstores though. I've started to notice editorial selection everywhere now. I'd never thought about it that way before. Even a restaurant has an editorial selection. Some restaurants try to be all things to all people. This makes sense in a New Jersey diner, but that's not how you get to be the French Laundry. Or In-n-Out burger, for that matter. It's not by being all things to all people.

Wish you were here...


Spamgen english->spanish->english

Willy Wonka and the factory of the chocolate is a film 1971 cradle in the book Charlie of the 1964 children and the factory of the chocolate by Roald author Welsh Dahl. The film dug in generally the revisions received critical foxhole on its opening in 1971, but it was only a low commercial success, despite on the years the film has become one of more the beloved, well-known affluent family always films fact - and in spite of its age, and fable original creative attempt as kinematic musical comedy for the children - also has to you from grown in an important classic work of the cult with the children and the adults.

Try this for a random spam blog from google blogspot. :) I've seen a number of wordgen produced blogs out there. Not sure if anyone is using machine translation to wash text yet though.

November 13, 2007

Gee whiz

I've just been offered $125k for rt.com, the first domain I ever registered. I got it back in 1991, before there was even a web. It was for email hosted on my uucp Amiga Unix node.

The domain market sure looks hot right now. That seems like a good offer, but the domain has sentimental value, and who knows what domains might be worth in another 10 or 20 years...

Spice Girls VC

So one day a few years ago I'm sitting in a VC's office having a chat. I had a few ideas rattling around in my head but the VC had his eyes on a then-current space which was hot. He tossed a business plan for one of the leading startups into my lap.

"Where'd you get this?" I asked.

"They gave it to me."

He went on to talk about how he wanted to launch a company into the space as well, and I'd be a great vp eng. He said he knew a guy with some technology who could be cto, had a vp marketing in mind, and then we'd just need a world class ceo to round out the band.

I formed a theory that the process of seeking VC ended up calling your own competitors into existence. You'll meet with many more VCs than the 1-2 who end up funding you. But after seeing a company or two get funded in your space, the VCs who passed or weren't able to get in decide they want to have a bet in the space too. Fortunately they have the benefit of having heard your pitch and the opportunity to personally grill you at length on your approach.

But doing the Spice Girls or N' Sync thing to put a startup together can be tricky. Startup founders can be so cranky / eccentric.

November 15, 2007

Netscape's "Tin Man" rocket

If you used to work in Netscape building 23 or 24, you probably remember the giant rocket-ship shaped tower between the buildings. The purpose of the tower (as described by a FAQ on the Netscape intranet) was to pump a poison called TCE out of the groundwater and spray it into the air, supposedly so it would "dissipate":

The Tower a.k.a. The Tin Man a.k.a. The Rocket...

This is an Air Stripper. It is used to remove VOCs (Volatile Organic Compounds) from ground water wells located under our campus. In summary, water is pumped to the top of the tower, and then released over 'wiffle balls' to increase the surface area that it flows over. At the same time, a powerful stream of air is sent up the tower, where the high velocities pull the VOC's out of the water. They are then ejected high into the atmosphere, where they evaporate & diffuse immediately. For more information on how the Air Stripper operates, please visit The Air Stripper FAQ Page.

Building 23 Has an Interesting View

Yes, building 23 is right next to the tower, but do not despair. The VOCs are discharged from the air stripper at a very high rate of speed, & they evaporate into the atmosphere immediately. The EPA, the BAAQMD, the RWQCB, the previous owners, and Netscape have & will continue to monitor the processes and the quality of the environment across our entire campus. Similarly, much of the Bay Area, including many residential areas, have ground contamination issues & are undergoing remediation.

What's Down There Anyway?

Primarily Trichloroethene (TCE), and derivatives of that chemical. (TCA, PCE, DCE, DCA, trace amounts of Freon 113, Phenol, Vinyl Chloride, DCB) The area where these are found is approximately a half mile wide and 2 miles long, much of which is covered by Moffet Airfield.

There was a little putting green between the two buildings, and if you were anywhere near it or walking between the buildings you could feel the mist raining town from the tower. The building HVAC intakes buildings were also nearby.

We were moved to building 24 in 1999, and wondered about this hare-brained scheme to rid groundwater of poison by spraying it onto people and into ventilation intakes. Bryn's dad is a PhD chemist, so we asked him about TCE. His opinion was that it was bad stuff and we likely didn't want to be soaking in it. So we asked Netscape for TCE testing of our air quality, but predictably got the runaround. Shrug.

A few years later, tragi-predictably, the EPA reclassified TCE to be far more harmful than they previously had claimed:

The U.S. Environmental Protection Agency will require 10 Silicon Valley high-tech companies that once operated manufacturing plants in Mountain View to conduct -- for the first time -- air-quality testing for a toxic substance inside several offices that were later built on the land.

The same companies are suspected of having leaked into the ground a substance called trichloroethylene, known as TCE, a widely used solvent that cleans machine parts.

Now, the EPA believes TCE might be 60 to 70 times more dangerous to humans than previously thought, and it is concerned that contamination in groundwater is seeping into the air inside office buildings constructed in areas vacated by those companies.

    -- old unlinkable merc story

Might be seeping up? Yah right. It was pumped out of the ground and sprayed all over the place, on purpose. Sheesh.

I'm too apathetic about such things to worry over what kind of increased risk of god-knows-what I might have in the future, but if you care check out the TCE blog.

November 26, 2007

Unix should have a newfd call

So you can open a file and then unlink() it. The fd hangs onto the file, even though the file isn't visible in the filesystem anymore. Once you close the file, it goes away.

This is kinda cool and gets gets used every so often, but much more useful would be the reverse.

Cleaning up files in the process of being created before they are renamed() into place is a pain. If you could create a new filesystem fd in the unlinked state first, and then link it into the filesystem once it was ready, all the temp file and unlink-on-error nonsense could be done away with.

newfd() would have to take a path to associate the fd with a particular filesystem, like statvfs(), but that's easy.

A lot of newbie programmer errors where a partially-written file is put into place over the existing file would be eliminated too. I bet this could have saved a lot of trouble over the years.

Kinda late to be adding calls like this though. 30+ years late.

November 28, 2007

Apple's "1984" ad: Rejected by the board

So the best commercial ever - literally, the best commercial ever - shows up with Ridley Scott as the director, and half the folks who see it, including Apple's board, want to give the time back to the network rather than run it.

The ad was the pride of the entire agency. They were confident that 1984 would generate a tremendous interest in not only the Macintosh, but all Apple products.

Unfortunately, Apple's board didn't concur. When the board was shown the ad, cofounder Mike Markula suggested that Apple drop Chiat/Day altogether. The rest of the board was not impressed either.

Sculley was discouraged by the board's reaction and asked Chiat/Day to sell back both the timeslots to CBS (the commercial was to air uncut during a minute spot, and an abreviated version would be aired during a thirty second spot). If a buyer could not be found, Manuals [an ad featuring a stack of manuals] would be run instead.
    -- Tom Hormby

But a few people could see the ad was great stuff and wouldn't give up:

Chiat/Day defied Sculley and only sold the thirty second spot.

Steve Wozniak, who was still friends with Jobs at the time, heard about the board's refusal to support the adfrom Jobs, who also showed it to him. Wozniak loved the ad and offered to pay for the spot personally if Jobs was unable to get Apple to air the ad.

Amazing. The ad agency ignores the CEO's instructions, and Woz the founder steps in to offer to pay for the ad out of his own pocket if they don't run it. That's so cool...

Popular history remembers successful efforts being destined for greatness from the start. But there's usually a messier story behind the scenes.

Interesting tidbit: the models who tried out for the ad were physically unable to throw the sledgehammer. They had to hire an actual competitive discus thrower to play the part. Cool. :)

November 29, 2007

If you can't read this...

The IP address of this blog changed from to Windows for some reason doesn't seem to honor dns ttl at all and I had to reboot my windows machines to see the update. So if you're not reading this post, you have bad dns... eh. well. hmm.

When we've moved big sites we always leave behind an http redirector on the old IP for a few weeks. It's surprising to me just how many clients out there will continue to use old dns weeks after an update to an entry with a 15-minute ttl. I didn't bother with the redirector for this ip change on my blog though, seemed like overkill.

About November 2007

This page contains all entries posted to Skrentablog in November 2007. They are listed from oldest to newest.

October 2007 is the previous archive.

December 2007 is the next archive.

Many more can be found on the main index page or by looking through the archives.

Powered by
Movable Type 3.33