Netflix wins 2006 User>Driven Hall of Fame

Readers had the most fun picking products and services for the Hall of Shame. My nomination of the Comcast HD DVR touched off a string of posts (and private emails) from frustrated users. On the other hand, everyone had their own opinion on nominations for the Hall of Fame, with only a couple of things receiving more than one vote.

Netflix received the most Hall of Fame votes, though, and I think it's fitting they should win. My wife and I are subscribers and regular users, as are many people we know. The basic DVD-rental-through-the-mail subscription service was a real paradigm-shifter some years ago when they started up. But what has kept Netflix number one over Blockbuster's efforts and the efforts of other imitators? My theory is that Netflix is User>Driven.

Check out this article by Joshua Porter of User Interface Engineering about how Netflix uses fast iterations to continually improve their user experience. According to Porter's interviews with Netflix designers, they change some aspect of the site on average every two weeks! And it is hardly change for change's sake. They test each change in isolation for a couple of weeks to see what effect it has on user behavior. If a change results in more people who attempt to add a DVD to their queue succeed, then the change is a success and it stays. If not, no matter how "well designed" a feature is, it goes. As I discussed in my entry on paper prototyping, more and faster iterations (combined with user feedback) is the best way to get quickly to an optimal product design.

Lending support to this idea is Eric S. Raymond's Web essay, The Cathedral and the Bazaar, wherein he admonishes us to "release early, release often." Raymond cites the rise of Linux as an example of fast iteration and user participation in development and describes something he calls the Delphi Effect (more properly the Delphi Method). This notion, that the properly aggregated inputs of many random people are better than the isolated input of any single expert, is described at length in the excellent book The Wisdom of Crowds by James Surowiecki.

Fast iterations and the abandonment of designs people worked hard is an emotional business. Good designers take pride in their work and want to see it succeed. Users don't know designers, though, and have no emotional investment in the work. And since it is they that must live with the product, data about how useful it is to them must be the deciding factor. The path to success is to learn to embrace failure. Or, as former IBM Chairman Thomas Watson, Sr., once said, "If you want to increase your success rate, increase your failure rate."

Links

See Ben Brophy's thoughts on an improvement NetFlix could make:

http://www.flickr.com/photos/lfb/359015964/ 

Comments

[Diana Carroll]

I've never used Netflix so I can't comment on the comparative usability of their site, but I've been pretty happy with Blockbuster's site, and they have a feature Netflix doesn't: Total Access. Which means I can either mail my movie back OR I can bring it to the store; if I do, then I get a free extra movie from the store in return. This means I get all the benefits of a movie queue and movies magically showing up in my mailbox, AND I get the spontaneaity and instant gratification of picking up a movie at the store at no extra cost. (The kids especially love this option; a trip to the movie store to get a movie for that night feels like a big treat to them)

I also recently read that I can get a game rental once a month from the store as part of the package, which is an option I plan to take advantage of just as soon as we've played through all the games we got for Christmas. :-)

[Bruce McCarthy]

Netflix is also a great place to work, apparently.

http://www.mercurynews.com/sitemap/ci_5493698