Netflix Prize lessons for the rest of us

For the last few weeks I’ve been in the throes of running a contest as part of my day job. Perhaps that was why this look at the disappointing results of one high profile contest caught my eye.

Netflix’s Xavier Amatriain and Justin Basilico provide the history:

In 2006 we announced the Netflix Prize, a machine learning and data mining competition for movie rating prediction. We offered $1 million to whoever improved the accuracy of our existing system called Cinematch by 10%….A year into the competition, the Korbell team won the first Progress Prize.

Amatriaan and Basilico go on to explain why Netflix will never use the code that won the prize. Basically, technology, and consumer preferences, progress, and “did not seem to justify the engineering effort.” “It wasn’t just that the improvement was marginal,” said Mike Masnick, “but that Netflix’s business had shifted and the way customers used its product, and the kinds of recommendations the company had done, had shifted too.”

The Netflix Prize was quite different from most philanthropic contests. Still, I wonder if we might discern lessons for those running contests today:

  • Adapt quickly to market, technological and audience behavioral changes. 
  • You don’t always get what you’re looking for. 
  • Don’t outsource your core business. (Borrowed from my weekend reading of Tony Hsieh’s Delivering Happiness.)
    (I’m streaming Ballerina as a write this.)

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s