Earlier this week Netflix awarded $1 million to a team of researchers calling themselves BellKor’s Pragmatic Chaos. At issue is Cinematch, the part of the website that comes up with a set of recommended movie rentals personally tailored to each customer based on their rental history and movie ratings. Netflix announced the contest three years ago, offering the prize to anyone who could find a way to improve Cinematch by 10 percent. Incredibly, another team of 30 programmers also found a way to do this but lost out on the prize because they submitted their proposal 20 minutes after BellKor’s did. What must it be like to spend three years on a project and fall short by 20 minutes? (“Dudes, I knew we shouldn’t have taken time off to play Guitar Hero back in March of 2007.”)
As this article from last year indicates, Cinematch did fine in some ways: It recommended horror movies to people who rented lots of horror, and romantic comedies to those who checked out lots of that. The problem came with offbeat indie films like Napoleon Dynamite; the system couldn’t figure out what renters who liked those movies would prefer. BellKor’s solved the problem by rejiggering the site’s evaluations of customers’ ratings of individual movies, taking into account when a customer submitted their ratings. (Apparently, people’s ratings of a given movie on Monday tend to be horrible indicators of their overall taste. Who knew?) Hard-core statistics nerds can sample some of BellKor’s methodology here.
This event is more than just another significant point in the history of crowdsourcing. It’s also a steal for Netflix, when $1 million is weighed against the costs of employing these researchers and programmers for the three years that they were working on the problem. More companies probably should turn to the wisdom of crowds in instances like these. In the meantime, Netflix is now announcing a second prize to make further improvements on Cinematch. Unfortunately, some users are up in arms about their privacy being violated, because this new prize will give researchers the ages, genders, and ZIP codes of customers. It won’t give out the customers’ names, but some privacy advocates were able to take the other data and still identify the customers anyway, with a success rate of 80 percent. What do you think? Do you not care as long as Netflix’s movie recommendations get better? Or would you prefer that no one know about you renting Ghosts of Girlfriends Past?
