Kaggle, competitions and competitive intelligence

Today, economist and blogger Nicholas Gruen announced that he had become chairman of Kaggle.com. Kaggle is a service that allows companies to run data-modelling and prediction competitions. Two type of competitions are supported – competitions to predict the past and competitions to predict the future.

“Predicting the past requires contestants to build models that are evaluated against a past event (the idea being that highly performing models can then be adopted by the competition host) … Predicting the future requires contestants to make predictions about a future event. Organizations may then choose to act on predictions made by contestants who have a history of forecasting accurately. ” (Kaggle.com 14/04/2010)

Nicholas cites a couple of sources of inspiration for Kaggle. First is the Netflix Prize, where an online movie provider offered $1m to whoever could  improve it’s movie recommendation algorithm by 10%. Second is Innocentive, a service that enables companies to post technical  and business challenges online, with cash rewards for the best answers received.  Kaggle also seems to borrow a little from prediction market sites like Intrade, especially with respect to its competitions for predicting the future.

So why should those with an interest in competitive intelligence (CI) be interested in Kaggle?

Firstly, these services may serve as a new source of  insights . Kaggle and Innocentive are tools that may be used to help meet an organisations intelligence needs.  Especially where access to outside expertise is beneficial, an Innocentive challenge or Kaggle prediction could prove valuable. Competitions and challenges might also be an innovative way for organisations to identify new talent in a particular field of interest.

The second  reason for  CI practitioners to pay attention to these services is the opportunity to be part of teams participating in these competitions.  Perhaps start-ups will form specifically to compete in these challenges – building teams to compete on a project by project basis? Perhaps traditional consultancies will leverage their expertise in to these new competition markets? Both these scenarios could see CI professionals being recruited to help give teams a competitive edge in these competitions.

Services such as Kaggle and Innocentive provide CI professionals with both a new tool for answering their intelligence problems, as well as a potential new source of employment.  Keeping an eye on the evolution of these services would therefore be a worthwhile activity for those with an interest in competitive intelligence.

Leave a Reply

This blog is kept spam free by WP-SpamFree.