"I Wanted to Predict Elections with Twitter and all I got was this Lousy Paper"

Full download available from the arXiv.org website:

"I Wanted to Predict Elections with Twitter and all I got was this Lousy Paper" -- A Balanced Survey on Election Prediction using Twitter Data

Daniel Gayo-Avello

(Submitted on 28 Apr 2012)

Predicting X from Twitter is a popular fad within the Twitter research subculture. It seems both appealing and relatively easy. Among such kind of studies, electoral prediction is maybe the most attractive, and at this moment there is a growing body of literature on such a topic. This is not only an interesting research problem but, above all, it is extremely difficult. However, most of the authors seem to be more interested in claiming positive results than in providing sound and reproducible methods. It is also especially worrisome that many recent papers seem to only acknowledge those studies supporting the idea of Twitter predicting elections, instead of conducting a balanced literature review showing both sides of the matter. After reading many of such papers I have decided to write such a survey myself. Hence, in this paper, every study relevant to the matter of electoral prediction using social media is commented. From this review it can be concluded that the predictive power of Twitter regarding elections has been greatly exaggerated, and that hard research problems still lie ahead.

Prediction is difficult, especially if you are depressed

A well known quip holds that “Prediction is difficult, especially about the future”. It is attributed to various people including Yogi Berra, Niels Bohr and, for all I know, Yogi Bear too. Nonetheless it has been widely observed that people are often pretty bad at making predictions. This is bad news for those economists who believe in the Rational Expectations hypothesis. Although they will probably say they expected to hear that.

So what factors might cause people to predict badly? In a new study, from INSEAD, researchers find that depressed individuals are particularly bad at prediction. The subjects consisted of 1,100 soccer fans asked to forecast how teams progressed in various competitions. In particular depressed people tend to over-weight unlikely events. So if you are a depressed individual and an Arsenal fan (and you can see why those two would go together) be prepared for disappointment.

WePredict

Using Twitter for macro-level analysis has been discussed on the blog before:
(i) Sample Selection, Twitter's Public Timeline, TweetScan and Quotably
(ii) Twilert (re-launched this month), Summize Labs, Twitter's acquisition of Summize
(iii) Life Analytics: Sentiment on the United States Economy
(iv) The Google-Index of Social Media, and the apparent superiority of Bing for deciphering real-time breaking trends

So it was interesting to read a recent article in the Irish Times about two students who used Twitter to predict that Joe McElderry would win the recent X-Factor competition... before the results were announced. "Ben McRedmond (17) and Patrick O’Doherty (16), two fifth years from Gonzaga College, Dublin demonstrated the power of their social networking analysis system, We Predict, at the BT Young Scientist and Technology Exhibition. It was developed over more than five months and is based on storing and studying a growing database of 24.5 million tweets which hold clues about what people are thinking."

A quick search led me to the WePredict website. The information there states that: "WePredict is a showcase of several technologies we have built: a data mining application, capable of mining data from multiple social networks; and a complex suite of analysis tools for analyzing this data...WePredict's large database of over 22 million status updates growing at 20 a second makes it the largest user survey ever done... using the collective intelligence of the whole internet, a mere 1.5 billion people, we can predict the outcomes of elections, talent shows or who will be the christmas #1 and analyze the public reaction to new legislation or medical epidemics."

An exciting endeavour such as this one reminds me of Hal Varian's famous quote that "the sexy job in the next ten years will be statisticians." A two-minute YouTube clip of Google's Chief Economist is shown below, discussing this very issue.