"I Wanted to Predict Elections with Twitter and all I got was this Lousy Paper"

Full download available from the arXiv.org website:

"I Wanted to Predict Elections with Twitter and all I got was this Lousy Paper" -- A Balanced Survey on Election Prediction using Twitter Data

Daniel Gayo-Avello

(Submitted on 28 Apr 2012)

Predicting X from Twitter is a popular fad within the Twitter research subculture. It seems both appealing and relatively easy. Among such kind of studies, electoral prediction is maybe the most attractive, and at this moment there is a growing body of literature on such a topic. This is not only an interesting research problem but, above all, it is extremely difficult. However, most of the authors seem to be more interested in claiming positive results than in providing sound and reproducible methods. It is also especially worrisome that many recent papers seem to only acknowledge those studies supporting the idea of Twitter predicting elections, instead of conducting a balanced literature review showing both sides of the matter. After reading many of such papers I have decided to write such a survey myself. Hence, in this paper, every study relevant to the matter of electoral prediction using social media is commented. From this review it can be concluded that the predictive power of Twitter regarding elections has been greatly exaggerated, and that hard research problems still lie ahead.

Google Trends, Polls, Betting, Prediction Markets and the British Election

Those interested in Google Trends may find it worthwhile to bear the first chart (directly below) in mind, as the results of the British Election come through. The red line represents "Gordon Brown", the orange line "Nick Clegg", and the blue line "David Cameron".


A different picture is illustrated in the second chart (directly below). The red line represents "Labour Party", the orange line "Liberal Democrats", and the blue line "Conservative Party".


Of course, this type of information may have no predictive value. The UK Polling Report (an independent survey and polling news website) shows that the majority of polls paint a closer picture, but with a distinct lead for the Conservative Party. Could the higher search (and news) reference volume for "Gordon Brown" be due to an incumbency effect? Could the higher search (and news) reference volume for "Liberal Democrats" be due to a novelty effect? Of course, we can't attempt an answer until later in the week. Many punters have already placed their bets though: bookmakers estimate that up to £40m will have been bet on this election, smashing previous records.

Finally, the Intrade prediction market indicates that the Conservative Party have roughly a 90% chance of winning, as shown in the third chart (directly below).


The Inkling Prediction Market indicates here that there is a 66% chance of a hung parliament, and a 32% chance of a win by the Conservative Party.

Hal Varian, Chief Economist at Google, discusses prediction markets here in the NYT, from a few years ago. The emphasis in the article is on the "Pentagon-sponsored futures market in terrorism indicators (that) was announced and squashed in all of two days."