"I Wanted to Predict Elections with Twitter and all I got was this Lousy Paper"

Full download available from the arXiv.org website:

"I Wanted to Predict Elections with Twitter and all I got was this Lousy Paper" -- A Balanced Survey on Election Prediction using Twitter Data

Daniel Gayo-Avello

(Submitted on 28 Apr 2012)

Predicting X from Twitter is a popular fad within the Twitter research subculture. It seems both appealing and relatively easy. Among such kind of studies, electoral prediction is maybe the most attractive, and at this moment there is a growing body of literature on such a topic. This is not only an interesting research problem but, above all, it is extremely difficult. However, most of the authors seem to be more interested in claiming positive results than in providing sound and reproducible methods. It is also especially worrisome that many recent papers seem to only acknowledge those studies supporting the idea of Twitter predicting elections, instead of conducting a balanced literature review showing both sides of the matter. After reading many of such papers I have decided to write such a survey myself. Hence, in this paper, every study relevant to the matter of electoral prediction using social media is commented. From this review it can be concluded that the predictive power of Twitter regarding elections has been greatly exaggerated, and that hard research problems still lie ahead.

Whatever Will Tweet Will Be?

A story on China Daily USA came to my attention this evening: about Zhong Lin, a graduate of Tsinghua University; and Zhao Siqi, from Hong Kong University. Both are engineers at Rice University, who designed a computer program that analyses tweets in real time. They hope to use it to predict the winner of the next US presidential election. To date, Lin and Siqi have been working on a project called SportSense, which examines tweets posted by NFL fans to infer what is happening in a game, and how excited the fans are. "It does so in real-time and provides visualized results for live games."

"SportSense is part of a larger project that aims to utilize people as sensors to infer what is happening in the physical world and what people feel about it." SportSense is not the first project to harness the power of Twitter, of course. I posted on the old Geary blog about a Dublin-based start-up called "WePredict". Kevin posted on this blog a few months ago about a study in Science which shows that work, sleep and the amount of daylight people are exposed to all affect mood. There has also been work (by InboxQ) on where the highest concentration of tweeters with the most knowledge about a specific topic are located. WiseWindow is a marketing firm that uses social-media activity to forecast demand for products.

Another group to keep an eye on is Derwent Capital Markets. They use Twitter sentiment to manage their hedge fund. There's been a lot of demand for the fund, according to this article. The strategy is based on an academic study by Johan Bollen (Indiana University), Huina Mao (Indiana University), and Xiao-Jun Zeng (University of Manchester) that established the connection between emotion-related words appearing in Twitter posts and subsequent movements in the Dow Jones Industrial Average. Here's the original research paper.

This article is critical of the approach taken by Derwent. It makes a number of points, one of which is: "Beyond the difficulty of assigning sentiment to tweets, there's a much bigger issue at play. If you look at the patterns of tweets what you find is that most are reactive rather than proactive... Twitter sentiment is likely to be a lagging indicator, at least in the real-time world of algo trading."

Nonetheless, this is an area which has garnered a lot of interest. This BBC story mentions a PhD student at Munich who has done similar work on predicting the stock market (and elections) with Twitter sentiment. The Economist had a piece on the topic in their second Technology Quarterly for this year. That article raised a number of interesting issues; such as the role of meaning in Twitter updates:

Humans excel at extracting meaning and sentiment from even the tiniest snippets of text, a task that stumps machines. To a computer, a tweet that reads “Feeling joyful after my trip to the dentist. Yeah, really” says that the author has been to the dentist and is now happy. Researchers have recently made strides in teaching machines to recognise such sarcasm, as well as double meanings or cultural references. In February Watson, a supercomputer devised by IBM, trounced two human champions at “Jeopardy!”, an American quiz show renowned for the way its clues are laden with ambiguity, irony, riddles and puns. But, for the most part, processing natural language remains a challenge.

While there may be skepticism about the predictive power of Twitter (especially for use in the domains of marketing and finance), there is no doubt that the medium produces a lot of user-generated information. While I am not (yet) a Twitter user, I have been keen to tap into it as a source of information for some time now. Recently, I found the means to do so: inagist is a Twitter-based news-service; probably as useful to both Twitter users and non-users alike. Even better is TweetMinster (due to its automatic updating): it's essentially a twitter-feed about current affairs (London-orientated). I'm following their live feed on breaking news.

TweetMinster tracks "the content most shared between expert users on Twitter and (we) use that data to discover and organise content for our news platform... this is what politicians, civil servants, activists, academics, business analysts and journalists think is the most important news of the day... We also feature live feeds of relevant twitter posts by the expert networks we track, so that you can follow the breaking news stories, big events and trending topics live – even if you’re not on Twitter..."

Addendum: It turns out that Twitter is only part of the story. I just read about a company called "Recorded Future" and blogged about them here: Using the Internet to Predict the Future.

Twitter shows we get up happy but get grumpier as the day wears on

Is there anything that we cannot learn from Twitter? It seems from a study of 509 million posts (wonder what p-value they used for the testing?) on Twitter that while we start the day happy, its downhill all the way. I am going back to bed.

Data Links

Some data related stories in the news recently:

Central Statistics Office investigates early release of data on Twitter

Opening our processes of democracy to scrutiny

Hugh Linehan argues for greater transparency with data that is currently subject to Freedom of Information requests. He mentions the website http://thestory.ie/ which is “… dedicated to sharing documents, combing and combining data and promoting transparency in public life: An experiment in journalism and crowdsourcing hoping to shed light on the government. If you’re spending the Irish taxpayers’ money, you’re on the radar.”

On his website Harvard economist Raj Chetty links to “An Open Letter on Expanding Access to Administrative Data for Research in the United States.” It goes without saying that we have a lot of catching up to do in this regard in Ireland.

WePredict

Using Twitter for macro-level analysis has been discussed on the blog before:
(i) Sample Selection, Twitter's Public Timeline, TweetScan and Quotably
(ii) Twilert (re-launched this month), Summize Labs, Twitter's acquisition of Summize
(iii) Life Analytics: Sentiment on the United States Economy
(iv) The Google-Index of Social Media, and the apparent superiority of Bing for deciphering real-time breaking trends

So it was interesting to read a recent article in the Irish Times about two students who used Twitter to predict that Joe McElderry would win the recent X-Factor competition... before the results were announced. "Ben McRedmond (17) and Patrick O’Doherty (16), two fifth years from Gonzaga College, Dublin demonstrated the power of their social networking analysis system, We Predict, at the BT Young Scientist and Technology Exhibition. It was developed over more than five months and is based on storing and studying a growing database of 24.5 million tweets which hold clues about what people are thinking."

A quick search led me to the WePredict website. The information there states that: "WePredict is a showcase of several technologies we have built: a data mining application, capable of mining data from multiple social networks; and a complex suite of analysis tools for analyzing this data...WePredict's large database of over 22 million status updates growing at 20 a second makes it the largest user survey ever done... using the collective intelligence of the whole internet, a mere 1.5 billion people, we can predict the outcomes of elections, talent shows or who will be the christmas #1 and analyze the public reaction to new legislation or medical epidemics."

An exciting endeavour such as this one reminds me of Hal Varian's famous quote that "the sexy job in the next ten years will be statisticians." A two-minute YouTube clip of Google's Chief Economist is shown below, discussing this very issue.

The Sentiment on the US Economy from Twitter

The Analytic Bridge blog describes an exercise conducted by Life Analytics where 10000 tweets containing the word economy were collected with the purpose of finding out what people think and how they feel about the US Economy and the economic crisis. The exact details of the methodology don't seem to be documented, but the following are some results:

- US President tells that the economy gets better but people don't feel the same.

- Economy cannot be getting better while at the same time there are layoffs.

- People expressing very negative feelings after losing their jobs.

Geary Institute on Twitter

will largely be used to distribute information about upcoming GI events.

link here

Twitter

more or less converted but this is an interim assessment. some potentially useful aspects

- many major organisations now message about seminars, new documents and so on.

- useful way of distributing seminar information to a self-selected audience. in that sense certainly useful for any reasonable size research group who like to get audiences to their seminars or keep people in touch who have a particular interest in what they are doing. for example, i notice some former team members here who are following the tweets and this is a nice way of keeping people in touch who want to be.

- if you pick the sites you wish to follow carefully, you get a neatly filtered overview of important things that have happened during the day or week.

- feel free to defend facebook but twitter is a much less awkward interface for me in that you don't have a lot of private material stored and you dont have to ask people's permission to follow them or vice versa. the twitter market seems a lot smoother in terms of following and dumping contacts as you wish and hard to offend anyone! the emotional investment seems a lot lighter and more appropriate for less personal interactions.


some sites i have found good include Stephen kinsella, Richard Florida, The Economist, Bill Easterly, Brad Delong, New York Times Economix, World Health Organisation. I can do a bigger list at some stage. the ones I follow are on my page below.

link here

Twitter

Tyler Cowen started an interesting discussion on the potential utility of Twitter. I am sceptical enough about it's utility for people involved in academic research, but I am going to give it a try again particularly following Cowen's eloquent defence.

My twitter page is now below and I will be using it almost exclusively for communicating on research and related issues. I will post on it if/when I give up and decide it doesn't have much value. Some potential benefits it may have include:

- keeping people uptodate with events that I am involved with or that I hear about that might be useful

- circulating some interesting documents for discussion

- tapping into very current debate on policy issues

- informing lectures with online discussions

link

Does Facebook Hurt Your Grades? Can Twitter Make You Smarter? What About YouTube?

Does Facebook hurt your grades? This may be the question that parents everywhere will soon be asking their children. There is the argument that students cannot blame Facebook because if Facebook did not exist, they would find another distraction. However, Facebook users under parental supervision will probably have scant praise for the researchers from Ohio State University who have shown that the majority of students who use Facebook every day are underachieving by as much as an entire grade, compared with those who shun the site. See the story in the Times here.

The researchers discovered that students who spend their time accumulating friends, chatting and “poking” others on the site may devote as little as one hour a week to their academic work. “It is the equivalent of the difference between getting an A and a B,” said one of the researchers, Karpinski. Of course, this does not account for selection bias and unobserved characteristics. Also, the researchers have not yet analysed whether a student’s grades continue to deteriorate the longer he or she spends on Facebook. So the same individuals have not been tracked over time.

A new YouTube offering may take up as much of students' time as a Facebook account, though it may help their grades. "YouTube University" has gathered together all the video channels of the universities and colleges it's partnered with over the years and put them under the YouTube EDU banner. One of the videos (below) is from the University of Minnesoata and is about using Twitter, Wikis and instant messaging in the classroom.

Educational researcher Cynthia Lewis says that the use of social media in the classroom serves two purposes. The first is catching the attention of students and increasing their participation in the classroom. The second is preparing them for communicating in tomorrow's new media landscape. She says that "21st century literacies are all about digital media".





While it may be the case that too much Facebook can hurt your grades, doing the right amount of social networking through online platforms can help your career and is also a lot of fun. Plus, it may help learning in the classroom.