Using Internet Search Data

Martin Ryan posted recently (here and here) on the potential predictive power of Twitter and the Internet. Liberty Street Economics, a blog on the Federal Reserve Bank of New York's website has a new post looking at the predictive power of Internet search data.

They look at two potential uses of the data: "now-casting", which aims to provide information on current conditions which bypasses the normal lag periods of official economic and financial data, and more traditional forecasting.

For now-casting, they use Google search data to anticipate a weekly index of mortgage refinancing. They find that "the search index increases the R² by about 10 percentage points and is highly statistically significant, which suggests that the search data have information not captured by the model’s other variables". Real-time information on a weekly index may not be particularly useful, however the appendix cites what may be more interesting research:

"Askitas and Zimmermann (2009) show strong correlations between search data and German unemployment. D’Amuri (2009) of the Bank of Italy finds that an Internet-search-based measure is superior to other leading indicators in predicting Italian unemployment. D’Amuri and Marcucci (2009) find that augmenting models of the U.S. unemployment rate with an Internet job-search indicator outperforms traditional forecasting methods and the Survey of Professional Forecasters. Suhoy (2009) of the Bank of Israel finds search data to be a good predictor of labor market conditions in that country."

They find search data less useful when attempting to forecast: "using Internet search data to predict financial market movements is a more fraught exercise. We could not forecast gold prices, European sovereign spreads, interbank rates, and equity market implied volatility with models using search data". However they have better results in markets that may contain less open information, such as renminbi (the Chinese currency) forecasting. This seems an interesting area, however countries which have restricted economic and/or financial information are also more likely to place restrictions on Internet access and usage.

The post and the appendix contain many more references.

On a side note, the post also helps with language skills: 人民, or rénmín, is people's and 币, or bì, is currency. 人民币 = people's currency, or renminbi. Pronunciation is left to the reader.

One Step Closer to Perfect Information

"MySpace and Facebook have both signed deals with Google to allow publicly available status updates to be indexed in real-time by the search giant...Up till now Google had only signed a similar deal with Twitter and was missing agreements with Facebook and MySpace...This means that when somebody searches for a particular topic on Google they will receive real-time updates from a variety of social media sites, as well as the usual list of search results." Full story here.

The Economics of Attention

Your attention is a scarce resource. In the Information Economy, this is more critical than ever. Before I elaborate on the economics of attention, I want to motivate its importance based on previous posts about scarcity in resources that are not land, labour, capital or enterprise. We've heard about the knowledge economy, information economics and time-allocation. While knowledge, information and time are important, it may be time to turn our attention to, well... attention - the new scarce resource.

To illustrate how economists think about 'non-traditional scarce resources', I'll begin with a discussion on time use (or time-allocation). This resource is the closest in essence to attention (with knowledge and information being more similar in their nature; also, attentive use of time is needed to acquire knowledge and sift through information). Time-allocation is the first-stage decision in various production functions; on top of this comes the application of attention. As time-use has been discussed quite a bit on this blog, it's a good place to start. Here's a paper about student time use and academic performance; here's one about whether the recession is making students study harder.

More recently there was a link to the presentation at Google by Zimbardo and Boyd on time perspectives. In the MetaPsychology review of Zimbardo and Boyd's book, the point is emphasised that time cannot be replenished - in contrast to other goods, such as gold, diamonds, etc. Therefore, you have to be more careful about how to spend your time. Zimbardo and Boyd ask: "Why do we often spend our money more wisely than our time?"

I agree with Michael Daly that the ability to recognise our time perspectives and to switch between perspectives efficiently is important (see a brief summary of the perspectives here). Aaron Swartz suggests that "time perspective” is a concept which Zimbardo has essentially come up with himself, but which he feels (not without some justification) is rather important.

I'm inclined to agree - after reading about and working with the consideration of future consequences scale (CFC) a lot - "time perspective" seems to offer a more comprehensive theory of how we interact with "time". There of course may be scope to use both the CFC and the Zimbardo Time Perspective Inventory in a research project - a recent article by Adams and Nettle in the British Journal of Health Psychology does just that.

In addition to time-allocation and time perspectives, 'attention' plays an important role in how human behaviour produces outcomes. The most illustrative example I can think of is the recent debate on student time use and online social networking. The question that we debated was "does Facebook hurt your grades?". The conclusion offered by Aryn Karpinski (at Ohio State University) is that her research does not suggest that Facebook directly causes lower grades, merely that there's some relationship between the two factors. "Maybe [Facebook users] are just prone to distraction. Maybe they are just procrastinators".

An attempt has been made to replicate the results of Karpinski's research. Eszter Hargittai, associate professor of communication studies at Northwestern University and a fellow this year at Harvard's Berkman Center for Internet & Society, says the following: "We found no evidence that Facebook use correlates with lower academic achievement". More details on this are available here.

The main point I want to make is based on the opening comment in this post: that attention is a scarce resource. While students may allocate their time across activities such as study and online social networking, how much attention are they giving to each activity? Are they so zoned into Facebook that that they can't cope with the Twitter Mania, but when it comes to study is the intensity a lot less? Is designated study time interrupted by SMS text messages, tweets and the like? Or is there just less concentration applied to the activity of study?

On his MindBlog, Derek Bownds states that concentration is the issue of our time(s) (sic). He mentions a NYT article by John Tierney on the science of concentration. A clip from this is below:

"You can lead a miserable life by obsessing on problems. You can drive yourself crazy trying to multi-task and answer every e-mail message instantly...Or you can recognize your brain’s finite capacity for processing information, accentuate the positive and achieve the satisfactions of... the focused life."

With attention and concentration such critical issues in the production functions for study, research and a whole range of activities that feed into the knowledge economy, I have taken a renewed interest in mindfulness - something which Michael has mentioned on the blog. A number of times. Liam also linked to a lecture by Kabat-Zinn on mindfulness - originally presented to staff at Google.

The framework I am most interested in, however, is what this blog post is essentially about - the economics of attention. Attention economics is an approach to the management of information that treats human attention as a scarce commodity, and applies economic theory to solve various information management problems. It's not new - Herbert Simon was perhaps the first person to articulate the concept of attention economics when he wrote:

"...in an information-rich world, the wealth of information means a dearth of something else: a scarcity of whatever it is that information consumes. What information consumes is rather obvious: it consumes the attention of its recipients. Hence a wealth of information creates a poverty of attention and a need to allocate that attention efficiently among the overabundance of information sources that might consume it" (Simon 1971, p. 40-41).

Simon noted that many designers of information systems incorrectly represented their design problem as information scarcity rather than attention scarcity, and as a result they built systems that excelled at providing more and more information to people, when what was really needed were systems that excelled at filtering out unimportant or irrelevant information (Simon 1996, p. 143-144). We have such systems now - you can skip the ads on TV if you want! You can also block out email spam.

"Attention economics" today is largely concerned with the problem of getting consumers to engage with advertising (sellers compete for scarce attention). However, it would be worth developing a stream in attention economics that relates to academic production functions, the knowledge economy, health behaviour and well-being. A final thought: we should all be careful with those little drops of dopamine. And bear in mind that business is going to get your attention using style, for style is what competes for our attention amidst the din and deluge of the new media (see Richard A Lanham - The Economics of Attention).


Simon, H. A. (1971), "Designing Organizations for an Information-Rich World", written at Baltimore, MD, in Martin Greenberger, Computers, Communication, and the Public Interest, The Johns Hopkins Press, ISBN 0-8018-1135-X.

The Text Tax

It has been suggested by the Green Party that a 1c tax on text messages would raise some much needed funds for the public purse, in the current economic crisis. This has been referred to as as “unfair and injust” by Tommy McCabe, director of the Irish Cellular Industry Association (ICIA) - see story here.

How much revenue do we think this would generate? According to the Irish Examiner, a record two billion text messages were sent by Irish mobile phone users in the final three months of last year. Say we assume that this is a steady state level of texting, and that a 1c tax would not deter anyone to send a text. With these assumptions made, then the 1c text tax would produce 8 billion cents in revenue per annum, or an annual sum of 80 million euro.

This is all well and good, but I would like to know more about how this tax would be collected. What I have been able to find is a news story from 2006 which suggests that European Union lawmakers have already considered tax on e-mails and text messages as a way to fund the 25-member bloc in the future. Also, a text message tax was introduced in Sacramento, California last December (see story here). The city sent out letters to telecommunications companies to instruct them to levy the tax on customers' bills.

This is an interesting development in the economics of information. While I don't yet have any fears about negative consequences for the widespread distribution of information, comminication taxes could be undesirable if they prevent useful information exchange. Especially in the so-called Information Economy. On a related note, it was announced yesterday that UCD won SFI strategic research cluster funding of €3.56 million - which will be focused on "data analytics".