Workshop on Adjusting for Non-Ignorable Missing Data using Heckman-Type Selection Models

Workshop on Adjusting for Non-Ignorable Missing Data using Heckman-Type Selection Models

Harvard University, September 8^th 2015 0900 – 1800

Background

Missing data is common problem in survey data, and standard approaches for dealing with this issue rely on the strong and generally untestable assumption that data are ignorable (missing at random) once we condition on the observed characteristics of respondents. The assumption of missing at random is often implausible, including in contexts where the outcome itself may be a predictor of survey participation. For example, estimates of HIV prevalence which rely on data collected from blood tests taken from respondents in nationally representative household surveys may be affected by selection bias if those who are HIV positive are less likely to participate in testing. Then, conventional adjustments for missing data, such as using imputation or inverse-probability weighting, will result in biased estimates because of an incorrect assumption of missing at random. Standard approaches are also likely to result in confidence intervals which are too narrow because they ignore the uncertainty surrounding the unknown relationship between participation and the outcome, which needs to be estimated.    

Workshop

This workshop will introduce the use of Heckman-type Selection models for adjusting for non-ignorable missing data with the goal of making this approach easily accessible to researchers working with survey data affected by non-participation. A non-technical introduction to different approaches for dealing with missing data will be provided, and we will discuss the implications of not correctly adjusting for missing data which are not missing at random. We will provide an overview of the statistical rationale for the use of selection models, and the R package SemiParBIVProbit will be presented. This software allows researchers to implement this approach in a straightforward and transparent manner in a variety of different contexts affected by missing data. A simulation study will also be used to demonstrate the properties of the model. The final session will be interactive where participants are invited to bring their own datasets, and the audience and presenters will work together on implementing this approach in their own research. Alternatively, the organizers will provide example data. Throughout, we will illustrate the key concepts using data from HIV research.

Invitation

The workshop is free and open to all interested parties, however space is limited so if you would like to attend please register with Mark McGovern (mcgovern@hsph.harvard.edu). The workshop will take place at Harvard on September 8^th, exact location to be confirmed. Unfortunately we do not have the funds to cover expenses of participants.

Organizers

Harvard University: Till Bärnighausen, Guy Harling, Mark McGovern

University College London: Giampiero Marra

University of London Birbeck: Rosalba Radice

Agenda

Time	Topic
0900-0930	Introductions and Background
0930-1015	Implications of Non-Ignorable Missing Data for Parameter Estimates
1015-1030	Break
1030-1130	Introduction to Selection Models
1130-1230	Overview of Applications of Selection Models
1230-1300	Lunch
1300-1330	Optional Session on Getting Started with R
1330-1415	Introduction to R Package SemiParBIVProbit
1415-1445	Simulation Studies
1445-1500	Break
1500-1800	Interactive session with Data from Participants or Data Provided by Organizers

Key References

Bärnighausen, T., Bor, J., Wandira-Kazibwe, S., & Canning, D. (2011). Correcting HIV Prevalence Estimates for Survey Nonparticipation using Heckman-type Selection Models. Epidemiology, 22(1), 27-35. http://www.ncbi.nlm.nih.gov/pubmed/21150352

 

Marra, G., Radice, R., Till, B., Wood, S., McGovern, M., 2015. A Unified Modeling Approach to Estimating HIV Prevalence in Sub-Saharan African Countries. Research Report 324, Department of Statistical Science, University College London. http://www.ucl.ac.uk/statistics/research/pdfs/rr324.pdf

McGovern, M., Bärnighausen, T., Marra, G., Radice, R., 2015. On the Assumption of Bivariate Normality in Selection Models: A Copula Approach Applied to Estimating HIV Prevalence. Epidemiology 26, 229–327. http://www.ncbi.nlm.nih.gov/pubmed/25643102

 

Marra, Giampiero, and Rosalba Radice, 2015. A Regression Modeling Framework for Analyzing Bivariate Binary Data: The R Package SemiParBIVProbit. http://www.homepages.ucl.ac.uk/~ucakgm0/SemiParB.pdf

 

McGovern, M. E., Bärnighausen, T., Salomon, J. A., & Canning, D. (2015). Using Interviewer Random Effects to Remove Selection Bias from HIV Prevalence Estimates. BMC Medical Research Methodology, 15(1), 8. http://www.biomedcentral.com/1471-2288/15/8/

Robustness in health research: Do differences in health measures, techniques, and time frame matter?

Robustness in health research: Do differences in health measures, techniques, and time frame matter?
Paul Frijters, Aydogan Ulker
Journal of Health Economics
Volume 27, Issue 6, December 2008, Pages 1626–1644

Abstract
Survey-based health research is in a boom phase following an increased amount of health spending in OECD countries and the interest in ageing. A general characteristic of survey-based health research is its diversity. Different studies are based on different health questions in different datasets; they use different statistical techniques; they differ in whether they approach health from an ordinal or cardinal perspective; and they differ in whether they measure short-term or long-term effects. The question in this paper is simple: do these differences matter for the findings? We investigate the effects of life-style choices (drinking, smoking, exercise) and income on six measures of health in the US Health and Retirement Study (HRS) between 1992 and 2002: (1) self-assessed general health status, (2) problems with undertaking daily tasks and chores, (3) mental health indicators, (4) BMI, (5) the presence of serious long-term health conditions, and (6) mortality. We compare ordinal models with cardinal models; we compare models with fixed effects to models without fixed-effects; and we compare short-term effects to long-term effects. We find considerable variation in the impact of different determinants on our chosen health outcome measures; we find that it matters whether ordinality or cardinality is assumed; we find substantial differences between estimates that account for fixed effects versus those that do not; and we find that short-run and long-run effects differ greatly. All this implies that health is an even more complicated notion than hitherto thought, defying generalizations from one measure to the others or one methodology to another.

JEL classification C23; C25; I31; Z1
Keywords Morbidity; Mortality; Lifestyle; Income

Ungated Version

Avoiding Invalid Instruments and Coping with Weak Instruments

Useful advice for when trying to implement IV.

Avoiding Invalid Instruments and Coping with Weak Instruments
Michael P. Murray
Journal of Economic Perspectives—Volume 20, Number 4—Fall 2006—Pages 111–132

Archimedes said, “Give me the place to stand, and a lever long enough, and I will move the Earth” (Hirsch, Kett, and Trefil, 2002, p. 476). Economists have their own powerful lever: the instrumental variable estimator. The instrumental variable estimator can avoid the bias that ordinary least squares suffers when an explanatory variable in a regression is correlated with the regression’s disturbance term. But, like Archimedes’ lever, instrumental variable estimation requires both a valid instrument on which to stand and an instrument that isn’t too short (or “too weak”). This paper briefly reviews instrumental variable estimation, discusses classic strategies for avoiding invalid instruments (instruments themselves correlated with the regression’s disturbances), and describes recently developed strategies for coping with weak instruments (instruments only weakly correlated with the offending explanator).

http://www.eui.eu/Personal/Guiso/Courses/Econometrics/Murray_IV_jep_06.pdf

Discrete Choice Course in NUIG

Discrete Choice Modeling

Professor William Greene

Stern School of Business, New York University

at

National University of Ireland, Galway

with funding from NUI Galway's Millennium Fund

July 4-6, 2012

The National University of Ireland Galway, J.E. Cairnes School of Business and Economics is delighted to host a three day intensive course on 'Discrete Choice Modeling' with Professor William Greene of the Stern School of Business at New York University. Discrete choice models have become an essential tool for the analysis of individual choice behavior and can be applied to choice problems in a wide variety of diverse fields including environmental management, urban planning, transportation, energy, telecommunications, and healthcare. This course will present the most recent developments in theory and methods of estimation for discrete choice models. A number of applications from different areas of the professional literature to illustrate these techniques will be discussed.

The presentation will include roughly ten morning classroom meetings. In the afternoon of each day, we will do some hands on analysis using “live” data sets and a familiar computer package.

Course Fee

The course fee is €350 if registered on or before May 15th 2012 and €420 thereafter. Places are limited to 50 so early booking is recommended. The fee includes refreshments throughout the course.

Conference Venue

The conference will be held at the National University of Ireland, Galway in the Aras Moyola Building.

Conference 2012

Immediately prior to the econometrics summer school on July, 3rd 2012, NUI Galway is holding the Applied Microeconometrics and Public Policy Conference. Please click here for details.

Volvo Ocean Race

Galway is a vibrant and exciting city all year round. However, if you need an added incentive for your trip, the final leg of the Volvo Ocean Race is due to arrive in Galway a few days before the start of the course. This will attract tens of thousands of visitors to Galway with many special events planned over a two week period. Delegates are recommended to book early to obtain best value in hotel rates.

Further Information

Contact conference secretariat: Trish Carney
Email: p.carney4@nuigalway.ie

AEJ Journal Applied

The January edition of the AEJ: Applied is available on this link. The journal policy means that the data-sets and do-files used to create the results are made available. As well as promoting transparency, this is also potentially a very good resource for graduate teaching.

Greene Microeconometrics at Galway

NUIG have done a good service by organising this. I could not recommend this highly enough to any graduate student in Europe who is working on applied microeconometrics. The lecturer is among the most highly regarded econometrics professors in the world and has written some of the most widely used textbooks and software. For those outside of Ireland, Galway is a gem of a place to visit and this would be a good excuse to do so.

Topics in Microeconometrics

Professor William Greene

Stern School of Business, New York University

at

National University of Ireland, Galway

June 1-3, 2011

The National University of Ireland Galway, J.E. Cairnes School of Business and Economics is delighted to host a three day intensive course on 'Topics in Microeconometrics' with Professor William Greene of the Stern School of Business at New York University.

This course will introduce the student to methods and models used to analyze cross section and panel data. We will depart from the linear regression model to specifications for binary and censored data, ordered choices, count data and multinomial choices. The discussion will present basic models for cross section data then introduce theory and methods for extensions to panel data and stated choice experiments.

The presentation will include roughly ten morning classroom meetings. In the afternoon of each day, we will do some hands on analysis using “live” data sets and a familiar computer package.

Course Fee

The course fee is €350 if registered on or before April 15th 2011 and €420 thereafter. Places are limited to 50 so early booking is recommended. The fee includes refreshments throughout the course.

Conference Venue

The conference will be held at the National University of Ireland, Galway in the Aras Moyola Building.

Further Information

Contact conference secretariat: Sinéad Keogh
Email: s.keogh4@nuigalway.ie

Tesco Metrics: Every Little Bit of Data Helps

Liam linked to an article in the Guardian earlier this week, which was all about Nudge. One comment in the article was that "while shopping, working, or even deciding on who to share their lives with, individuals are less thoughtful and less calculating than modern-day economists... typically assume." This blog-post zones in on shopping, in particular the data-analysis of consumer purchasing behaviour at Tesco. The Guardian article linked above also suggests that "any critic who points out that that's hardly news to the women...(and) the men at Tesco... is spot on." Indeed, Tesco have been conducting interesting micro-level analysis on individual behaviour for many years now.

An informative article on this topic was written by Jenny Davies in the Sunday Times last year. According to Davies, Tesco gets its data from its loyalty clubcard scheme; this was launched 15 years ago with much fanfare - the advert below may jog memories for some readers. Davies also informs us that around this time last year, Tesco was tracking "the shopping habits of 16 million families across Britain, delivering an extraordinary insight into their lives — not only for itself but for companies such as Coca-Cola, Nestlé and Unilever, which buy the rights to the data." Readers in the Republic of Ireland might also remember that the Tesco Clubcard was launched there on the 13th. Oct 1997. To date almost 800,000 members have joined in the Republic.



Jenny Davies also tells us that: "Each bill detailing every item in a customer’s shopping basket is logged in a data centre in London Docklands and decoded by Dunnhumby, the marketing firm that is in charge of the scheme. It has to process 100 baskets a second — six million transactions a day. This helps Tesco to decide which products should go on to the shelves at what times, and in early trials it increased sales by as much as 12% in some of the supermarkets." According to the Guardian (in this article), the power of the clubcard was demonstrated in 2009, "when Tesco harnessed the card's database to halt the exodus of shoppers to cheaper retailers because (of) the recession, by doubling the points available to shoppers."

In a blog-post on Tesco data from two years ago, Tony Hirst desribes the early analysis conducted by Dunnhumby, and how this has changed over the last 15 years. A couple of months ago, Dunnhumby (and its recently departed co-founders) were profiled in the Guardian. The article says:

According to company lore, there was a 30-second silence after Humby presented the initial trial's results to the Tesco board, until the then chairman, Lord MacLaurin, declared: "What scares me is that you know more about my customers after three months than I know after 30 years."

One question that readers might have is: what's in it for club-card holders? According to Tony Hirst, a good place to get an answer to this question is the book: Scoring Points: How Tesco Continues to Win Customer Loyalty. Hirst describes the "Clubcard customer contract: more data means better segmentation, means more targeted/personalised services, means better profiling. In short, the more you shop with us, the more benefit you will accrue." According to the Marketing Week magazine, "from the day of its launch in February 1995 the Tesco Clubcard was immediately embraced by customers attracted to the 1% discount off their shopping bills. But its long term success has not been built on discounts alone, rather on the personalisation of the shopping experience."

However, perhaps the last word should go to UCD social psychologist Ken McKenzie, writing on his A Head in Business Blog: "I don’t have a loyalty card, and every time I’m in Boots, Tesco or Dunnes, and they ask if I have one, I feel a slight sense that I should justify why I don’t, as it it’s odd to not have one. And according to rational actor theory in Economics, it is odd to not have a loyalty card and avail of discounts. However, there’s a small but growing body of work in the overlapping area between Psychology and Economics that might explain why (some) people might behave like me."

On a lighter note

Fixed or Random Effects?

New IZA Working Paper

The Choice Between Fixed and Random Effects Models: Some Considerations for Educational Research

Paul Clarke, Claire Crawford, Fiona Steele, Anna Vignoles

Abstract:
We discuss fixed and random effects models in the context of educational research and set out the assumptions behind the two approaches. To illustrate the issues, we analyse the determinants of pupil achievement in primary school, using data from the Avon Longitudinal Study of Parents and Children. We conclude that a fixed effects approach will be preferable in scenarios where the primary interest is in policy-relevant inference of the effects of individual characteristics, but the process through which pupils are selected into schools is poorly understood or the data are too limited to adjust for the effects of selection. In this context, the robustness of the fixed effects approach to the random effects assumption is attractive, and educational researchers should consider using it, even if only to assess the robustness of estimates obtained from random effects models. When the selection mechanism is fairly well understood and the researcher has access to rich data, the random effects model should be preferred because it can produce policy-relevant estimates while allowing a wider range of research questions to be addressed. Moreover, random effects estimators of regression coefficients and shrinkage estimators of school effects are more statistically efficient than those for fixed effects.

http://ftp.iza.org/dp5287.pdf

Economics at Yahoo!

I've blogged before about Hal Varian, Chief Economist at Google, and how his role in the organisation has been central to Google's business model. I have also mentioned pioneering work at Yahoo! Research on the effectiveness of online advertising, the battle between Google, Yahoo! and Bing in the economics of internet search, and the statement by Yahoo! Research that they routinely compete for talent with the top ten economics departments in the world.

However, it has only now come to my attention that Varian has a counterpart at Yahoo! The economist in question is Preston McAfee, on leave from the position of J. Stanley Johnson Professor of Business, Economics, and Management at the California Institute of Technology. At Yahoo!, Professor McAfee is Vice President and Research Fellow at Yahoo! Research in Burbank, CA, where he leads a group focused on microeconomics research. Here is a list of recent publications from the Microeconomics and Social Systems cluster at Yahoo!

Professor McAfee wrote Introduction to Economic Analysis, a free, open-source text that spans both principles and intermediate microeconomics. In 1994, the FCC in the USA auctioned access to a number of radio frequencies for new communications services, using an auction designed by Paul Milgrom, Robert B. Wilson, and Professor McAfee, and raised over $17 billion. This auction design was copied around the world. McAfee, Milgrom, Wilson and John McMillan (1951-2007) formed a company, Market Design, Inc., that advises governments on how to maximize the return from sales of radio frequencies, mineral rights, airports, and other assets.

Returning to Yahoo!, The Register magazine describe the company's Right Media exchange — a display advertising marketplace that matches advertisers with publishers and ad networks — as (by one measure) the largest exchange in the world, running over nine billion auctions each day. Finally, the Yahoo! Advertising Blog is also an interesting read; the current post - Mad Men No More - features a discussion by "advertising’s new guard" on how they are re-defining the industry.

A Reminder About The Dangers of Interpreting Interaction Effects in Non-Linear Models

"The magnitude of the interaction effect in nonlinear models does not equal the marginal effect of the interaction term, can be of opposite sign, and its statistical significance is not calculated by standard software. We present the correct way to estimate the magnitude and standard errors of the interaction effect in nonlinear models."

Ai, Chunrong & Norton, Edward C., 2003. "Interaction terms in logit and probit models," Economics Letters, Elsevier, vol. 80(1), pages 123-129, July.

The Stata programme inteff is recommended.

John Frain - STATA with Econometricians in Mind

Thanks to Enda Hargaden for sending on this useful link to a recently released guide on STATA for Econometrics written by John Frain. Recommended for beginners in particular.

Abstract

This paper is an introduction to Stata with econometrics in mind. One aim of the proposed methodology is the keeping of appropriate records so that results can be easily replicated. These records should meet the requirements of management and internal audit functions in policy making bodies and be sufficient for submission to journals that require such material. The paper describes the Stata desktop, shows how to organise an analysis, how to read and transform data and covers the OLS regression command in detail. It includes details of various post-estimation commands, specification tests, model verification procedures, calculation of elasticities and other marginal effects, forecasting and the use of various statistics used by Stata during the estimation procedure. As all estimation commands in Stata share a common structure the detailed study of the OLS command will assist in the use of other commands.

World Econometric Congress Programme

Colm sent the link to the programme for this year's Econometric Society Congress. Even if you can't make it to Shanghai, the programme itself is well worth looking through and many of the papers are available in various working paper series that can be found with some googling. There are very many sessions relevant to different people who read this blog. Below are three examples.

HUMAN CAPITAL (LAB)
Chairperson: Sebastian Buhai, Northwestern University and Aarhus University

EXPLAINING PERSONALITY PAY GAPS IN THE UK
Alita Nandi, ISER, UNIVERSITY OF ESSEX
Co-Author: Cheti Nicoletti, ISER, University of Essex

NONPARAMETRIC BOUNDS ON RETURNS TO EDUCATION IN SOUTH AFRICA: OVERCOMING ABILITY AND SELECTION BIAS
Martine Mariotti, Australian National University
Co-Author: Juergen Mienecke, Australian National University

JOB HAZARD PAY AND WORKER RISK ATTITUDES
Sebastian Buhai, Northwestern University and Aarhus University
Co-Author: Elena Cottini, Catholic University Milan

EDUCATION DECISIONS (LAB)
Chairperson: Chao Fu, University of Pennsylvania

RISK AVERSION AND SCHOOLING DECISIONS
Marco Leonardi, University of Milan
Co-Author: Christian Belzil, Ecole Polytechnique

MODELING COLLEGE MAJOR CHOICES USING ELICITED MEASURES OF EXPECTATIONS AND COUNTERFACTUALS
V. Joseph Hotz, Duke University
Co-Authors: Peter Arcidiacono, Duke University and Songman Kang, Duke University

EQUILIBRIUM TUITION, APPLICATIONS, ADMISSIONS AND ENROLLMENT IN THE COLLEGE MARKET

EARLY CHILDHOOD INTERVENTIONS (LAB)
Chairperson: Gabriella Conti, University of Chicago

UNDERSTANDING THE MECHANISMS THROUGH WHICH AN INFLUENTIAL EARLY CHILDHOOD PROGRAM BOOSTED ADULT OUTCOMES
Peter Savelyev, The University of Chicago, Department of Economics
Co-Authors: James Heckman, University of Chicago, Lena Malofeeva, University of Arizona and Pinto Rodrigo, University of Chicago

THE IMPACT OF IODINE DEFICIENCY ERADICATION ON SCHOOLING: EVIDENCE FROM THE INTRODUCTION OF IODIZED SALT IN SWITZERLAND
Dimitra Politi, University of Edinburgh

A STRUCTURAL MODEL OF CHILD CARE CHOICES, MATERNAL TIME AND CHILD’S COGNITIVE DEVELOPMENT FOR SINGLE MOTHERS IN THE U.S.
Raquel Bernal, Universidad de los Andes
Co-Author: Michael Keane, University of Technology Sydney

EARLY ENDOWMENTS, EDUCATION AND HEALTH
Gabriella Conti, University of Chicago
Co-Authors: James Heckman, University of Chicago and Sergio Urzua, Northwestern University

Aldrich - Econometrics and Psychometrics– Rivers out of Biometry

Econometrics and Psychometrics– Rivers out of Biometry
Date:    2010-06-01
By:    Aldrich, John
URL:   link here
At the beginning of the 20th century economists and psychologists began to use the statistical methods developed by the English biometricians. This paper sketches the development of psychometrics and econometrics out of biometry and makes some comparisons between the three fields. The period covered is 1895-1925.

Keywords; History of econometrics, statistics, biometry, factor analysis, path analysis.

JEL Classification: B816.

JEP Symposium on Causal Econometrics

The JEP symposium linked below discusses Angrist and Pischke's (AP) paper on IV and other experimental and quasi-experimental methods as key developments in modern microeconomics (the "taking the con out of econometrics" paper). I have put together a few slides on this and we will meet next Friday 2nd July at 11am in Geary boardroom to talk through the papers. This session will run till 1pm. It is completely informal and a chance for people working here to get their heads around the key issues in these crucial debates. It is not essential to read all of the papers in advance though it would be good to read the AP paper in detail and at least one of the other papers. I will put up two slides on each of the replies and attempt to summarise them.

JEP Symposium (requires subscription)

Clustering standard errors

Using standard errors as taught to undergraduates can lead to incorrect inference if information is replicated in some way, such as applying GDP statistics to household-level data. Stata's useful cluster option is a common method of addressing this issue.

As much of the work produced by the Geary Institute is microeconometric in nature, researchers may be interested in two recent papers on the topic.

Andrew Gelman cites a paper that suggests cluster'ing is inadequate and that multi-level modelling should be preferred, while Barrios, Diamond, Imbens and Kolesar (2010) suggest researchers should also be wary of spatial correlations.

Important statistical development: ordinary least squares a mistake

In an important new development, an archivist in the University of Gottingen has discovered a "typo" in a manuscript by Gauss. It transpires that he actually proved that OLS is neither unbiased nor efficient thus rendering 99.5% of econometrics invalid.

Angrist and Pischke: NBER Paper, The Credibility Revolution

The Credibility Revolution in Empirical Economics: How Better Research Design is Taking the Con out of Econometrics

Joshua Angrist, Jörn-Steffen Pischke

NBER Working Paper No. 15794*
Issued in March 2010

This essay reviews progress in empirical economics since Leamer’s (1983) critique. Leamer highlighted the benefits of sensitivity analysis, a procedure in which researchers show how their results change with changes in specification or functional form. Sensitivity analysis has had a salutary but not a revolutionary effect on econometric practice. As we see it, the credibility revolution in empirical work can be traced to the rise of a design-based approach that emphasizes the identification of causal effects. Design-based studies typically feature either real or natural experiments and are distinguished by their prima facie credibility and by the attention investigators devote to making the case for a causal interpretation of the findings their designs generate. Design-based studies are most often found in the microeconomic fields of Development, Education, Environment, Labor, Health, and Public Finance, but are still rare in Industrial Organization and Macroeconomics. We explain why IO and Macro would do well to embrace a design-based approach. Finally, we respond to the charge that the design-based revolution has overreached.

A Few Links

1. MySpace has allowed a large quantity of bulk user data to be put up for sale: user playlists, mood updates, mobile updates, photos, vents, reviews, blog posts, names and zipcodes. Friend lists are not included.

2. InfoChimps is a bulk data marketplace with more than 5000 data sets in its catalog so far. The vast majority are free.

3. A Mulley Communications/NCI study shows that people do not pay much attention to more than three results on a Google search result page. Also that the ads on the right hand side of the results page are barely looked at.

4. Here is a video of heatmaps being generated based on eye movements.

5. It's worth looking at data-series on capital expenditure: in this case in the UK. A pick-up in this series should lead improvements in employment figures.

6. Ellerdale, still in alpha testing, tracks data sources from around the web, primarily Twitter, and examines what topics are being discussed. It then organizes these conversations into categories like "people," "sports," "politics," "music," "television," and more.

7. The most rented movie in Chicago last year? "The Curious Case of Benjamin Button." This app on the NTY website allows for examination of Netflix rental patterns, neighborhood by neighborhood, in a dozen cities.

8. Netflix have announced that they canceling plans for a second Netflix Prize contest, one that would have involved the release of more information than the first.

9. Privacy concerns were an issue in the Netflix decision. Among the first to draw attention to the issue was University of Colorado law professor Paul Ohm, who said: "Researchers have known for more than a decade that gender plus ZIP code plus birthdate uniquely identifies a significant percentage of Americans (87% according to Latanya Sweeney's famous study)."

10. Here's a recent paper by Ohm on re-identification of individual data.

11. The deadline for Facebook's Ph.D. Fellowship Program has passed, but it's interesting to note that they have a particular focus on Internet Economics.

How Google Does Business...

1. "What could be more baffling than a capitalist corporation that gives away its best services, doesn't set the prices for the ads that support it, and turns away customers because their ads don't measure up to its complex formulas?". Read how economics underlies every aspect of the Google business model: here in Wired.

2. AdWords is a pioneering variation on a second-price auction.

3. AdWords was such a hit that Google used auctions to place ads on other websites: AdSense.

4. Hal Varian, Chief Economist at Google, has been mentioned on the blog before: here and here.

5. "But the really gutsy move," Hal Varian says, "was using it in the IPO." In 2004, Google used a variation of a Dutch auction for its initial public offering.

6. The Google equivalent of the Consumer Price Index is called the Keyword Pricing Index. Here's a link about Fathom Online's version. Examples of very competitive keywords are 'flowers' and 'hotels'.

7. Quality Scores are important: a penalty is invoked when the ad quality is too low. In such cases, the company slaps a minimum bid on the advertiser.

8. Hal Varian says: "The people working for me are generally econometricians—sort of a cross between statisticians and economists". He's currently hiring a senior economist. The London office also has other opportunities.