Avoiding Invalid Instruments and Coping with Weak Instruments

Useful advice for when trying to implement IV.

Avoiding Invalid Instruments and Coping with Weak Instruments
Michael P. Murray
Journal of Economic Perspectives—Volume 20, Number 4—Fall 2006—Pages 111–132

Archimedes said, “Give me the place to stand, and a lever long enough, and I will move the Earth” (Hirsch, Kett, and Trefil, 2002, p. 476). Economists have their own powerful lever: the instrumental variable estimator. The instrumental variable estimator can avoid the bias that ordinary least squares suffers when an explanatory variable in a regression is correlated with the regression’s disturbance term. But, like Archimedes’ lever, instrumental variable estimation requires both a valid instrument on which to stand and an instrument that isn’t too short (or “too weak”). This paper briefly reviews instrumental variable estimation, discusses classic strategies for avoiding invalid instruments (instruments themselves correlated with the regression’s disturbances), and describes recently developed strategies for coping with weak instruments (instruments only weakly correlated with the offending explanator).

http://www.eui.eu/Personal/Guiso/Courses/Econometrics/Murray_IV_jep_06.pdf

Month of birth & SES

Liam's post below points to differing patterns in the seasonality of birth by family background. Using PISA 2000, I plot month of birth according to the data's measure of SES - using a median split.
There does appear to be differences although they don't look dramatic to me. Around Autumn/Fall they seem fairly similar and this is usually when the cut-off dates for school entry are.

Season of Birth as an Instrument

The use of season of birth as an instrument for education is something that has been debated in Economics for the last 20 years or so, with the basic idea being that season of birth is randomly distributed but can effect school attainment through timing of compulsory school and so on (see the original Angrist and Krueger paper and recent work referenced in the paper below for a much more eloquent summary). A number of papers, particularly Bound and Jaeger (1995) argue the relationship between season of birth and schooling is too weak for it to be used as an instrument. A recent NBER paper by Buckles and Hungerman provides a new argument as to why this instrumentation strategy may not be valid, namely that people with different socioeconomic characteristics time their conceptions differently.

How not to do Instrumental variables

Instrumental Variable estimation, Generalized Method of Moments and related techniques are part of the standard toolkit for applied economists. They are also increasingly used in other fields such as health.
What everyone knows, or should know, is that while one can think of these models as a two stage process this is not actually how you do it. But this paper which looks at how systolic blood pressure depends on anti-hypertensive drugs in Japan, published in the Bulletin of the World Health Organization 2008, gets it badly wrong. As they note, a simple regression of blood pressure on medication is likely to get a positive slope so you need to instrument or do something.
They estimate a logit and then stick the predicted values into an OLS model. Aside from the identifying assumption (which isn't discussed & looks pretty dodgy to me), this is not IV as usually defined and it is not clear that the estimate is consistent or that the standard errors are correct. The model also includes controls for exercise but these are also likely to be endogenous but this is ignored.

On a lighter note

Stata resources for treatment effects

There are a large number of resources within Stata for the estimation of treatment effects. Some are part of official Stata and others are user written that can be easily downloaded.

To estimate regression discontinuity models, there is a download rd due to Austin Nichols. Further details at Nichols, Austin. 2007. "Causal Inference with Observational Data." Prepublication draft available at http://pped.org/stata/ciwod.pdf. It is published in the Stata Journal now I think.

To estimate IV models there are several options in Stata.
ivregress is the main program. A download ivreg2 due to Baum, Schaffer & Stillman is very useful - I recommend it. Make sure you get the latest version. Their paper should be used in conjunction with it: http://ideas.repec.org/a/tsj/stataj/v7y2007i4p465-506.html. xtivreg2 is the equivalent program for panel data.

ivtobit and ivprobit do what their names suggest. If using them, you need to satisfy yourself that they are consistent estimators. Caution is appropriate where the instrumented variable is binary. In the latter case biprobit may be better.

cmp (due to David Roodman) allows you to estimate using MLE a wide range of simultaneous models with combinations of linear and non-linear equations provide they satisfy a recursive structure.

treatreg allows the estimation of what Stata calls "treatment effects models". This is something of a misnomer since it only for a very specific model: a linear regression with an endogenous dummy.

condivreg estimates IV models with a single endogenous variable and provides an exact confidence interval for the slope as opposed to the usual asymptotic one. It is particularly useful if weak instruments are a concern.

For estimating Treatment effects using Propensity Score matching there are several downloads including: psmatch2 (Leuven & Sianesi) which does a wide range of matching estimators and nnmatch which does nearest neighbour matching. psbalance allows you to test covariate balance after matching - something that is recomended.

Economic Journal: New Empirical Analysis in the Economics of Education

This month’s Economic Journal features a series of articles on the economics of education, including:

The Impact of Diagnostic Feedback to Teachers on Student Learning: Experimental Evidence from India*
Karthik Muralidharan Venkatesh Sundararaman

Are Educational Vouchers Only Redistributive?*
Eric Bettinger Michael Kremer Juan E. Saavedra

'Every Catholic Child in a Catholic School': Historical Resistance to State Schooling, Contemporary Private Competition and Student Achievement across Countries*
Martin R. West Ludger Woessmann

You Get What You Pay For: Incentives and Selection in the Education System*
Thomas Dohmen Armin Falk

Ethnicity and Educational Achievement in Compulsory Schooling*
Christian Dustmann Stephen Machin Uta Schönberg

From an Irish perspective, the article on the effects of private schooling which uses historical prevalence of Catholicism as an instrument is particularly interesting. There is also a book review of Akerlof and Shiller’s `Animal Spirits’. Link.

Genetic markers as instrumental variables

This looks pretty interesting, a nice combination of econometrics, genetics & epidemiology

Genetic Markers as Instrumental Variables:An Application to Child Fat Mass and Academic Achievement
Stephanie von Hinke Kessler Scholder, George Davey Smith, Debbie A. Lawlor,Carol Propper,Frank Windmeijer

The use of genetic markers as instrumental variables (IV) is receiving increasing attention from economists. This paper examines the conditions that need to be met for genetic variants to be used as instruments. We combine the IV literature with that from genetic epidemiology, with an application to child adiposity (fat mass, determined by a dual-energy X-ray absorptiometry (DXA) scan) and academic performance. OLS results indicate that leaner children perform slightly better in school tests compared to their more adipose counterparts, but the IV findings show no evidence that fat mass affects academic outcomes.

Instrumental Variables in the Economist

The Economist picks up on the debate about the use of IV in economics

link here

The Effect of Child Weight on Academic Performance: Evidence using Genetic Markers

von Hinke Kessler, Scholder, S
This paper examines the relationship between children’s weight and academic outcomes using genetic markers as instruments to account for the possible endogeneity of body size. We use medically assessed measures of body size which are more appropriate than the generally used BMI measures. OLS results indicate that leaner children perform better in school tests compared to their heavier counterparts, but the IV results, using genetic markers as instruments, show no evidence that fat mass affects academic outcomes. We compare these IV results to those using the instruments generally adopted in this literature. We show that the results are sensitive to the instrument set and argue that several of the commonly used instruments do not meet the exclusion restrictions required of a valid instrument.

http://d.repec.org/n?u=RePEc:yor:hectdg:09/25&r=edu

Movie Attendance and Violent Crime

A QJE paper by Stefano Della Vigna puts forward the provocative result that (after controlling for temporal effects) the release of violent films actually reduces assaults. The reason for this is that although the films do stimulate aggression, they also keep aggressive people off the street and not only keep them off the street but do so at a time when they would quite likely be drinking and getting into trouble otherwise. Bear in mind that many of these films are disproportionately viewed by young men and that many of them have several million tickets sold for times that would be associated otherwise with drinking then the results become a lot more intuitive.

While this paper cannot say anything about the long-run effects of exposure to violent images, it is certainly a very interesting result and methodology and the idea has many potential applications.

http://www.econ.berkeley.edu/~sdellavi/