Evidence on how the quality of professors matter

Applied economists have become enthusiatic users of natural experiments to generate exogenous variation in key independent variables although it has been argued that this approach has severe limitations: particular instruments may not identify parameters of interest (see Heckman passim). The paper below I think is quite an imaginative use of a "natural experiment" from a dark period in history to see how the quality of professors matter for their graduate students.

Quality Matters - the Expulsion of Professors and Ph.D. Student Outcomes in Nazi Germany
Fabian Waldinger

I investigate the effect of faculty quality on Ph.D. student outcomes. To address the endogeneity of faculty quality I use exogenous variation provided by the expulsion of mathematics professors in Nazi Germany. I find that faculty quality is a very important determinant of short and long run Ph.D. student outcomes. A one standard deviation increase in faculty quality increases the probability of publishing the dissertation in a top journal by 13 percentage points, the probability of becoming full professor by 10 percentage points, the probability of having positive lifetime citations by 16 percentage points, and the number of lifetime citations by 6.3.

Adopted children as a natural experiment ?

Many applied micro-economists like natural experiments to deal with selection on unobservables and the use of data on adopted children (compared to non-adopted) seems a nice way to deal with with unobserved genetic effects. For example see Erik Plug's AER paper "Estimating the effect of mother's schooling using a sample of adoptees" (2004), 94(1), 358-368.
Such a model essentially relies on the assumption that the only difference between biological and adopted children is that the parents are genetically related to one and not the other.
But adopted children may differ systematically for other reasons and the behaviour of parents towards such children may also differ so its unclear to me that this is a good natural experiment. Parents do have biological children by accident for example whereas adoption takes a big investment of time and energy and also requires the consent of others. Consistent with my skepticism about the merits of this "natural experiment" (I think) is the following paper:
Differential parental investment in families with both adopted and genetic children Kyle Gibson
Abstract:
Stepchildren are abused, neglected and murdered at higher rates than those who live with two genetically related parents. Daly and Wilson used kin selection theory to explain this finding and labeled the phenomenon “discriminative parental solicitude.” I examined discriminative parental solicitude in American households composed of both genetic and unrelated adopted children. In these families, kin selection predicts parents should favor their genetic children over adoptees. Rather than looking at cases of abuse, neglect, homicide and other antisocial behavior, I focused on the positive investments parents made in their children as well as the outcomes of each child. The results show that parents invested more in adopted children than in genetically related ones, especially in educational and personal areas. At the same time, adoptees experienced more negative outcomes. They were more likely to have been arrested, to have been on public assistance and to require treatment for drug, alcohol or mental health issues. They also completed fewer years of schooling and were more likely to divorce. In adoptive families, it appears that “the squeaky wheel gets the grease.” Parents invest more in adoptees not because they favor them, but because they are more likely than genetic children to need the help. I conclude that discriminative parental solicitude differs in adoptive and step households because adoptive families generally result from prolonged parenting effort, not mating effort like stepfamilies.
Evolution and Human Behavior 30 (2009) 184–189

Evaluating State Programmes: “Natural Experiments” and Propensity Scores

DENIS CONNIFFE, VANESSA GASH, PHILIP J. O’CONNELL

The Economic and Social Review, Vol. 31, No. 4, October, 2000, pp. 283-308

Abstract: Evaluations of programmes — for example, labour market interventions such as employment schemes and training courses — usually involve comparison of the performance of a treatment group (recipients of the programme) with a control group (non-recipients) as regards some response (gaining employment, for example). But the ideal of randomisation of individuals to groups is rarely possible in the social sciences and there may be substantial differences between groups in the distributions of individual characteristics that can affect response. Past practice in economics has been to try to use multiple regression models to adjust away the differences in observed characteristics, while also testing for sample selection bias. The Propensity Score approach, which is widely applied in epidemiology and related fields, focuses on the idea that “matching” individuals in the groups should be compared. The appropriate matching measure is usually taken to be the prior probability of programme participation. This paper describes the key ideas of the Propensity Score method and illustrates its application by reanalysis of some Irish data on training courses.

Natural experiment designs

This paper by Michael Rutter is a couple of years old but has some useful guidelines on various kinds of natural experiment designs. One I liked is the children of twins design where the children of monozygotic and dizygotic twins are compared to assist in separating genetic and environmental factors that contribute to the physical or mental health of the children. In this case it's expected that the children of the MZ twins (comparing the children of MZ twin 1 with the children of MZ twin 2) should have more similar problems than the children of DZ twins if the inheritance is genetic. If parents transfer risk factors to their children socially/through environmental factors then it is expected that the differences between the children of MZ twins and between the children of DZ twins should be comparable. You may have to consult the article for this one!

Opiates for the Matches

A working paper (2008) by Jasjeet S. Sekhon from the Department of Political Science at UC Berkeley: "Opiates for the Matches - Matching Methods for Causal Inference".

Abstract

In recent years there has been a burst of innovative work on methods for estimating causal effects using observational data. Much of this work has extended and brought a renewed focus on old approaches such as matching, which is the focus of this review. The new developments highlight an old tension in the social sciences between a focus on research design versus a focus on quantitative models. This realization along with the renewed interest in field experiments has marked the return of foundational questions as opposed to a fascination with the latest estimator. I use studies of get-out-the-vote interventions to exemplify this development. Without an experiment, natural experiment, a discontinuity, or some other strong design, no amount of econometric or statistical modeling can make the move from correlation to causation persuasive.