The econometrics of the death penalty

The death penalty is an emotive and complex subject. The degree to which it is accepted in some countries, especially in the USA, seems bizarre to many Europeans. An argument for the death penalty is that it acts as a deterrent. But is this supported by the evidence? One would hope that if the state is going to kill people, that at least the decision should be evidence based. The analysis reported below shows that the evidence for the deterrence effect is not that robust and indeed there is even evidence that the death penalty could actually increase the murder rate.

Deterrence and the death penalty: partial identification analysis using repeated cross section

C F Manski , J V Pepper

Researchers have long used repeated cross sectional observations of homicide rates and sanctions to examine the deterrent effect of the adoption and implementation of death penalty statutes. The empirical literature, however, has failed to achieve consensus. A fundamental problem is that the outcomes of counterfactual policies are not observable. Hence, the data alone cannot identify the deterrent effect of capital punishment. How then should research proceed? It is tempting to impose assumptions strong enough to yield a definitive finding, but strong assumptions may be inaccurate and yield flawed conclusions. Instead, we study the identifying power of relatively weak assumptions restricting variation in treatment response across places and time. The results are findings of partial identification that bound the deterrent effect of capital punishment. By successively adding stronger identifying assumptions, we seek to make transparent how assumptions shape inference. We perform empirical analysis using state-level data in the United States in 1975 and 1977. Under the weakest restrictions, there is substantial ambiguity: we cannot rule out the possibility that having a death penalty statute substantially increases or decreases homicide. This ambiguity is reduced when we impose stronger assumptions, but inferences are sensitive to the maintained restrictions. Combining the data with some assumptions implies that the death penalty increases homicide, but other assumptions imply that the death penalty deters it.

NBER working paper W17455

From the "correlation is not causation" department

This article discusses a recent paper which summarizes a ton of papers that find a positive correlation between happiness and longevity. It also concluded that anxiety, depression, and pessimism were linked to higher rates of disease and a shorter lifespan. Interesting correlations except that the lead author of the paper makes a causal connection (happiness etc causes greater longevity) which doesn't follow for reasons that are all too familiar.
Of course its plausible that being a happy dude makes you live longer but it's also plausible that not expecting to live long is a bit of a downer. I, for one, am delirious that I am going to outlive you all. Alternatively, you could have "the right stuff" which makes you both happy and healthy.

On a lighter note

Causal inference in econometrics: structure vs. program evaluation

Those interested in the JEP symposium linked below but would like something a bit more technical should look at Heckman's recent NBER paper:

Building Bridges Between Structural and Program Evaluation Approaches to Evaluating Policy
J J Heckman , NBER 16110, June
This paper compares the structural approach to economic policy analysis with the program evaluation approach. It offers a third way to do policy analysis that combines the best features of both approaches. We illustrate the value of this alternative approach by making the implicit economics of LATE explicit, thereby extending the interpretability and range of policy questions that LATE can answer.

The paradoxical effect of omitted variables

Omitted variables are terrible. If you are beset by them then (& unless you are lucky that they are orthogonal to whats included) you are condemned to regression hell: your coefficients are biased and inconsistent, you cannot derive policy relevant conclusions and your girlfriend won't love you anymore.
So the conclusion is to get better data. So say you do and you now have a previously omitted variable in your data. You should include it, right?
Wrong actually, if the paper below is correct which it looks like being. The problem is that the standard results in this area are based on there being only one omitted variable. If you have two omitted variables the bias on whats included depends in a messy way on all the correlations between the X's.
Say the model is:
Y=b1*X1 + b2*X2 + b3*X3 [ignoring the constant & disturbance term]

So you don't observe X2 and X3 initially so your estimate of "b1" is biased. It may seem counter-intuitive but adding X2 does not necessarily get you a better estimate of "b1". Actually, its quite intuitive: say omitting X2 was biasing b1 upwards and omitting X3 was having the reverse effect. So its quite possible you could have a small [or even zero] bias and adding in one of them makes things worse. I don't think you don't actually need these opposing biases for the result to hold because there is also the X2,X3 correlation.
Its rather analogous to the Second Best Theorem in Welfare Economics due to Lipsey & Lancaster.
The practical problem is that there may always be an "X3", that is typically you cannot be sure that you have all the relevant variables. Its all rather disturbing.

The Phantom Menace: Omitted Variable Bias in Econometric Research , Kevin Clarke
http://www.rochester.edu/college/psc/clarke/CMPSOmit.pdf

The Effect of Smoking in Young Adulthood..

Evidence based on the Vietnam Era Draft Lottery

A thought provoking paper that I overlooked last year but which has recently come to my attention. Using neat causal techniques without strong identifying assumptions this paper suggests that smoking in young adulthood does not lead to large negative health effects later in life. The paper deals convincingly with obvious issues like mortality and other attenuating factors and although it might be reasonable to have some LATE concerns it remains a striking finding.

Basketball & endogeneity

Basketball is very important here in Kentucky. While at the mall today, on the back of a guy's hoodie today I saw the slogan "Basketball doesn't build character. It reflects it!" So this guy, or the person who designed his apparel, had a strong awareness of how unobserved heterogeneity can lead to misleading inferences about causal relationships. I thought about congratulating him on this, perhaps inviting him to reflect on Pearl's recent work on causality, but thought better of it. Nonetheless, he is way ahead of many more educated people. Go Wildcats!

Mostly Harmless!

DON'T PANIC! The core methods in today's econometric toolkit are linear regression for statistical control, instrumental variables methods for the analysis of natural experiments, and differences-in-differences methods that exploit policy changes.

This is the refrain of Joshua Angrist and Steve Pischke in the preface to their new book: "Mostly Harmless Econometrics". We mentioned it on the blog before here. In advance of Professor Angrist's visit to Geary on Friday, people might be interested in a preview of the Mostly Harmless book that is available here. For anyone who doesn't have a copy of the book (or even if you do), it's worth looking at the paper by Angrist and Krueger on "Empirical Strategies in Labour Economics". This is more a primer than a paper; it featured here (takes less time to to load up) in the 1999 Handbook of Labour Economics.

Abstract below:

This chapter provides an overview of the methodological and practical issues that arise when estimating causal relationships that are of interest to labor economists. The subject matter includes identification, data collection, and measurement problems. Four identification strategies are discussed, and five empirical examples -- the effects of schooling, unions, immigration, military service, and class size -- illustrate the methodological points. In discussing each example, we adopt an experimentalist perspective that emphasizes the distinction between variables that have causal effects, control variables, and outcome variables. The chapter also discusses secondary datasets, primary data collection strategies, and administrative data. The section on measurement issues focuses on recent empirical examples, presents a summary of empirical findings on the reliability of key labor market data, and briefly reviews the role of survey sampling weights and the allocation of missing values in empirical research.

Finally, there is also a range of "Mostly Harmless" t-shirts for sale; see below for a sample!

Causality and Econometrics

There have been various lectures and discussions on causality here over the last while, the culmination being Professor Heckman's session. Below are some background readings.

The starting point for discussion is a recent paper by Angus Deaton. This paper
was delivered as the Keynes lecture. Deaton argues strongly against the use of RCT and LATE methods on their own as useful tools for development analysis. A similar theme is picked up in the Heckman
Urzua paper below that.

http://ideas.repec.org/p/nbr/nberwo/14690.html

http://ideas.repec.org/p/iza/izadps/dp3980.html

Professor Heckman's ideas on causality are expressed in a recent International Statistical Review paper that is available below.


http://ideas.repec.org/p/nbr/nberwo/13934.html

His work on Treatment effects with Vytlacil is developed in a number of handbook of econometrics articles available on his IDEAS page. A good paper is below:

http://ideas.repec.org/a/tpr/restat/v88y2006i3p389-432.html

The use of LATE is defended in the paper below by Guido Imbens, who along with Josh Angrist was a key figure in the development of these methods.

http://www.nber.org/papers/w14896

Also, the work of Esther Duflo should be consulted in this regarded as a strong proponent of randomised trials in development analysis.

http://ideas.repec.org/e/pdu166.html

Correlation, causation & God

The statement that "Correlation does not imply causation" is hardly uncontroversial or, indeed, particularly deep. Nonetheless it is a matter of concern that people who should know better commit the fallacy that one implies the other. In this context you may have come across recent media attention to a report, released by the Iona Institute, which claims that religion makes you happier. It is written by Patricia Casey , Professor of Psychiatry at UCD.
I cannot find the report but the press release below makes clear that the claim is that religion is beneficial i.e. there is a causal link to people's well-being. However it is hard to imagine a research design that would allow one to interrogate this question to a standard that would be considered satisfactory in any scientific (including medical) journal. Finding a correlation is not surprising: maybe happy people see their happiness as evidence of God or the disillusioned turn away from religion or there is some other common unobserved factor. Any empirical economist hardly needs to see the arguments rehearsed. In the absence of randomization or some quasi-experimental design there is no reason, a priori, to think of the correlation as being any more than just that.

http://www.ionainstitute.ie/

Advertising Works - According to Yahoo! Research

We have discussed the possibility of evaluating adevertising campaigns on this blog before (here and here). So it is interesting to read that Yahoo! Research is measuring the effects of advertising on sales through a controlled experiment (more details available here). Researcher David Reiley has recently collaborated with Yahoo!’s Marketing Insights team in their ongoing efforts to help advertisers evaluate the effectiveness of their advertising campaigns.

Reiley joined Yahoo! Research because of his longstanding interest in field experiments and he points out a potential weakness of a study reported in a recent Harvard Business Review article, which measured large increases in sales due to online advertising. The study used large quantities of data from comScore, a key online information provider that logs the Internet browsing behaviour of two million users worldwide. By comparing the purchases of those who saw a given online ad with the purchases of those who do did not see it, the study concluded that there are large positive effects of online advertising.

However, "the population of people who sees a particular ad may be very different from the population who does not see the same ad,” says Reiley. Reiley's research made use of a database match between Yahoo! and a nationwide retailer by identifying users who registered the same email address with both companies. After finding over one million matched users, the researchers randomly assigned them to treatment and control groups for one of the retailer’s online advertising campaigns.

The project then tracked sales each week at the retailer, both online and in stores. In a paper co-authored with summer intern and MIT PhD student Randall Lewis, Reiley found that the online display advertising increased total revenues by approximately 5% for those users exposed to the ads, with 93% of the total effect happening in offline sales. They also observed online ads to have a large impact on sales even when the ads are not clicked: 78% of the increase in sales came from those who viewed, but did not click, the ads.

Could this have implications for the "pay-per-click" model in online advertising?

Correlation is not Causation

This cartoon is too good not to put up here aswell. Thanks to Aleks Jakulin from the Columbia Statistics Blog for sharing it. For a discussion on the issue, follow this link to the Columbia Statistics Blog.

MIT Poverty Lab Executive Education Programmes

For policymakers and executives interested in understanding the causal effect of the programmes they are running or financing, the following courses look like very good options.

http://www.povertyactionlab.org/course/

Opiates for the Matches

A working paper (2008) by Jasjeet S. Sekhon from the Department of Political Science at UC Berkeley: "Opiates for the Matches - Matching Methods for Causal Inference".

Abstract

In recent years there has been a burst of innovative work on methods for estimating causal effects using observational data. Much of this work has extended and brought a renewed focus on old approaches such as matching, which is the focus of this review. The new developments highlight an old tension in the social sciences between a focus on research design versus a focus on quantitative models. This realization along with the renewed interest in field experiments has marked the return of foundational questions as opposed to a fascination with the latest estimator. I use studies of get-out-the-vote interventions to exemplify this development. Without an experiment, natural experiment, a discontinuity, or some other strong design, no amount of econometric or statistical modeling can make the move from correlation to causation persuasive.

Mostly Harmless Econometrics

DON'T PANIC! The core methods in today's econometric toolkit are linear regression for statistical control, instrumental variables methods for the analysis of natural experiments, and differences-in-differences methods that exploit policy changes.

This is the refrain of Joshua Angrist and Steve Pischke in the preface to their new book: "Mostly Harmless Econometrics". A preview is available here. My econometrics professor, Paul Devereux, tells me that there are many references throughout to the Hitchhikers Guide to the Galaxy. There is also a range of "Mostly Harmless" t-shirts for sale; see below for a sample!