Several months ago the ESRI released a study of a FAS programme which showed that the individuals on it were significantly less likely to return to work than those who were not on it. It got a lot of attention understandably.
Its natural to ask whether this is an unusual finding. A lot of people saw the finding as a reflection on FAS as an organization. In this regard it is well worth looking at a study of a training program in Norway by Aarvik, Heckman & Vytlacil (J. Econometrics 2005). This paper develops and applies some very nice (but challenging) techniques to look at treatment effects for discrete outcomes. One of the reasons why simple comparisons can be unreliable is because of cream-skimming: those running the programs may choose to take on those who are likely to get jobs anyway as it will make them look better. Clearly this will bias upward an estimate of the treatment effect.
What they find for Norway is that relatively simple comparisons (like matching on observables) imply a positive treatment effect: the probability of being employed raises by 3 or 4%.
However, once one allows for unobservables – using their sophisticated unobserved heterogeneity technique or a simpler Instrumental Variable approach- that the treatment effect is actually negative. None of the estimates are very precise. They also look at how the treatment effects vary with some observed characteristics of individuals. For example the effects are larger for those who are more likely to be unemployed.
All of this shows that careful strutiny of labour market policies is necessary. One needs to use the best econometrics available: the cost of that is small relative to the money one might waste on ineffective policies.
No comments:
Post a Comment