Penalty points and road safety: is there really a link

According to the Irish Independent, drivers in Ireland face a raft of new penalty offences – to be introduced by the end of 2012. In the article the minister, Leo Varadkar justified the clampdown, saying: “Each measure we take to advance road safety increases the likelihood that lives will be saved.” The article also quotes Conor Faughnan of the AA as saying that the penalty points system had changed driver behaviour.

So it is true? Road safety is a big topic on which there is a fair of research internationally and there have been a few papers on the Irish experience – penalty points were introduced in 2002. Here is a quick summary of a few papers I found.

Hussain et al. (British Journal of Oral and Maxillofacial Surgery 2006) look at maxillofacial injuries. They simply compare the number of various injuries in the year before and after the 2002 reform. They assert that “The introduction of legislation led to a 61% reduction in the need for emergency maxillofacial operations.” Clearly the design does not support such an assertion and subsequently in the paper they seem to acknowledge that a causal link cannot be firmly established. Yes indeed. Just comparing means before and after tells you nothing. Maybe the series just goes up and down a lot. Maybe something else happened. Lots of the other outcomes they consider don’t change at all. And of course if you look at enough outcomes you are bound to find some differences (though the paper does not appear to contain a single p-value or confidence interval).

At least two other papers were generated by this reform.

Healy et al. (Injury 2004) look at spinal injuries. This paper follows a similar methodology as the previous one but at least does some inference “In the first 6 months of the new speed penalty system the number of RTA (road traffic accident) related spinal injuries was 17. This was significantly lower then the same period in the previous 4 years when on average of 33 admissions occurred (P < 0:05, Chi-sq=3:96, d:f: =1)”. The same issue arises ‘though: how can one attribute the change to the reform? Interestingly in the second 6-month period after the reform, the number of RTA related injuries rose again so it looks, to me at least, that any effect was shortlived. Looking at their Table 2 it is far from clear the reform had any effect. Again, simply comparing means before and after is uninformative.

Butler et al. (Irish Journal of Medical Science 2006) also looks at spinal injuries (the authorship of the two papers overlap) and seem to report much the same result: an initial fall in RTA related spinal injuries that were not sustained.

Based on these 3 papers (there are others which I haven’t looked at) the evidence that the introduction of penalty points in 2002 was effective seems pretty underwhelming although a casual reading of these papers would not leave you with that impression. As the system changed in 2003, with additional offences introduced, it would be difficult to estimate the long run effect of the reform.

The possibility that such a reform had a temporary effect with people regressing to their usual driving standards quite quickly is plausible to me. I came across a paper that looked at whether conviction for driving offences lowered risk of death subsequently (Redelmeier et al. The Lancet 2003). They find it reduces the risk of death sharply in the immediate aftermath of a conviction but the effect was much smaller after 2 months and not significant after 3-4 months.I wonder is this a general finding.

Road safety is an emotive topic and it is essential that policy is based not on anecdote or gut instinct but on good evidence. Inferring whether the points system improved road safety is probably difficult or impossible given the data available. That does not mean one shouldn’t consider such policies – my guess is that the loss function is asymmetric: introducing penalty points probably can’t do any harm. But exaggerated claims of efficacy are never justified.