Behavioural Insights Team Update

The Behavioural Insights Team within the UK Cabinet office has been working on the application of a number of literatures surrounding behavioural, including behavioural economics, to policy (brief summary from a previous post here). Their annual update, released recently, is available on this link. The report outlines policies they have been involved with in the areas of organ donation, healthy food consumption, consumer empowerment, tax and environment. There has been an ongoing debate in the UK about the rationale behind the approach including the recently released House of Lords report. Interestingly the Behavioural Insights team report gives a lengthy nod to Ben Goldacre's suggestion that they be sacked and replaced with a policy trials unit devoted to randomised control trials on unresolved policy questions. In essence, as they note, a fully worked out behavioural policy would have trial like this embedded into policy.

How should we evaluate policies?

My purpose in writing this post is to pick up on a debate we have had on this blog and some of us have debated on other forums in the context of the beginning of the new government. I don’t think any of us really believed that a beleagured government, knowing fully that it was on the way out and facing a major crisis, was really going to be a vehicle for serious economic policy change in Ireland at micro level. But the new government has the type of majority and popular appetite for reform that may never be seen again and it is a good time to start this discussion anew.

A key feature of an economic recovery should be a more intelligent policy design and analysis framework for assessing policy changes in areas such as health, social insurance and education. At least part of this will involve greater economic skills on behalf of existing staff, more public assesment frameworks and less direct political interference in the actual design and implementation of policies. Another part will come from a willingness to engage with developments in economics and economic methodologies, particularly those emanating from microeconomics, the half of economics that hitherto has mostly been ignored by Irish governments.

A technical question of sizeable importance is how we should best actually evaluate public policies and who should take responsibily for doing this. By public policies, I mean any action of the state, including direct provision of funds, legislation, the creation of regulatory agencies and so on. One approach relies on measuring relevant economic variables in micro-units and examining the causal effect of the change and the level of the change. Various quantities can be calculated, including the average effect of the change across the population of interest and the effect on people specifically affected by the policy.

Estimating these types of quantities is technically and practically complex. One solution that has been advocated for several decades by various researchers particularly in the US and has found new force among development economics researchers in particular is the use of Randomised Controlled Trials (RCTs) to examine the effect of policies. A large number of researchers have been advocating the use of RCTs in public policy, with their main putative benefit being that they allow the isolation of the causal effects of policy.

There has been somewhat of a backlash against the more evangelical view of RCTS. In particular, people like Heckman and Deaton have pointed out that the use of RCTS brings new problems to the table, including the potential for Hawthorne effects, the difficulty in externally applying results that come from model trials, the cost of implementing trials and so on.

It is worth the while of the new government to engage with this debate. We have talked before about the fairly limited engagement, for example, of the aid programme with developments in Economics and the international debate in development economics would certainly help sharpen the case for spending money in these areas. One thing that should be kept in mind is the potential for the rapid deployment of randomised trials in the development of the surface features of tax and social welfare policy. An increasing literature in behavioural economics is showing impressive effects of fairly cheap changes in things like the nature of forms used in different policies and default options. This is spawning an increasing literature, some of which is convincing. As the Irish government moves to policies such as automatic enrolment for pensions, it will be increasingly important to examine how the surface features of these policies will influence people’s behaviour.

A few things I would say that I believe are reasonable conclusions from the current debate:

1. Randomised Trials should not be overhyped but they have strong potential in specific areas and the government should examine this potential.

2. A greater focus on defining the outcomes of public policies and engaging in sufficient longitudinal tracking to enable assessment of these outcomes should characterise the main decisions made by the new government. The types of radical changes coming down the tracks to pensions and healthcare should be matched by an effort to increase the level of intelligence used to assess these changes.

3. While the development of an independent fiscal council is obviously a positive step, the new government should have greater engagement with developments in microeconomics, including a greater emphasis on peer-review of major policy evaluations.

4. A cultural norm needs to emerge placing greater pressure on government departments and agencies to provide solid rationales for large-scale policy changes. In the US, various legislation mandates the deployment of rigorous methodologies to assess policy changes.

5. Policymakers need to become more aware of the basic nature of modern economics. Many recent Irish policy documents are noted by the real failure to engage with the type of research that actually drives the fields under discussion e.g. the innovation taskforce document was particularly weak in its research foundations and strategic documents are routinely published that do not seem to draw in any way from what has been learned in the relevant literatures.

6. One potential solution would be the development of a fast-track economics group within the Irish civil service, something that Karl Whelan, Colm Harmon and others have spoken about. In my view, such a group would need to place a heavy emphasis on ability to synthesise economic knowledge on sectoral and specific micro-policy issues. This does not neccesarily mean recruiting outside of the existing service but it does mean breaking norms within the civil service and prioritising the development of a group who would be given a mandate to engage with the wider economics world. The lack of economic expertise in areas such as health, social protection and finance is genuinely frightening and damages the credibility of the state in a major way, something that will be increasingly exposed in a less opulent environment. The almost complete absence of anyone with a PhD Economics qualification from the Irish civil service is something that cannot continue.

Harmon on Policy Decisions

Colm Harmon has an article in the Times last week on changing the way policy decision are made in Ireland.

And Budget 2011 did make some bright moves – the changes in the transit taxes for air travel is framed like an experiment – change designed to have impact, reversible if it fails, embedded if it works. We need to get into the same thought-space on microeconomic policy on all domains.

Colm references the Travel Tax reduction in the budget. Basically, Minister Lenihan played the part of Solomon and instead of leaving the 10 euro flight tax as it was or abolishing it totally, he has reduced it to 3 euro for the next year when it will be reviewed. There is definitely a progression here over the usual way of doing policy, one that seems calculated to call the bluff of Ryanair in particular. Of course, as an experimental design, it is going to be a tricky one to look at. Macroeconomic conditions are changing rapidly and this is a once-off change. Worth talking through for the econometrically minded of the group.

Who is in charge of the economy?

I think this quote from Paul Seabright's The company of strangers: a natural history of economic life is well worth thinking about:
"Politicians are in charge of the modern economy in much the same way as a sailor is in charge of a small boat in a storm. The consequences of their losing control completely may be catastrophic (as civil war and hyperinflation in parts of the former Soviet empire have recently reminded us), but even while they keep afloat, their influence over the course of events is tiny in comparison with that of the storm around them. We who are their passengers may focus our hopes and fears upon them, and express profound gratitude toward them if we reach harbor safely, but that is chiefly because it seems pointless to thank the storm. (p. 25)"

(via Bill Easterly via Greg Mankiw via Peter Gordon)

Evidence and Policy special issue

'Evidence and Policy' have a special issue and the editorial, 'Evidence and policy in six European countries: diverse approaches and common challenges' is available free online.

Revised Programme for Government

The RPG is available below. For people interested in Irish policy, it will be useful to read this in order to understand the extent to which evidence and careful analysis plays a central role in the formation of policy in Ireland.

link here.

Evidence Based Policy: Andrew Leigh

Andrew Leigh examines the extent to which randomised controlled trials are the way forward for policy in Australia

"In Australian policy debates, the term „evidence-based policymaking‟ has now become so meaningless that it should probably be jettisoned altogether. The problem in many domains is not that decision makers do not read the available literature, it is that they do not set up policies in such a way that we can learn clear lessons from them. In employment policy, the
early 1990s recession saw Australia spend vast amounts on active labour market programs, without producing a skerrick of gold-standard evidence on what works and what does not. In Indigenous policy, there are as many theories as advocates, but precious few randomised experiments that provide hard evidence about what really works."

We have already debated extensively the papers by Deaton, Heckman, Imbens and others outlining the pros and cons of RCT's as a policy tool. Across health, education, social welfare, crime, aid and other policy areas in Ireland it is certainly long past time to ask exactly what we are trying to do with many of our policies and whether they could be improved with these approaches. Institutional innovation in an environment where budgets are closing must partly take the form of using academic knowledge directly in policy.