Saturday, July 23, 2022

Some Recent Posts

 I will use this blogpost to update on job opportunities that people send me. Obviously, no posting is an endorsement though I will try to post ones that will be interesting to the type of people who keep an eye on the blogposts. 

23rd July 2022 

Public Perceptions of Decarbonising Domestic Heat Options for Climate Change: Cardiff University School of Psychology seeks to appoint a Research Associate in the area of environmental and/or social psychology to work with Professor Nick Pidgeon (Cardiff University) and Dr Christina Demski (Bath University) on a major ongoing project as part of the UK Energy Research Centre (https://ukerc.ac.uk/). The successful candidate will administer, analyse statistically, and report a major nationally representative survey of how the public view options for the future decarbonisation of domestic heat in the UK. The research will focus upon public responses to heat decarbonisation in the light of the urgent challenge of climate change as well as the current energy bills crisis. The post would suit a statistically literate person with online survey experience (SPSS, Qualtrics) ideally nearing or recently awarded a PhD. For informal inquiries contact pidgeonn@cardiff.ac.uk or cd2076@bath.ac.uk and for post details and how to apply see Research Associate | Cardiff University (brassring.com)   

Several positions at CogCo, a really interesting new company that is employing quite a few behavioural science graduates. 


Two postdocs working with computational social science team led by Prof Taha Yasseri at UCD.

Thursday, July 07, 2022

Speaking Notes Behavioural Transformations Workshop

Below are speaker notes for a talk I gave at LSE as part of the workshop entitled "Behavioural Transformations in the 21st century" on July 7th 2022. The talk itself is interactive and not fully scripted. Hopefully the notes below will help if anyone wants to source various material or follow up on any points.

What is behavioural science? Historically. Current field definitions.

How can we incorporate well-being into the evaluation of behavioural public policy? More generally, clarifying interaction between normative foundations, evidence and practice.

How can we deal with the complexity of cultural variation? Moving beyond just pointing out the WEIRDNESS of behavioural science? Global team based science.

What are the emerging ethical issues in the integration of behavioural science? How do these manifest in public and private sectors as well as NGOs?

What issues emerge when we scale behavioural science ideas? When we go from the lab or from theory to the world, what sort of dynamic feedback loops might arise? What have we learned in this regard from covid? Social representations of behavioural science.

What structures will emerge nationally and globally in these areas? BPP Journal, IBBPA, JBPA. Professional structures BSPA, GAABS, Government networks, Private Sector Networks.

References:

Banerjee, A., Banerji, R., Berry, J., Duflo, E., Kannan, H., Mukerji, S., ... & Walton, M. (2017). From proof of concept to scalable policies: Challenges and solutions, with an application. Journal of Economic Perspectives, 31(4), 73-102.

Bavel, J. J. V., Baicker, K., Boggio, P. S., Capraro, V., Cichocka, A., Cikara, M., ... & Willer, R. (2020). Using social and behavioural science to support COVID-19 pandemic response. Nature human behaviour, 4(5), 460-471.

Chater, N., & Loewenstein, G. (2022). The i-Frame and the s-Frame: How Focusing on the Individual-Level Solutions Has Led Behavioral Public Policy Astray. Available at SSRN 4046264.

Dolan, P., & Galizzi, M. M. (2015). Like ripples on a pond: behavioral spillovers and their implications for research and policy. Journal of Economic Psychology, 47, 1-16.

Dolan, P., Hallsworth, M., Halpern, D., King, D., & Vlaev, I. (2010). “MINDSPACE: Influencing behaviour through public policy” Institute for Government and Cabinet Office.

Lades, L. K., & Delaney, L. (2022). Nudge FORGOOD. Behavioural Public Policy, 6(1), 75-94.

Michie, S., Van Stralen, M. M., & West, R. (2011). The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implementation science, 6(1), 1-12.

Milkman, K. L., Patel, M. S., Gandhi, L., Graci, H. N., Gromet, D. M., Ho, H., ... & Duckworth, A. L. (2021). A megastudy of text-based nudges encouraging patients to get vaccinated at an upcoming doctor’s appointment. Proceedings of the National Academy of Sciences, 118(20).

Muthukrishna, M., Bell, A. V., Henrich, J., Curtin, C. M., Gedranovich, A., McInerney, J., & Thue, B. (2020). Beyond Western, Educated, Industrial, Rich, and Democratic (WEIRD) psychology: Measuring and mapping scales of cultural and psychological distance. Psychological science, 31(6), 678-701.

OECD (2017), Behavioural Insights and Public Policy: Lessons from Around the World, OECD Publishing, Paris, https://doi.org/10.1787/9789264270480-en.

Sunday, March 13, 2022

Speaking Notes: Irish Health Econ MasterClass

Below are speaker notes for a talk I gave at UCD as part of the Irish Health Economics masterclass titled "Behavioural health policy: mental health, measured human experience, and ethics" on March 14th 2022. The talk itself is interactive and not fully scripted. Hopefully the notes below will help if anyone wants to source various material or follow up on any points.  

I am going to speak about some directions in integrating mental health and human experience measures into policy appraisal and evaluation. At a very broad level, I am interested in the context whereby mental health and measures of human experience are entering back into economic analysis after quite a while being separate. But also there is a very practical context whereby many people from health economics and related backgrounds are working in the development of behavioural policies across a vast array of areas. The talk draws from papers with PhD students Karen Arulsamy and Lucie Martin as well as colleagues Leo Lades, Orla Doyle, Simon McCabe, and Conny Wollbrant.

i) One very simple idea is that policies will have differential treatment effects across levels of mental health. This could be due to differential decision making strategies but also due to the very widely studied phenomenon of poor mental health disrupting economic progression. Put simply, people with poor mental health live more complex administrative lives. In work I have conducted with Karen Arulsamy we show that mental health is a very strong predictor of pension coverage prior to autoenrolment in the UK. This is of course partly because of employment but also even within full-time employed individuals, pension coverage is far lower among people with poor mental health, controlling for a wide range of other factors. This gap disappears after autoenrolment. 

ii) The second aspect I wanted to speak about more generally is the experience of administrative environments. In work with Lucie Martin and Orla Doyle, we look at how people experience administrative burden more generally. As can be seen, various markers of disadvantage select people into administrative environments that are more emotionally taxing. We also show that administrative burden is experienced as more discouraging for these groups. But focusing on experience itself is something worth reflecting on. Even if it did not alter the behaviour, there is an interesting ethical issue surrounding the type of psychological reaction we generate.




As a brief interlude, this is Brendan Bracken. The inspiration for Big Brother. The Ministry for Information was a sort of wartime Nudge Unit that generated a lot of discussion. Has anyone seen these before? They generated quite a bit of controversy at the time (excellent short paper on that here). Keep Calm and Carry on is often represented in modern media as exemplifying the stroic British spirit. It was deeply unpopular. Some of this may have reflected psychological reactions to the Ministry for Information itself amplified through media (something that indeed might have happened recently in the UK with covid messaging). But I also think if you look at it carefully, it is not hard to see why people living in wartime conditions would have seen it as patronising to be given messages like this.


iii) With this in mind, along with Stirling colleagues Conny Wollbrant and Simon McCabe we have been looking more at how people experience different types of public messaging relating to health behaviour and the extent to which this would create both psychological reactions but also potentially unintended behavioural consequences. The example discussed in the talk focused on a proposal by the British government to include messages on medication bottles about the price and funding of the medication. As can be seen in the paper, likely engagement might be higher but so also would reported feelings of indebtedness and being a burden. 


This is currently only illustrative and we are working on integrating such measures into live policy roll-outs. But in general, there is now a vast enterprise unfolding in deploying various types of psychological emotion modifications to influence people's behaviour. It will be important to examine potential for them affecting people in different ways. One thing we have examined is whether some type of ethical pre-mortem could be embedded to the appraisal process of large-scale behavioural work. FORGOOD. While we have not attempted something that would create a very generic scaled template for this, some scope for identifying particularly aversive psychological reactions is important and arguably an important capacity for behavioural teams working in these areas to have. 

Thank you, hopefully there are some things of interest there. By way of conclusion, Mental health and human experience measures were largely taken out of economic analysis because they were too messy. Some would probably want to leave them out. But I think we will be moving rapidly in the direction of seeing appraisal and evaluation literatures that incorporate mental health and measured human experience in terms of ethical appraisal, heterogeneity analysis, and behavioural mechanisms. 

Those of you who are working on these areas will face a lot of challenges so I will just finish by saying Keep Calm and Carry On. 

References 

Arulsamy, K., & Delaney, L. (2020). The impact of automatic enrolment on the mental health gap in pension participation: Evidence from the UK (No. 202004). Geary Institute, University College Dublin.

Lades, L. K., & Delaney, L. (2022). Nudge FORGOOD. Behavioural Public Policy, 6(1), 75-94.

Lades, L. K., Martin, L., & Delaney, L. (2022). Informing behavioural policies with data from everyday life. Behavioural Public Policy, 6(2), 172-190.

Martin, L., Delaney, L., Doyle, O. (2022). Everyday administrative burdens and inequality. WP22/05 UCD School of Economics: University College Dublin. Available at: http://hdl.handle.net/10197/12778

Saturday, March 12, 2022

Reading Groups: Scaling in Social and Behavioural Science

During reading groups with students here at LSE, we will discuss papers from a number of emerging literatures at the interface of behavioural science. I am making the papers and reading lists available on this blog as they may be of interest to wider readers. 

Clearly, many disciplines grapple with issues of scaling and systemic effects more generally. I am happy to represent that in this reading list where I can meaningfully make connections. Suggestions for other papers welcome. Many of the papers in the ethics reading list address related questions such as institutional legitimacy and discussions interlinking these are welcome. 

John List's recent book "The Voltage Effect" summarises a range of factors that can lead to differential success in scaling up trials. 
Countless enterprises fall apart the moment they scale; their positive results fizzle, they lose valuable time and money, and the great electric charge of potential that drove them early on disappears. In short, they suffer a voltage drop. Yet success and failure are not about luck - in fact, there is a rhyme and reason as to why some ideas fail and why some make it big. Certain ideas are predictably scalable, while others are predictably destined for disaster. In The Voltage Effect, University of Chicago economist John A. List explains how to identify the ideas that will be successful when scaled, and how to avoid those that won't. Drawing on his own original research, as well as fascinating examples from the realms of business, government, education, and public health, he details the five signature elements that cause voltage drops, and unpacks the four proven techniques for increasing positive results - or voltage gains - and scaling great ideas to their fullest potential. By understanding the science of scaling, we can drive change in our schools, workplaces, communities, and society at large. Because a better world can only be built at scale.
The AER paper "From Proof of Concept to Scalable Policies: Challenges and Solutions, with an Application" by Banerjee and a wide group of co-authors is one of the most useful papers I have read on issues with scaling data from trials. As well as discussing practical issues across the scaling process, it relates these nicely to the type of parameters being estimated in trials. 
The promise of randomized controlled trials is that evidence gathered through the evaluation of a specific program helps us—possibly after several rounds of fine-tuning and multiple replications in different contexts—to inform policy. However, critics have pointed out that a potential constraint in this agenda is that results from small "proof-of-concept" studies run by nongovernment organizations may not apply to policies that can be implemented by governments on a large scale. After discussing the potential issues, this paper describes the journey from the original concept to the design and evaluation of scalable policy. We do so by evaluating a series of strategies that aim to integrate the nongovernment organization Pratham's "Teaching at the Right Level" methodology into elementary schools in India. The methodology consists of reorganizing instruction based on children's actual learning levels, rather than on a prescribed syllabus, and has previously been shown to be very effective when properly implemented. We present evidence from randomized controlled trials involving some designs that failed to produce impacts within the regular schooling system but still helped shape subsequent versions of the program. As a result of this process, two versions of the programs were developed that successfully raised children's learning levels using scalable models in government schools. We use this example to draw general lessons about using randomized control trials to design scalable policies.

The BPP paper "Successfully scaled solutions need not be homogenous" provides an account of scaling based on machine learning and micro-level heterogeneity. 

Al-Ubaydli et al. point out that many research findings experience a reduction in magnitude of treatment effects when scaled, and they make a number of proposals to improve the scalability of pilot project findings. While we agree that scalability is important for policy relevance, we argue that non-scalability does not always render a research finding useless in practice. Three practices ensuring (1) that the intervention is appropriate for the context; (2) that heterogeneity in treatment effects are understood; and (3) that the temptation to try multiple interventions simultaneously is avoided can allow us to customize successful policy prescriptions to specific real-world settings.

PNAS paper "Scaling up behavioral science interventions in online education" also very useful on the theme of large-scale iterative field trials. 

Online education is rapidly expanding in response to rising demand for higher and continuing education, but many online students struggle to achieve their educational goals. Several behavioral science interventions have shown promise in raising student persistence and completion rates in a handful of courses, but evidence of their effectiveness across diverse educational contexts is limited. In this study, we test a set of established interventions over 2.5 y, with one-quarter million students, from nearly every country, across 247 online courses offered by Harvard, the Massachusetts Institute of Technology, and Stanford. We hypothesized that the interventions would produce medium-to-large effects as in prior studies, but this is not supported by our results. Instead, using an iterative scientific process of cyclically preregistering new hypotheses in between waves of data collection, we identified individual, contextual, and temporal conditions under which the interventions benefit students. Self-regulation interventions raised student engagement in the first few weeks but not final completion rates. Value-relevance interventions raised completion rates in developing countries to close the global achievement gap, but only in courses with a global gap. We found minimal evidence that state-of-the-art machine learning methods can forecast the occurrence of a global gap or learn effective individualized intervention policies. Scaling behavioral science interventions across various online learning contexts can reduce their average effectiveness by an order-of-magnitude. However, iterative scientific investigations can uncover what works where for whom. 

PNAS paper by Milkman et al "A megastudy of text-based nudges encouraging patients to get vaccinated at an upcoming doctor’s appointment

Many Americans fail to get life-saving vaccines each year, and the availability of a vaccine for COVID-19 makes the challenge of encouraging vaccination more urgent than ever. We present a large field experiment (N = 47,306) testing 19 nudges delivered to patients via text message and designed to boost adoption of the influenza vaccine. Our findings suggest that text messages sent prior to a primary care visit can boost vaccination rates by an average of 5%. Overall, interventions performed better when they were 1) framed as reminders to get flu shots that were already reserved for the patient and 2) congruent with the sort of communications patients expected to receive from their healthcare provider (i.e., not surprising, casual, or interactive). The best-performing intervention in our study reminded patients twice to get their flu shot at their upcoming doctor’s appointment and indicated it was reserved for them. This successful script could be used as a template for campaigns to encourage the adoption of life-saving vaccines, including against COVID-19.

Deaton and Cartwright "Understanding and misunderstanding randomized controlled trials" gives a very strong summary of a wide range of points both authors have been making as a critique of the recent literature on randomised trials in policy applications.

Randomized Controlled Trials (RCTs) are increasingly popular in the social sciences, not only in medicine. We argue that the lay public, and sometimes researchers, put too much trust in RCTs over other methods of investigation. Contrary to frequent claims in the applied literature, randomization does not equalize everything other than the treatment in the treatment and control groups, it does not automatically deliver a precise estimate of the average treatment effect (ATE), and it does not relieve us of the need to think about (observed or unobserved) covariates. Finding out whether an estimate was generated by chance is more difficult than commonly believed. At best, an RCT yields an unbiased estimate, but this property is of limited practical value. Even then, estimates apply only to the sample selected for the trial, often no more than a convenience sample, and justification is required to extend the results to other groups, including any population to which the trial sample belongs, or to any individual, including an individual in the trial. Demanding ‘external validity’ is unhelpful because it expects too much of an RCT while undervaluing its potential contribution. RCTs do indeed require minimal assumptions and can operate with little prior knowledge. This is an advantage when persuading distrustful audiences, but it is a disadvantage for cumulative scientific progress, where prior knowledge should be built upon, not discarded. RCTs can play a role in building scientific knowledge and useful predictions but they can only do so as part of a cumulative program, combining with other methods, including conceptual and theoretical development, to discover not ‘what works’, but ‘why things work’.

"RCTs to Scale: Comprehensive Evidence From Two Nudge Units". Fascinating paper by DellaVigna and Linos pooling date from many policy trials and comparing effect sizes to those in academic literature. 

Nudge interventions have quickly expanded from academic studies to larger implementation in so-called Nudge Units in governments. This provides an opportunity to compare interventions in research studies, versus at scale. We assemble a unique data set of 126 RCTs covering 23 million individuals, including all trials run by two of the largest Nudge Units in the United States. We compare these trials to a sample of nudge trials in academic journals from two recent meta-analyses. In the Academic Journals papers, the average impact of a nudge is very large—an 8.7 percentage point take-up effect, which is a 33.4% increase over the average control. In the Nudge Units sample, the average impact is still sizable and highly statistically significant, but smaller at 1.4 percentage points, an 8.0% increase. We document three dimensions which can account for the difference between these two estimates: (i) statistical power of the trials; (ii) characteristics of the interventions, such as topic area and behavioral channel; and (iii) selective publication. A meta-analysis model incorporating these dimensions indicates that selective publication in the Academic Journals sample, exacerbated by low statistical power, explains about 70 percent of the difference in effect sizes between the two samples. Different nudge characteristics account for most of the residual difference.

Chater and Loewenstein's recent working paper "The i-Frame and the s-Frame: How Focusing on the Individual-Level Solutions Has Led Behavioral Public Policy Astray" advances the view that the last 20 years of development of behavioural public policy has been captured to an extent on a focus on micro-interventions that have not delivered on promises for transformative change. 

An influential line of thinking in behavioral science, to which the two authors have long subscribed, is that many of society’s most pressing problems can be addressed cheaply and effectively at the level of the individual, without modifying the system in which individuals operate. Along with, we suspect, many colleagues in both academic and policy communities, we now believe this was a mistake. Results from such interventions have been disappointingly modest. But more importantly, they have guided many (though by no means all) behavioral scientists to frame policy problems in individual, not systemic, terms: to adopt what we call the “i-frame,” rather than the “s-frame.” The difference may be more consequential than those who have operated within the i-frame have understood, in deflecting attention and support away from s-frame policies. Indeed, highlighting the i-frame is a long-established objective of corporate opponents of concerted systemic action such as regulation and taxation. We illustrate our argument, in depth, with the examples of climate change, obesity, savings for retirement, and pollution from plastic waste, and more briefly for six other policy problems. We argue that behavioral and social scientists who focus on i-level change should consider the secondary effects that their research can have on s-level changes. In addition, more social and behavioral scientists should use their skills and insights to develop and implement value-creating system-level change.

Tuesday, March 01, 2022

Reading Group: Sludge Audits

During reading groups with students here at LSE, we will discuss papers from a number of emerging literatures at the interface of behavioural science. I am making the papers and reading lists available on this blog as they may be of interest to wider readers. 

Sunstein, C. R. (2020). Sludge audits. Behavioural Public Policy, 1-20.

Sunstein, C. R. (2021). Sludge: What Stops Us from Getting Things Done and What to Do about It. MIT Press.

Reading Group: Behavioural Science & Voting

During reading groups with students here at LSE, we will discuss papers from a number of emerging literatures at the interface of behavioural science. I am making the papers and reading lists available on this blog as they may be of interest to wider readers. 

The readings for this one on behavioural science and voting are below. 

Allcott, H., & Gentzkow, M. (2017). Social media and fake news in the 2016 election. Journal of economic perspectives, 31(2), 211-36.

Bond, R. M., Fariss, C. J., Jones, J. J., Kramer, A. D., Marlow, C., Settle, J. E., & Fowler, J. H. (2012). A 61-million-person experiment in social influence and political mobilization. Nature, 489(7415), 295-298.

Wilson, R. K. (2011). The contribution of behavioral economics to political science. Annual Review of Political Science, 14, 201-223.

Reading Group: Plastic Waste and Environmental Charges

During reading groups with students here at LSE, we will discuss papers from a number of emerging literatures at the interface of behavioural science. I am making the papers and reading lists available on this blog as they may be of interest to wider readers.

The readings for this on plastic waste and environmental charges are below. 

Poortinga, W., Nash, N, and Poortinga, W., Nash, N., & Hoeijmakers, L. (2019). Rapid review of charging for disposable coffee cups and other waste minimisation measureExpert Panel on Environmental Charging and Other Measures: Scottish Government.

Team, B.I. (2021). Net Zero: Principles for successful behaviour change initiatives. Cabinet Office: London, UK.