April 15th Workshop on Behavioural Science, Measurement and Policy

This workshop addresses innovations in measurement in the social and behavioural sciences. It is the eight and final workshop in our series. We examine a number of key themes in the development of rich measurement tools and pragmatic survey designs including: the integrated use of brief psychometric measures, experience sampling, and wearable devices to measure behaviour, attitudes, well-being and health in a brief-yet-precise manner. In addition, this workshop will consider ethical and privacy considerations, issues of response bias, the extent to which participants will give accurate responses, the potential impact of implementing this measurement in a policy context, and the traits and behaviours that are particularly important to measure in different policy contexts. It will also address problems of statistical inference and publication bias that relate to the presence of widespread secondary data and private researcher decisions. 

Please register here to attend the workshop. Registration is free of charge. It takes place in the Court Room, on the fourth floor of the Cottrell Building in Stirling.

Workshop on Behavioural Science, Measurement and Policy

845 to 915 Registration 

915 to 930 Welcome 

930 to 1015am

Dr. Leonhard Lades (Stirling University)

Measuring self-control in everyday life: Implications for present bias and subjective well-being. 

  

Abstract: It is difficult to name a problematic behaviour in our lives that is independent of a self-control failure. Accordingly, self-control has received massive attention in economics and psychology. In ongoing research, we combine state-of-the-art theories and methodologies from both disciplines and measure self-control failures in people's everyday lives. Applying this novel approach to measuring everyday decision making, firstly we test whether individual differences in present bias predict self-control failures in everyday life. Secondly, we identify episodes in which study participants have self-control failures and compare subjective well-being across episodes.

1015 to 11am

Dr. Mark McGovern (Queen's University Belfast)

Designing RCTs and Observational Studies to Account for Missing Data not Missing at Random

Missing data is a common feature of both survey data and RCTs, which has the potential to greatly impact on the policy recommendations we derive from empirical studies. Non-response can lead to biased estimates if the characteristics of respondents systematically differ from those who decline to participate. In practice, if any adjustments for missing data are made, they tend to be based on either multiple imputation or inverse probability weighting. Conventional methods such as these all rely on a key assumption: missing data must be missing at random, or missing at random conditional on observed covariates. This is a strong and generally untestable assumption which is unrealistic in many settings, especially where some respondents have an incentive not to participate. An alternative approach, Heckman-type selection models, can be used for dealing with missing data. This method can provide consistent estimates even when the assumption of missing at random does not hold, and respondents systematically opt out of survey participation on the basis of unobserved confounders. Using examples from research on HIV, I illustrate the consequences of imposing an unrealistic missing at random assumption on survey data. I conclude by discussing how to design RCTs and observational studies to facilitate the implementation of this selection model approach.

11 to 1130am Break 

1130 to 1215pm

Professor Marjon Van Pol (Aberdeen University)

Improving the measurement of time preferences

The interest in measuring individuals’ time preferences is growing in both economics and psychology. Time preferences describe individuals’ preferences over when outcomes occur and are a determinant of a range of important life outcomes such as health and education. Surprisingly little attention has been paid to the robustness of the design of the elicitation methods. In standard economic theory, the design of the elicitation method is irrelevant as individuals have fully formed and highly articulated preferences which they can quickly and accurately access and which are not affected by design features. However, evidence from stated preferences methods such as contingent valuation suggests that individuals construct their preferences and may use decision heuristics in experiments. The design of the elicitation method matters in this case and may lead to different policy recommendations. In this paper we test the internal validity of the most commonly used time preference elicitation method, the multiple price list, drawing on insights from the contingent valuation literature.  We test for both an order effect and the effect of a truth telling oath in an online survey. We compare the % of ‘theoretically inconsistent’ responses, response times and average rates of time preference.  The preliminary results suggests a strong order effect and a weaker effect of a truth telling oath.

1215pm to 1pm   

Dr. Daniel Powell (Aberdeen University)

Real-time tracking of state inhibitory control and health behaviour in daily life: an overview of the SNAPSHOT study

Several contemporary theories of health behaviour, including temporal self-regulation theory and the various dual process theories, suggest that variations in cognitive efficiency should have important consequences for health behaviour. However, little research has tested how health behaviours may be influenced by dynamic fluctuations in executive functioning within individuals in daily life. The SNAPSHOT (SNAcking, Physical activity, Self-regulation, and Heart-rate Over Time) study is a 7-day ecological momentary assessment (EMA) study incorporating various wearable devices, including accelerometers, heart-rate monitors, GPS trackers, and a wrist-mounted electronic diary. In a community sample of 68 participants, hourly self-reports of snacking behaviour and – uniquely – objective measures of inhibitory control (the Go/No-Go test) were requested via the diary. This talk will (i) provide an overview of the findings we have in relation to associations between inhibitory control and snacking behaviour, (ii) detail clear individual differences in the contextual correlates of snacking in daily life, and (iii) outline some of the methodological challenges, particularly with the Go/No-Go test.

1pm to 2pm Lunch 

2pm to 245pm 

Dr. Stephan Bruns (University of Kassell)

P-curve and p-hacking in observational research

The p-curve, the distribution of statistically significant p-values of published studies, has been used to make inferences on the proportion of true effects and on the presence of p-hacking in the published literature. We analyze the p-curve for observational research in the presence of p-hacking. We show by means of simulations that even with minimal omitted-variable bias (e.g. unaccounted confounding) p-curves based on true effects and p-curves based on null-effects with p-hacking cannot be reliably distinguished. We also demonstrate this problem using as practical example the evaluation of the effect of malaria prevalence on economic growth between 1960 and 1996. These findings call recent studies into question that use the p-curve to infer that most published research findings are based on true effects in the medical literature and in a wide range of disciplines. p-values in observational research may need to be empirically calibrated to be interpretable with respect to the commonly used significance threshold of 0.05. Violations of randomization in experimental studies may also result in situations where the use of p-curves is similarly unreliable.

245pm to 330pm 

Professor Alex Bryson (UCL)

The Biometric Antecedents to Happiness

Abstract

Happiness is beneficial to individuals and society. Happier individuals are more productive, more resilient to illness and disease, and live longer. However, little is known about its antecedents and, in particular, its relationship with biometric indicators of wellbeing. What  is known is based largely on cross-sectional data. We contribute to the empirical literature by examining the independent association between various aspects of biometric wellbeing measured in childhood and happiness in adulthood. We find only one of the eight biomarkers we consider predicts happiness in adulthood: serum triglycerides, which are a type of fat found in the circulation, are negatively associated with subsequent happiness. The finding is robust to controls for age, sex, body size, family background, nutritional intake, physical activity, income, education and labour market experiences, as well as other biomarkers measured in childhood. It suggests higher levels of serum triglycerides in childhood can be damaging to one’s happiness in adulthood.

330pm to 415pm 

Dr. David Comerford (Stirling)

Agency: Its role in the measurement of preferences and utility

 Abstract:

We distinguish between agentic preference (preference regarding outcomes that the consumer can actively influence); and non-agentic preference (preference regarding outcomes that are passively received). We theorize that agentic preference is informed by the signalling value of endorsing an outcome, and by the reputational value of being responsible for that outcome. Non-agentic preference is not informed by either of these sources of value. Often, policymakers need to measure non-agentic preferences, for instance, when measuring the costs inflicted by an externality. We present experimental evidence that agentic preference orderings over a given choice set can differ systematically from non-agentic preference orderings. We present examples from the literature where agentic preferences are used to infer non-agentic utility, and where non-agentic preferences are used to infer agentic utility. We conclude that our typology of agentic and non-agentic preference can clarify utility measurement.