Using behavioural science in the classroom to unlock motivation

Everyone starts with an A

Applying behavioural insight to improve performance and narrow the socioeconomic attainment gap in education.

“Imagine a classroom where everyone started off an academic year with an “A” grade, and in order to keep the grade, a pupil had to show continuous improvement throughout the year. In this classroom, the teacher would have to dock points from a pupil’s assessment when his or her performance or achievement was inadequate, and pupils would work to maintain their high mark rather than to work up to it. How would this affect effort, expectations, performance, and assessment relative to current practice?”

This is one of the questions the RSA (Royal Society for the encouragement of Arts Manufacture and Commerce) pose in their report Everyone Starts with an A, which explores the application of behavioural insight to educational policy and practice.

Using research from behavioural science and our evolving understanding of human nature, the report explores how effort, motivation, learning enjoyment, resilience, and overall performance at school can be influenced in ways not often traditionally recognised.

Download EveryoneStarts with an 'A' full report in English (PDF 393.6KB)
Download Everyone Starts with an 'A' poster in English (PDF 71.4KB)
Download Everyone Starts with an 'A' full report in German (PDF 411.4KB)
Download Everyone Starts with an 'A' poster in German (PDF 56.3KB)

Connecting theory to practice

Supported by the Vodafone Foundation Germany, this report is intended to start a conversation among educators and includes practical tips to help connect theory to real-life practice in the classroom.

Three concept areas are covered:

·         Growth mindsets: the belief that intelligence and ability are not a fixed and innate trait, but rather they can be improved and strengthened through effort and practice.

·         Cognitive biases: our thinking patterns can have systematic influence on motivation, evaluation, and teacher and pupil expectations about performance. Anchoring, the halo effect, confirmation bias, and loss aversion may all play a role. 

·         Surroundings: the physical environment of the classroom can affect various cognitive and non-cognitive skills - such as attention levels and self-control - which are important for learning.

These three points are promising areas for further research. An improved understanding of how these concepts affect pupil learning might be especially valuable to disrupt patterns of assumption about performance levels, for those who self-identify as being part of a stigmatised group, such as those from a low socioeconomic background. The RSA hopes that practitioners will continue the discussion started here by trialling the tips and techniques in their own schools and sharing their experiences with peers and colleagues.

For more information, download the paper here (or choose your preferred version above), read blogs about the launch of the report here and here, or view the press release here. 

Nathalie Spencer is a Senior Researcher in the RSA’s Social Brain Centre, and a longtime reader of the Stirling Behavioural Science blog.  

Survey Questions for Research on Non-Cognitive Skills

The Decision Making Individual Differences Inventory (DMIDI) is a catalogue of over 170 individual difference measures commonly used in judgment and decision-making research. Basic descriptive information (including references & scale information) is available for all measures. Topics covered include attitude to risk, cognitive ability, motivation and personality (which includes time orientation). DMIDI is maintained by Kirstin Appelt, Kerry Milch, Michel Handgraaf and Elke Weber (all affiliated to the Center for Research on Environmental Decisions, Columbia University, New York).

A similar initiative is the Social-Personality Psychology Questionnaire Instrument Compendium (QIC), which links only to pages where (a) the full instrument is shown, and (b) the instrument was put on the web by the person who created the instrument, thus ensuring that the instrument was intended to be put into the public domain. QIC is maintained by Alan Reifman from Texas Tech University.

The (U.S. National Institutes of Health) National Cancer Institute Grid-Enabled Measures (GEM) database is a website of behavioral and social science measures. Researchers have the ability to search, download, submit and provide feedback on measures. The purpose of GEM is to provide a resource that enables researchers to:
# Use standardized, vetted measures; and
# Share data that results from using these measures

What Do Students Learn in College?

They might attend their lectures, do additional hours of study and get good grades; but... what are students learning in college? A recent article in the New York Times asks just this question: "what are America's kids actually learning in college?". Thanks to Peter C for sending on the link. According to the NYT article, "a provocative new book, Academically Adrift: Limited Learning on College Campuses, makes a strong case that for a large portion of the nation’s seemingly successful undergraduates, the years in college barely improve their skills in critical thinking, complex reasoning and writing." The authors are Professors Richard Arum of New York University and Josipa Roksa of the University of Virginia. A Google-Books preview is available here.

Arum and Roska's research draws on survey responses, transcript data, and the Collegiate Learning Assessment, a standardised test administered to students in their first semester and then again at the end of their second year. More details on the authors' research project are available here. According to the authors' analysis (of more than 2,300 undergraduates at twenty-four institutions), 45 percent of these students demonstrate no significant improvement in a range of skills, including: critical thinking, complex reasoning and writing (during their first two years of college). According to the authors, "many students come to college not only poorly prepared by prior schooling for highly demanding academic tasks that ideally lie in front of them, but — more troubling still — they enter college with attitudes, norms, values, and behaviors that are often at odds with academic commitment."

In relation to academic commitment, education economists would draw attention to the idea of non-cognitive skills. Pedro Martins has shown that a targeted program related to non-cognitive ability – study skills, motivation, and self-esteem – can improve student achievement at state schools in Portugal. In work that I have done with Liam and Colm, there is a clear demonstration of academic advantage for Irish university students who are more future-orientated and more conscientious. A targeted program (providing academic and social supports) was evaluated in the setting of an Irish university by Denny, Doyle, O’Reilly and O’Sullivan (2010); the results suggesting it was an effective means of improving academic outcomes for socio-economically disadvantaged students attending Irish universities.

Returning to the NYT article, it interprets Arum and Roksa's research to indicate that "students are hitting the books less and partying more. Easier courses and easier majors have become more and more popular. Perhaps more now than ever, the point of the college experience is to have a good time and walk away with a valuable credential after putting in the least effort possible." A study by three Cornell economists found a large increase in enrollment in courses with a median grade of A; after a new policy which introduced the publication of course median grades on the Internet. This is suggestive of students enrolling in classes where it easier to obtain higher grade-scores; an issue which I raised before in relation to the Irish Leaving Certificate: the final examination at the end of second-level education Ireland.

Recent research by Brian Jacob, Brian McCall and Kevin Stange (all from the University of Michigan) specifically addresses the issue of the consumption-value of college education. Jacob, McCall and Stange report that students "appear to value several college attributes which we categorize as consumption because their benefits arguably accrue only while actually enrolled, including college spending on student activities, sports, and dormitories." In addition, research by Babcock and Marks (discussed on this blog before) documents that academic time investment among full-time college students fell from 40 hours per in 1961 to 27 hours per week in 2003. Another article from the NYT tells us that Arum and Roksa's research is "consistent with the findings of the National Survey of Student Engagement, which has polled more than 2 million students at more than 1,000 colleges and universities over more than a decade — and reported that many spend little time studying or writing."

According to this article in the Huffington Post, the Collegiate Learning Assessment "has its share of critics who say it doesn't capture learning in specialized majors or isn't a reliable measure of college performance because so many factors are beyond their control." Nonetheless, it looks like this debate is set to grow more widespread in the future. It also resonates with last year's debate in Ireland about the issue of grade inflation. Introducing a research component into the undergraduate curriculum could be one way to improve students' skills in critical thinking, complex reasoning and writing. In 2005, Trinity College Dublin launched a summer programme offering undergraduate students, typically from Science and Engineering, the opportunity to work in research for an 11 week period, on projects in laboratory environments.

Resilience

One of the non-cognitive skills that recent research has emphasized is resilience to adversity, for example Heckman & Cunha, 2009. We could do with a bit of that here & indeed some folks are doing their bit to that end.

Can Targeted, Non-Cognitive Skills Programs Improve Achievement?

This is the title of a new research paper by Pedro Martins from the University of London. Martins’ findings show that a targeted program related to non-cognitive ability – in this case study skills, motivation, and self-esteem – can improve student achievement at state schools in Portugal. In work that I have done with Liam and Colm, there is a clear demonstration of academic advantage for Irish university students who are more future-orientated and more conscientious. Given all of the above, there is a case for considering whether the Portugese program can translate (with obvious modifications) to Irish universities, and Irish schools. Of course, it must be borne in mind that there is an existing Access program in Irish universities (and Irish schools) which provides academic and social supports. The Access program was evaluated in the setting of an Irish university by Denny, Doyle, O’Reilly and O’Sullivan (2010); the results suggesting that Access programs can be an effective means of improving academic outcomes for socio-economically disadvantaged students attending Irish universities. One question that I plan to investigate further is whether the design of Access programs in Ireland exlicitly considers the concept of non-cognitive skill formation. I would be interested to hear from any readers who have any information on this.

IZA Workshop: Cognitive and Non-Cognitive Skills

From the workshop website:

Organizers: Steffen Altmann (IZA), Armin Falk (University of Bonn and IZA)
Place: IZA, Bonn
Date: January 25 - January 27, 2011

Submission Deadline: October 15, 2010
Notification of Acceptance: November 15, 2010
Complete Papers Due: December 31, 2010

The workshop will bring together leading scholars and young researchers pursuing research at the intersection of economics, psychology, and biology. Acknowledging the importance of interdisciplinary research for understanding the foundations and development of cognitive and non-cognitive skills, we invite contributions from various methodological backgrounds: theoretical contributions as well as empirical research using field data, experiments, or neuroeconomic methods.

# Topics of the workshop will include (but are not limited to): Measurement of cognitive and non-cognitive skills
# Skill formation during the life-cycle
# Economic and social consequences of (non-) cognitive skills
# Economic, biological, psychological and environmental determinants of skills
# Policy interventions to enhance skill formation, particularly during early childhood

A highlight of the meeting is the Keynote Lecture which will be held by James J. Heckman (University of Chicago and IZA).

The conference and lodging will be free of charge for presenters, and travel expenses will be reimbursed according to IZA travel guidelines.

Smile Like You Mean It

Thanks to Eoin McL for giving me a new insight into what we consider to be non-cognitive ability (or non-cognitive skill). I've discussed this concept before in relation to labour market earnings, graduate education, life expectancy and development of cognitive skill.

Eoin pointed me towards the author Samuel Smiles, who according to Wikipedia, was editor of the Leeds Times from 1838-1845. In this role, he advocated radical causes ranging from women's suffrage to free trade to parliamentary reform. Wikipedia reports that in the 1850s he seems to have completely given up on parliamentary reform and other structural changes as a means of social advance. For the rest of his career, he advocated individual self improvement. This is the link to what we think of now as non-cognitive skills. Smiles is best known as the writer of what can be considered as self-help books, some of which are listed below:

Self-Help (with Illustrations of Conduct and Perseverance), London, 1859
Character, London, 1871
Thrift, London, 1875
Duty, London, 1880
Life and Labour, London 1887

I'm currently reading the first book; electronic copies of this and many others are available here on the Project Gutenberg website. The Smilesian view on the importance of education is provided here by James Stansfield.

Completing an Economics PhD in Five Years

Thanks to Christian for pointing out this paper published in the AER today (by Stock, Finegan and Siegfried). Endogeneity concerns aside, finishing your PhD within the designated time is positively affected by:

- larger 1st year PhD classes
- shared offices
- being male
- whether someone went to a top tier university for undergrad

Factors that have a negative effect are:

- doing your PhD with a top tier instution
- high attrition in the 2nd year
- pre-thesis research work requirement
- having an undergrad degree in economics

The authors conclude that "many considerations unique to individual students and faculty that we cannot measure—such as ambition, motivation, persistence, organizational skills, the creativity of students, and interest in students’ success as well as mentoring and motivational skills among graduate faculty—matter more than the myriad characteristics we were able to measure, which collectively account for less than 15 percent of the variation in completion among students."

Some insights on how non-cognitive personality constructs (such as ambition, motivation, persistence and organisation) apply to graduate education are provided in the Educational Assessmnet Journal (2005) by Patrick Kyllonen, Alyssa Walters and James Kaufman from the Princeton Educational Testing Service. We discussed this research on the blog before: here.

Stock and Siegfried (2006) reported on time-to-degree for economics Ph.D.'s in the United States in the AEA Papers and Proceedings. That research motivated me to consider that the duration of the Ph.D. process (or time-to-degree) may be a source of comparability problems in self-rated skills matching for Ph.D. graduates (see a previous post on skills-matching here).

The idea is that the more the individual has committed to the process of atatining a Ph.D., the more he or she will want to view the outcome of that process favourably. Taking a year longer during Ph.D. training implies a very particular opportunity cost. There is a precedent for this type of comparability-bias in the anchoring vignettes literature.

Buckley (2007) used the anchoring vignettes technique to investigate the "rose-coloured glasses" effect, which refers to parents reporting higher levels of satisfaction with a school solely or partially as a justification for the effort expended in the choice process. The analogy to 'time-to-degree' is about the amount of time expended in the Ph.D. process. (See a previous discussion of Buckley's research here).

Using Measures of Non-Cognitive Ability in Economics

Cognitive ability (as measured by test scores) only determines part of a person's success in the labour market. This came into sharp focus in the economics profession at the 2001 meeting of the American Economics Association. At this meeting a number of papers were presented about the importance of non-cognitive ability (also referred to by some authors as 'non-cognitive skills'). An example is Heckman and Rubinstein (2001) who mention non-cognitive skills such as "persistence, reliability and self-discipline". Most often though, the phrasing of "non-cognitive ability" is used, for example: Heckman, Stixrud and Urzua (JLE, 2006): "The Effects of Cognitive and Noncognitive Abilities on Labour Market Outcomes and Social Behavior."

Heckman and Rubinstein (2001) identified the importance of non-cognitive abilities with their observation that high school equivalency recipients earn less than high school graduates despite the fact that the high school equivalency recipients are smarter. They attribute this to the negative non-cognitive attributes of equivalency recipients originally dropping out. Their conclusion is that individuals with higher amounts of persistence and self-discipline may be more likely to attain academic qualifications.

In relation to further evidence, Heckman, Stixrud and Urzua (2006) model the influence of young individuals' cognitive and non-cognitive abilities on schooling and earnings. They find that better non-cognitive abilities lead to more schooling, but also have an earnings return over and above this. Kern and Friedman (2008) de-compose (overall) conscientiousness into a range of non-cognitive abilities, including persistence, industriousness, organisation and discipline (read previous blog discussion on this here).

The trait of conscientiousness is taken from the "Big Five" set of personality characteristics. Kyllonen, Walters and Kaufman (2005) review the literature on noncognitive constructs (such as the "Big Five"), and conclude with a discussion of how non-cognitive constructs (or personality factors) might be used in admissions and guidance applications for graduate education (read previous blog discussion on this here).

Braakmann (2009) has used the German Socio-Economic Panel to show that differences in various non-cognitive traits, specifically the Big Five, contribute to gender inequalities in wages and employment (this was previously mentioned on the blog here). Mueller and Plug (2004) shows that the labour market values conscientiousness and openness to experience for women (previous discussion on the blog here --- in relation to non-cognitive personality, education and earnings).

Kyllonen (2008) is perhaps the most detailed assessment of how to measure non-cognitive abilities; he associates the non-cognitive abilities shown below with the "Big Five" personality traits. Kyllonen is based at the Princeton Educational Testing Service (ETS) and put forward the framework below ("Enhancing Noncognitive Skills to Boost Academic Achievement") at a 2008 Washington conference entitled 'Educational Testing in America: State Assessments, Achievement Gaps, National Policy and Innovations'.

Noncognitive Constructs and Their Assessment in Graduate Education

Some insights on how non-cognitive personality constructs apply to graduate education are provided in the Educational Assessmnet Journal (2005) by Patrick Kyllonen, Alyssa Walters and James Kaufman from the Princeton Educational Testing Service and Department of Psychology, California State University at San Bernardino. The authors review the literature on noncognitive constructs, as well as personality as it specifically relates to graduate education.

In the first section, they review measures typically used in studies of graduate school outcomes, such as attrition and time to degree. They also review which student qualities faculty and administrators said they desired and cultivated in graduate programs. (They noted that there are many qualities faculty ranked high in desirability but which could only imperfectly be gleaned from sources such as letters of recommendation and personal statements).

In the second section, they review general personality factors (such as the “Big Five”), and conclude with a discussion of how personality factors might be used in admissions and guidance applications for graduate education.