New evidence on class size effects

Class size effects: evidence using a new estimation technique

Kevin Denny, Veruska Oppedisano
This paper estimates the marginal effect of class size on educational attainment of high school students. We control for the potential endogeneity of class size in two ways using a conventional instrumental variable approach, based on changes in cohort size, and an alternative method where identification is based on restriction on higher moments. The data is drawn from the Program for International Student Assessment (PISA) collected in 2003 for the United States and the United Kingdom. Using either method or the two in conjunction leads to the conclusion that increases in class size lead to improvements in student’s mathematics scores. Only the results for the United Kingdom are statistically significant.

Education cuts, the National Plan and class sizes

Because of the dire fiscal situation, it seems some cut-backs to educational spending would be inevitable. The National Plan is pretty vague on education (amongst other things). Discussing the plan in the Irish Times Colm McCarthy remarked “The plan reflects successful lobbying to exempt the education budget from severe cuts. This is being justified in terms of the importance of holding with existing targets for pupil-teacher ratios, notwithstanding the dearth of evidence that reducing these ratios weakens educational outcomes in any measurable way.”

Is this really true? Well no. Few parameters in the economics of education have been so well studied as the effect of class size on educational outcomes. There are dozens and dozens of studies. So what’s the answer then? Well this is where it gets complicated. Firstly, we have no good evidence for Ireland that I am aware of. If this is what Colm McCarthy means then he is correct but then we don’t have any evidence on lots of things for Ireland and absence of evidence is not evidence of absence. What does the international evidence say then? The first complication is that one should not expect one answer. Primary schools are different from secondary schools, a class of 40 is different from one of 20 and Korea is not Bangladesh so variation in measured effects is to be expected. A further problem, which non academics may not care about but is important, is that methods for estimating these effects vary widely and this partly explains some of the variation.

The most well studied country is the US. The STAR experiment in Tennessee is generally considered a well designed study and points to significant benefits from smaller classes but in that case the reductions were big (around 9 pupils on average). A “natural experiment” in Connecticut came up with a “precisely estimated zero” effect (Hoxby). The famous Maimonides Rule study for Israel (Angrist & Lavy) found positive effects of smaller classes but similar work for the Netherlands found the opposite (papers by Levin, Dobbelstein et al). A cross country study using TIMSS data (Woessman & West) found a mixed bag of results. Some reviews of the evidence point to negligible effects over all (see the work by Eric Hanushek) while other meta-analyses point to clear benefits from reducing classes. So rather than a dearth of evidence there is too much of it or at least there is not enough consensus and you can pick a study to suit your prejudice (or “prior” to give it its scientific name).

What’s striking about this literature is its near obsession with one variable, class size. Other measures of quality are almost entirely ignored. Ask yourself or someone else was their school good and they will quickly you reasons why it was or wasn’t. Class size tends not to be prominent a reason in my experience. This isn’t scientific but it does remind us that lots of things, some hard to measure, go into making a good school. One factor that everyone mentions is their teachers. Curiously, measuring the quality of teachers and the effect it has on outcomes does not feature much in the policy debates.

In the absence of clear evidence it probably makes sense that any damage from increased class effects is minimized by favouring more socially disadvantaged schools and those schools with the biggest class sizes.

Class Size in Irish Primary Schools: 1967 to 2006

I referenced the class size debates in primary-level and higher education recently on this blog. Below is an abstract (also the first item on this link) from a research paper from the ERC in Drumcondra (Irish Journal of Education: Vol. 38, 2010). The paper discusses Govt. policy relating to reduction in primary-school class size over the last 50 years. During the time-period 1967/68 to 2005/06, overall class size dropped from 32.4:1 to 16.6:1. This of course masks variation, which can be examined for the last four years here: by primary-school and class-room.

CLASS SIZE AND PUPIL-TEACHER RATIO: POLICY AND PROGRESS
Susan Weir, Peter Archer, and Laura McAvinue
Educational Research Centre, St Patrick’s College, Dublin

Government policy relating to reduction in class size and pupil-teacher ratio over the past 50 years is outlined. Since the 1980s, the policy has focused on positive discrimination towards schools serving pupils in disadvantaged areas. An examination of statistics reveals a decline in overall pupil-teacher ratio from 32.4:1 to 16.6:1 over the period 1967/68 to 2005/06. Statistics for single-grade class size for the period 1985/86 to 2005/06 also show a decline, more so in junior than in senior classes. Time-series analyses indicate that the introduction of schemes involving reduction in class size to address disadvantage was associated with a perceptible departure from overall trends. Class size in the urban dimension of Giving Children an Even Break (GCEB) was found to be smaller than class size in other urban schools.

Class Size in Higher Education

The role of class-size in primary education has between discussed before on this blog. Earlier this year Kevin and Mark both posted interesting comments on the analysis of secondary data. In addition, I recently mentioned that data on primary-school class sizes in Ireland are available for the last four years by primary-school and class-room. Those looking for an introduction to how economists view the importance of class size in primary education could do worse than investigate this debate between Alan Krueger, Eric Hanushek and Jennifer King Rice.

"Alan Krueger maintains that smaller class sizes can improve students’ performance and future earnings prospects. He challenges Prof. Hanushek’s widely cited analysis of the class size literature, arguing that it gives disproportionate weight to single studies that include a large number of estimates... Jennifer King Rice brings a third-party perspective to the debate. She addresses each author’s arguments and focuses on the policy implications of the class size literature."

As educational achievement in higher education is often discussed on this blog, it seems salient to ask what we know about the importance of class size in higher education. To begin, it is helpful to point out that much of the work using education production functions has concentrated on the educational attainment of pupils in compulsory schooling, with less attention paid to higher education (Arulampalam, Naylor and Smith, 2009). The common inputs in education production functions are things like school resources, teacher quality, and family attributes, and the outcome is student achievement (Hanushek, 2007). However, there is a precedent for the theoretical consideration of higher education production functions (Freire and Silva, 1975; Johnson, 1978; Hopkins, 1990; Douglas and Sulock, 1995). There is also a much wider empirical literature on higher education production functions, in which researchers give attention to student inputs, in particular: lecture attendance and additional hours of study.

Looking at the evidence on class size in higher education, one result is that smaller classes do not translate into gains in achievement (Martins and Walker, 2006). Looking at economics students only, Kennedy and Siegfried (1997) find the same result i.e. that class size does not affect student achievement. Other work by Gleason shows that the same holds for mathematics students.

The opposite result (that class size matters) is found in a study examining peer effects and class size in higher education; Machado and Vera-Hernandez (2010) find that class size negatively influences medium ability college students. Dillon and Kokkelenberg (2002) show that class size "has a negative logarithmic relationship to grades and that the effect on class size on grades differs across different category of student."

In a recent Vox article, Bandiera, Larcinese and Rasul (2010) state that the effect of increasing class size in tertiary education is not yet well understood. Drawing on their article forthcoming in the Economic Journal, Bandiera, Larcinese and Rasul describe how they estimate the effects of class size on students’ exam performance by comparing the same student’s performance to her own performance in courses with small and large class sizes. "Going from the average class of 56 to a class size of 89 would decrease the mark by 9% of the observed variation in marks within a given student. The effect is almost four times larger for students in the top 10%."

It seems that the debate on class size in higher education is just as lively as the debate on class size in primary education.

The Long Run Effects of the Tennessee STAR Experiment

The Tennessee STAR experiment (where school children were randomly assigned to different class sizes) is one of the most well known RCTs in social science, and is frequently referenced on this blog. On his site, Greg Mankiw links to an article in the NYT which discusses new research by Chetty et al. Up to now STAR was mainly used to examine the effects of class size on test scores, with the conclusion generally being that the treatment (of smaller class size) had a contemporaneous positive effect on test scores, however this effect did not last beyond a couple of years. This phenomenon will be familiar from other early intervention studies such as the Perry Preschool programme, where test scores in the treatment group showed an initial jump before returning to the same level as the control group. The following graph is taken from James J. Heckman & Dimitriy V. Masterov, 2007.

So if you are just looking at school test scores, the impact of these types of interventions can be debatable. However the authors of this recent research are able to examine more long run effects by linking data on STAR participants (now adults, it was conducted in the 1980s), to tax records. Their conclusions appear to be (the link above is to a presentation, I look forward to reading the full paper) that the contemporaneous rise in test scores is a good measure of the effect of the treatment on adult outcomes. According to their analysis, class size and teacher quality have an important impact on adult earnings. Again, this mirrors the findings from the Perry Preschool programme (shown below), where despite the same "fade out" in test scores, a significant positive effect was found on adult outcomes.

Lazear on class size effects

The link below has a nice summry of Ed Lazear's work on class-size effects (h/t Alan Fernihough)
http://www.hoover.org/publications/digest/3476106.html
A more technical version is at:
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=194830

Some unpleasant educational arithmetic

Given the major pressures on the public finances and hence on public spending on education, the question of class-size is likely to rear its ugly head again. We don't have good evidence for Ireland though there is some good potentially data. So what does it tell us - at least at first blush?
Using PISA data for Ireland I show a kernel regression of the students (15 year olds) against class-size. The results show that as you lower class size from 40 to about 35 there is a higher score- as you might expect. After that however, its down-hill all the way: smaller classes are associated with worse scores. This is actually robust to inclusion of school effects and various other controls. In work in progress with a colleague it even seems to be robust to controling for endogeneity (using two distinct identification strategies).

Evidence on Class size effects at university

The econometric evidence on the effects of class size on academic achievement is mostly at 2nd or 3rd level. The paper below considers the case of Italian universities:

Effects of Class Size on Achievement of College Students
De Paola, Maria & Scoppa, Vincenzo
In this paper we investigate the effects of class size on the achievements of a sample of college students enrolled at a middle-sized Italian public university. To estimate the effects of class size we exploit the exogenous variations in class size determined by a maximum class size rule introduced by the 2001 Italian university reform. From our analysis it emerges that large teaching classes produce negative effects on student performance measured both in terms of the grades obtained in exams and the probability of passing exams. These results are robust to the use of a matching estimator.
http://d.repec.org/n?u=RePEc:pra:mprapa:16945&r=edu