Monday, October 04, 2010

The Lake Wobegon Effect in Student Self-Reported Data

The Lake Wobegon effect, as described by Wikipedia, is also known as illusory superiority or the above average effect. It is a "cognitive bias that causes people to overestimate their positive qualities and abilities and to underestimate their negative qualities, relative to others. This is evident in a variety of areas including intelligence, performance on tasks or tests and the possession of desirable characteristics or personality traits." The phraseology of the Lake Wobegon effect comes from Garrison Keillor's fictional town, Lake Wobegon, where "all the children are above average".

In 1994, Maxwell and Lopus published an article in the American Economic Review: The Lake Wobegon Effect in Student Self-Reported Data. According to the authors, overstated achievement may produce biased estimates of the relationship between achievement and educational inputs, if overstatement is correlated with achievement. Maxwell and Lopus produce evidence for biases which stem from two sources. First, below-average students tend to inflate their academic achievements, and second, they often fail to report their inferior accomplishments.

In a more recent paper (Journal of Economic Education, 2010), Haley, Johnson and McGee examine whether using student-survey data in place of official records data meaningfully biases regression estimates. They motivate their contribution by noting a useful statistical feature of overreporting on bounded variables such as grade point average. "Specifically, the misreports will be negatively correlated with the true grade point average, yielding a form of nonclassical measurement error that actually counteracts the bias." The authors connect this observation to reliability ratios used in labour economics. In two applications, they find that it is unnecessary to correct for the bias from the Lake Wobegon effect because it is so small.

2 comments:

Liam Delaney said...

very interesting Martin. It is such a crucial area more generally given how difficult it is to link to admin records and how much more you can measure in a survey compared to just looking at admin files. If self-reported measures like grades yield very similar estimates to having the "true" measure this opens up a lot of research avenues.

Peter Carney said...

Martin, the key line you quoted doesn't make any sense to me;

"Specifically, the misreports will be negatively correlated with the true grade point average, yielding a form of nonclassical measurement error that actually counteracts the bias."

Sounds like a double bias - not sure if i would be as happy about the finding but i'm probably just not getting it.