Wednesday, July 22, 2009

Intrinsic Incentives and Evaluation

Related to the last post, I give a lecture in Behavioural Economics on alternative theories of motivation other than financial incentives.

Lecture Here

The intrinsic incentives literature has stressed the extent to which individual's intrinsic motivations to perform their tasks can be crowded out if they are excessively monitored or made to comply with different types of identity-conflicting beurocratic tasks.

In the context of a more behaviourally driven model of researcher and academic productivity, it is worth considering the potential effect of using incentives, punishments and centralised allocation models on the desire of individual researchers and academics to push to perform at very high levels. Related to this, the extent to which cultural and peer environments drive individual academic and researcher performance needs to be thought of a lot more also. A comment on a previous post suggests that just allowing academics to function without any accountability processes or centralisation will create better outcomes than costly central mechanisms that potentially create distortions and crowd out motivation. As against that though, what if people are gaming the system and are not that motivated to perform? What if whole institutions that are receiving public money are not producing anything of value other than lobbying? And perhaps more crucially to the current debate, what if researchers are very intrinsically motivated but have no interest in wider commercial or policy implications of their work? In some circumstances this may create outcomes and Im sure people will reference many geniuses who work oblivious to any practical applications. But is it really a sufficient model of human motivation to power lower level technological and research innovations?

Bruno Frey has written on this topic in a number of papers. Operationalising in the context of the current situation made pose challenges but these forces should not be dismissed.


Ernie Ball said...

You write:

What if people are gaming the system and are not that motivated to perform? What if whole institutions that are receiving public money are not producing anything of value other than lobbying?

I have some similar questions: What if some of the staff are witches? What if all of them are witches and our universities are very well-disguised gigantic covens?

Seriously, though, your questions are only slightly less paranoid than mine. This is why, in my responses to that other post (I was anonymous there, I'm pseudonymous here) I kept asking where the suspicion comes from. What motivates it? Better still: who is pushing it? And do you think that such paranoia is a sound basis for public policy? Does the country need a new McCarthy report, not Colm this time but Joseph? Are you now or have you ever been a slacker?

Given the kinds of sacrifices one has to make to become a university lecturer or researcher, the rigours of the hiring process, as well as the tremendous risk of failure (the beaches are white with the bones of talented researchers who didn't "make it"), it would be surprising to say the least if more than a tiny fraction were taking advantage and not intrinsically motivated. I'd wager there are fewer than in just about any other profession. Are there some? Sure. But is it really smart to force the vast majority who are not to jump through hoops in order to weed out a few? Is that a wise use of resources? Is it a worthwhile use of your time to try to devise the hoops at which the slackers and only the slackers will fall short?

I would like to point out, once again, that the best universities in the world are also the ones that impose the least of these sorts of monitoring and accountability exercises. Not all of them are private institutions. Rather, they have confidence in the people they hire and they are able to hire better people because they have the reputation of having confidence in the people they hire. They let them get on with it.

It is rarely pointed out the sort of pernicious effects such monitoring exercises like the RAE in the UK have on hiring. Like it or not, the academic labour market is global. I say this as someone who was recruited internationally myself. Contrary to what is often said, that is one of the only areas where Irish universities actually do have to compete. Our island is too small to provide all the research and teaching talent we need. Given all the disadvantages involved in immigrating to Ireland (and this was so even before we became the basket case of Europe), do we really want to have the reputation of being the place where universities impose all kinds of pointless accountability exercises on their staff? That will not only make it impossible to attract talented staff, it will make it difficult to retain native-born Irish staff who would otherwise be inclined to stay.

You ask another question that is somewhat less paranoid: What if researchers are very intrinsically motivated but have no interest in wider commercial or policy implications of their work?

Well, then, they are doing exactly what they are supposed to be doing and are to be applauded. Even in the neo-liberal money-is-the-end-of-all-existence sort of way of thinking that everyone in this country seems to have adopted, it will always make sense to preserve the universities as the only places where disinterested (a more charitable way of saying "not interested in the wider commercial applications, etc.") research can take place. Allowing this research to take place is more likely to result in revolutionary breakthroughs and, yes, profitable product categories (if that's your thing) than having everyone always thinking only of the short-term bottom line. The list of discoveries made by such disinterested research and their subsequent applications (think: nuclear power or MRIs) is far more impressive than anything that those who think they can pick winners might come up with.

Liam Delaney said...

You are arguing against something other than my posts Ernie. This is an interesting discussion so try to keep it polite. I am a committed academic myself and have spent a huge amount of time promoting my field so accusations of being paranoid or trying to do anything to harm research and academic activity are unfair.

I am raising a question as to the appropriate form of accountability for academics and colleges. You are talking specifically about pretty intensive exercises that I have not endorsed or called for in any way. Your comments suggest that no form of accountability is appropriate for anyone with a university academic contract. Surely there must be a way for a university to independently demonstrate the value deriving from its activities and thus obviate the need for exercises such as RAE that indisputably generate volumes of form-filling that are maddening for most serious academics.

Similarly, there has to be ways of sorting between different people when making decisions. How would you allocate research funding in your model Ernie? Would every academic be given an equal amount of money? Similarly, does every academic teach the same number of hours in your model? Or is there a basic system for recognising that research takes time and that it may not be possible to take on a full undergraduate teaching load if one is conducting intensive research?

For example, in these top universities you speak of it is completely normal for academics who are judged very highly to be paid extra and to have a lower teaching load (though again I am not fully endorsing the latter as it is desirable to have good research professors in front of students). You make continuous reference to off-the-scale salaries in specific Irish institutions but, again, that's an argument for a different thread. Where in my comments or posts have I agreed with that system? I am not proposing that the current systems in Ireland are the best ones. In fact, a large motivation for my posts is to ask whether there are fairer and better ways of allocating university resources to people that will use them better for students, research and society.

With regard to the view that university staff should have no explicit policy or commercial remit, get down off the high-horse for a minute. I absolutely agree that a society should reserve funding for what would now be called basic research, that has no explicit mandate other than to discover the truth. But applied research is also needed for many areas. In Ireland and in many countries funding is made available explicitly for universities to conduct applied research. You are reducing down a complex question as to the relative mix of applied and basic research that should be conducted in a university down to a platitude about academic independence. There are about 3,000 contract research staff in Irish universities, with a large number of them working on applied research topics. It is a very interesting argument as to how much resources should be devoted to basic research but don't reduce it down to such a simplistic discussion.

The European Framework process is interesting in this regard. For the most part, it is interested in applied research but has recently created a purely academically driven component called the European Research Council. As far as I can see, this is almost entirely driven by academics and the criteria for being awarded resources are entirely connected to academic assessments of the quality of the work. It funds both "starting" and "advanced" grants that are substantial and require a minimum of paper-work. In relation to what you are saying, I think it will be very interesting to see whether the ERC research in the long run is the research that actually has the most societal value long after the applied research has fizzled out. I might even attempt to test this!

Ernie Ball said...


Just to avert a misunderstanding, I should have made clearer that my comments about paranoia were not specifically directed at you. Rather, the paranoia seems to be at large in the society as a whole and I see such questions as those you raise (which you are not alone in formulating, obviously) as symptomatic of this. Which is why I keep asking where the suspicion comes from. That should make it clear that I don't think that you are the source of the suspicion.

I'd like to write a more thorough response when I find time.

Liam Delaney said...

Sure - I will post something else on this issue in the near future. I recommend that posting on this blog with relation to this issue starts from the assumption that none of the posters have any particular axe to grind with relation to maintaining the current Irish systems. The question of interest raised is how should universities be funded and evaluated. The answer may very well be that all evaluation be completely internal to the university academics but its worth some public discussion of such a model.

Liam Delaney said...

Perhaps I spoke too soon on ERC

Nature Article