Coming up with ways of making colleges accountable for the money they spend is something that has been long debated across many literatures. The Economix blog discusses this in a number of places including a discussion of a recent report on this issue. Using degrees awarded per money received is an interesting place to start but, as acknowledged, has a pretty strong set of flaws. Obviously, one would prefer some measure of the quality of education given per money received.
http://economix.blogs.nytimes.com/2009/07/21/investing-in-colleges/
Some of the key quotes are fascinating and likely indicative of a potential response to such research in Ireland. I have seen several hundred if not thousand of potential performance indicators that could be used in a university context and Im not sure I have ever seen one that couldn't be gamed in a way to achieve the target while not being in the spirit of what the target is about. There are certainly many in academia who argue that imposing these types of indicators crowd out intrinsic motivation but against that having no accountability also creates bad incentives. If anyone has a resolution to this dilemma, it would be good to hear!
From Economix:
“Some say there’s no way in this business to gauge quality,” said Mr. Kelly, a senior associate at the National Center for Higher Education Management Systems. “For most of the folks on the academic side of things, that’s typically their big rub. They say, ‘Yeah, we can just turn higher education into Wal-Mart.’ The truth is, Wal-Mart may be good for a lot of people.”
Some states — those that ranked well on Mr. Kelly’s productivity metric, of course — have been pleased with the research.
“Folks from Colorado and Florida love it,” he said. “It gives them ammunition to go to their legislatures and say, ‘We’re doing as good as anyone else with our resources! Please don’t cut our funding anymore!’”
5 comments:
Accountability is needed only where there's a suspicion that the payer might be getting ripped off. This of course assumes that the payer is well-informed, knows what it wants and that what it wants is commensurate with the raison d'être of the institutions themselves. If the payer wanted to turn universities into widget factories and demanded a totting up of the number of widgets produced, this wouldn't lead to the universities being held accountable.
My questions are: are there grounds for suspicion? If not, where does this suspicion come from?
My own view is that the suspicion of front-line academics and consequential "performance management systems" with their crude bibliometrics, etc. is completely misplaced. Academics that I know are committed to their jobs to a degree that one doesn't find in areas where the reasonable presumption is that the job is disagreeable and therefore that people will attempt to get out of doing it. That is why such neo-Taylorist management by control is completely inappropriate.
However, when it comes to university management, I think the suspicion is well-founded. When the locally-based member of a purportedly "international" presidential search committee resigns from the committee and then is appointed president, it arouses suspicion. When he appoints friends and courtesans all the way down to the lowest managerial levels and installs a centralised form of management worthy of Louis XIV, that arouses suspicion. When he offers those people off-the-scales salaries, that arouses suspicion. When he spends millions on consultants with no cost-benefit analysis, that arouses suspicion.
It is there that the value-for-money question is appropriately raised. Unsurprisingly perhaps, none of the "performance management" systems are applied at that level. Instead they are misapplied only to the front-line staff.
Which gives us a partial answer to the question of where the suspicion of front-line staff comes from. It comes from the administrators who are so in need of performance management themselves and finds a willing echo in the press and among politicians.
With respect that sounds like a stock quote rather than a specific reaction to the post. do you not acknowledge any sense that a college should be able to point to what benefits the public is deriving from publicly funding its activity? I agree with you that intrinsically motivated driven academics do not need to be chased around with poorly constructed accountability exercises in order to perform their tasks to a very high level but I would like to think that there is a basic set of things that a university does with public money that can be quantified to provide some metric. Ideally, it should be one that respects good teaching, good research and public service.
What "benefits"? What benefits do you think they get from it? If the public didn't think they were getting benefits from third level, would they be so keen to have their children go to third-level institutions? This reveals the cynical double game that the public and the press play with the universities. "Show us we're getting value for money but I sure hope my kid gets in." And it is interesting to see the charge for accountability being led by economists who otherwise trumpet the infallibility of the market...
As for research, start asking the 'value for money' question insistently and you'll find that the humanities as a whole are eradicated. Sorry, but any accountability exercise that jeopardises the historical heart of the university is not about measurement but rather about transforming universities into something else (the handmaidens of business). If you think this concern is idle or overstated, try extrapolating from the remarks of the Fine Gael spokesperson on education in the context of his statements about the "impact" of research funding: "I think the idea that we offer courses on subjects where there are few available jobs is not viable." You can be sure that he holds similar sentiments with regard to research: if it isn't profitable, it isn't viable.
Further, as you mention, any metric can be gamed. I don't think that this is an incidental consideration since another way of saying a metric has been gamed is to say that the university's mission has been perverted. Start counting up articles and you'll find that academics do nothing but produce make-work articles that nobody reads and that do nothing but decrease the signal-to-noise ratio in any given field. You'll fill the libraries with so much chatter and no academic will devote his life to the patient composition of a few masterworks of the sort that are much more likely to influence the course of intellectual and human history than whole reams of make-work articles. This, incidentally, is why the universities that are universally acknowledged to be the very best generally don't engage in such crude metrics. Rather, they rely on the professional judgement of those who know. Ireland is not unique in trying to remove qualitative judgement from the entire process, but it is an egregious offender.
So, again, I must ask: is there grounds for suspicion and, if so, where should the suspicion be directed? Clearly the answer to this is that anyone in charge of 7 and 8-figure budgets blown on consultants ought to be able to show why we need such consultants. Those receiving off-the-scales salaries ought to be able to justify such payment. More generally, those engaged in areas of university life that they (wrongly) see as being "just like a business" ought to be judged by the criteria used in business. But that's far different from asking all front-line academic staff to justify their entire disciplines or their salaries. There may be a few rare exceptions to this (newfangled disciplines that were brought in for political reasons, for example and that are no longer pertinent) but there can't be many.
Another way of saying this: philosophy, literature, history and classics (to take only a few examples) may not be "cost-effective" in crude terms. They may not lead to the sort of "impact" that ministers and would-be ministers think is important. They may never establish any theses at all incontrovertibly. They probably don't generate enough money (in the funny-money, cook-the-books sort of way these things are calculated internally these days). They are nevertheless core disciplines in any university worthy of the name with long histories the importance of which dwarfs the on-the-spot opinions of a bunch of febrile arrivistes who know nothing of that history and think they've made important discoveries when all their doing is parroting the latest groupthink (e.g. Eureka! Spread the news! we've discovered Innovation as a third strand alongside teaching and research!).
Let's say I agree with you entirely on the issue that humanities, philosophy and related disciplines should not receive their funding on the basis of economic impact (this is an overstatement but a working assumption!). Let's say you then have some humanities departments that are using the budgets allocated to them wisely by employing staff that are good researchers, good teachers and are promoting humanities in public forums. And then you have some where the standard of teaching is poor, where little research is conducted and the academics contribute very little by way of public speaking, promotion of their discipline and so on.
If we restrict the discussion to this aspect (thus fully taking on board your point about economic impact being a poor measure for these disciplines) is it not fair to ask whether there is a metric that would reward departments that were benefiting humanities at the expense of departments where money was simply being squandered? Why assume that simple bibiometric exercises are the only way to rate departments. Perhaps the reason universities resort to these is that academics haven't proposed an alternative way other than "trust us we're professionals". Personally, I have worked around professionals all my adult life who work up to 80 hours a week driving their research and teaching agenda often out of deep personal conviction of the value of their work. I share at least some of your views that some of the best things that happen in academia are when intrinsically driven people are given full support and bring others with them.
"And it is interesting to see the charge for accountability being led by economists who otherwise trumpet the infallibility of the market..."
Jaysus Anonymous - I hope, given that the subject matter of this blog is behavioural economics, that you weren't aiming that at us!
Post a Comment