Dos breves reflexiones de A. Usher sobre los rankings universitarios aparecidos ayer y hoy en el Blog de HESA.
Rankings Touchiness (Part 1)
The last decade or so has seen a lot of brouhaha about rankings, especially those of the global variety. Loads of books have been written about how rankings are driving consumerism in higher education (mostly an anglo-American complaint, it should be said), and how they are altering (for the worse) policy-making in the sector.
But one question which, to my knowledge, has not been addressed, is this: if rankings are so god-awful, why is higher education the only sector that screams so loudly about them?
Rankings are everywhere. In the automobile industry, JD Power and Associates uses survey data to measure glitches in new cars, and ranks different models accordingly. Law firms get ranked. So do hospitals. Countries get ranked all the time on things like competitiveness, transparency, and development. Cities get ranked in a way very similar to universities, by global media outlets like The Economist and Monocle.
Many of these rankings attract their share of criticism (Joel Kotkin’s takedown of city rankings is particularly good). But I’d venture a guess that the academic literature on the evils of ranking universities, and the methodological wickedness that goes on therein, is several times the size of the literature on all those other rankings, combined.
A reasonable question follows: why are universities so much touchier about rankings than other organizations?
One obvious possible reason is that, unlike lawyers and car manufacturers, universities are social entities with multiple objectives, which don’t have a profit motive, and are thus more difficult to rank. Fair enough, but the same could be said for cities and countries. Why is the UN Human Development Index (HDI) OK, but the Shanghai Rankings not?
The usual objection at this point turns to some form of criticism on the lack of measure for “value-added” – of course Harvard comes top of the rankings: they’re so rich! Again, a reasonable comment, but nobody handicaps the HDI by gross domestic product, and the mayor of Hanoi doesn’t whine about Munich having an unfair advantage due to being rich.
There are, as I see it, two reasons why university rankings are uniquely controversial. One is that many faculty view the university as a collection of individual departments rather than as a single entity, and they resent the fact that their own reputation is dragged down by being bracketed with the clowns in (insert department here). The second reason is that fighting over rankings is actually a proxy fight over the hierarchy of value within academia; people who care about teaching get really cheesed-off about the primacy given to scientific discovery in most rankings models.
Of course, there are solutions. But more on this, tomorrow.
Rankings Touchiness (Part 2)
As I noted yesterday, the basic fight over rankings in higher education boils down to two questions: should institutions be judged as whole entities, or on the basis of their constitutent parts? And, should rankings give primacy to the existing hierarchy of values of higher education (i.e. research and publication), or to something else?
Let’s start with the first question. There’s absolutely nothing stopping us from ranking individual bits of the university, as opposed to the entire institution. We’ve had law school and MBA rankings for almost two decades now, and that same approach could – with a bit of modification – be applied to the rest of the academy as well. In Europe, it’s a quite common approach – Germany, Switzerland, Italy, the Netherlands, and the UK either have, or have had, rankings which make comparisons at the departmental level.
In Canada, the main thing stopping this kind of approach is a lack of data; though, with the Globe and Mail, we did manage to do some of this a few years ago in the Canadian University Report, and its online version. But if universities ever wanted to provide data to move in this direction, it would be easy enough to do.
To the second question: why can’t rankings challenge the hierarchy of values in academia, rather than re-inforce them? Well, the fact is that they can. Universities can get ranked on things like their commitment to sustainability, or their online offerings. In the US, Washington Monthly magazine includes metrics of social ability and service in their analysis.
But those are all somewhat traditional in the sense that they all assume there is a single “best” institution. If we get rid of the notion that you need to aggregate and sum individual indicator scores, we can have, what some people call, “multi-dimensional” rankings. The new European “U-Multirank” systems works along exactly these lines (as did the online version of the Globe’s Canadian University Report, which we at HESA developed).
There is, in short, lots of scope to address essentially every single criticism of rankings. The issue is whether there’s a will to change. Universities tend to like “multi-dimensional” rankings (and so do I!) because they’re less judgmental and more balanced, but given that they’re somewhat less intuitive than Maclean’s-style rankings, it’s not clear whether, given a choice, students and parents actually prefer them. Nor is it clear that universities outside Europe dislike existing rankings sufficiently to do the necessary work to provide data that would make improved rankings possible.
In short, we’re at an equilibrium. Better rankings are possible, but neither consumers nor data providers seem to be using their influence to make them so. Expect stasis, and continued kvetching.