Rankings not so rated

Robert Johnson Jr proposes a consumer guide to university rankings and suggests that the Times Higher Education-QS World University Rankings would deserve to be highly rated in any such consumer guide ("We know who's top - but now we must rate the rankings", 2 October).

We heartily agree that the proliferation of international rankings, across many fields of public services and not just higher education, calls for a credible Kitemarking system for consumers.

In fact, we have developed and applied a method for doing precisely that and it is published in the current issue of the International Public Management Journal. In the article, we rate international rankings on six criteria of validity and reliability.

However, from what we have been able to learn about it, we do not think the Times Higher Education-QS World University Rankings would achieve a top validity score on our index because the logic behind the selection of indicators appears obscure and the placement of cases highly dependent on small changes in weightings. In addition, we cannot find any consideration of confidence intervals or measurement error. But that is all the more reason for proper Kitemarking of such rankings.

Christopher Hood and Ruth Dixon, University of Oxford.