The effect of methodological variations on university rankings and associated decision-making and policy

This paper explores the effect of methodological choice on global Higher Education Institutions’ (HEI) rankings, and the effect of choices regarding League Table and Ranking Systems (LTRSs) upon decision-making and policy. LTRSs are identified as heuristics to aid decision-making about HEIs. The paper examines the methodological choices commonly made within ‘rankology’. It describes the methodologies used in six popular LTRSs and the manner in which the ranking formulae were constructed. An analysis of the ranking of global HEIs by the six LTRSs for 2015 revealed that whilst there was a degree of agreement across these LTRSs regarding the ranking of two ‘elite’ institutions, the level of agreement across the LTRSs rapidly broke down as one progressed to consider institutions further down the ranking order. The study shows that the use of LTRSs to aid decisions is highly sensitive to one’s choice of LTRS. © 2017, © 2017 Society for Research into Higher Education.