Why university rankings are pointless exercises for S’pore students

A few weeks ago, I received an e-mail asking me to participate in an academic reputation survey for an organisation producing international university ranking tables. Soon after, The Straits Times reported that the National University of Singapore (NUS) and the Nanyang Technological University (NTU) had been rated the top universities in Asia based on the QS or Quacquarelli Symonds rankings by subject areas involving 1,594 universities in 93 countries and territories. A similar report in 2022 recapped how both also ranked well in the QS World University Rankings.

There are several other international ranking tables – like those published by Times Higher Education, Shanghai Jiao Tong University and US News & World Report. A key claim made by these organisations producing such tables is that they serve as guides to help students select universities and courses. 

Many universities around the world await the publication of each round of ranking tables anxiously. They know the results come with high stakes, and may serve to boost a university’s international standing and inform its strategic goals. 

Prospective employers may watch these rankings closely as they make hiring or deployment decisions. They may also shape some prospective students’ decisions over what and where to study. 

These choices are increasingly vital as universities compete for students in the lucrative international student market. Market intelligence firm HolonIQ estimates that six million to nine million international students will enrol in foreign higher education institutions, with spending amounting to US$433 billion (S$575 billion) by 2030. 

Move to jettison international university rankings

Yet in more recent years, a number of prestigious US university law and medical schools – like those at Harvard (both of which ranked among the top 10 in their respective fields) – as well as a few undergraduate schools have withdrawn from the US News & World Report ranking tables.

The ranking system simply ran against key institutional values, including a commitment to equity, diversity and inclusion, they argued. 

Professor Stephen Joel Trachtenberg, former president of George Washington University, was also cited in a 2019 CNN report as saying that “schools feel pressure to game the rankings”.

How international university rankings are conducted

The reality is that such rankings are rarely scientific studies drawing clear conclusions. The trouble lies with the computation of scores.

Most focus on broadly similar performance indicators, albeit with differing weightages being accorded to the various indicators. 

The QS 2023 rankings used indicators relating to academic reputation (teaching and research quality), employer reputation (how well universities prepare students for successful careers, and which universities provide competent, innovative and effective graduates), faculty/student ratio, academic citations per faculty member, international student ratio and international faculty ratio. 

Part of this rankings data is also obtained through potentially subjective surveys based on sentiments or incomplete knowledge. The QS academic reputation survey is sent to thousands of academics around the world each year. Respondents are asked which country/territory and geographical region they are most familiar with. 

They are then asked to provide, among other things, their individual nominations of top domestic and international institutions they think produce top research in their faculty area.

Brewing criticism over university rankings

Such international ranking tables have attracted a great deal of criticism from academic researchers and universities in recent years. 

In 2020, Malaysian academics Muhammad Ashraf Fauzi, Christine Tan, Mahyuddin Daud and Muhammad Mukhtar Noor Awalludin demonstrated how the methodologies employed in ranking exercises can be prone to flaws, in a journal article in Issues In Educational Research. 

Historically prestigious universities such as Cambridge and Oxford tend to have higher ratings. Western universities operating in English tend to receive higher rankings than their non-Western counterparts – even domestically prestigious institutions such as the National Taiwan University (ranked 77 in the latest QS tables) – reflecting the possibility that survey respondents tend to think of the more internationally prestigious universities that teach in English as superior. 

A second problem arises because these tables do not reflect the varied purposes of higher education. Differences in each university’s key objectives in research, teaching and community service, and the great diversity of programmes offered in universities worldwide make it a pointless exercise to compare apples with oranges, durians and tomatoes. 

A more poorly ranked university – such as Universiti Malaya – may excel in teaching or in other qualities contributing to nation-building compared with universities with higher rankings. Yet, such rankings compare different types of universities as though they are identical and accord equal weightages to all of their functions and educational offerings.

Even if we assume they are directly comparable, making the composite score meaningful, the nature of ranking tables magnifies minuscule differences in scores and separate institutions with similar scores by many numerical positions in the tables.

A third issue is that the tables fail to adequately indicate the processes and outcomes related to quality teaching and learning, and that students’ opinions are hardly, if ever, solicited in this regard. 

A fourth major criticism is the heavy weightage accorded to research and institutional reputation rather than focusing on teaching quality. Academic Woo Jun Jie argued in a 2018 CNA commentary that this skewed emphasis on research may come at the cost of teaching quality as universities seeking to ascend the ranking tables are incentivised to devote extensive resources towards boosting research outcomes at the expense of adult learning.

Some perspective on university rankings

Such rankings are rarely helpful for prospective students. The process of selecting a university, course of study and concentration, whether local or overseas, depends on a complex mix of factors.

Besides examination scores, personal interests and passions, parental aspirations and the amount of comprehensive information about the range of available options both within and outside of Singapore, other factors may include financial circumstances, family circumstances and peer influence. Some students may also consider fields that they believe are emerging areas of demand in the job market. 

Much also depends on a prospective student’s life stage and purpose in pursuing higher education. Individuals with full-time jobs deciding whether to embark on a degree programme might likely give more consideration to the pros and cons of a full-time versus a part-time degree, in view of the opportunity cost involved in forgoing their income, compared with students fresh out of junior college or polytechnic.

Adult students must think more carefully about the implications for their families and financial commitments in giving up a job for a full-time degree programme.

And what are students to think of those among the six publicly funded autonomous universities, as well as the programmes in the autonomous universities, that have not been highly ranked, vis-a-vis those that have? It would be patently ridiculous to assume that those in the former category are somehow inferior and not worth serious consideration on the basis of this one point.

In their 2017 CNA commentary, academics Pang Eng Fong and Linda Lim highlghted the lack of empirical studies linking university rankings with student outcomes such as post-graduation job placement or salary rates. They also noted that international students did not appear to prefer the National University of Singapore (NUS) and the Nanyang Technological University (NTU) to the Singapore Management University (SMU) and other unranked universities, choosing their universities instead for a range of other reasons including costs and scholarship availability.

Singapore’s varied university landscape

University rankings are also less useful in aiding Singapore students to navigate the diverse local university landscape. 

The Ministry of Education (MOE) website indicates that these six universities can be grouped into two categories: more academically focused, research-intensive universities (the NUS, the NTU, the SMU and the Singapore University of Technology and Design, with the first two being comprehensive and the next two being specialised) and those that provide applied-degree pathways, with more hands-on experience and industry exposure (the Singapore Institute of Technology and the Singapore University of Social Sciences). 

In addition, there are also a number of programmes by private educational institutes such as the Singapore Institute of Management and Curtin University.

The diversity of providers within Singapore in terms of their institutional missions and programme offerings cannot possibly be captured in the ranking tables.

Thankfully, the QS website acknowledges the rankings are merely a starting point that cannot replace individual decision-making based on further research and information-gathering through exploring university websites, speaking to alumni and attending open-day events. Students should do their own homework rather than rely on ranking tables to light the way for them.

Similarly, employers today consider a range of other factors besides these institutional rankings when assessing the merits of job applicants and current employees. A range of non-academic skills and attributes – such as leadership qualities and communication abilities – not covered by the ranking tables matter. 

Growing emphasis on upskilling and mental health

These ranking leagues are even less useful for local autonomous universities, as the educational landscape evolves to embrace larger goals of creating more pathways to success and empowering individuals to reach their fuller potential through upskilling.

A prominent theme emerging in recent years in Singapore discourse is that of broadening the definition of success, and moving away from a meritocracy of grades towards a meritocracy of skills. 

There is an official drive to encourage multiple entry points to study for Singaporeans from varying ages and backgrounds, while making sure that a university education remains accessible and affordable. MOE has encouraged universities to provide students with options to customise degree programmes and expand the range of modules for adult learners. Another theme is the need for individuals to take charge of their learning.

In addition to the need to promote an understanding of, and interaction with, the world beyond Singapore, other issues include the encouragement of interdisciplinary learning and the need for universities to nurture graduates who define success in terms of their contributions to the wider society. 

For this to work, cooperation rather than competition among the local universities is needed.

Speaking at the Straits Times Education Forum in February 2022, Education Minister Chan Chun Sing urged the six autonomous universities to operate as a single team, collaborating with one another and building on one another’s strengths. His remarks contrast with the tendency for the ranking tables to engender a zero-sum game mentality, where one university’s rise is accomplished only at another’s fall.

On another front, the recent Covid-19 pandemic has brought into focus the importance of addressing issues related to students’ mental well-being. The pandemic has also left in its wake questions about the adequacy of home-based learning as an alternative to face-to-face instruction, thus highlighting the importance of examining the merits of diverse instructional modes.

The autonomous universities have to, therefore, move beyond the limits imposed by international ranking tables, pursue their distinctive educational missions and find fields where cooperation with the rest is of mutual interest. 

While the results of ranking exercises may prove helpful in the short term in terms of boosting institutional prestige, attracting research funding and attracting prospective students, it makes more sense instead to set independent priorities in the light of more pressing national priorities.

A Singapore ranking table?

Some have asked whether MOE should devise its own ranking tables for the autonomous universities based on criteria more attuned to the Singapore context. 

An immediate question would be the extent to which these desired outcomes can be quantified, when such ranking tables have proven poor in lending themselves well to quantification and measurement.  

A related question concerns the desirability of subjecting the universities to yet more ranking, a practice likely to leave universities, students and employers grappling with yet more issues related to the adequacy of the ranking tables, at a time when secondary school and junior college rankings have been abolished.

In the final analysis, it is probably time for all of us to break the habit of reading more into the international ranking tables than they can tell us about what really matters in higher education. 

This includes higher education’s primary functions at both the individual and collective levels: as a means of personal growth and development, a vehicle of social mobility and national prosperity, and a driver of service for the wider public good.

• Jason Tan is Associate Professor in Policy, Curriculum and Leadership at the National Institute of Education.