Do universities teach students to think critically?

OECD researchers offer evidence that students aren’t getting ‘generic skills’ needed for world of work – with potentially big implications

September 6, 2022
Tough Guy 2004 adventure event at Mr Mouse Farm for Unfortunates in Perton, South Staffordshire
Source: Getty

Professional services giant PwC’s recent announcement that new recruits will no longer require at least a 2:1 degree was seen by many as the latest sign that some of the world’s largest employers are losing faith that a good university qualification guarantees a candidate of a certain quality.

The firm is by no means the first to look for new ways of determining the talent and potential of recent graduates as employers become increasingly vocal about the supposed failures of even the top universities to ensure that those entering the workforce have obtained the status of being “job-ready”.

In response, governments and policymakers around the world have emphasised the need for more practical, vocational degree courses that are closely tied to real-world experiences. But a new publication from the Organisation for Economic Cooperation and Development (OECD) argues that it is in the teaching of more generic critical thinking skills where universities can make the most difference.

“There’s a discrepancy in that people are qualified – they have the stamp from universities that says they can do certain occupations – but then employers find they don’t have the skills needed for the workplace,” said Dirk Van Damme, who co-edited the new book and recently retired as the OECD’s head of innovation.

“The assessment done by universities doesn’t guarantee that candidates have the problem-solving skills that employers think are important, and so they have to find ways to test this themselves.”

The notion that institutions are lacking in this regard has long been suspected, and the researchers behind the study think they may finally have come up with a way to prove it.

If all this sounds familiar, it is because it is. Many of those involved in the research also worked on the OECD’s aborted Assessment of Higher Education Learning Outcomes (Ahelo) project, which sought to establish a global system for assessing students’ skills at the end of their degrees.

Billed as a university-level equivalent of the highly influential Programme for International Student Assessment (Pisa) tests for school pupils, the scheme faced stiff opposition from elite institutions – some of which were arguably motivated by fears for their positions at the top of the hierarchy if teaching outcomes were to become better known.

More fundamental questions were also raised about whether such skills could accurately be assessed across institutions and borders, and the project fell apart in 2015.

A handful of countries remained committed to the idea, however, and have been testing students’ critical thinking skills ever since using the Collegiate Learning Assessment (CLA+), developed by the Council for Aid to Education (CAE), a US-based non-profit. The assessment includes a performance task and set of questions designed to test a student’s cognitive thinking, rather than their ability to recall knowledge.

“There’s no way that any one specific assessment can measure all of critical thinking,” acknowledged Doris Zahner, CAE’s chief academic officer and the co-editor of the new book.

“What we do really well is measure a specific, well-defined component of critical thinking: namely, analytical reasoning and evaluation and problem-solving,” she said.

“That includes data literacy, understanding quantitative information, being able to gather information from various sources and then making a decision based on this and crafting an answer that supports your argument and refutes the opposite – that’s what the assessment does.”

The results of the tests, published by the OECD on 30 August in the book Does Higher Education Teach Students to Think Critically?, are stark: on average, only 45 per cent of tested university students were proficient in critical thinking, while one in five demonstrated only “emerging” talent in this area.

What’s more, the “learning gain” of students between the start and the end of their courses was found to be small on average, while there were big discrepancies between courses, with those studying fields closely aligned to real-world occupations – such as business, agriculture and health – scoring the worst.

For Dr Van Damme, the results reflect a move away from the teaching of critical thinking in higher education, with less emphasis being placed on engaging with content and with some sectors abandoning exercises such as essay writing.

“Critical thinking is a skill that I think [many people] just assume is taught,” Dr Zahner said. But she pointed out that it has never been reported in university transcripts, so there has traditionally been no way of knowing if a student has developed these skills. “Universities, at least the ones that we have talked to, have said ‘It is not our job; they should have learned these things in high school’…everyone feels like it is somebody else’s responsibility to teach these things,” she said.

The authors recognise the limitations of the research, particularly the self-selecting sample of students, confined mostly to campuses in the US, with only a fraction coming from the other five countries taking part – Chile, Finland, Italy, Mexico and the UK – meaning that data for these countries could not be said to be representative.

But the authors believe they have demonstrated that “an international, cross-cultural, comparative assessment of generic learning outcomes of higher education is feasible”.

While the OECD does not yet seem to have mustered the will for another go at instigating an Ahelo-type project, the study’s repercussions could be major.

“What I personally believe this will do is lay the foundations for placing greater weight on the quality of teaching in higher education,” Andreas Schleicher, the OECD’s director for education and skills, told a launch event for the book in Hamburg.

He said that employers had “seen through” the degree system and that students were becoming more discerning consumers because they were having to shoulder more of the cost of their education.

Therefore, he continued, it was getting harder to “hide poor teaching behind great research”, while demand for skills that were easiest to test and teach – such as memorising and regurgitating knowledge – were exactly the areas that were losing value most quickly.

“Teaching excellence needs to obtain the same status – the same recognition – as academic research, which is still the dominant metric for valuing academic institutions, whether you look at rankings, research assessment frameworks or performance-based funding,” Mr Schleicher said.

To critics, all this sounds suspiciously like groundwork for the creation of a new ranking, something that was never an aim of Ahelo even though many thought its data were likely to eventually feed into institutions’ scores in global league tables.

Dr Van Damme said that while many criticisms of rankings were justified, there should be a recognition that they are not going to go away and, therefore, it would be better to find ways to ensure that they accurately reflect the quality of teaching – something that could change the complexion of league tables completely.

“In an ideal world, where you have as much transparency for teaching and learning as you have for research, there would be a profound impact not only on rankings but the hierarchy and landscape of the system,” he said.

“It is certainly not the case that universities that are excellent in research are also automatically excellent in teaching and learning; and if you placed greater weight on teaching, you would get different results [in rankings].”

As well as an upheaval in institutional reputation, greater focus on the teaching of critical thinking could fundamentally alter the types of courses that are seen as necessary for societies and economies to thrive, according to Dr Van Damme.

Politically influenced drives towards utilitarian approaches to education that produce students who are immediately employable in a certain occupation – which tend to favour the STEM subjects – neglect to consider volatility in the labour market and the need to train young people for their entire lifetime, he said.

“The economy and labour market are in transformation because of digitalisation, and so the job reality in 10 years’ time will be completely different from today. There should be more interest in teaching the generic skills that matter in the long term,” he added.

In this world, it is the much-maligned humanities that truly come into their own, and the CLA+ results showed that those students pursuing these fields displayed much higher levels of critical thinking, according to Dr Van Damme.

He said studies have demonstrated that while vocational training produces better employability results in the short term, these wane after five years and “those with better generic skills have much better employability and earning prospects over a lifetime”.

Dr Zahner said universities would likely come under increased pressure from industry and governments to address these issues, whether they like it or not.

“Hopefully the universities will hear this messaging. It’s great if you can graduate your students, but it is not so great if you graduate all these students and they don’t have success in their careers. We’re hoping being able to increase critical thinking skills will be able to close that gap.”

tom.williams@timeshighereducation.com

POSTSCRIPT:

Print headline: Universities aren’t instilling critical thinking, finds OECD

Register to continue

Why register?

  • Registration is free and only takes a moment
  • Once registered, you can read 3 articles a month
  • Sign up for our newsletter
Register
Please Login or Register to read this article.

Related articles

Reader's comments (14)

Academically Adrift all over again? https://www.insidehighered.com/news/2013/05/20/studies-challenge-findings-academically-adrift (but doesn't really undermine those findings at all.)
In ancient Greece, the great philosophers taught their students how to think first before teaching them any knowledge. Not sure since when, the modern world education, from kindergarten to university, seems to have gone the exact opposite. Being content-focus is a bigger issue beyond higher education. Indeed, the critical-thinking focus International Baccalaureate (IB) curriculum gives a breathe of fresh air to the rather stale knowledge-stuffed outdated A Level. However, the uptake of IB in secondary school is scarce in the UK. Even if students have gone through the IB education, the UK higher education might fail to continue to nurture these youngsters' critical thinking ability and creativity. In all, if we really want to get students career-ready, a reform to the entire education system is needed at all levels, and not just in higher education.
It is depressing to see education again reduced to preparation for work. Training should be the responsibility of employers but they will naturally be happy if the taxpayer picks up the tab for it. I am not sure that employers really want critical thinking since they will then get plenty of disgruntled employees frustrated by hierarchies, outraged by inefficiencies and hemmed in by conformist cultures.
Well what did they expect given the current level of grade inflation for both A level and HE students ? Teaching can only be as good as the student's ability to absorb it. Critical thinking tells me the problem needs to be addressed from both ends.
Education - at any level but particular at University - ought to be about equipping students with open and enquiring minds. If they know how to learn and can think critically, they can acquire knowledge about any subject when the need arises. Drawback is, it is not so easy to figure out how to actually teach these things, or assess them once taught. Curiosity seems innate in some people and lacking in others, but if you are not curious it can be hard to learn especially independently. As a computer science academic, we have the scope to teach problem-solving, that's really what computing is about. As for formulation of arguments to support the solution you've come up with, I try to teach it in the 'Ethics for Computing' class I teach... a class some students adore and others find dull & tedious. My first degree however is in botany. Did they teach me to have an open and enquiring mind, that was able to manage a change in discipline to the level where I am now? Or was this more innate... I am insanely curious & always ready to run and find out by nature? How can I inculcate this into my students?
This is where work experience and year placements come into play. We cannot necessarily teach students everything needed to be "work ready" (a phrase that I really dislike). Whilst in the workplace students develop skills and confidence that they would otherwise be lacking.
Since professors are regularly canned -- or face a deluge of death threats and hate-mail -- in the US for daring to teach critical thinking skills, it is hardly surprising that they largely avoid threatening the ideological bubbles of their students and their parents. These threats and attacks are overwhelmingly generated from right wing ideologues and their cult followers, while the media smiles and winks, and play "both sides" games. So until and unless the Corporate University system actually takes steps to protect those who teach critical thinking skills, you don't get to complain about their absence.
Universities have both Left and Right wing forces that are hostile to free speech. Especially on the Left there are many 'Studies' fields where the existing orthodoxy mustn't be challenged and students have to protected from opposing views via 'safe spaces'. Not to say that the Right are not problematic too, but it is not mainly them as you suggest.
Could it be argued that if you haven't "learned to think" by the time you get to 1st year of University, it is too late? And that the job of Uni is to specialise?
An interesting contribution to both, What are Universities for? and How well are Universities performing? I wonder if "critical thinking" is something that can be taught or is it more like a "gift of nature" similar to musical ability, athletic ability and an innate curiosity? Perhaps we will never know. For me it indicates the need to have an open mind and, in terms of problem solving, the need for diversity in the membership of teams created to find the best solutions to challenging and important questions like Climate Change.
The first sentence of this article says: "Professional services giant PwC’s recent announcement that new recruits will no longer require at least a 2:1 degree was seen by many as the latest sign that some of the world’s largest employers are losing faith that a good university qualification guarantees a candidate of a certain quality." Does PwC really think they'll find better candidates among people with lesser degrees? I have my doubts about critical thinking in its HR department.... I agree with comment #1 that the new OECD report is in essence a rehash of Arum and Roksa's 2011 Academically Adrift. The media liked back then, and still likes, the 45% figure (BTW, there were some statistical problems with it; see https://www.davidmlane.com/hyperstat/essays/academically_adrift.html). I think even more important is that all but the premier education media overlooked a second conclusion from Arum and Roksa's study: that students of liberal arts programs did improve significantly on the CLA-test whenever their schools featured a large number of courses that required more than 50 pages of significant text to read per week and at least 20 pages of writing in the course of a term. I think most liberal-arts professors know this intuitively, but apparently administrators do not even when it is spelled out in a report. So we know what the solution to the problem is, but there has been no rush to implement it.... Instead, we get STEM, STEMM, STEMM+, etc.
It is frustrating that we're in this position when the value of critical thinking in education and employment has been established for many years. It's also frustrating that there is considerable evidence on how the skills of critical thinking should be taught (including the value of discrete courses in the subject). It can be taught from kindergarten upwards, and can (indeed, should) be included explicitly within every subject taught in schools (and beyond in higher education). It should be no surprise that a country like Singapore (which heads the PISA tables) has, for some years, required that critical thinking is developed throughout its education system.
Critical thinking doesn't exist in a vacuum and is easiest to teach in the context of the particular material in a given course. I wrote and taught a course in Scientific Methods many years ago and found that the less I assumed about the students abilities at the start, the more effective I was in teaching the fundamentals of the scientific method. It isn't easy for young academics to make their thinking explicit to the students. Revealing how exactly we think about a problem can be intimidating for people who still lack confidence in their own abilities. Another major problem is the reticence that we have in questioning the reasoning of our students and encouraging them to question us on our assumptions. The current climate in many institutions views any sort of intellectual challenge as threatening to the mental wellbeing of our charges. I think the problem in some sectors is that "critical theory" has replaced critical thinking in a way that Michel Foucault or Jacques Derrida never intended.
I agreed with one or more readers, we need to go basic. Don't teach them but let them learn by making mistakes. Mistakes are experiences that useful knowledge accumulated in process of learning. As for lecturer or educator, don't feed them useless knowledge. A knowledge become useful when they put them into practice. So, educator, give them real life problem and ask them search the knowledge and apply it. The educator, in this case, becomes facilitator or enabler.

Sponsored