European bid to compare teaching standards moves forward

A European Union-backed project to compare student skills across borders has unveiled what it says are the “most sophisticated” guides ever published to what students should learn during their degrees.

The Measuring and Comparing Achievements of Learning Outcomes in Higher Education in Europe initiative, known as Calohee, will now develop tests designed to measure how undergraduates in different universities and countries perform against each other – potentially revealing whether prestigious institutions live up to their reputations.

But the project will have to overcome university opposition and student apathy that has felled several other attempts to apply standardised tests across different institutions and national systems.

Calohee’s new guides, released at the end of October, have been created by teams of European academics and students and cover five subjects: history, nursing, teacher education, civil engineering and physics.

They stress that as well as mastering subject knowledge and academic skills, students need to be prepared for employment and a social, civic and cultural role.

History students, for example, should be able to debate contemporary issues “using sound arguments based on historical evidence”, the new guidelines say.

Robert Wagenaar, the project coordinator, said that Calohee recognises that higher education is about “much more” than just knowledge and skills. “This has not been done before” in similar projects, he added.

The next step for Calohee is to use these guidelines to create test banks of potential exams. Essay questions, for example, could judge whether students are prepared for civic life after graduation, he said. Qualities such as leadership or cooperation could be assessed by presenting students with a tricky situation – a doctor-patient encounter, for example – and asking them what they would do. 

When Calohee is complete, test results for individual universities and entire countries could be made public. “We can actually compare systems,” said Dr Wagenaar, director of the International Tuning Academy at the University of Groningen.

Yet Calohee must convince universities – especially prestigious ones, which could have their reputations punctured by unexpectedly poor results – to take part and allow the results to be made available. The Organisation for Economic Cooperation and Development blamed resistance from “elite” institutions for the failure of its own learning gain project, the Assessment of Higher Education Learning Outcomes (Ahelo), in 2015, which the UK refused to sign up to. However, Calohee differs from other learning gain projects because it will not test a single student cohort at different points during a degree – only measuring performance at the end of bachelor’s and master’s levels. 

Hamish Coates, a professor at Tsinghua University’s Institute of Education, who led a feasibility study on Ahelo, said that “the sector’s usual change-resistant barricades will quickly rise”, which was a “key risk” for Calohee.

Another hurdle is getting enough students to take part, which has scuppered previous learning gain studies in England and Brazil. To combat student test fatigue, Calohee could split up day-long tests between different groups of students, meaning that they would only have to give up a couple of hours of their time, said Dr Wagenaar. 

“We realise that it’s very ambitious, but we are looking for better evidence,” he said, and argued that universities “want to know whether [they] are doing a good job”. About 70 universities have been involved in the project so far, which has been a “bottom-up” effort, he said.

But it is still likely to be about four years until tests are conducted, he cautioned. 

david.matthews@timeshighereducation.com