‘World standard’ review for Australian research assessment

Experts will be enlisted to recalibrate Australia’s research assessment exercise, and Indigenous studies will be evaluated as a stand-alone field, following a review of Excellence in Research for Australia (ERA) and the Engagement and Impact (EI) assessment.

The Australian Research Council (ARC) has vowed to implement all 22 recommendations of the review, including changes to boost the meaningfulness of ERA’s “world standard” rating – a benchmark met or exceeded by 90 per cent of Australian research in the most recent assessment in 2018.

The reviewers’ report cites Organisation for Economic Cooperation and Development data and Times Higher Education World University Rankings results as evidence of Australian research’s rising quality since ERA was introduced in 2010. But the improvement has occurred alongside a “rapid increase in the number of publications generated worldwide”, the report concedes.

“Many of these publications…attract few citations, resulting in a widening gap between Australian performance and the world standard. There is a risk that the ERA rating scale will become less capable of differentiating between high performers, as university ratings cluster at the top of the scale.”

The report adds that ERA’s world standard benchmark is “not well-defined” and “may not be applied consistently”. It urges the ARC to convene an expert group to undertake the “technical work” needed to “reach a robust and equitable solution”.

The group will revise the ERA rating scale, the benchmarks for citation and peer review assessment, the definition and appropriateness of “world standard” and guidance material for ERA assessors. The new rating scale and benchmarking information will be published alongside ERA submission guidelines in the first half of 2022, according to an “action plan” outlining how the ARC intends to implement the recommendations.

Former University of Melbourne deputy vice-chancellor Frank Larkins said the review had acknowledged the “incremental creep” in Australia’s performance. “There’s a suspicion that the improvement has been largely because of the movement downwards in world standards,” he said.

“That was a central criticism, because some of the outcomes in the last round were a bit incredulous.”

Eight recommendations are aimed at “streamlining” the assessments through “smarter use” of data and technology. Measures include adopting a “collect once” policy across several data compilations, and harvesting existing data to “partially pre-fill” universities’ ERA submissions.

The review was conducted between March 2020 and April this year, as Covid ravaged the sector – a point not lost on advisory committee chairman Michael Brooks. “Constrained budgets and reduced international enrolments have affected university resources and research workforces,” he notes in the report’s foreword. “It is more important than ever to minimise the reporting burden for universities.”

ARC chief executive Sue Thomas said the review would spawn “major improvements to both assessments for universities, government and research end users”.

Indigenous studies will be assessed as a distinct discipline in the next ERA and EI assessment rounds in 2023 and 2024, in a change made possible by revisions to the Australian and New Zealand Standard Research Classification. “Indigenous research [is] a pillar of Australian research which, until now, has not been possible to assess as a discipline in its own right,” the report says.

“This would improve national recognition and visibility of Indigenous research and ensure that ERA and EI accurately reflect the Australian research landscape.”

john.ross@timeshighereducation.com