Research impact is more than a multimillion-dollar question

The recent announcement by Australia’s education minister, Simon Birmingham, that universities will be assessed on their research impact and engagement has long been telegraphed. The Australian Research Council (ARC), which will run the exercise, beginning in 2018, had already conducted a pilot and it was really a question of when, rather than if, this would be mainstreamed.

The Engagement and Impact Assessment (EIA) is a companion piece to the Excellence in Research for Australia assessment: the Antipodean version of the UK’s research excellence framework, which also assesses impact. It will allow Australia, for the first time, to demonstrate impact underpinned by excellence in research. That is not necessarily a bad thing; Deloitte has already calculated that 10 per cent of Australian gross domestic product is derived from university research – although, as in the UK, the ARC has strived to make clear that impact can be environmental, social or cultural, as well as economic. Yet, unlike with the REF, there will be no explicit link between EIA results and institutional funding. Nor is there any indication of how this exercise will make research better or more impactful.

National success rates in ARC competitive funding rounds are already below 20 per cent, entailing a lot of wasted effort by applicants. Measuring the impact of this research post hoc will only add to the burden on the sector, requiring the generation of REF-style case studies and narrative statements – again, unfunded and without explicit return.

However, in recent interviews the minister has asserted that taxpayers want to see more from their investment. And, coincident with the announcement of the EIA, a number of somewhat pejorative stories ran prominently in the press, citing details of carefully selected ARC-supported projects where no direct economic return on investment was evidenced, and questioning the value to the Australian taxpayer of such work. All this amounts to a not-so-subtle shot across universities’ bows regarding return on investment – in spite of the wider definition of impact announced.

This attitude is not unique to Australia, of course. The swing of the tolerance pendulum away from supporting knowledge creation for its own sake has happened in many jurisdictions over the past decade and it is easy to understand why. But when I was chair of the Irish Research Council, I would at this point in the debate segue into a discussion of Hamiltonian quaternions and the 100-year lag between that aspect of mathematical theory being tackled and its impact on the Apollo moon landings.

The risk that I perceive most often relates to the impact of measuring impact: a near-immediate pivoting of sectoral focus to a short-termism that inevitably results in an evolution of funding schemes and assessments. If it goes too far, this can squeeze out the bluer-skyed research. There is that wonderful quip often attributed to Einstein – that it would not be called research if we knew what we were doing.

My point is that we should not dress up “development” as “research” in a truly comprehensive national research and development system. The move to measure “impact” in Australia, as it has been defined, does not overtly seek to do this – but the possible evolution of the funding landscape, telegraphed through soundbites about return on investment, just might.

My institution, the University of South Australia, has had engagement with end users (and impact) at the heart of its research strategy since its foundation, so I am confident that we will perform very strongly against the new metrics. But I also have fundamental concerns around the comparability of data and statements around impact and engagement from diverse research fields – and even from within related fields. Does an engineering project that generates direct end-user value to industry partners of A$1 billion (£577 million) in additional profitability over five years have more “impact” than a study that transforms science education in the classroom and increases the number of female engineering students over a decade?

In a utopian assessment exercise, cultural, social, environmental and economic impacts would all be created equal. In practice, some are already positioned in the media as more equal than others. Dollars, after all, are easy to understand.

David Lloyd is the vice-chancellor of the University of South Australia.