Proposed metrics for knowledge exchange framework announced

England’s forthcoming knowledge exchange framework will include metrics on graduate start-up rates and the amount of time that academic staff commit to public and community engagement but will not rank universities based on a single score, Research England has proposed.

In its consultation on the KEF, published on 9 January, Research England says that universities should be measured on their performance across seven equally weighted broad areas or “perspectives” and each area will include one to three metrics. The metrics under each perspective would also be equally weighted and composed of the average of the most recent three years of data.

The perspectives are: research partnerships; working with business; working with the public and third sector; skills, enterprise and entrepreneurship; local growth and regeneration; intellectual property and commercialisation; and public and community engagement.

Proposed metrics include co-authorship with non-academic partners as a proportion of total outputs, consultancy income with businesses per academic, contract research income with the public and third sector per academic, graduate start-up rate per student, and time per academic staff committed to public and community engagement.

The consultation document confirms that universities would be banded into clusters and that each institution would be assessed relative to the average of its peers within its group.

It proposes that institutions should be assigned a decile rank for each area of performance. This means, for instance, that the top 10 per cent of institutions in each cluster would be assigned a decile rank of 10 and the bottom 10 per cent of institutions would be assigned a decile rank of one.

However, the ranks for each set of metrics would not be aggregated to provide a single score. Instead the performance of each institution would be “expressed in a radar chart with a scale in deciles, relative to the average decile rank of the peer group”. It would be presented through a set of interactive, online dashboards.

Meanwhile, the decile ranks on public and community engagement and local growth and regeneration would be supplemented with narrative statements from institutions.

Research England plans to run a pilot KEF with 12 to 16 institutions between February and April 2019. Meanwhile, the consultation will close on 14 March.

Once the first full KEF is conducted in 2019-20, the results are expected to be used to inform the allocation of sources of support such as the Higher Education Innovation Fund.

Richard Jones, professor of physics at the University of Sheffield and chair of the KEF technical advisory group, said that he hoped that as the KEF evolved it would “recognise the wide diversity of types of knowledge exchange that universities do, using robust metrics where they already exist and are appropriate, and stimulating the development of better metrics where necessary”.

ellie.bothwell@timeshighereducation.com

Proposed knowledge exchange framework metrics

Research partnerships

• Contribution to collaborative research (cash and in-kind) as proportion of public funding (HE-BCI table 1a)
• Co-authorship with non-academic partners as a proportion of total outputs (data provider TBD)

Working with business

• Innovate UK income (KTP and grant) as proportion of research income (Innovate UK)
• Contract research income with businesses per academic FTE (HE-BCI table 1b)
• Consultancy income with businesses per academic FTE (HE-BCI table 2)

Working with the public and third sector

• HE-BCI contract research income with the public and third sector per academic FTE (HEBCI table 1b)
• HE-BCI consultancy income with the public and third sector per academic FTE (HE-BCI table 2)

Skills, enterprise and entrepreneurship

• HE-BCI CPD/CE income per academic FTE (HE-BCI table 2)
• HE-BCI CPD/CE learner days delivered per academic FTE (HE-BCI table 2)
• HE-BCI graduate start-ups rate by student FTE (HE-BCI table 4)

Local growth and regeneration

• Regeneration and development income from all sources per academic FTE (HE-BCI table 3)
• Additional narrative/contextual information

IP and commercialisation

• Research resource (income) per spin-out (HEBCI table 4)
• Average external investment per formal spinout (HE-BCI table 4)
• Licensing and other IP income as proportion of research income (HE-BCI table 4)

Public and community engagement

• Time per academic staff FTE committed to public and community engagement (paid and free) across: events, performances, museums and galleries (HE-BCI table 5)
• Additional narrative/contextual information

Note: HE-BCI = Higher Education Business & Community Interaction Survey