Can contributions to a positive research culture be measured?

Next month, the results of the research excellence framework will tell UK universities where they currently stand on the traditional scale of research prowess. But as they await the verdict, with both fear and excitement, there are rumblings that it may be time to review what that scale looks like.

Research England’s Future Research Assessment Programme, which will evaluate REF 2021 and look beyond it, includes the questions: “How can the assessment system contribute to a positive research culture that enables talented, diverse people and teams to thrive?” and “How can the system better recognise diversity in approaches to research, rewarding a broad range of outputs and impacts, within and across disciplines?”.

Both the UK and Scottish governments recently reviewed research in the context of sustainable Covid recovery, and they recognised the importance of a positive research culture. But what is a positive research culture? Do we have it? How do we measure it? And how do we encourage it?

For the past two years, we at the University of Stirling have held annual research culture awards in seven categories, including “outstanding research leader”, “outstanding collaborator” and “outstanding early career researcher”; the word “outstanding” replaced “best” this year because it is less competitive. Self-nomination is not allowed, but we share the texts of their nominations with nominees, and we celebrate everyone at the awards ceremony, identifying some “highly commended” nominations.

The awards give us an interesting insight into what those within the research community value. This has allowed us to identify several elements of good research culture. Not surprisingly, activities and actions that fostered inclusion, teamwork and community were particularly important this year; examples mentioned include away days, peer review groups for papers, regular inclusive meetings, early career researcher events of various types – and cake!

Attitude was also singled out. Willingness to share expertise, knowledge and networks is valued, but so are more general characteristics, such as kindness, good humour and the ability to make time to see people (despite busy diaries).

Formal and informal mentoring is universally appreciated, and even many of the early career researchers nominated were recognised for their skills in this area. Among the best mentor nominations, the ability to listen came up several times. Overall, a good local research culture is seen to derive from good role models, as well as a strong sense of community and teamwork.

Of course, the issue with all these factors is that they can be time-consuming to sustain, but we have to acknowledge that if we want to create a good research culture then we need to make sure staff have the time to contribute to it. The trick will be to replicate examples of good research culture even where it is not happening naturally or instinctively.

Critically, institutions have to determine how to recognise (in its most literal sense) those activities or actions that contribute to a positive research culture and then share, encourage and reward them. However, these are not straightforward things to measure.

The emergence of the narrative CV should help, by allowing people to identify their contributions to teams. Its encouragement for research assessment to look beyond standard bibliographic outputs will also allow recognition of informal organisation of activities, such as research group meetings or reading groups. 

Another approach would be to be specific about what is expected of senior staff in terms of being research enablers and supporting more junior colleagues. For example, some universities now require applicants for professorial status to describe how they have mentored or championed people. It is brilliant that this is now recognised as a valued activity, but it would also be nice to ensure the quality of that mentoring, and to recognise that it is not necessarily the same thing as supervision. We could add metrics on how many mock interviews have been done with ECRs, for instance, or how much internal peer reviewing has been done of grants/papers on which the professorial applicant is not a co-author or co-investigator.

More challenging – because it is so dependent on subject or seniority – would be to look at diversity of collaborations and research groups, where the concept of diversity includes career stage and discipline, as well as protected characteristics.

As with any metrics, these indicators must be used responsibly, so as to not inadvertently introduce bias. Moreover, there is always the risk that people will simply aim to tick the box rather than embrace the purpose of the metric. To tackle this, there should be validation – such as 360-degree feedback – from other members of the team, who can attest to the positive research culture being created (or otherwise).

As long as the overall outcome is that good practice is recognised, shared and rewarded, I am convinced that all our research environments will be more enjoyable to work in. And this, in turn, will make us all more creative and successful researchers.

Rachel Norman is institutional dean of research engagement and performance at the University of Stirling.