Key questions to help universities measure societal impact

Societal impact is a hot topic in higher education. The powerful actors who control funding push universities to demonstrate that they are impactful – but each has their own definition of what impact means and how it should be measured. 

Faced with conflicting demands, some institutions avoid commitment to any metric of societal impact. They stick to the established definition of research impact, which focuses on academia, and claim that by influencing science they also influence broader society. This strategy might work for elite universities that can rely on their strong local reputations or international rankings in attracting students, employers, partners and donors, but it can be risky for less self-sufficient institutions.

Universities can also face a mismatch between what staff consider to be impactful activities and the institutions’ narrow definition of societal impact. Institutions that try to demonstrate impact to multiple external actors – for example, funders, government agencies, corporate partners and alumni – often implement an array of overlapping measurement systems. These might leave the staff wondering what counts as impact and make it difficult for an institution to support societal impact creation in a consistent manner.

• How to demonstrate the economic and societal value of higher education
• Advice for enacting a ‘circular’ approach to research and impact
• Resources on research excellence: what is it and how can universities achieve it?

To avoid these undesirable outcomes, institutions need to find a coordinated approach to measuring societal impact that has sufficient level of:

• External legitimacy (it is recognised by governments, funders and other external stakeholders); and 
• Internal legitimacy (academics and professional service staff can see their own contribution in it). 

We highlight that the level of legitimacy should be sufficient, because any approach selected is unlikely to satisfy everyone’s highest standards. Hence a realistic aspiration is to have an approach that is legitimate enough in the eyes of both internal and external stakeholders to ensure that it is not dismissed summarily by them.

Here, we offer advice (based on our recent publication) on how academic leaders can approach this challenging task.

Three key questions to help measure and demonstrate societal impact

In developing an approach to measuring and demonstrating the societal impact of their institution, academic leaders need to answer three questions:

1. What does impact mean for us?

Resolve the understanding of “social/societal impact” from the outset. Do not assume that everyone has the same idea in mind. Disagreements about the meaning of impact often stem from disciplinary traditions, so ensure that all disciplines have a voice in the discussion. Without common ground, you will have more confusion and stronger resistance to ways of measuring impact. Aim for an inclusive set of potentially measurable definitions that capture pathways to impact rather than a singular definition (even if you have one that is easy to measure and is favoured by most powerful stakeholders).

2. What are the reasonable ways to measure this impact?

You might have tension between quantitative and qualitative approaches, and will need to put together a reasonably diverse bundle of measures or ways to communicate impact. To select specific metrics, consider their validity (how well a metric reflects impact on broad society) and their ability to inform busy decision-makers’ thinking in a succinct way. While each individual metric will not be perfect, a diverse bundle has a better chance of capturing the many “faces” of societal impact.

3. How much should the university spend on impact measurement?

Think about return on investment. Given the reputational (and economic) benefits that communicating impact can create for the university, how much can it invest in data collection about impact? Depending on your resource envelope, you will need to make choices based on the cost of data collection for different types of metric (which can differ wildly) and the value that a particular metric can deliver (some can be used for external benchmarking; others are only suitable for internal decision-making).

Three stages of measuring impact

Putting in place a system for measuring societal impact is a three-stage process, with each stage being necessary for establishing legitimacy. We recommend this sequence:

Stage 1: Figure out the impact for the institution and where it is created

At this stage, consult with many stakeholders across the university. Although a top-down process (where leadership decides and imposes its vision of impact on the university community) can work in the short term (given sufficient authority of university leaders), in the long term it will result in a window-dressing exercise, where reports about impact are dutifully submitted, but nobody cares about impact activities or uses impact metrics to inform decision-making. A broader, bottom-up approach will give impact staying power. 

Stage 2: Choose the right metrics 

Consider existing approaches to impact measurement in the institution’s regulatory environment. For example, what do local funders, government bodies and accreditors consider to be impact? There is extensive literature on impact measurement, but given substantial disciplinary differences, it is important to seek input from academic units on appropriate metrics for their discipline. Do not forget that societal impact can be created, in interconnected ways, through research, teaching and other university activities.

Stage 3: Make resource-based choices 

Establish a working group with the expertise and decision-making authority to weigh the costs and benefits of measuring societal impact using specific approaches. On this working group you might have:

• A university representative to tell the group which metrics the institutions already collects for regulatory bodies and other purposes
• A research librarian to advise on relevant bibliometric and altmetric indicators
• An admissions office representative to advise on the value of impact communication for attracting future students 
• A communications office representative to advise on which impact-related evidence makes the most powerful narratives
• An academic and other staff representative to keep the group connected with the voices of those who generate societal impact
• Someone with financial decision-making power to commit resources to the choices made by the group.

This process should deliver an approach to societal impact measurement that has sufficient to keep a university’s critical stakeholders invested in it. Genuine engagement of academic leaders in co-creating a legitimated impact framework with external and internal stakeholders gives the leadership a stronger mandate for coordinating impact activities. Ultimately, by relying on such a legitimate system, a university can demonstrate its societal impact in a more effective way. In the short term, this happens due to improved coordination in impact creation and communication. In the longer term, the impact framework enables the development of organisational structures and strategies that facilitate impactful activities and make societal impact creation a shared responsibility of individuals and their institutions. 

Olga Ryazanova is associate dean for strategy and governance and Peter McNamara is dean, both of the Faculty of Social Sciences at Maynooth University. Tatiana Andreeva is associate professor in management and organisational behaviour and a former research director of Maynooth University School of Business.

If you would like advice and insight from academics and university staff delivered direct to your inbox each week, sign up for the Campus newsletter.