Demystifying impact: a glossary of key terms

Impact is a complex and diverse topic that challenges much conventional understanding about the end results of activities, particularly in academic research.

What do we mean by “impact”? It can be seen as a wide-reaching concept that refers to changes in areas such as society, culture, the environment, and the economy due to the activities of an individual or organisation. Often referred to as research impact in academia, it forms a key further stage in a research journey, along with the outputs and outcomes.

We define research impact as the demonstrable change or benefits to the wider society, economy or the environment that arises through research. This is not about journal factors, bibliometrics or H indices but what difference is being made, outside academia, in the real world. However, the exact definition can vary depending upon what you read or who you speak to.

The sheer quantity of vocabulary surrounding this subject can be overwhelming even for those who have started their impact journey. This glossary explains common terms associated with impact to help anyone in academia gain an understanding of some initial concepts. The terms are not explained in great technical depth but are accessible and provide a useful starting point for those interested in impact.

Beneficiary: An individual, informal group or legal organisation that benefits from impact work. Beneficiaries can include communities, ecosystems and taxpayers.

Chief Impact Officer (CIO): A member of the executive leadership team, sitting at the C-level of an organisation such as a not-for-profit or business, responsible for the overall direction of its impact work. CIOs can be accountable for developing the impact strategy and framework against which progress towards societal and environmental goals can be measured. Prince Harry’s appointment as the CIO of coaching and mental health firm BetterUp raised the role into public consciousness in 2021.

Economic cost-benefit analysis: A decision-making tool that considers what society has to give up for the benefits of a given project to be realised. It is useful for policymakers when determining which initiatives are most in the interests of society. It is distinct from an economic impact analysis, which attempts to predict but not evaluate the effects of a project.

Economic and financial impact: Effects that can be understood in terms of revenue generation, exports, job creation, GDP contributions and similar results. University activities can have wide-ranging economic and financial impacts, for example by employing members of staff, attracting international students or creating value through research.

Environmental impact: This can include changes in biodiversity, pollution levels, water or emissions, and many other similar environmental factors. Environmental impact can be measured through a number of well-established systems such as the Carbon Disclosure Project, which has provided a framework to manage carbon outputs of cities and companies for more than 20 years; the water footprint, for the amount of water consumed during certain activities; and the UK Government’s Town and Country Planning guidelines.

Universities can play a key role in improving environmental impact; by encouraging more sustainable daily practices among staff and students such as riding bicycles rather than driving cars, but also at a strategic level by supporting teaching and research towards programmes rooted in sustainability.

Impact Case Study (ICS): A document that demonstrates the impact of a research project or body of work. In higher education, these include detailing how the underpinning research has been used beyond academia to have social, environmental, economic or cultural impacts. ICSs are critical during submissions to certain national assessments, such as in the UK, Australia and Hong Kong. Importantly, ICSs must be supported with clear evidence of the impact, alongside a compelling narrative. In some national assessments these play a critical role in determining the level of funding a university secures.

Impact manager: The staff member responsible for embedding an impact strategy across an institution. This often includes adoption and development of systems and processes to help with understanding and demonstrating research impact, coordinating evidence collection and producing impact case studies.

Alongside impact officers (see below), this is often a relatively new role, created in response to impact being introduced into national assessments and thus may be more frequent in countries that operate national assessment frameworks.

Impact officers: Staff members who work with academic and impact-related staff to help improve the understanding and application of impact, develop impact case studies for national assessments, and develop an understanding within an institution of how to create and communicate impact “beyond academia”. They often work alongside the impact manager who is more focused on working with senior management to incorporate impact into the organisation’s strategy.

Impact pathways: A mechanism for illustrating how research is, has or could be applied beyond academia to have real-world impact, such as collaborating with a community group to embed a new approach to their work. Impact pathways demonstrate how you could, or did, engage with potential beneficiaries. They are useful to demonstrate what role the research may have had in achieving an outcome.

Logic models: A graphical representation of a theory of change, illustrating the links between programme resources, activities, outputs and audiences and the short, intermediate and long-term outcomes related to a specific problem or solution. They provide a useful framework for mapping and understanding what outcomes might be achieved and how. The Kellogg Foundation, an independent, private philanthropic foundation that works with communities to create conditions for vulnerable children to reach their full potential, provides guidance on the development of logic models.

National assessment: In a higher education context, national assessments are frameworks designed to assess the quality of research across the institution. Their purposes range from purely analytical to critical in determining the amount of funding an institution can achieve from the relevant decision-making body. Impact has begun to play an increasingly critical role in national assessments which are usually carried out in cycles over the course of several years, and assess research carried out during this period.

• In the UK, the research excellence framework (REF) helps determine the allocation of more than £2 billion every year from funding body UK Research & Innovation (UKRI) to institutions. The REF is carried out by Research England, the branch of UKRI that oversees functions relating to university research and knowledge transfer. Research England’s counterparts for the rest of the UK are the Higher Education Funding Council for Wales, the Scottish Funding Council and the Department for the Economy in Northern Ireland. Impact was introduced to the REF in 2014 as 20 per cent of the total mark of each submission, rising to 25 per cent in 2021.

• In Australia, the engagement and impact assessment (EIA) is designed to “assess how well researchers are engaging with end users of research and shows how universities are translating their research into economic, social, environmental, cultural and other impacts”. The first EIA took place in 2018 to incorporate impact for the first time; the next round is scheduled to take place in 2024.

• In Hong Kong, the research assessment exercise (RAE) is carried out by the Universities Grants Commission. The results of the RAE 2020 were published in June 2021 and evaluated the impact case studies of the eight universities in Hong Kong.

Negative impact: We tend to think of impact as a positive concept, however it is also possible for activities have negative effects. For example, a new runway at Heathrow Airport in London might create jobs and business opportunities, but its construction could destroy local ecosystems and additional flights could reduce air quality and increase use of hydrocarbon fuels. These would be considered negative environmental impacts.

Outputs, outcomes and impacts: There are several stages of the research journey, which can be seen to culminate in impact. Outputs, outcomes and impacts can be classified as follows:

• Research outputs are the immediate products of the research and can include theses, data, research papers, lectures and more. Non-traditional research outputs can include original creative works, recorded or rendered works or public exhibitions.

• Research outcomes are shorter-term expected and measurable benefits of activities; which could lead to longer-term and more widespread impacts. Researchers will specify desired outcomes, but these are limited by their nature. An outcome could include a new product, a new treatment or a change in public policy.

• Research impacts examine how outcomes influence society, the environment, the economy, culture and more in the longer term. Impacts for most national assessments must have a reach and significance beyond academia.

Reach: The extent or breadth of the beneficiaries of the impact. For example, does the impact affect only a handful of individuals, or does it have widespread national implications?

Significance: How much research has informed or improved policies, procedures, practices, understanding or otherwise. See REF panel guidelines for more information. Taking the example of a disease, the significance might be the efficacy of a cure – whether it partially or fully prevents illness.

Social impact: The impact on society, people or communities resulting from actions, activities, policies and programmes. An example would be creating educational opportunities to support individuals out of poverty and into fairly paid work. Universities can have numerous social impacts, such as creating equal opportunities for diverse students and staff to progress within education and career pathways.

SROI (social return on investment): A method for measuring value of activities beyond purely financial metrics, used by organisations – particularly not-for-profits – to incorporate social, environmental, economic and other values into decision-making. By considering SROI, organisations calculate the holistic value of a project or enterprise and it can play an important role in decision-making by involving stakeholders and encouraging transparency.

According to Social Value UK, SROI is a framework for measuring and accounting for a broader concept of value; it seeks to reduce inequality and environmental degradation and improve well-being by incorporating social, environmental and economic costs and benefits.

Theory of change: According to US scholar Carol Weiss, theory of change is “a theory of how and why an initiative works…the set of assumptions that explain both the mini-steps that lead to the long-term goal and the connections between project activities and outcomes that occur at each step of the way.”

It provides a framework for enabling change, building on four pillars – is the project credible, achievable, supported and testable? A theory of change identifies long-term goals and plans all the conditions and activities required to meet them. It can form the basis for reports to funders, policymakers and boards. In terms of research, impact can be seen as the goal of a theory of change that outlines causal links between planned activities and desired impact, assisting in the measurement and understanding of this impact.

Alex Smith is marketing manager and Laura Tucker is founder and CEO at Vertigo Ventures.