How can impact strategies be developed that better support universities to address twenty-first-century challenges?

Engagement and partnerships

Building and maintaining relationships is established as one of the most valuable precursors to achieving research impact (Stanley, 2016), and so it is not surprising that the most prevalent theme across all the strategies reviewed was partnerships and stakeholder and public engagement (it was a key theme in 51 out of 77 strategies). However, approaches to engagement and partnerships varied widely across the strategies reviewed. This can be dictated by considerations of place or impact types, and while most contained descriptions of planned activities, some addressed the importance of needs analysis or planning to improve the nature of relationships on an ongoing basis.

University strategies identified partnerships with organisations across the local region, and at national and international scales. Engagement within the university’s city and region was a common theme, and this was particularly prevalent in London-based universities. For example, the strategy for King’s College London aspired by 2029 to ‘be regarded throughout the world as London’s leading civic university … making a valuable contribution to the capital’s health and success through a wide range of collaborations that both draw London into King’s and put King’s expertise to work in productive ways that have meaning for London’.

Partnerships with business and industry partners featured in strategies across the sample. Research and innovation clusters (also named ‘precincts’) were mentioned in strategies for Monash University and the University of Melbourne. The University of Oxford was engaging in expansion of innovation districts in and around Oxford, a number of universities collaborated in science parks (for example, the Universities of Lincoln, Exeter and Durham were investing in Lincoln Science Park, Exeter Science Park and NETPark respectively, and City University of Hong Kong was running an incubation programme at Shatin Science Park), and Anglia Ruskin University was one of many universities that ran shared spaces for the co-location of start-ups and applied research groups. These are geographical areas, typically in the same city or state as the university, where universities and companies cluster (often including co-location in dedicated buildings), pool facilities and expertise, and connect with start-ups and business incubators, facilitating economic and social development. In the UK, similar proposals were made for engaging with Local Enterprise Partnerships to establish collaborative spaces where researchers and business could drive the local economy, or the creation of ‘catapults’ and incubators to drive innovation in collaboration with local industry. The University of Melbourne strategy explained how this works:

To foster productive research collaborations, a key starting point is the development of strong clusters of research activity which bring together people and infrastructure in productive ecosystems. … With appropriate settings, these clusters can become ‘research precincts’ – a powerful means of harnessing collaborations and boosting innovation effectiveness. … Precincts offer a way to reach across and beyond organisational boundaries to generate far greater impact on challenging problems than the University could achieve alone. They can have a physical centre and be linked to nodes in different geographical locations.

In some cases, generation of new income streams motivated the development of partnerships. For example, La Trobe University explained how ‘existing and new partners will actively seek out our researchers because of their proven track-record … [and this] strategic engagement will increase our end-user (Category 2-4) income stream’. Imperial College London’s strategy included a goal to ‘diversify funding by engaging with new public and third sector collaborators’. Danish strategies were less specific about business engagement; for example, Syddansk University simply sought to ‘promote research areas that match positions of strength in the regional business community’, and Aalborg University’s ‘carefully selected knowledge-sharing partnerships are based on mutuality and a shared focus’.

University strategies were also more likely to include civic and/or public engagement. The majority of public engagement in the strategies aimed to provide benefits to society, but often articulated one-way knowledge transfer and communication methods. For example, Goldsmiths, University of London offered a ‘range of short courses, our public lectures and events programme and our library and archives’ to make knowledge widely accessible to their publics. Similarly, Sheffield Hallam and Ulster University proposed ‘marketing’ their research to the public via social media, driving engagement with events and archives. The University of Copenhagen emphasised the importance of schools as key stakeholders, and sought to develop teaching materials and contribute towards curriculum development. Similarly, Lingnan University sought to support STEAM (science, technology, engineering, arts and mathematics) education in primary and secondary schools.

Some strategies framed public engagement as capacity building, including more two-way knowledge exchange mechanisms. For example, Memorial University of Newfoundland aimed to ‘[build] greater capacity for our external partners and collaborators through public engagement activities’. The University of Calgary described how community engagement actually shapes their research, as they have a: ‘responsibility to engage our communities that we serve and lead in discussions about important issues where evidence is required to better understand those issues or even resolve them. Our community engagement significantly influences our research directions.’

Strategies also aimed for international partnerships, and these were wide-ranging, including with: business and industry; governments and agencies; UN organisations and other international institutions and convention bodies; cultural organisations; community organisations and other civil society organisations; practitioners; experts and researchers in think tanks and academia; and not-for-profit organisations and philanthropic groups. It was common for strategies to identify partnerships with other universities internationally, but few linked this to impact. The Stockholm Environment Institute did, however, and they explained why: ‘We partner with other knowledge-providers for multiple reasons: to access expertise, to ensure our research is firmly grounded by consulting with local and regional research partners, and to create alliances for achieving greater impact on policy and practice.’

More rarely, strategies referenced partnerships with funders (for example, Research England, which coordinates the UK’s Research Excellence Framework). The Stockholm Environment Institute explained their reason for including their funders as partners, to help them to identify ‘knowledge frontiers, [provide] scientific approaches to their problems, and [deliver] outputs and results that are accessible and actionable’, but emphasised the need to ‘operate at arm’s length’ to avoid conflicts of interest.

There was not always a specific strategy to develop partnerships with particular sectors or types of organisation; for example, the University of Wollongong proposed a needs analysis to prioritise future partnerships in relation to the university’s strengths and stakeholder needs. Similarly, the University of Exeter proposed systematically mapping industry needs on a sector-by-sector basis ‘against current expertise and academic hires’ to provide ‘introductions, facilitate initial meetings … and build industrial interactions and engagement’. The University of Auckland proposed hosting industry/stakeholder days to identify stakeholder needs, which they argued would ‘provide opportunities for our local communities and other stakeholders to share with us what impact means to them’. Although partnerships were sometimes focused on seeking funding from external organisations to create new infrastructure or capacity within the university, they were more often focused on delivering benefits to collaborating groups and those they served in society, in some cases referencing the achievement of United Nations Sustainable Development Goals.

Strategies referenced a number of mechanisms for maintaining and deepening partnerships. For example, the University of Ottawa emphasised networking events, with a knowledge mobilisation (KMb) hub to facilitate networking and identify networking events and consultations run by their stakeholders in which researchers could participate, alongside the provision of tools and services to facilitate networking. The University of Queensland had a Special Studies Program to encourage researchers to engage with industry, government and the wider community, and funded industry placements for research students. The University of Lincoln funds ‘staff placements in, and exchanges with, industry to ensure that our … research is relevant and up-to-date’. The University of Glasgow had ‘outcome agreements’ with the Scottish Government ‘to support their ambitions in maximising the opportunities afforded to the economy through the exploitation of research undertaken by our world-renowned academics and scientists’. King’s College London proposed ‘a civic engagement programme that will deliver a coordinated approach to student volunteering, credit-bearing modules in partnership with local community organisations, and an annual Civic Challenge’. The University of Portsmouth strategy talked about its ‘portfolio of CPD [continuing professional development] programmes for government, industry and third-sector partners in response to emerging workforce needs’. The WISERD project used an annual conference and ‘evidence symposia across key themes of policy and academic interest’. Calgary University suggested ‘pitch competitions on thematic problems [to] exchange information with external partners’. Hong Kong University of Science and Technology aimed to ‘promote community service: by raising civic awareness and social entrepreneurship and setting up an ecosystem to support social enterprise start-ups’ and by reaching out ‘to more students and faculty and [encouraging] them to actively participate in community service programs’. Similarly, Roskilde University and the Technical University of Denmark taught their students to become ‘change agents’, to develop businesses and to ‘take leadership of sustainable change’. Aalborg University used: ‘problem-based learning as a valuable means to establish successful collaboration relationships between our surrounding community and our students. This approach enables the students to gain practical knowledge while the business community gains an insight in the most recent theoretical knowledge.’

Finally, there was a strong emphasis on partnerships with Indigenous populations and their representative bodies across the Australian, New Zealand and Canadian strategies, with commitments often positioned prominently. In many cases, these were framed as acknowledgements, ‘paying respects’ to Indigenous elders, knowledge and land, and they were not all linked to research impact. Where impacts were identified, they tended to focus on ‘Indigenous advancement’, ‘providing opportunities’, ‘reconciliation’ and ‘empowering’ Indigenous staff and students. Some, however, made explicit links to research and impact, for example the University of Wollongong included: ‘embedding Indigenous principles and practice … into our research processes. A holistic and inclusive institutional view of impact along with academic leadership will also safeguard against instrumentalising university research (directing research solely towards utilisation or political priorities).’ Massey University provided ‘support for and facilitation of mentoring networks designed to support researchers working in Pasifika research and development, especially those working in partnership with external stakeholders and Pasifika communities’. Queen’s University, Canada ‘support the diversity of perspectives across First Nations, Métis and Inuit communities, while working with the Indigenous research community to examine how Indigenous ways of knowing impact research across the university’. They did this through a Principal’s Implementation Committee on Racism, Diversity and Inclusion, community-based participatory research partnerships with Indigenous groups to examine issues of mutual interest, and recruiting more Indigenous scholars through initiatives such as the Queen’s National Scholar Program and the Canada Research Chairs Program. Similarly, the University of Regina aimed to:

build and strengthen our relationships with urban, rural, and remote Indigenous communities with an aim of accountable and reciprocal research; enhance Indigenous engagement in the research enterprise … ; [and] enhance professional development opportunities and supports for units and faculties to learn to Indigenize and decolonize pedagogy, curricula, policies, procedures, and processes.

Co-production and boundary organisations

Although co-production and boundary organisations are closely related to engagement and partnerships, some strategies took a distinct approach that moved beyond engagement. Many of the strategies discussed more interactive co-construction of knowledge with partners and stakeholders (Campbell and Vanderhoven, 2016), where researchers work in partnership with knowledge users. Co-production has been embraced ‘because of its potential to improve the quality and relevance of research and its effect on policy and practice’ (Redman et al., 2021: 1), and this was a stronger theme across more specialist units. Given their focus, they tended to provide more detail about specific groups and organisations linked to the organisation’s strategic impact goals and scale of operation. For example, the Institute for Development Studies focused on learning partnerships which enable them to learn from their stakeholders to better understand the contexts in which they produce knowledge, so they could co-produce more relevant findings:

Learning partnerships enable us to better understand the environment in which development happens and map out desired changes, key stakeholders and policy processes. Achieving impact means not just producing evidence, but engaging with the politics of knowledge – who it is produced by and for, and whose voice counts …

The Stockholm Environment Institute went on to describe some of the co-production methods they used: ‘We often build engagement into research, through methods such as citizen science or participatory scenario development, co-production processes and workshops and dialogues of different types, as well as through tools and platforms that users can work with independently.’ The Plymouth Marine Laboratory strategy provided support to researchers to identify stakeholders early in the research process so that they can co-develop proposals together. CGIAR took this a step further to propose work to get ‘greater strategic clarity on where CGIAR lies in the development landscape’, in order to match stakeholder partnerships to challenges, building in ‘greater diversity in the range of research and scaling partners’. They went on to specify three types of partnership they wished to target: (1) ‘partnerships along the impact pathway … to co-deliver on innovations in technology, institutions and policy’; (2) ‘partnerships with the private sector’; and (3) ‘multi-stakeholder platforms’, which they describe as ‘structured alliances of stakeholders from public, private and civil society convened in the international development community to address complex global problems enshrined in the SDGs [Sustainable Development Goals], with CGIAR participating in those whose architecture and activities are best designed to link global policy and local action, and whose actions are informed by research’.

Some of the university strategies also emphasised co-production of knowledge, but they rarely considered how this would be done. One exception was the University of Glasgow, which, in addition to providing training for community and public engagement, sought to ‘identify innovative and effective models of community engagement and co-creation from within the institution and provide mechanisms through which these models can be shared, adapted and adopted’.

Universities did, however, regularly work with boundary organisations, brokers, intermediaries and boundary spanners (for definitions, see Neal et al., 2021). The most common approaches were focused on interfaces with industry via innovation precincts/districts, science parks and co-location facilities (see above). In the creative sector, Norwich University of the Arts positioned its East Gallery NUA as a boundary organisation designed ‘to develop a formal network of partner galleries across the UK and Europe with the potential to host collaborative exhibitions, it becomes a key resource for NUA academic staff and PGRs [postgraduate researchers] as well as external colleagues as a forum through which research activity can be organised and disseminated’. The University of British Columbia’s KMb Unit employed brokers to operate ‘as a gateway of access between UBC and other communities/general public … [It] brokers partnerships between communities at UBC and communities outside UBC and operates as a connector for currently existing knowledge mobilization groups and personnel, enhancing cross-pollination, synergy and innovation.’ Some strategies mentioned working with organisations and networks that could connect them with other universities working on impact or public engagement, as well as with stakeholders, for example, Research Impact Canada and the UK’s National Co-ordinating Centre for Public Engagement.

In contrast, specialist units were more likely to position themselves as boundary organisations. For example, the ClimateXChange Centre of Expertise was designed to act ‘both as a knowledge broker between researchers and policy, and as a research provider’. It operates in two modes. First, it ran a research programme that is co-produced with the Scottish Government and its agencies ‘to respond to questions and requests for evidence, identify upcoming evidence needs, and then independently plan our research and analysis to meet policy timelines’. Second, it provided a knowledge brokerage service to ‘facilitate conversations and broker knowledge across sectors, disciplines and institutions to provide new insights for policy’. Similarly, CGIAR engaged in ‘innovation systems’, which they described in an early draft of their strategy as follows: ‘The concept of an innovation system stresses that the exchange of technology, information and other inputs among people, enterprises, and institutions is key to an innovative process. Within innovation systems, there are no hard boundaries where CGIAR stops and starts.’

Resourcing for impact

Alongside the externally focused engagement and co-production strategies was a set of more inward-looking activities designed to support researchers from inside the organisations, and to improve capacity and capability for delivering impact. The majority of university strategies included resources for dedicated professional services impact staff to provide this support. These teams were typically separate from commercialisation teams or other knowledge exchange departments which may have a more external focus (Ward, 2017), although both clearly worked closely together. Although implicit in most strategies, it was clear that commercialisation teams and organisations such as Warwick Ventures and Oxford University Innovation focused on industry engagement and economic impacts, while broader impact teams focused on other sectors and types of impact. These teams provided a range of functions, including:

• helping researchers develop pathways to impact (including for funding applications)

• running events, including stakeholder workshops and networking events

• administering impact funding (see below)

• administering systems and providing training in impact monitoring and evaluation

• regularly reviewing impact capabilities in relation to their strategy

• monitoring evolving external agendas to recommend appropriate responses

• sharing good practice in impact generation

• coordinating impact generation-related activities across the institution

• coordinating and delivering training

• providing or facilitating specialist support for engaging with industry and policy

• identifying and managing impact case studies

• providing impact stories to communications teams and press offices

• creating and managing online resources for researchers to generate impact, in particular, impact toolkits, guides, training resources and case studies.

Some activities which crossed over more directly with the commercialisation or knowledge exchange teams included:

• managing relationships with stakeholders and alumni (in some cases using Client Relationship Management systems)

• providing a gateway and/or clearly identified contact points for external organisations.

Another common approach was to appoint ‘impact champions’ (these featured in strategies from the University of Wollongong, La Trobe University, the University of Auckland, the University of the West of England, London Metropolitan University, Ulster University, the Chinese University of Hong Kong and the Glasgow Institute of Health and Wellbeing). The champion role included: helping build skills and approaches to impact relevant to the discipline or unit; sharing the knowledge of how to translate and communicate research impact within the discipline or unit; helping prioritise effort and resources for impact; recognising and celebrating impacts occurring in their area; sharing good practice with other champions across the university; and, in the case of London Metropolitan University, being responsible for actually generating impact for research centres. Champions were responsible for these tasks at the scale of departments, schools or faculties, with the goal of providing insights into the unique challenges and opportunities faced by colleagues in their disciplinary area. With the exception of the University of Auckland, which specified both academic and professional services champions, the role was either academic or not specified in the strategy.

A number of universities ran internal impact funding schemes to generate new impacts or to extend existing impacts, and in the UK and Australia, in particular, also to facilitate evidence collection activities. The scope of the Warwick Impact Fund (University of Warwick) was particularly broad:

Internal investment will be available to support a range of innovation activities, from industry–University secondments, industrial fellowships in areas of relevance to the Government’s Industrial Strategy, collaborative research projects with practitioners in the public and private sector and with creative industries, ‘industry engagement days’, and ‘proof of concept’ and commercialisation funding.

Funds in other universities were more specialised, for example, the University of Edinburgh’s College of Arts, Humanities and Social Sciences ran ‘a venture fund for investment in commercial exploitation of research (through external partnership with Arthurian Life Sciences), [with] strong links to the SET Squared innovation and business incubator, of which we are a member’. In the UK, availability of these funds may depend on an institution’s eligibility for government funding from UK Research and Innovation via the Higher Education Innovation Fund, or Impact Acceleration Accounts from the UK’s disciplinary research councils.

Many universities had internal structures to provide accountability and to link the activities of impact teams to wider activity in the institution; for example, the University of Wollongong’s strategy recommended the creation of an ‘impact and engagement steering committee reporting to the University Research Committee … compris[ing] senior research staff, relevant support staff and impact champions’. The University of the West of England had a similar structure designed to provide ‘clear reporting lines for research impact within faculties and between faculties and the university’. This was often operationalised through networks of academic and professional services staff with specific responsibilities for impact, including departmental or school impact officers and facilitators from professional services, and academic coordinators and directors of impact or impact champions.

In contrast to the emphasis of university strategies on impact teams, roles, organisational structures and internal funding schemes, impact strategies from more specialised units were more likely to provide training and support and facilitate best practice at the scale of individual research projects or teams, or to integrate across projects thematically to achieve economies of scale in specific impact domains. For example, CGIAR’s strategy included a mechanism to ‘build a shared portfolio of research for development based on pooled funding … [to] achieve the levels of partnership required to scale impact, and attract the best minds to the challenges facing our food, land and water systems’. There was limited information on resourcing and structures for impact in Danish and Hong Kong strategies.

Impact training

Training was the main alternative (or supplement) to impact support in the strategies reviewed, aiming to build ‘impact literacy’ (as the University of Auckland called it), presumably referring to Bayley and Phipps’s (2019) framework for understanding: what impacts happen, for whom, and how researchers can demonstrate it; how research can be mobilised into action; and who is needed, with what skills, to make that happen. In addition to the necessary knowledge and skills to underpin an impact culture, some strategies mentioned the need to change perceptions of impact, for example, ‘socialis[ing] the importance of KMb for research impact among graduate students and faculty’ (the University of British Columbia), ‘promot[ing] the benefits of designing research to enhance impact’ (the University of Auckland) and providing opportunities to connect with researchers already engaged with impact via ‘networks to connect those who are interested in impact or who are engaging in impact activities already’ (the University of Auckland). The University of the West of England and the University of Calgary proposed going beyond a network to establish an impact ‘community of practice’, presumably referring to Wenger-Trayner and Wenger-Trayner’s (2015: 2) concept of a group of people who ‘share a concern or a passion for something they do and learn how to do it better as they interact regularly’.

As a comparatively new research skill, building capacity for research impact through training is a standard approach, so it is perhaps surprising that impact training was only explicitly included in a minority of strategies (17 out of 77 reviewed). In some cases, these were passing mentions, or restricted to certain skills or group. For example, the University of Queensland highlighted communications training for graduate researchers, and the University of Wollongong, La Trobe University, Manchester Metropolitan University, Memorial University of Newfoundland, Plymouth Marine Laboratory and the WISERD project included mentoring or coaching programmes. Although few strategies provided details, topics and skills covered in training programmes included:

• foundations and principles

• planning and designing research for impact

• building and maintaining stakeholder relationships

• tracking impact pathways

• measuring impact success and evidencing impact

• communication and dissemination, including media and online engagement skills

• public engagement

• public policy impact

• ethics of research and impact

• entrepreneurial skills for researchers and commercialisation routes

• co-production skills.

The University of Wollongong’s training on co-production sought to ‘understand and recognise relevant forms of evidence (including those outside traditional forms valued solely by academics)’. They also emphasised the need to take a coordinated approach to training, revisiting a ‘researcher development needs analysis’, and identifying training priorities at school and faculty levels. A number of strategies included the development of online toolkits, guides, training resources and case studies, including definitions, templates and signposting to specialist sources of support. As might be expected, there were often links between the content of these published resources, training and the functions for which impact teams were responsible (see previous section); for example, all three covered monitoring and evaluation of impact.

The independent research institutes tended to take a different approach to impact training compared to the other organisations and units in the sample. While recognising the importance of building capacity in their research community, their focus was on training and building capacity among stakeholder communities, for example, training them in the use of decision-support or analytic tools arising from research or in the use of collaborative data platforms (Stockholm Environment Institute), and ‘targeted capacity development … ranging from training-of-trainers at the farmer level through to ongoing institutional support to national partners … and technical advice to policy-makers at global level’ (CGIAR). The only parallel to this in the university strategies was the Memorial University of Newfoundland, which sought to build stakeholder capacity for engaging with researchers, and the University of British Columbia, which ‘trains and builds capacity for research utilization in partnerships with different stakeholders’.

Monitoring and evaluation

Few of the strategies included key performance indicators or success measures (16 out of 77), or any more detailed information about monitoring and evaluation of impact (15 out of 77). Of all the jurisdictions analysed, Australian strategies were most likely to mention the word ‘evaluation’, but only 3 strategies from this country contained substantive monitoring and evaluation activities (described below). However, 5 out of the 12 specialised units, programmes and institutions had a strong focus on monitoring and evaluation, and these were most likely to use impact planning tools, such as theory of change and logic models.

Strategy implementation was included in some strategies via a theory of change (5 out of 77) or logic model (or similar) (4 out of 77) to visualise or tabulate actions, assigning responsibilities and deadlines. For example, Massey University visualised their implementation plan as a matrix (Figure 1), the University of South Wales used a simple logic model, and N8 AgriFood drew on logical framework analysis to plan and track progress towards their impact goals (Figure 2). The N8 AgriFood logic model included assumptions, whereby risks might be identified. Generally, however, there was limited consideration of risks across the strategies reviewed. Notable exceptions were Norwich University of the Arts and the Third Sector Research Centre, which both considered a range of risks in their strategies. Both focused primarily on non-delivery risks, while the University of Auckland and the University of British Columbia reported on staff surveys which also highlighted risks of negative unintended consequences arising from impact generation activities (incidentally, the University of British Columbia was the only strategy to mention the need for more research on impact). Plymouth Marine Lab included an implementation timeline with milestones linked to a theory of change (Figure 3). In others, reference was made to committees responsible for overseeing progress towards strategy goals (for example, see the committee structure proposed by the University of Wollongong in the ‘Resourcing for impact’ section above).

Figure 1.

Implementation plan for Massey University’s strategic plan, including enterprise and knowledge transfer (KT) in the far-right column (Source: Massey University, Research Strategy 2018–2022)

Figure 2.

Examples of logic models from the University of South Wales (top) and N8 AgriFood (bottom) (Sources: University of South Wales, Research Strategy 2018–2028; N8 AgriFood Theory of Change and Logic Model)

Figure 3.

Theory of change (top) and implementation plan (bottom) from Plymouth Marine Lab (Source: Plymouth Marine Lab Research Impact Plan 2020–2025)

Table 2 shows the most common indicators used to track progress towards strategic impact goals. The most frequently cited indicator was income from consultancy and industry (used in 11 strategies). These mainly came from the UK and Australia, which might reflect the inclusion of industry funding as an indicator in Australia’s Engagement and Impact Assessment and in the UK’s Knowledge Exchange Framework. There were some nuances in the indicators included in this category; for example, the University of Liverpool aimed to achieve specific numbers of new funded strategic partnerships with industry, and the University of Queensland aimed to ‘rank first in Australia for attracting research income from industry’. Whereas this indicator focused specifically on funding from industry, 3 strategies also included indicators based on funding for impact from any source.

Although only 2 impact strategies included monitoring and evaluation in their success measures, 15 strategies included material on this topic elsewhere in the document. In many cases, this pertained to the investigation of options for monitoring systems (for example, La Trobe University and the University of Auckland) or aspirations to ‘[develop] processes to capture, collate and celebrate our impact’ (Sheffield Hallam University). Investment in impact monitoring and evaluation was often linked to research assessments and the generation of case studies in the UK and Australia, and in other cases, it was sometimes linked to the generation of annual reports on impact. Some mentioned specific tools, such as Vertigo Ventures Impact Tracker (University of Wollongong), and aimed to increase the use of these tools by researchers. However, it was recognised that impact evaluation was likely to require additional input. For example, the University of Wollongong suggested that despite their investment in the impact tracking software:

without appropriate support mechanisms in place, researchers will be unable to adequately capture or quantify impact. Some of this could be achieved through evidence gathering support from research assistants or professional staff, but in some cases specific expertise (e.g. interviews or surveys) or analytics support (e.g. website demographic analysis) may be required.

As a result, some strategies also aimed to increase capacity for impact evaluation among researchers, for example, developing and promoting the use of specific evaluation tools (University of Ottawa). Independent research institutes were the only organisations who employed independent external impact evaluators, partly in response to requirements from their funders. The Stockholm Environment Institute (SEI) had a strong culture of learning around monitoring and evaluation:

SEI has a scheme of learning activities to ensure that the monitoring and reporting on outputs and outcomes from research activities are fed back into the organization … We regularly assess our effectiveness in achieving outcomes, capturing key success factors and the dos and don’ts of, for example, stakeholder engagement.

Plymouth Marine Laboratory’s approach to monitoring and evaluation was also particularly comprehensive, including: the identification of information gathering points at key project stages, reporting structures, identification of impact indicators at project proposal stage, monthly science area reports providing updates on impact and potential case studies, annual impact reporting for all projects, systems to review all new publications for their impact potential, training for staff on monitoring and evaluation, maintenance of an impact database, timely requests for testimonials from research users, proactive case study monitoring and support, and regular monitoring against the organisation’s theory of change (Figure 3) and impact performance indicators. There was limited attention given to monitoring and evaluation in New Zealand, Hong Kong and Danish strategies, compared to the other jurisdictions in the sample.

Impact culture

Pressure on research productivity from assessment, precarious contracts, increased competition for research funding and a focus on outputs at the cost of all else has led to a recent rise in interest in ‘research culture’ or, as the Royal Society (n.d.: n.p.) defines it, ‘the behaviours, values, expectations, attitudes and norms of our research communities’. Work by the Wellcome Trust (Moran et al., 2020) and the Association of Research Managers and Administrators (ARMA, 2020) in the UK considers factors in research activity that can give rise to ‘disruption’ of research, and how this is manifested in ‘poor’ research culture. Wellcome deduced that ‘Factors identified as disruptive to research culture included chasing impact, increased competition, proliferation of metrics, job insecurity and rigid career pathways’ (Moran et al., 2020: 1). The UK Government released an R&D People and Culture Strategy in summer 2021 (Department for Business, Energy and Industrial Strategy, 2021), which aims to ensure that ‘everyone’s contribution is valued, and the UK has an outstanding research culture that truly supports discovery, diversity and innovation, and offers varied and diverse careers that bring excitement and recognition’ (Kwarteng, 2021: n.p.), although most actions are currently reviews and consultations.

Many of the institutional strategies we examined explicitly considered research culture, and, as a subcategory within this, much of this content could be implied to apply to impact culture. For example, much of the content in strategies about research ethics, responsible research and innovation, open access policies, equality, diversity and inclusion, and staff health and well-being would apply to impact. For the purposes of this study, however, only material pertaining specifically to impact culture was analysed.

First, it should be noted that many references to culture were non-specific and ill-defined, and in reality were probably referring to an ‘approach’ rather than to a culture, if culture is understood in relation to how people ‘find meaning as individuals (on the basis of their own perceptions), collectively (on the basis of social norms and shared perceptions) and through their relationship with objects’, or if impact culture specifically is understood as ‘communities of people with complementary purpose who have the capacity to use their research to benefit society’ (Reed and Fazey, 2021: 2, 4). For example: La Trobe University aimed to ‘develop a culture that values and generates impact from research through industry engagement’; Queen’s University, Canada, stated that their ‘culture of sustainability will be underpinned by our commitment to the United Nations Sustainable Development Goals’; the University of Lincoln aimed ‘to promote a culture of enterprise and innovation across our communities’; Anglia Ruskin University aimed to ‘deliver a culture and working environment in which collaborative and multi-disciplinary research and innovation thrive’; and London Metropolitan University aimed to ‘embed impact and knowledge exchange within our research culture’. Very rarely did these and other strategies like them define what they mean by culture or explain how they would achieve their cultural goals.

Impact culture was often associated with values, for example, the Hong Kong Polytechnic University aimed to ‘align departmental and individual performance measures to better reflect a congruent value system that properly addresses the balance among education, research and community impact through KT [knowledge transfer] and entrepreneurship’. The Brunel University strategy described a ‘culture of mutual respect and equality of opportunity, placing the health and welfare of individuals at the heart of our ethos’. The University of Edinburgh described ‘a shared culture that values people and provides leadership within a supportive working environment built upon collaboration, communication and coordination’. Queen Mary University of London sought to ‘build on our core values to further embed a culture of engaged research practice’. The Stockholm Environment Institute provided one of the richest descriptions of an impact culture rooted in clearly articulated values and principles:

Our organizational culture lays the foundation for the way we work with partners and with each other. Our culture is grounded in our development ethos and commitment to resolving sustainability challenges, from local to global. It stands for transparent and inclusive decision making, for building and maintaining trust, for empowering our partners, for giving space to diverse voices, and for delivery of the highest quality. It holds us to ethical standards of integrity, collegiality and respect in all our professional interactions.

Hong Kong University’s strategy sought transformational change in its culture to deliver impact: ‘We will make a paradigm shift to focus on and reward … research innovations that benefit communities and transform global technologies … There will be a shift from activity to value, from output to outcome, and from strength to leadership.’ To do this, they proposed to:

provide more opportunities for outcomes-driven translational and transformational research; support research that transcends intellectual output to meeting an innovative outcome that has value and impact, and driven by societal needs or enterprise; [and] deliver demonstrable and significant outcomes to our social communities and the technological world through research, innovation and enterprise development.

Aligned with research culture and values is the issue of intrinsic and extrinsic drivers of behaviours. Extrinsic drivers are those where external demands or incentives provide motivation for researchers and organisations, while intrinsic drivers build on personal values and a fundamental self-motivated desire to meet the needs of society (Ryan and Deci, 2000). These drivers are the building blocks of research and impact culture.

UK strategies were more likely than strategies from other countries to make explicit links between impact and formal assessments of research and impact (16 strategies, compared to 2 in Australia, 1 each in New Zealand and Canada, and none in Denmark; 2 of the Hong Kong strategies referenced the Research Assessment Exercise, with 1 referencing the UK’s system as a benchmark). Although these represent a minority of UK strategies, links to the REF were particularly prominent in some of these. For example, the REF was mentioned in the first point in the first main section of Manchester Metropolitan University’s strategy, with the document later implying that investment in research for impact was typically dependent on a return on investment via quality-rated (QR) funds from the REF: ‘Internal resources will be directed at research that meets our ethical standards and usually: a. Generates academic outputs of sufficiently high quality to attract external income … and b. Generates beneficial social, economic, environmental or cultural impact (sufficient to attract QR funding).’ Similarly, two out of six criteria for the establishment of new Research and Knowledge Exchange Centres were linked to REF performance, key performance indicators were due to be linked to the REF (similar to a number of other institutions – see Table 2) and workload allocations across the institution were linked to the REF: ‘Faculties will set a specific target for the proportion of workload allocation directed at 3* and 4* work and measure against it as a lead indicator of progress with the research strategy.’

A total of 17 out of the 77 strategies reviewed included various kinds of extrinsic incentives designed to increase researcher engagement with impact (Table 3). These were largely absent from the five independent research institute strategies, and they were not present in Danish strategies. There were examples of impact being included in promotions exercises in the UK, Canada, Australia and New Zealand (note that this includes the University of Auckland, where inclusion in promotions was under consideration only). The need to ‘recognise’ impact generation activities was noted in two of the Hong Kong strategies, but no mechanisms were proposed (for example, Hong Kong Polytechnic University aimed to ‘enhance the benefit-sharing policy and recognition mechanism to encourage academic staff to identify and pursue high-impact KT endeavours’). Impact was only included in annual appraisals in 4 strategies (2 each from the UK and Australia), and it was only included in recruitment criteria in 2 strategies (both from Australia). Although there were general comments about the need to recognise and celebrate impact across all six jurisdictions and the independent research institutes, formal impact award ceremonies were only mentioned in 1 Australian and 1 Canadian strategy.

In two universities that did not yet offer formal rewards for impact, strategies included findings from staff surveys that highlighted issues arising from the lack of incentives. For example, the University of Auckland strategy noted that, ‘Many of our researchers are already conducting impactful research but are often not recognised or rewarded for this, or their work is perceived negatively’, and the University of British Columbia highlighted a number of issues linked to a lack of incentives for impact, including the recognition that impact is ‘something Professors do off the side of their desk’ and ‘not yet part of the tenure and promotion discussions [or] recognized in most faculties’.

There was limited consideration of the potential negative unintended consequences of providing extrinsic incentives for impact. One notable exception to this was La Trobe University’s strategy, which emphasised the need to reward impact without disadvantaging non-applied researchers, recognising ‘that there are researchers undertaking pure or fundamental research that may not yield “real-world” change in the immediate future but will profoundly influence the course of knowledge and the ability of other researchers to achieve future impact’.

In contrast to these extrinsic incentives to promote engagement with and reward impact, strategies also sought to engage with the intrinsic motivations of researchers. In an oblique reference to the limitations of incentivising impact via research assessments, the University of the West of England was ‘keen to ensure that a research impact culture extends beyond the REF and that as much of our research as possible is focused on being of value to society’. Connecting with the intrinsic motives of their researchers to innovate to tackle real-world challenges, the Stockholm Environment Institute (SEI) describes itself as:

a trust-based organizational culture, and our people breathe life into and carry out our mission. We put high levels of confidence in our colleagues around the world, which enables SEI researchers to innovate, take initiative, and engage with key arenas of decision making. SEI is innovative and adaptive in order to respond to new challenges.

Research ethics statement

Not applicable to this article.

Consent for publication statement

Not applicable to this article.