Enabling interdisciplinary research capacity for sustainable development: self-evaluation of the Blue Communities project in the UK and Southeast Asia

Questionnaire

The questionnaire was based on the Research Capacity and Culture Tool [18], which gathers information on participant’s perceptions of the research capacity and culture of their institution, team and self across a range of generic research capacity markers. This questionnaire was adapted by the authors to be relevant to the researchers in this project. Specifically, additional markers for assessment were added, including on interdisciplinary and international working, carrying out research that has an impact and a question about the effect of the coronavirus (Covid-19) pandemic. Further open and closed questions were added to gain more in-depth insight into the perspectives of the project participants and how these aligned with the overarching aims of the project and the work that was carried out during the project. The questionnaire was held on the JISC online platform, and the link distributed by email to the members of the BC project. Project members were mainly from academic institutions and non-governmental organisations in the UK and in four SE Asian countries – Malaysia, the Philippines, Indonesia and Vietnam. Researchers within the project ranged from those with little research experience to those with long careers in research, and categories in the survey were chosen to capture all of these career stages. The survey was distributed in February 2022 and was open for two weeks. The timing of the distribution of the survey coincided with the final two months of the four-year BC grant and therefore captured perceptions at this point in time. The survey was written in the English language and consisted of questions in four parts: (1) demography, (2) individual research capacity, (3) team level research capacity (participant’s BC team at their own institution) and (4) institution level research capacity. Questions included those with a numeric scale response to rate skills on various aspects related to research capacity and rating scale responses to assess change in research capacity. See Appendix for the full survey.

Data analysis

The demographic factor of main interest was the broad region of the respondent. To explore overall perceptions of research capacity and whether these differed between groups based on region (Global South and Global North), quantitative data were summarised based on the country of participant, or UK (/European) vs. SE Asian. Other demographic variables (gender, age, career stage/research experience and contract type) were also explored for associations with different responses to perceptions of research capacity. Due to small cell sizes, Fisher’s exact test was used to explore associations between variables throughout, with p values reported and significance taken at the 0.05 level.

To compare across unequal groups of responses to questions on what activities people participated in, what resources they benefited from, what are their motivators and barriers to carrying out research, and what they valued most from the project, responses were weighted according to the total number of individuals per group. That is, the frequency of responses is shown as the proportion of participants in a group who responded. These are presented as bar plots. Where response rates were low in certain groups, categories were combined as indicated (e.g., undergraduate plus MSc research experience).

The responses to a number of statements regarding participants’ experience in the project is visualised in side-by-side matrix plots where the size and colour of squares represent the frequency of responses against each score to each aspect of research capacity for UK (and other European) and SE Asian respondents. Matrix plots were produced using Raw Graphs 2.0.³

The relationship between the current research capacity (current success or skill across a range of aspects) and perceived improvement in capacity of these, was explored through Spearman rank correlation for the UK (and other) and the SE Asian regions. Correlation plots, R and p values indicate the strength of association between the current perceived research capacity and the perceived improvement of each aspect as a result of involvement in the Blue Communities project. A significant positive association (significant p value and positive R value) may indicate that higher levels of research capacity have resulted from involvement with the project. These were produced using ggplot2 (Wickham [19]) in R (R Core Team [20]). Significance was taken at the 0.05 level.

Demographic information

A total of 56 people responded to the survey, out of approximately 115 researchers who were involved over various time periods throughout the project. Of these, most (57%) were female and aged between 31 and 50 years of age (64%) (Table 1). The largest group of respondents came from the UK (or other European countries) and the smallest from Indonesia.

Most respondents to the survey came from academia (88%), though non-governmental organisations (NGOs) and government agencies were also represented (Table 2). Fifty-five per cent of researchers had fixed-term contracts and 74% had multiple work commitments. All career stages from early, mid and later career were represented in the survey, with 53% from the broader early career categories (students and PhD + five years or less experience).

There was evidence of an association between age and gender (p = 0.01), with more younger researchers being female; and age and experience (p < 0.01), with older researchers having more experience (for full results see Table A1). There was also an association between experience and country (p = 0.01) or region (i.e., UK and other vs. SE Asia; p = 0.02), with researchers with less experience being more likely to be from SE Asian countries.

Individual research capacity

Respondents took part in a broad range of activities throughout the project, with most people involved in publishing, presenting, analysing quantitative data, collecting data and designing studies (Fig. 1). There was no evidence of an association with the type of activities carried out and gender (p = 0.987), age (p = 0.984), experience (p = 1), contract type (p = 0.998) and country (p = 1) or region (p = 0.811) (see also Table A2). Most researchers were involved in particular with writing reports (86%) and publications (82%), collecting (61%) and analysing (61%) data and designing studies (61%). Fewer people overall were involved with applying for and securing research funding (41%), submitting financial claims (32%) and submitting health and safety assessments (21%).

Figure 1

Research activities respondents have been involved with as part of the BC project. Respondents could choose as many options as were relevant. The bars are weighted according to the total number of respondents from each country/region (e.g., if every respondent chose an option, each bar segment would have a value of 1).

The resources researchers benefited from were associated with the region (p = 0.002, Table A2). Respondents across all regions benefitted the most from knowledge exchange resources such as seminars (80%), networking (79%), training (79%), access to expertise (73%) and mentorship (70%) (Fig. 2). Resources such as protocol development (38%), library access (34%), health and safety guidance (30%), database management (30%) and software (27%) benefitted fewer respondents overall, but of those, benefits were felt mostly by the SE Asian respondents.

Figure 2

Resources respondents benefited from through the BC partnership. Respondents could choose as many options as were relevant. The bars are weighted according to the total number of respondents from each country/region (e.g., if every respondent chose an option, each bar segment would have a value of 1).

When asked what the respondents valued most from their BC experience, respondents across all across regions and career stages most valued interdisciplinary (61%) and international working (43%), publishing papers (34%) and improving their subject understanding and knowledge (30%) (Fig. 3). There was evidence of an association between age and the skills and opportunities valued (p = 0.023, Table A2), younger researchers in particular valued publishing papers and further employment opportunities. Country (p = 0.030) and region (p = 0.005) also had an association with values, with SE Asian researchers being more associated with valuing developing a positive attitude to research.

Figure 3

Research skills or opportunities respondents valued the most from their experience in BC. Respondents could choose up to three options. The bars are weighted according to the total number of respondents from (a) each country/region, and (b) their age (e.g., if every respondent chose an option, each bar segment would have a value of 1.

Many of the top barriers to research that respondents identified were related to time constraints in general [e.g., ‘Lack of time for research’ (54%), ‘Desire for work/life balance’ (41%), ‘Other work roles take priority’ (38%) and ‘Lack of suitable backfill’ (38%)] (Fig. 4). There was an association with the contract type (p = 0.009, Table A2), with those on fixed-term contracts particularly identifying lack of long-term employment and personal motivations as barriers. Covid-19 pandemic restrictions were also identified as a key barrier by 48% of respondents, particularly for SE Asian researchers (p = 0.001). Other barriers were a lack of long-term employment (27%), personal commitments (23%), fear of getting it wrong (21%) and lack of skills (20%). English language was identified by 13% of respondents as being a barrier.

Figure 4

Barriers to research, according to participants of the BC project. Respondents could choose as many options as were relevant. The bars are weighted according to the total number of respondents from (a) each country/region, and (b) their contract type (e.g., if every respondent chose an option, each bar segment would have a value of 1).

When asked what personally motivates them to carry out research, respondents indicated developing skills (79%), advancing their career (64%), making an impact (a problem that needs solving) (61%), increased job satisfaction (54%) and scientific curiosity (46%) (Fig. 5). These options were indicated across gender, age, contract type, regional and career stage groups showing the motivations for research were common across this group of researchers (Table A2).

Figure 5

Personal motivators to research, according to participants of the BC project. Respondents could choose as many options as were relevant. The bars are weighted according to the total number of respondents from (a) each country/region, and (b) their career stage (e.g., if every respondent chose an option, each bar segment would have a value of 1).

Across both broad regions, 91% of respondents agreed that they worked with interdisciplinary teams, with 66% in strong agreement with this statement (Fig. 6E); 91% agreed or strongly agreed that they feel positive about working with people from different disciplines in the future (Fig. 6O) and 89% that they had the opportunity to lead research (Fig. 6M). Sixty-eight per cent of respondents agreed or strongly agreed that they had the chance to lead a publication (Fig. 6K), of these 76% were from SE Asia. Leading publications was associated with age (p = 0.012; Table A3) and career stage (p = 0.021), with the youngest and least experienced, and oldest and most experienced not having led publications. On the whole, respondents from SE Asia responded more positively across all statements. Ninety-seven per cent of respondents from SE Asia agreed or strongly agreed that their research was relevant for making an impact in their region (making a difference to society), but this was less clear for UK respondents with 56% in agreement with this statement (Fig. 6A; p < 0.001 Tables A3 and A4). Ninety-two per cent of SE Asian respondents also agreed that they led on their own research questions (Fig. 6L; p = 0.008), compared to 56% of UK respondents. Ninety-five per cent also agreed they learnt new skills (Fig. 6J, p < 0.001), compared to 61% for UK respondents. SE Asian respondents also perceived that their career progressed, and prospects improved [Fig. 6C (88%), Fig. 6H (95%); p = 0.041, p = 0.015]. Fifty-six per cent and 67% of UK respondents agreed to the same markers on career progression and prospects.

Figure 6

Level of agreement to a number of statements from (a) SE Asian, and (b) UK (and other European) respondents. A five-point scale was used: Strongly disagree (−2), Disagree (−1), Neither agree nor disagree (0), Agree (1) and Strongly agree (2). Larger square and darker colour indicates higher frequency of responses in the matrix plot. Statements A–Q are abbreviated in the figure, full statements and percentage breakdowns are given in Table A4 in the Appendix.

At the individual level, across both broad regions, most respondents were confident in their success and/or skill on most aspects of research capacity, with 64% of ratings across skills being at a score of 7 or higher (Fig. 7), and with no sufficient evidence of a difference in success or skill between the regions on any aspect (Table A5). Respondents in both regions were most confident in finding and critically reviewing literature (Fig. 7E, G) with 84% scoring themselves 7 or higher. Seventy-nine per cent of respondents scored 7 or higher in presenting research (Fig. 7J) and 77% in protocol/study design (Fig. 7T). Sevent-five per cent scored 7 or higher in understanding interdisciplinary approaches and issues (Fig. 7P). Areas of lower confidence for respondents were in submitting a health and safety assessment (Fig. 7M; 32% scored 7+), financial claims (Fig. 7O; 41% scored 7+), in securing research funding (Fig. 7L; 45% scored 7+) and in submitting ethics applications (Fig. 7N; 52% scored 7+).

Figure 7

The relationship between SE Asian respondent and UK (and other European) respondent perceptions of their personal (individual level) current success or skill level for each aspect of research capacity (1 = no success/skill and 9 = highest possible success/skill) and change in success or skill level for each aspect as a result of involvement in the BC project (rating scale categories converted to numbers where –2 is ‘Much worse’, 0 is ‘no change’ and +2 is ‘Much better’). Correlation line, R and p values indicate Spearman’s rank correlation. Note that discrete data points are ‘jittered’ for visualisation purposes. Research capacity aspects A–V are abbreviated in the figure, full statements given in Table A6 in the Appendix.

Self-assessed success or skill in the different aspects generally was not associated with demographic variables, except in a few circumstances. There was evidence of association with age and data collection (p = 0.05, Table A5), where the 31–50-year-old age category scored themselves highest; and age and reviewing literature (p = 0.04), where older age categories scored themselves higher. Early career researchers (up to PhD student) scored themselves lower on finding literature (p = 0.02) and on publishing (p = 0.04). There was an association with gender and the scores on quantitative analysis, where some female researchers scored themselves very low (p < 0.001).

In terms of change following involvement with the BC project, all but one respondent saw improvement in the understanding of overseas issues (Fig. 7Q). SE Asian partners indicated higher improvement across 14 out of 22 markers of research capacity compared to UK partners who mainly indicated no change or a smaller degree of improvement across most markers (Fig. 7, Table A5). SE Asian respondents saw greater improvement in collecting data (Fig. 7D, p < 0.001), finding and critically reviewing literature (Fig. 7G, p < 0.001, Fig. 7E, p < 0.001), questionnaires (Fig. 7F, p < 0.001), managing projects (Fig. 7H, p = 0.018), presenting research (Fig. 7J, p = 0.008), networking (Fig. 7I, p < 0.001), referencing and data management systems (Fig. 7R, p = 0.001, Fig. 7S, p = 0.027), research reports and publications (Fig. 7U, p = 0.002, Fig. 7V, p = 0.008) and understanding interdisciplinary approaches and issues (Fig. 7P, p = 0.001). Similar to UK respondents, they mostly saw no change submitting health and safety applications (Fig. 7M, p = 0.51) and in financial claims (Fig. 7O, p = 0.12). There was no association between other demographic variables and the degree of improvement reported.

There was evidence to suggest a significant positive correlation between the current success or skill of individuals and the degree of improvement during the BC project in 18 out 22 aspects for SE Asian respondents and in two aspects [providing advice (Fig. 7K) and submitting finance claims (Fig. 7O)] for UK participants (Fig. 7). Together this evidence indicates that SE Asian respondents, on most aspects, perceived that they had improved from a lower success or skill level to achieve a success or skill level that was in line with what UK respondents had assigned themselves from the start of the project.

Team level research capacity

At the team level (the participant’s BC team at their own institution), most respondents across both broad regions were confident in the success or skill of their team across most research capacity markers, with 74% of ratings across skills being at a score of 7 or higher and with insufficient evidence of a difference in success or skill between the regions on any aspect (Fig. 8, Table A7). Eighty-six per cent of respondents scored their team 7 or higher for publications (Fig. 8U), 82% for research opportunities (Fig. 8R) and 80% for having leaders that support research (Fig. 8Q). On other aspects, there was lower confidence with 63% scoring their team 7 or higher for having incentives and support for mentoring (Fig. 8N) and for availability of software to support research activities (Fig. 8P), and 64% for having adequate resources to support staff training (Fig. 8J). There was evidence of an association with career stage and disseminating research (Fig. 8B, p = 0.044), with early career groups (up to 5 years post PhD) scoring their teams highly on this; their team’s success in providing expert advice (Fig. 8K, p = 0.010), with MSc/PhD students scoring their teams lower on this, and scholarships (Fig. 8T, p = 0.041), with MSc/PhD students and those up to 5 years post PhD scoring their teams lower on this. More experienced researchers (p = 0.007) and those on permanent contracts (p = 0.035) scored their teams higher on software (Fig. 8P). Male researchers were associated with a lower team score for engaging with external partners (Fig. 8L, p = 0.025).

Figure 8

The relationship between SE Asian respondent and UK (and other European) respondent perceptions of their team’s current success or skill level for each aspect of research capacity (1 = no success/skill and 9 = highest possible success/skill) and change in success or skill level for each aspect as a result of involvement in the BC project (rating scale categories converted to numbers where –2 is ‘Much worse’, 0 is ‘no change’ and +2 is ‘Much better’). Correlation line, R and p values indicate Spearman’s rank correlation. Note that discrete data points are ‘jittered’ for visualisation purposes. Research capacity aspects A–U are abbreviated in the figure, full statements given in Table A8 in the Appendix.

In terms of change following involvement with BC, there was disparity between groups, with SE Asian partners finding most aspects to be better or much better and UK respondents mostly reporting no change (Fig. 8). SE Asian respondents reported significantly higher improvement than UK respondents on all aspects except scholarships (T) (Table A7). There was no association with age, gender, career stage or contract type and the level of improvement.

There was evidence to suggest a significant positive correlation between the current success or skill of teams and the degree of improvement during the BC project in 11 out 21 aspects for SE Asian respondents and in two aspects [staff being involved in research planning (Fig. 8D) and staff training (Fig. 8J)] for UK respondents (Fig. 8). Together this evidence indicates that SE Asian respondents, on around half of research capacity markers, perceived that their teams had improved from a lower success or skill level to achieve a success or skill level that was in line with what UK respondents had assigned their teams from the start of the project.

Organisational level research capacity

At the organisational level, again most researchers rated their organisation’s success or skill highly across all or most research capacity markers in both broad regions, with 66% of ratings across skills being at a score of 7 or higher (Fig. 9). Seventy-seven per cent of respondents scored their institutions 7 or higher for accessing external funding for research (Fig. 9A), encouraging research activities relevant to creating impact (Fig. 9B), and for supporting the peer-reviewed publication of research (Fig. 9T). While only 54% of respondents scored their institutions 7 or higher for ensuring organisational planning is guided by evidence (Fig. 9D) and ensuring staff career pathways are available in research (Fig. 9E). Only for having adequate support for staff training (Fig. 9K), did UK respondents score their institutions higher than SE Asian respondents (p = 0.049, Table A9). For this aspect, 72% of UK respondents and 47% of SE Asian respondents scored their institutions 7 or higher. There was an association with career stage and scores attributed to some aspects. Later career researchers (more than 15 years post PhD), scored their institutions higher on getting external funding (Fig. 9A, p = 0.046), their institution’s access to software (Fig. 9Q, p = 0.011) and on its interdisciplinary approach (Fig. 9S, p = 0.041).

Figure 9

The relationship between SE Asian respondent and UK (and other European) respondent perceptions of their organisation’s current success or skill level for each aspect of research capacity (1 = no success/skill and 9 = highest possible success/skill) and change in success or skill level for each aspect as a result of involvement in the BC project (rating scale categories converted to numbers where –2 is ‘Much worse’, 0 is ‘no change’ and +2 is ‘Much better’). Correlation line, R and p values indicate Spearman’s rank correlation. Note that discrete data points are ‘jittered’ for visualisation purposes. Research capacity aspects A–T are abbreviated in the figure, full statements given in Table A10 in the Appendix.

In terms of improvement following involvement with BC, SE Asian respondents reported some improvement (‘Better’) across all markers and overall higher improvement than UK respondents across all markers, who reported mostly no change (Fig. 9, Table A9). There was evidence of an association with gender on degree of improvement on two aspects, with females more likely to report no improvement at their institution for research development policy (Fig. 9F, p = 0.006) and ethics (Fig. 9H, p = 0.005).

There was evidence to suggest a significant positive correlation between the current success or skill of institutions and the degree of improvement during the BC project in 11 out 20 aspects for SE Asian respondents and in one aspect [publication (Fig. 9T)] for UK participants (Fig. 8). Together this evidence indicates that SE Asian respondents, on around half of the research capacity aspects, perceived that their institutions had improved from a lower success or skill level to achieve a success or skill level that was in line with what UK respondents had assigned their institutions from the start of the project.

Successes, or what worked well for current and future research capacity

The skills and opportunities valued most by the respondents of this study were interdisciplinary (chosen by 43%) and international working (chosen by 61%) to make a difference to society and 91% felt positive about continuing to work in this way in the future; one respondent reflected on ‘working with amazing international partners on issues that matter’ (BC project participant, UK) and another could see an impact in their local community: ‘the great response of the communities to our engagements’ (BC project participant, Philippines). Almost all (97%) respondents from SE Asia could see that their research was relevant for making an impact in their region, while 61% of the full group identified a problem that needs to be solved as one of the motivations for their research. While some researchers recognised the challenges and benefits of this type of working, ‘Having differing disciplines within the team is enriching and engaging despite the conflicts that came with it’ (BC project participant, Malaysia), building trusting relationships between partners, with integration and collaboration, is one of the key requirements of a successful interdisciplinary capacity building project and keeping people engaged in the process [8,9,21,22]. Capacity building is not only about transferring traditional skills but also about ‘a process of strengthening relationships that enable innovation and resilience in communities, organisations and societies’ [9], thus, the process of collaborating and working together builds capacity in itself [17]. The results of this survey suggest that the researchers involved are enthusiastic, passionate and engaged in working collaboratively and making a difference to society. Importantly, respondents expressed their hopes for continuing to work this way in the future, with 77% hoping to build upon the networks and relationships that were developed through the project. As one respondent stated: ‘I hope to continue to cooperate in the future, to develop the research direction of the project’ (BC project participant, Vietnam).

One clear example of ‘learning-by-doing’ in action was in carrying out evidence synthesis and systematic reviews. During the project a team of UK researchers who are very experienced in systematic reviews ran a series of training sessions and provided ongoing guidance and support to SE Asian researchers in developing their own systematic reviews with research questions relevant for their region. This approach was clearly successful in that researchers in SE Asia identified critically reviewing literature as being a factor they are particularly skilled or successful at and identified this as an area of much improvement because of involvement with the project. Three systematic reviews were carried out for three of the SE Asian partner countries, all led by SE Asian researchers (publications in progress). In addition, protocols for carrying out reviews were also developed and published [23,24]. Furthermore, participants in the workshops have since gone on to teach the method to others in their institution, demonstrating the sustainable nature of this capacity building.

Notably, lead authorship in the BC project amongst the respondents was distributed between those from different countries, leaning more towards those from SE Asia, with 76% of SE Asian and 50% of UK respondents agreeing they had the opportunity to be a lead author. This was clearly appreciated by some, as one respondent described their team’s motivation as being ‘the independence granted to develop and pursue research questions’ (BC project participant, Indonesia). This is in contrast to many studies that show disparity in lead authorship between high- and low-income partner countries. For example, in the Future Climate for Africa programme, Harvey et al. [8] found only 14% of 230 publications were led by a researcher from an African institution. Interdisciplinary research, by nature, requires input from a diversity of partners coming from different knowledge backgrounds but power imbalances can mean that these different actors do not always contribute sufficiently [21]. A key feature of BC was that it was decided from the outset that early career researchers, in particular those from SE Asian partner institutions, would be prioritised in terms of leading research and publications, and were supported by more senior staff in doing this. In addition, the project established an Early Career Researcher Network, which encouraged members to apply for additional funding to support their own research questions, host seminars and share skills. Having this set out clearly and supported with leadership meant these power imbalances were explicitly addressed.

The Covid-19 pandemic restrictions presented a challenge, as reported by respondents, especially SE Asian participants (58% of SE Asian respondents identified this as a barrier to research). This was through an inability or reduced time to visit field sites and collect new data, an inability to meet project partners in person, and potentially more difficulty with Internet or resource access, as well as other personal factors. This is likely to have impacted capacity building through impacting development of personal relationships. Despite this, SE Asian partners responded positively in terms of improvement due to their involvement with the project across 18 out 22 research capacity markers. Teams adapted quickly to the new situation and in some cases changed their focus. Indeed, partners in the project demonstrated good practice in moving activities online in a sensitive and structured way [25]. In some, but not all cases, project participants recognised that they were fortunate to have the pandemic come later in the project so that personal relationships were already well established. However, where this was not the case, partners demonstrated concerted effort in building relationships online. For example, Richter et al. [25] emphasised the importance of using icebreakers in the virtual environment. This made a relatively smooth transition to moving capacity building elements and research working online.

Most respondents agreed or strongly agreed that they had the opportunity to lead research questions (80%) and publications (68%), they learnt new skills (84%), that their career level progressed (77%) and that they would have more career opportunities available (86%) to them as a result of their involvement in BC. This shows that the respondents perceive concrete and sustainable capacity building has been achieved during the project, and that partners feel they can carry on with this type of research independently into the future. One respondent reflected: ‘my involvement at the Blue Communities has increased my visibility in the local academia. This program has also significantly impacted my research and project management skills. Most importantly, my involvement with the Blue Communities has paved my career path in significant ways’ (BC project participant, Malaysia).

Challenges for sustainable current and future research capacity

An issue identified previously in research projects that aim to create impact in solving global challenges and build capacity is the conflict between research aims (e.g., advancing knowledge and publishing papers), influencing policy and building capacity [8]. Harvey et al. acknowledge that a common strategy is often used to achieve these aims, but this may not be appropriate for all, and research aims can be given priority. This conflict clearly emerged during the BC project. Just over half of respondents to the survey were on fixed-term contracts and, traditionally, publishing papers is important for career advancement, while even established researchers depend on their publication record in winning further research funding. Younger researchers, in particular, valued publishing papers and further employment opportunities (56% and 67%, respectively, of 18–30 year olds valued these skills/opportunities), but publishing was important for many respondents, with several mentioning publishing papers as a motivator for their team, and one respondent describing the motivation to be the ‘Esteem and recognition for good research published, contributing to career development and attraction of further research funding for self-determined research pathways’ (BC project participant, UK). However, tension with these motivations and the aims of building capacity and achieving real impact in communities and how this is recognised for individuals, was also felt, as one respondent described: ‘I’d say some team members are too obsessed with papers as a marker of success, and universities do not sufficiently recognise the value of impact in their promotion criteria’ (BC project participant, UK).

This tension may be driven particularly by the UK side where researchers may feel under more pressure to publish for their career progression and to meet expectations of funding bodies. Fifty-six per cent of UK respondents agreed their career had progressed during the project compared to 87% of SE Asian respondents. One SE Asian respondent noted that ‘I’m now appointed as a Senior Lecturer at a local university, and one thing that got me into this job is because my employer values my networking with the international, multidisciplinary research team of BC’ (BC project participant, Malaysia) indicating that the values in UK universities differ from those that may be found in other cultures [15]. Overall, across almost all markers and at all levels, SE Asian participants reported more positive improvement than UK participants, who only identified improvements due to involvement with the project in, at most, two markers at individual, team or institutional level. Several factors may explain this, for example, the markers given may not capture adequately what UK participants may have benefited from nor what adequately evaluates interdisciplinary aspects of research capacity [21]. However, it could also be that in some cases participants felt capacity building was acting mainly in one direction. For example, only 56% of UK respondents agreed they had been able to answer some of their own research questions compared to 92% of SE Asian respondents. One respondent said ‘Compared to traditional research projects, the career progression opportunities for UK teams may have [conversely] advanced less. The focus was on capacity development, rightly, but this may have inadvertently reduced the scientific innovation and output from UK teams because of the amount of time needed to support the partner teams’ (BC project participant, UK). While UK respondents felt positively about some aspects, for example, 83% agreed that they project managed, if these attributes are not obviously valued in their career pathways, individuals may also not value these highly. Considering that interdisciplinary researchers tend to publish less at first and have greater difficulty in demonstrating research productivity than more traditional researchers [21], the perceived lack of career development in this type of project will only exacerbate the conflict between research aims, building capacity and making an impact. The increasing importance of impact in the UK’s evaluation of higher education providers through evaluations by funding bodies such as the UK Research and Innovation’s (UKRI) Research Excellence Framework and Knowledge Excellence Framework may go some way towards valuing and incentivising researchers who participate in capacity building research.

In some cases, within the project, researchers did prioritise research aims. Other studies of international consortia have reported that researchers in the Global South can feel like ‘data sources’ in that they are not heavily involved in planning or analysing data, but only in commenting on it; that responsibility stays in the North [8]. In the BC project, researchers from both regions were involved in the collection of data to some degree, and it was clear that SE Asian respondents were involved in all aspects of research, from planning, to collecting data, to analysing and interpreting. There were instances throughout the project where SE Asian partners sometimes deferred to UK partners to carry out complex analyses. For example, one respondent observed: ‘Some [sub-]projects, while providing training at annual meetings, ended up doing the analysis for the partners rather than training and then letting partners take ownership of the research. This is reflected in some [sub-]projects not having many papers lead authored by [SE Asian] partners’ (BC project participant, UK). Harvey et al. [8] emphasised the importance of being willing to fail as part of a learning-by-doing process, thus sometimes sacrificing high-impact research outputs to focus on capacity development.

It was unexpected that UK respondents did not feel more strongly that their research capacity improved due to their involvement with the project, in particular in relation to applying and understanding interdisciplinary approaches. A greater understanding of overseas issues was the only marker where all UK respondents identified improvement. This particular marker may encompass a multitude of factors, and it may be that the parameters provided in the survey do not adequately articulate what UK researchers did learn from involvement with the project. It is important to identify these parameters and ensure more active two-way dialogue in future collaborations, so that UK or other participants from HICs are mutually learning from their project partners. Although UK researchers may have seen themselves more in the role of delivering research capacity than receiving it, there are important reasons for mutual learning and capacity strengthening. Just over half of UK researchers identified the project as having an impact in their region. This is not totally unexpected since UK partners were not working directly with local communities as SE Asian partners were. However, there are areas that could have potential impact in the UK. For example, the current discourse in the UK on the need to decolonise the curriculum [26] would clearly benefit from researchers who have experience working with other cultures and introducing this diversity through their teaching and research citations. In addition, researchers working directly with communities in LICs on sustainability issues try to highlight the knowledge that is held in the Global South as ‘the limited Western view of sustainability is stifling progress’ [27]. SE Asian partners instigated a wealth of approaches throughout the project, working creatively with local communities and practitioners. For example, researchers in Indonesia carried out participatory film making with local communities addressing sustainability issues. This resulted in changes in environmental behaviours and the formation of a film making community group dedicated to making audio visual work on behavioural change related to plastic pollution and climate change. Another example from Malaysia saw engagement with local communities resulting in greater attendance to health centres and vaccine uptake. More work is needed to reflect on and recognise the learning of UK partners in this collaboration. However, this may become more apparent over the longer term than at the point this survey was carried out.

There was disparity in resources at organisational level between UK and SE Asia, with less than half of SE Asian respondents scoring their institutions highly for having adequate resources to support staff research training, while 72% of UK respondents reported their organisations were good in this. In other studies, participants have felt that it is important to recognise this organisational inequality to manage expectations and ensure a meaningful partnership [17]. The level of improvement at the institutional level was perceived by SE Asian respondents to be more limited than at the individual level, with improvement in only around half the markers correlating with the current success. Development is still needed at an institutional or organisational level to reduce inequality in these factors, as there can be a lack of investment at higher levels, beyond the individual [8]. Despite this, 79% of SE Asian and 72% of UK respondents felt that they would build upon the international networks and relationships developed through the project.

Many respondents felt lower confidence in submitting health and safety assessments, financial claims and ethics applications, though at an individual level, there were improvements in these for SE Asian respondents, and improvement in financial claims for UK respondents. At the team and institution level, these areas were not perceived to have improved. While not all respondents would have needed to participate in these aspects, and that may explain some of the variability, these aspects may reflect a lack of facilities or support for these within organisations but also that they can be complex administrative processes where rules can be unclear even where facilities are well developed. For example, one respondent mentioned the ‘bureaucracy of financial process’ (BC project participant, Philippines) as a barrier to their team. Additionally, ethics applications are often reviewed by individuals on an ethics committee and responses to applications can depend strongly on the individual reviewers, which can vary from organisation to organisation. Similar studies have also found efficiency of researchers to be inhibited by bureaucracy or technical and administrative support in time-limited research projects [8,17]. This project worked with organisations to develop their ethical approval processes, financial management and risk assessment, and there is variability in these depending on the specific location. One respondent mentioned a team barrier as being ‘lack of administrative support in the initial stage of project’ (BC project participant, Malaysia), indicating that things did improve. Despite lower confidence indicated by respondents on these aspects, from the personal observations of the principal investigator and project manager (authors MA and VC on this paper), there was substantial improvement of SE Asian individual, team and to some extent organisational capacity in financial claims and ethics processes. This project, through learning-by-doing, adapted a flexible approach, to meet the needs of researchers in different countries and organisations and adapt to their specific circumstances. This included, for example, providing advances on funding to allow participants to travel or take part in research activities and circumvent inhibitive administrative processes.

Study limitations

There are limitations to this study, specifically that almost 90% of respondents came from academia, and to fully evaluate a transdisciplinary project, the perspectives of other actors, such as community partners, are also needed [21]. The objectives of other actors, or their perceived markers of success in research capacity needed to reach complex sustainability goals, are likely to differ from those with an academic focus, such as in terms of how capacity may translate to making an impact in communities, and this has not been captured in the responses to this survey.

The survey was only available in the English language, and this would have excluded some potential respondents. It is likely that the response to the English language acting as a barrier is an underestimate for this project, and ideally the survey would be translated to local languages to reach and get perspectives of all participants. For example, Indonesian respondents were underrepresented in the survey, and we are aware that some of the participants from Indonesia would have been restricted by the language barrier as they are non-English speakers. The BC project largely operated through English and non-English speakers relied on information being passed on by their colleagues. From this survey, we cannot say to what degree this knowledge transfer benefitted non-English speakers or if their research capacity improved. Future work should aim to assess this. Projects should ensure that local researchers form part of capacity building teams, and that ways to deliver knowledge and capacity in local languages are embedded within projects.

A longer-term assessment of research capacity will be required to evaluate if it has sustained into the future beyond the life of the project [14,28]. A key measure of research capacity is if it is lasting and if it can spread more widely in society. While this survey captured respondents’ perspectives at a specific time, just as the project was ending, this perspective could change over time, following experiences with transferring skills and knowledge to other projects or work.

Lessons learnt and implications for future projects

This study provides a broader perspective on the success of a learning-by-doing approach to building research capacity than focussing on research outputs such as publications and funding alone. There are key lessons emerging from the outputs of this study that can be used to enhance or modify approaches used for capacity building in future collaborations:

• Identify the benefits that partners that are in the role of delivering research capacity training may receive from such partnerships, and the parameters to measure these benefits, to ensure that these are clearly recognised and therefore can be valued and incentivised in career paths.

• Explicitly address power imbalances. This can look like, at leadership level, deciding on a strategy that prioritises certain groups to be supported in leading research and publications, for example, researchers from LICs and early career researchers. This could also include taking a flexible approach and providing additional support for administration, for example, finance, ethics.

• Develop concrete tools/training that can be taught to and applied by participants within the time of the project, so that skills can then be passed on locally by those participants.

• From the outset, put effort into building relationships and establishing trust between partners. In the BC project, this was established through (i) sharing roles and responsibilities, for example, holding the kick-off meeting in SE Asia, co-organised by partners there, and early scheduling of presentations from all partners; (ii) establishing an inclusive project culture, for example, mixing of groups, listening, all questions valid, patience and understanding; (iii) finding common interests, for example, social interaction around food from different cultures; and (iv) maintaining communication, for example, with follow-up in-person and online meetings.

Research ethics statement

The authors declare that research ethics approval for this article was provided by the University of Plymouth ethics board with written support obtained from leaders of each institution where participants are based.

Consent for publication statement

The authors declare that research participants’ informed consent to publication of findings – including photos, videos and any personal or identifiable information – was secured prior to publication. Consent for this study was obtained from survey respondents on the basis that their anonymity and confidentiality is protected.