A typology of Twitter interactions to effectively analyse engagement and evidence research impact

Introduction: contextualising social media impact

In order to achieve impact, it is necessary to develop engagement with your research activities, but how do you know when people are engaging with you on Twitter, and whether that engagement leads to any impact? What types of evidence can indicate that impact has taken place? These are key questions for any researcher planning to use social media to share results, to create discussion, or for any of the many purposes the public engagement can have.

The @BarbieReports ArtActivistBarbie (AAB) Twitter account was set up by Sarah Williamson both to communicate her research around gender inequalities in museums and galleries, and to document and carry out art activism. In this article, the authors reflect on their attempt to understand whether the Twitter engagements can be said to constitute types of impact. The authors recognise that not all the engagements fully evidence impact, as defined by, for example, by the Research Excellence framework in the UK, but they still consider that it is useful to include these as demonstrating two-way engagement, which is part of the journey towards impact. So, rather than simply dismiss some of the Twitter engagements as not evidencing impact, we consider them in context to understand what they can tell us about the nature of the engagement, and to what extent they can or cannot be used as evidence of impact.

In this article, we discuss how Twitter engagements can demonstrate research impact, and the extent to which different types of engagement can be said to constitute impact or not. We start by exploring definitions of impact and public engagement, before presenting an analysis of a selection of Twitter interactions from the AAB account.

We create a typology of four categories to demonstrate types of engagement and impact, enabling an understanding of the extent of impact that can be claimed from contextual social media analysis. The typology can be used by researchers to analyse tweet interactions in three ways:

1. as indicators of two-way engagement

2. as evidence of impact

3. as part of a wider story of change and influence.

What constitutes public engagement?

The National Co-ordinating Centre for Public Engagement (NCCPE, 2020) defines public engagement as a two-way process for mutual benefit that involves sharing higher education and research activities with the public. Reed et al. (2018) outline a typology for public engagement that includes informing, inspiring and/or educating the public, consulting and actively listening to the public, and collaborating and working in partnership with the public.

Central to this definition is that activities that simply broadcast (typically defined as being one-way) to an audience are not considered to be public engagement, as they do not involve mutual interaction and benefit. There are challenges in evaluating public engagement and demonstrating the impact, as these are often ‘subtle (on understanding, attitudes or values)’ as well as ‘hard to measure’ and, furthermore, ‘change over time’ (Reed et al., 2018: 158).

Social media is often used by researchers for communicating their research, as can be seen from the Wellcome (2015) report. However, the NCCPE (2018: 11) guide suggests that one of the disadvantages of using Twitter is that ‘stimulating genuine engagement can be challenging and time consuming’. It is clear that there are challenges in using social media in an interactive way, and that simply posting updates on a platform or having a lot of followers does not demonstrate the required mutual benefit, or two-way aspect, of public engagement.

When does research impact happen?

The Economic and Social Research Council (ESRC) defines research impact as the demonstrable contribution that excellent research makes to society and the economy. Some examples of research impact are influencing the development of policy, practice or services, shaping legislation, and changing behaviour, contributing to the understanding of policy issues and reframing debates and technical and personal skill development. In the UK, the Research Excellence Framework defines impact as an effect on, or change or benefit to, the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia.

In a similar way to public engagement involving genuine engagement and mutual benefit, research impact must involve a demonstrable contribution in the form of evidence of having ‘made a difference’, changed something or benefited someone. The notion of ‘tangible benefits’ is also highlighted in Reed et al. (2018: 146), who consider that this is difficult to evidence. In this way, the concept of ‘benefit for people/organisations’ is central to both public engagement and research impact.

The potential of Twitter as a platform for engagement and impact

In this article, we look specifically at Twitter as a platform for engagement and impact, using the example of the AAB Twitter account. As a result of analysing a selection of tweets, and the subsequent interactions with the tweets by followers of the account, we have developed a typology with four categories of tweet or Twitter interaction; each involves a type of engagement linked to evidencing impact.

There are several options in terms of how audiences on Twitter can engage with or respond to tweets. A user can retweet (RT), sharing the tweet with their followers, which is an example of broadcast activity, or one-way communication. An RT does not mean much in terms of engagement, as there is no indication of why the user has chosen to RT a tweet. An RT is simply an indication that the tweet has been ‘seen’ and shared, so it can speak to the reach (the extent of the audience) of a tweet, but not to the significance (the impact or difference) it may have had. In addition, a Twitter user can engage with a tweet by sharing it as a quote tweet (QT), which is an option that allows a follower to add a comment ‘on top of’ the original tweet. There is also the ‘reply’ option, which allows a user to respond directly to the tweet, and which will show up in the mentions of the user who tweeted. As limited as these QT and reply functions can seem, they are examples of a two-way process, which is part of the definition of public engagement. These two options allow for a direct (reply) and more indirect (QT) engagement with tweets and Twitter users, which, we argue, can evidence, for example, changes in awareness, changes in behaviour, agreement, encouragement, engagement in public debate, sharing similar experiences and so on. Also, and by no means least important, the impact of the AAB Twitter account is often enjoyment or fun, in the sense that the account followers express a sense of joy or fun in response to the visual content of the account.

The Twitter account was set up by Sarah Williamson, a researcher at the University of Huddersfield, UK, as a way to share images of AAB in action, reporting on the way in which women are represented in art galleries and museums. Williamson (2020a) describes ArtActivistBarbie as a fearless, feminist Barbie doll who is staged and posed in art galleries and museums to draw attention to gender representation, inequality and injustice.

Williamson started the Twitter account in November 2018 with a tweet showing ArtActivistBarbie with a placard at the National Gallery in London, pointing out that the collection featured 2,300 works by men and only 21 by women (Figure 1).

Figure 1.

ArtActivistBarbie. The a/r/tographic redeployment of Barbie in museums and galleries as a feminist activist and pedagogue (Source: photo © Sarah Williamson)

Williamson crafts tweets using wit and humour to point out examples of gendered representations, and also comments on current debates, such as the controversy that arose over the statue of Mary Wollstonecraft, which was unveiled on 10 November 2020. Three-and-a-half years later, the Twitter account has more than sixteen thousand followers, and it has been featured in an article in The Guardian (Langham-Hooper, 2020), as well as in other international media. For Sarah, the AAB account has also led to many opportunities to be interviewed, to join a feminist podcast, to invitations to be a keynote speaker, and many other activities that mean that the research can reach a wider audience. As the Twitter account developed, it became important to understand what kind of impact AAB could be said to have on followers.

Literature review on social media use and impact

We identified a range of sources from 2016 to 2021 that focus on what societal impact can look like, particularly through social media. These range from how social media use can enhance readership and citations (Bardus et al., 2020; Breland et al., 2017; Jordan, 2019), to how to track hashtags (Copley, 2018), to using social media for communication of research (Schnitzler et al., 2016), to work that considers how digital platforms as a whole are ‘socio-technical infrastructures which shape the activity of their users’ (Carrigan and Jordan, 2022: 354). However, there is comparatively little work that discusses actual societal impact and how it can be measured.

If we accept that knowledge, awareness, opinions and behaviour can change as a result of online activity, how do we evidence this? Nicola Osborne (2016), a digital education manager at the University of Edinburgh, underlines the importance of presenting data with context and developing stories of change, success and influence.

We argue that researchers need to look at and analyse the interactional context of which a tweet is part to better understand the types of engagement and kinds of impact that tweets can indicate. Like a conversation, you will not understand much meaning by looking at what just one person is saying.

A typology of types of engagement

Some tweets directly evidence changes in behaviour, awareness and knowledge in cases where the content of a tweet response contains a statement that outlines the impact in response to an original tweet. These are few and far between, although researchers may be able to increase the ratio if they follow up on tweets that express engagement but do not directly articulate the type of impact. Doing follow-up is a time-consuming activity, and it requires researchers to closely monitor the account and consistently pursue interactions.

However, Twitter interactions are more than simply likes and RTs; the responses are nuanced, and they can indicate different types of engagement, which are perhaps not evidence of major impact, but which certainly challenge the idea of Twitter being mainly one-way dissemination. In order to more effectively evaluate impact, a typology offers a more nuanced way to identify and evidence impact in context.

We explored tweet interactions in the period 2019–21, and we selected about 30 tweets for further analysis. We then identified four categories of tweets or Twitter interactions which involve a type of engagement linked to impact (Table 1). There are overlaps between categories, as some interactions cover more than one category and the categories of ‘comment’ and ‘conversation’ are similar in nature but different in how engagement happens or is initiated. ‘Comments’ are more likely to be stand-alone statements in response to tweets, whereas ‘conversations’ are more likely to be contributions to a discussion of issues raised.

Discussion

The follower count and the Twitter reach in terms of RTs demonstrate some very impressive numbers for the AAB account, but the typology shows that the account also enables two-way interactions that are more qualitative, and that indicate meaningful engagements that have impact potential. As Sarah Williamson has shown with the AAB account, it takes a lot of carefully crafted content and time spent online to build followers and to respond to interactions. The AAB account involves very visual content related to building the AAB persona (for example, the AAB modus operandi, ‘small signs, big questions and a fabulous wardrobe’):

The visual aesthetic of the work is important and AAB’s wardrobe is much admired on Twitter. She wears vintage pieces lovingly made for my own Barbie in the 1970s by my mother. At 93 she can longer make the pieces, so my sister has taken up the couture mantle for AAB. Drawing on the perceived Barbie glamour aesthetic, the clothes declare that an interest in style does not preclude feminism or activism. The Barbies are artfully arranged to echo artworks, sometimes in parody but also in homage to female artists or to celebrate the positive representation of women. Curatorial labels, wall texts and statements are questioned, challenged and often mocked. (Williamson, 2020b: n.p.)

Setting up the AAB account with a focus on humour, even if it is often sharp and subversive, has engaged followers in a positive way, even though it is overtly feminist and dealing with often controversial issues (that is, criticism of the composition of museum collections).

The typology offers researchers categories to analyse tweet interactions as indicators of real engagement, examples of how tweets can evidence change that show impact, and a way to tell stories of change and influence. By looking at the context of the interactions, researchers can use these tweet categories as part of a larger story about their activities having impact, and to identify tweets to follow up for more context/clarification on impact. We also argue that the typology is useful beyond the Twitter platform, as it offers analytical categories of interaction that can be applied to other social media platforms.

Conclusion and limitations

With the examples of interactions from the AAB Twitter account, we have demonstrated how Twitter can offer two-way engagement with mutual benefits, although this can only be evidenced for a small proportion of followers. It is perhaps not possible to extrapolate from the analysed interactions to any conclusions about the impact on all the followers, but it does allow researchers a way to demonstrate the value of having a social media profile.

Having a social media account for a project involves a time commitment that not all researchers will be able to make. There is work involved in creating engaging, timely and ongoing content, as well as in responding to interactions, and capturing and filing the data, all of which is time-consuming. There is also the analysis of existing data, even without contacting followers for further information, which represents a time commitment for researchers. All of these activities are necessary to be able to evidence impact (rather than simply engagement).

Declarations and conflicts of interest