Developing arts-based methods for exploring virtual reality technologies: A university–industry case study

The need for signed consent from young person and guardian

All activities within the workshop would need to be explained as clearly as possible to students and guardians prior to the event to ensure that they were giving informed consent to their involvement. This placed a limit on the spontaneity of the session plans. However, some leeway was provided to the Kilter team by describing only the ’types’ of activity that would be involved in the workshops, without necessarily tying each activity to a specific single workshop or time period.

The requirement of up-to-date certification for working with under-18 year olds

In the UK, all adults working with children and young people within a school environment must have up-to-date government clearance via a background criminal records check. Several team members did not have this clearance, and the lead time to obtain it can be several weeks. Fortunately, the host school were able to make a dedicated member of their staff available to supervise the sessions. This staff member did not actively participate in the workshop sessions (although they were invited to do so, if they so wished), and monitored the sessions quietly and unobtrusively from the back of the room.

Data collection and storage

University ethical protocol mandates that all data collected during research work must be stored within university-held storage structures by specific (named) university employees. Access to this information is subject to screening, anonymization and the explicit written permission of the people involved. This requirement created a slight tension between the researchers and the Kilter teams in terms of what constituted ’research data’. From the perspective of the researchers, all audio, video and textual materials created during the sessions were ’research data’. From Kilter’s perspective, static ‘snapshot’ photographs taken by them in which faces could not be clearly seen, and materials written by the students that were not clearly attributed to them by name, were not ‘research data’, but simply supplementary materials that could, for example, be published online as a part of a blog detailing the workshops.

To manage this, the teams agreed to keep all materials confidential, and to seek explicit additional permission from the staff and students of the school before publishing any of the audio, visual or textual material generated during the workshops, either online or in print.

Exposing young people to virtual reality technology safely within a classroom environment

During one of our workshops, we provided the students with a short first-hand experience of virtual reality using commercially available VR headsets. For many of the students, this was their first experience of the technology, and for some it was a major draw for their participation in our workshops. At the time of writing, health and safety guidelines for virtual reality are relative sparse, and the effects of exposing younger adults and children to sustained sessions of VR remain unclear. However, the leading manufacturer of VR equipment recommends a minimum age of 13 (Oculus by Facebook, 2021).

From an ethical standpoint, we had to balance allowing the students to engage with the technology and maintaining a safe environment. This we achieved pragmatically by: (1) keeping their exposure to VR relatively short (five minutes); and (2) using a low-intensity VR experience by choosing an experience that was relatively benign and calm – a short immersive film of a coral reef.

In addition, students were supervised and remained seated throughout, and they were told to close their eyes and remove the headset immediately should they feel uncomfortable in any way. In practice, none of our workshop participants experienced any ill effects from their experience.

Quickly breaking down hierarchies: The roundtable discussion

The objectives of our first workshop were twofold: (1) to create an environment within which the students felt comfortable to speak and share (both with each other, and with the research team); and (2) to begin to understand how the students envisioned future societies and the role of VR technologies within them.

As a first activity, the Kilter team directed everybody (the students and the research team) to move from the tiered seating of the classroom to sit in a circle. The students were asked if they would be willing to move their chairs into a space in the centre of the room. The students were given autonomy over who they wished to sit next to, and the researchers also joined the young people in the circle. This would remain standard practice for the following workshops.

Each person was then asked to introduce themselves and generate a word describing their feelings about the future in order to introduce all parties to each other and to promote the notion that every participant had an equal voice. This immediately revealed a wide range of outlooks: optimists and pessimists, those curious and open-minded about future society, and others with subversive ideas of their own. The use of the circular seating arrangement seemed to support a reduction of the sense of hierarchy within the classroom environment, and granted everyone a platform to be heard. For instance, after the students were asked to introduce themselves and share their feelings about the future, it often opened up a dialogue with other members in the circle (with other students, not just the research team), who made their own comments and asked questions about the ideas being shared.

Introduction of methods for participant engagement

Kilter also exhibited the benefits of balancing group-based and individual-based methods. Within the circle, the Kilter team shepherded initial ideas into a group discourse around the future of society through local, national and global lenses. They then anchored the discussion by producing some conversation starters in the form of written predictions about the future that they had brainstormed before the session, before introducing their first novel ideation approach – the ‘washing line’ method. The ‘washing line’ approach was essentially a linear timeline. During the discussions, whenever a new prediction about the future was suggested, the author of the idea was asked to write it down on a Post-it Note and pin it to a chronologically labelled line of string that traversed the breadth of the classroom. The students were then asked to move in physical space to stand at the place on the line that they felt best represented the year that they believed their prediction would become a reality. The activity provided a temporal way for the students to represent their ideas, mapping their predictions for world and personal events from the present day until the end of the century. This was not only an effective way of engaging the students to envision their ideas, but it also served as a visual representation of consensus and discord, which we were able to reflect upon and adjust as a team following the workshops. Moreover, the Kilter team was particularly interested in uncovering ideas that represented the near (and believable) future, to be used in the final production script.

A second technique deployed by Kilter was the use of workshop journals. Each student was provided with their own journal within which to develop creative narratives inspired by previous workshop content. Within their journals, students were repeatedly tasked to reflect upon specific questions and ideas about the future role of VR technologies that were posed during the workshops. From a research perspective, these journals served as an artefact for communicating the students’ ideas about the design and ethics of VR in a future society – in a similar vein to developing a ‘design fiction’. Yet, the individual and insular nature of this activity was also a great lesson in participant management: it enacted a rapid de-escalation in the intensity of the group interactions during prior activities. Moreover, it afforded an opportunity for the less vocal students to contribute without feeling the pressure of direct attention by the whole group.

To close the session, one member of the Kilter team wrote down the concepts raised during the session on a whiteboard, before inviting the students to direct him to write down the ‘things that we discussed today’. The students shouted out concepts as he wrote them in real time. Following this, he asked the student group, ‘What was the most important topic that was raised today?’ As answers were expressed and opinions exchanged between the class members, he asked the students to use a show of hands in order to gauge the most impactful topics.

Workshop 1 outcome summary: Laying down some early roots of inspiration

Guiding the design of the final production, the activities of the first workshop identified several themes that appeared to be of particular concern to our cohort. These themes were developed further during debrief and planning meetings that were held between each workshop. Given the short time between the sessions, analysis of the previous workshop stopped short of a formal, academic qualitative method (such as thematic analysis). Instead, each member of the team brought to the debrief and planning meeting notes based on their experience of the previous workshop and any observation notes that they had taken during the session itself. Each member of the team shared what they believed to be the emerging themes from the workshop, with regard both to the workshop process and to the conceptual themes relevant to VR and ethics. A subset of themes that were noted by multiple members of the team were taken forward for further elaboration within the next workshop, and also as inspiration for the script of the final production.

Central among these was a perception that digital agents – for example, robots and artificial intelligence assistants such as Apple’s Siri – were seen somehow as a de facto solution to various challenges for the planet, yet they also came with several implications that were not necessarily all positive. The students were most vocal at the prospect of such digital agents replacing real human-to-human relationships, and this seemed to divide opinion. Some students were unconvinced that this would happen in the near future, while others disagreed, stating that digital agents were already present and replacing human beings in many industries, such as the education industry. Opinions fragmented even further when the discussion turned to whether such agents would be positive or negative for society.

Furthering participant learning

The second workshop session began with the Kilter team leading a five-minute discussion reflecting on the previous session, with several of the students’ journal ideas being shared as a basis for further ideation. Anonymity was provided to the students during this process; however, they were encouraged to take ownership of the idea if they wished. This activity served to anchor the session in the themes of the previous workshop (the ideas shared were those that had been highlighted during the debrief and planning meeting), as well as fostering group relationship building by allowing the students to build on each other’s thoughts. The students were additionally given an opportunity to share any further thoughts that had not been covered if they wanted to, reinforcing the circle as a place where everyone had a voice. Yet, while the first workshop had helped us to understand how the students might visualize the future and the role of technology within it, we now wished to narrow the scope further towards the role of VR within this world.

To do this, part of the second workshop was dedicated to equipping the students with a depth of knowledge about VR and its history. By teaching them about the past, we hoped to equip them to better understand the present and to imagine the future. Moreover, given that we were working in a school context, we felt it was important that we contribute to student learning in a variety of different methods – experiential and theoretical – in addition to the creative activities from the previous session. Kilter and the research team had originally conceived the delivery of this educational content about VR as a more constructivist activity, with a great deal of student self-determination and socially constructed learning, in line with the other content delivered in the workshops. However, two external pressures impacted our ability to do this. First, there was an extremely limited amount of time available within the workshop to allow for a full and socially constructed method for the students to learn about such a rich set of technical topics within 30 minutes. Second, the school and a number of the attending students interested in STEM subjects intimated a desire to hear the researchers involved talk about their work, and to share their expertise in VR.

Hence, following the recap discussion, a member of the HCI team provided the students with a 30-minute talk about the roots and emergence of VR, as well as about some notable projects that the researchers had undertaken in this space. To allow use of a projector to provide additional visual materials, this was presented with the students sitting in tiered rows, with the content delivered through a combination of visual slides and spoken content. Immediately following the talk, the students were provided with VR headsets to experience a short immersive film of swimming in a coral reef. The objective was to ensure that every student in the class had a baseline of at least one experience of using VR, which they could use as a launchpad for their imaginative design.

For many of the students, this was their first experience of VR. The opportunity to try it out allowed them to cement their understanding of the VR concepts that had been presented through practical experience. Although many of these students had experience of immersive virtual environments through video games, it was the first time that they had been fully encapsulated within a 360-degree environment of this kind. Nevertheless, this did not seem to obviously affect their enthusiasm, as several students from both sides of the STEM/arts subjects divide stayed behind for a few minutes after the session to ask the researchers questions about the technology.

Balancing research and creative goals

Providing an actual VR experience was clearly impactful for the students, yet it also highlighted some differences in prioritization for the HCI and Kilter teams. As there were only six VR headsets available for concurrent use (and only two researchers trained in their administration), tight time management and orchestration were required to keep students engaged while waiting for their turn. To ensure fairness, students selected among themselves the order in which they would take part in the VR experience. Meanwhile, the Kilter team ran a concurrent task where students who were waiting for their turn elaborated upon the future societies described in their journals in order to include a role for VR. While this kept the students engaged, the activities had different goals and were weighted with different importance for the HCI and Kilter teams.

For Kilter, the journal activity was an important part of expanding upon the students’ narrative ideation, providing them with more material to inspire the final immersive theatre performance around the ethics of VR. The HCI team felt that the opportunity to provide the young people with practical experience of VR, which was a selling point of the workshops to the school, and a strong basis of knowledge was equally important. They believed that this was required in order to scope students’ design in the not-too-distant technological future – which could support the students to deliver plausible ideas for future VR technologies and generate the most impactful content for the final production.

In practice, completing both tasks running in parallel placed severe pressure upon the limited time available in the session. Due to the set-up time required, each five-minute VR experience for a group of six students took ten or fifteen minutes to complete, causing an overrun. To complete both tasks, additional time would need to be carved out of the following session to enable the remaining students to undertake their VR experience. This additional logistical pressure was a source of frustration for Kilter, who had been very careful about the format and chaining of activities in order to manage the energy of the students. However, they were understanding, and all parties agreed that any future researcher-led methods in the workshops would be better co-designed with Kilter.

Workshop 2 outcome summary: Providing a practical reference point for virtual reality

Given the emphasis on providing the participants with a grounding in the history and direction of VR technologies during Workshop 2, there were limitations on establishing themes for use in the final production. Although Kilter had deployed their journal activity, relatively little time had been available for any firm narrative ideation. Hence, the next debrief and planning meeting focused on understanding the students’ perception of their VR experience, and thinking about how this could be used to inform further creative ideation activities.

All of the students had expressed positive feelings about their VR experience, but with seemingly little immediate concern about the potential ethical implications of the technology (at least in its current form). On first glance, it did not appear that VR caused the same trepidation for them as other technologies (such as digital agents) mentioned in the first workshop. Given the brief and somewhat benign VR experience that we offered, this was not entirely surprising. Nevertheless, one particular idea began to take shape – the use of VR to allow people to experience places and activities that they could not experience in real life – and this was taken through to the final workshop session.

Describing the role-play ideation method

Our final workshop built upon the students’ first-hand experience of using VR from the previous session, and reoriented their focus towards considering what (if any) ethical implications they felt it raised. The goal of the session was for the students to ideate their own application for VR, before using peer review and reflection to identify any ethical issues surrounding their application.

The students began by undertaking a refresher discussion about their VR experience in the circle, as the subgroup of students who were yet to undertake their VR experience were given the opportunity to do so. After the subgroup had finished their VR experience, they were invited back into the circle, where they were given a range of topics representing different societal lenses on A4 sheets of paper, which they would use as a basis for their initial VR design ideas. These topics had been curated by Kilter during previous projects, and included law and order, the environment, health care and history. The participants formed small groups depending on their area of interest, and they were asked to brainstorm their idea for a VR application, and then to elaborate upon their idea in a brief presentation to the rest of the group.

To facilitate their presentation, the Kilter team employed role play, with one member adopting the persona of a news anchor, and the other playing a ‘roving reporter’ who was presenting a news story about a VR start-up convention from the studio, before visiting ‘tech start-ups’ (that is, the student teams) ‘on the ground’. The ‘reporter’ interviewed each team in turn, guiding the students towards explaining their ideas with semi-structured questions. Noting the increasing tendency towards student-to-student interaction during the role play, the Kilter team later extended the activity further. They took votes from all the students on their favourite idea, with the teams that had conceived the two leading ideas then invited to take part in a ‘press conference’. Here, they fielded questions from the other students, who took on the role of journalists. The goal was to allow for a more student-led, expanded ideation of the created concepts and any ethical concerns.

Using role play to engage

As professional dramatists, the Kilter team were skilled at adopting convincing personas quickly and creating an entertaining narrative that would have been difficult for the researchers to have undertaken without considerable training and practice. The researchers observed the ability of the Kilter team to focus attention, provoke discussion and evoke audible and visible emotion from the students during the role play. For instance, the role play was accompanied by audible laughter from many of the students, while individual students began to mirror the Kilter team by appearing to adopt their own characters or amplified personas of themselves. Furthermore, students began engaging each other while in character, and attempted to break into role-play interactions by making comments and addressing the Kilter team from across the room.

Presenting ideas in the classroom in front of their peers can sometimes be intimidating for young people. This can be caused by a lack of confidence or self-efficacy with regard to the topic or idea being discussed, the social consequences of being incorrect and being subject to mockery, and personality and cultural factors, as well as teaching-centred learning environments and intimidating power imbalances between students and their teachers (Zhouyuan, 2016). Hence, Kilter’s use of spontaneous role play attempted to make the process informal, inclusive, engaging, and perceived as low risk for the students to actively participate. Key to this was the role of the reporter, who – as part of ‘the act’ – structured and supported the students’ explanations of their idea. This was designed to support students’ confidence, and to help them to improvise or play a role of their own. At the same time, the ‘press conference’ format allowed ideas to emerge quickly and to be explored in further depth for feasibility, without alienating or discouraging other participants by giving them a role to play. Through playing a role, students were granted deniability for any ideas that may have missed the mark – that is, such ideas could be attributed to the character, rather than the individual – something from which many ‘shy’ actors have benefited (Scott, 2017).

Workshop 3 outcome summary: New applications for virtual reality, and a greater understanding of the ethics of virtual reality technology

The role-play activity within Workshop 3 created a diverse range of ideas for future VR applications. These included the use of VR headsets to control environmentally friendly robots, to relive historical events for educational purposes, to simulate the physical and judicial consequences of carrying knives, and to extend the lives of terminally ill people. The students cast an even number of votes for the latter two ideas, with the press conference generating several ethical concerns. Notable among these was the risk that a VR experience of knife crime could be ‘triggering’ for those experiencing it – both for those experiencing these events for the first time within virtual reality, and for those who had been previously directly or indirectly involved with knife crime in real life.

The second idea, ‘Bucketlist VR’, described a VR-based system for extending the lives of terminally ill people. This became a major influence upon Kilter’s final production. Bucketlist VR was conceived as a broadly positive vision of a VR company that enabled terminally ill hospital patients to live their lives beyond their physical confines, doing anything that they wished to do and being able to spend more time with family members in a virtually constructed alternative environment to a hospital. However, the idea also raised a number of ethical issues for the students, including the notion of people preferring to live their lives in VR rather than their actual reality, and whether such an application would negatively (or indeed positively) affect the process of grief and grieving.

Understanding structural and procedural norms can be addressed through dialogue, but bridging cultural differences may require organizational immersion

Taking time to gain an understanding of each partnering organization’s working practices is requisite to the success of academic–creative industry collaborations, but this process took a great deal more time than we anticipated. Part of this was a result of the difference in size of the organizations, and the way in which time and resources are accessed. Kilter, for example, is a small–medium enterprise with a flat power hierarchy (Boardman and Bozeman, 2015). While many university-based HCI teams ostensibly hold a similar power structure (small groups of people of equal rank who are brought together from various areas of study, such as computer science), they are often subsumed by more complex hierarchies on which they must rely for key activities (such as resource procurement, ethics, finance and communication channels) (Schofield, 2013).

Consequently, within our collaboration, although the HCI team held a similar degree of autonomy to Kilter over the planning of collaboration activities, the implementation of these plans could not be undertaken with as much flexibility – not least due to a requirement for clearance by the university ethics committee for all work undertaken by its researchers. Echoing Rybnicek and Königsgruber’s (2019) model, we therefore suggest that it is vital that time is taken prior to beginning a university–industry partnership to understand each organization’s structure, as well as their logical, procedural and ethical rules and regulations. Reflecting upon our collaboration, a great deal of time early on was spent bringing Kilter up to speed on wider university structures and procedures in order to set realistic expectations, particularly with regard to university requirements around research ethics and data storage.

While this exchange of knowledge helped to bridge structural and procedural differences, fundamental diversions in organizational culture were harder to unroot. In particular, finding a balance between Kilter’s desire for spontaneity – to be able to respond quickly to the ‘mood of the room’ – and the HCI team’s desire to maintain a controlled analytical approach created friction between the HCI and Kilter teams. The ability to engage in spontaneous improvisation during workshop sessions was seen as a useful tool by Kilter to engage audiences, but, while the researchers could also see the potential value of this, they were anxious about the degree to which such unplanned activity impacted the informed consent of the participants.

With hindsight, and with more time, these frictions might have been eased with early, focused immersion in each other’s organizations. Ideally, both partners should be afforded first-hand experience of the other’s working practices and norms. During our collaboration, for example, all meetings were held at the university, and far too little time was afforded for building trust and experience around what ‘spontaneous’ activities actually entailed in practice.

Mutual openness to change can foster shared culture, but adherence to rigid research ethical procedures should be accepted early on as a university ‘red line’

The requirement for a rigorous and accountable ethical framework is culturally, procedurally and structurally inexpugnable for HCI researchers whose work involves members of the public. It is also critical, but sometimes forgotten, that ethical review by the host university serves to protect not only the participants, but also the researchers themselves. For the uninitiated, however, the rigour of the process involved in obtaining ethical clearance for researchers to conduct work outside a university can sometimes feel overzealous, if it is not sufficiently well explained. Hence, the importance of early planning for ethical review, and a realistic shared understanding of the time required for the process to be completed, should be communicated as early as possible.

In retrospect, a mistake was made in not involving Kilter more directly in the process of obtaining ethical clearance to conduct the workshops. As a result, the goals of ethical approval became somewhat abstracted, with confusion over what was and was not ‘research’, and what could and could not be done within the workshop sessions. This could easily have been avoided.

Collaborators should aim to progress from being multidisciplinary to interdisciplinary, but should also expect this process to be asymmetrical

Knowledge exchange partnerships are frequently a multidisciplinary endeavour, with experts from different fields approaching a shared problem through different lenses. As described in previous literature, frequent communication through a variety of channels (for example, face to face, through demonstrations and by email) is the basis for developing mutual understanding and common language (Hong et al., 2010; Lee, 2011; Rybnicek and Königsgruber, 2019).

In our collaboration, we found the process of knowledge exchange to be somewhat asymmetrical. Particularly in the early stages of the collaboration, knowledge exchange between the teams did not progress in a collective, linear trajectory. There were periods of the project where the balance of learning tipped towards one party, before later tipping back. For instance, our early knowledge exchange leaned heavily on the academic side of the collaboration to educate Kilter about VR, only later switching to Kilter upskilling the HCI team in immersive theatre when planning for the participatory workshops began.

Project leadership followed a similar pattern, with each team leading at different project stages and on different issues. Despite this, the gap in both expertise and leadership responsibilities soon began to narrow, and, as the collaboration evolved, it opened uncharted challenges and opportunities which necessitated a truly interdisciplinary approach. For example, the script ‘read through’ meeting for the VR100 performance became a setting for academic debate about which is the most prominent ethical issue of VR, as well as how it could be portrayed within the production, with opinions voiced on all issues by both Kilter and HCI teams. In this way, both knowledge and responsibility became shared over time. Thus, we posit that in HCI collaborations, partners should attempt to become more interdisciplinary – developing expertise in each other’s fields – rather than multidisciplinary – only bringing their own expertise. Interdisciplinarity is key to fostering innovation (Fontana and Corsatea, 2013) – the consideration of phenomena from new perspectives, or by using new tools to study them.

Even with maximum transparency, different stakeholder objectives can unexpectedly conflict

Rybnicek and Königsgruber (2019) highlight that collaborations in which individual (or groups of) team members intentionally prioritize their own agendas can lead to tension and a lack of trust. To mitigate this in our collaboration, time was taken to openly discuss and document individual, team and shared objectives prior to the project application – to ascertain how the project would be mutually beneficial. Despite this, however, some differences of perspective and external pressures were not immediately identified, and were hard to completely eliminate once the collaboration was under way.

For example, both teams shared the desire to raise public awareness regarding the ethics of VR. However, opinions on the means by which public awareness might be measured, and what constituted ‘value’ and ‘success’ in terms of project outputs, differed. For the most part, these differences in opinion coexisted without incident, although they sometimes and unexpectedly came into direct competition. A salient example arose during the workshops, when Kilter saw greater value in being able to undertake a more flexible approach that allowed for spontaneous improvisation, whereas the HCI researchers wanted to stick to a predefined schedule of activities that could be validated against ethical commitments and modelled in order to document the process in academic literature. As described previously, compromise needed to be made, and joint planning sessions before each workshop became an important forum to allow each team to speak candidly about their concerns, and to come to an agreed plan.

Within our collaboration, none of these conflicts were sufficiently severe as to derail the process of completing the necessary work. However, resolving them required additional meetings, which consumed valuable project time and relied heavily on the goodwill of both parties. Hence, we advocate for honest, transparent dialogue early in the project to establish where project objectives and measures of success converge and separate (Barnes et al., 2002). Collaborators should also consider putting in place a protocol to handle any unforeseen conflicts of interest, and being prepared to negotiate suitable compromises – for example: who is in charge of negotiations for each collaborator, and who else is involved, and what is the venue for negotiations?

What happens if collaborators cannot come to an agreement? Which factors (for example, financial or ethical) should be considered or prioritized during compromise? Most importantly, collaborators should describe and clearly define their own visions for achieving shared project goals (the inputs, methods and outputs) at the earliest convenience.

Developing a shared language can be complex and littered with assumptions

The development of shared language can help to bridge culture gaps between academia and industry, and also between individual disciplines (Canhoto et al., 2016). However, this process is not always straightforward, and assumed meanings can and do persist. As a consequence of the long-standing influence of the arts upon HCI, there was considerable overlap already with the terminology used by the Kilter team. However, nuanced differences in specific definitions were magnified in practice. A salient example of this in our collaboration involved both teams’ commitment to following a process of ‘co-design’ with the participants in our workshops. For Kilter, ‘co-design’ meant supporting the workshop participants to think about VR and its implications in a socially constructed way, with an emphasis on helping them to garner skills in creative narrative development and role play. Ultimately, however, Kilter would retain full control and ownership of the output of this exercise – the script of the VR100 performance – which would be written by them, but which would be ‘inspired’ by the workshop participants. For the researchers, however, ‘co-design’ was assumed to be analogous to ‘participatory design’ – a process that involved a much stronger emphasis on participant empowerment, shared ownership and continuous engagement in order to bolster the learning value of participatory processes. The HCI designers had imagined the workshop participants taking a role somewhere between ‘design partners’ and ‘design informants’ (Druin, 2002), empowered to share opinions, to give feedback, and to make key decisions within both the workshops and the final production process, which could be validated and supported by the HCI and Kilter teams. Literature suggests that such roles offer young people much greater opportunity to experience autonomy and empowerment, as opposed to feedback-only roles akin to ‘testers’ (Druin, 2002). In allowing this assumption to persist untested, the HCI team had envisaged the young people taking greater ownership of the process, for example, by directly helping to write the script, and perhaps by acting in the final production. In retrospect, a shared definition of co-design should have been more clearly established before the workshops took place.