Decarbonising existing homes in Wales: a participatory behavioural systems mapping approach

Summary of paper: this paper describes how a systems mapping approach which built an additional behavioural and actor focus into causal systems mapping was used to inform national policy recommendations for making existing homes in Wales more energy-efficient.

Summary of review: The paper is very clear and well-written, and is well embedded in the literature and policy debates. It presents an interesting and ambitious project, which clearly took a lot of hard work and thinking. The findings and discussion are interesting, and there are some honest and refreshing reflections. I felt there was an under-discussion of some key methodological elements, for example there was not enough discussion of whether systems and behavioural approaches are complements or alternatives, and there was no obvious analysis or deeper use of the map beyond creating it and interpreting it in a generic way. Overall, I felt the paper, despite these criticisms, was strong and a welcome addition to the literature in these domains.

Detailed comments:

• Abstract
  • Clear and well-written
  • The abstract could hint more at the connection of behavioural and systems worldviews – it reads as if these are natural companions, but to me it is more intuitive to think of them as alternatives (e.g. many systems thinkers are critical of the narrow focus of behavioural approaches). This comes out in the introduction a bit, but is missing in the abstract.
• Introduction
  • I would ideally like more discussion of the tension between behaviour and systems approaches – the critiques they make of each other, and whether they can fit neatly together. I think they can, but it warrants more consideration.
• Participatory Approaches to Systems Mapping
  • It would be helpful to acknowledge the distinction between complexity approaches and systems thinking here I think. ABM in particular, and arguably network analysis too, come more from a complexity science/theory angle than systems thinking. I think it is fine to lump them together in general, but I would acknowledge this distinction here.
  • There is some unfortunate duplication of the use of the term ‘Participatory Systems Mapping’ in the literature – it would be helpful to explain the difference between Artunes et al and Barbrook-Johnson and Penn more clearly. My understanding is that Artunes et al is causal loop diagrams, whereas Barbrook-Johnson and Penn is more like fuzzy cognitive mapping but using network analysis for analysis, not turning the map into a simulation. The paper skims over these nuances too quickly.
  • It would be helpful to make clearer the real difference between (i) the method used/developed here and (ii) annotated versions of causal loop diagrams or the PSM proposed by Barbrook-Johnson and Penn. In the early text of the paper, it seems like there is a big difference, but it is a little vague, then when you see the maps in results, it looks like an annotated causal loop diagram. More commentary on how this is helpful, but also the issues it introduces would be beneficial too.
• Process
  • Was any analysis of the maps done – how were the maps interpreted with and without stakeholders?
  • The researchers take a lot of control of the process, refining and merging maps, and asking stakeholders to build simple versions. Could you comment on any bias or shifts in focus this brought, if any?
• Discussion
  • Very clear discussion, and nice connection to policy process and behavioural framework.
  • Refreshing and good to see honest reflections in imperfections in the process/method.

Overall I welcome this article’s contribution to reflecting on the practical challenges, and opportunities, in tackling a complex systems problem with an urgent need for change!

The introduction of behavioural systems mapping to the field of retrofit discussion is important, and the use of an underpinning behavioural psychology theory, such as COM-B, adds a rigour which is not often present in this kind of highly applied and practitioner-facing work.

The close attention to the private rented sector is particularly needed, and interesting.

The clarity of thinking that this approach brought to a complex and urgent problem is very welcome.  This is described on page 34 as “explicitly specifying actors behaviours and influences on behaviour ”.  Such clear segmentation and analysis is vital if research understanding is to lead to actual change in practice!

Expanding on the level of validity of the study: I would be interested in more analysis and critique of the composition of the Advisory Group, whose knowledge was core to the maps produced, and therefore to the policy recommendations.  Does the advisory group represent all stakeholders?  Or rather all domains of expertise thought relevant?

Expanding on the level of completeness of the study: I believe that the most significant contribution of the research is methodological, but I think that the case for retrofit as a vital area which demands such methodological innovation would have been strengthened by more reference  to the technical requirements of retrofit (e,g, CCC 2019 “UK Housing: fit for the future”, CLC 2021 “National Retrofit Strategy v2.0) and a recognition that this can be measure by measure, whole house, or over time (e.g. FAWCETT, T. 2013. Exploring the time dimension of low carbon retrofit: owner-occupied housing. Building Research & Information, 42, 477-488.)

While I fully accept that owner occupiers, private landlords and Welsh policy makers are the major actors whose behaviour is being analysed, I think that putting the whole of the supply chain into a single box in the prototype systems diagram, labelled ‘suppliers’ is open to some challenge and could be further refined in future work.  Materials and equipment manufacturers have different behaviours to merchants (e.g. Killip G, Owen A, Topouzi M. 2020. Exploring the practices and roles of UK construction manufacturers and merchants in relation to housing energy retrofit. Journal of Cleaner Production. 251), who in turn have different behaviours to the construction firms working in RMI, which are the main focus of the research cited at the moment.

Other observations on the article

1. The understanding the sociotechnical nature of the retrofit delivery system is set out well.  Might the problem statement be strengthened further by returning to some of the core frameworks in STS – such as the multi-level perspective or the co-evolutionary systems?
2. The participatory mapping methodology is very well described.
3. I strongly support the need for methodological innovation in this area of research/practice, and we therefore need to explore how transferable are new methodologies.  In this case, I am keen to know how are the results obtained using this method validated?  On page 18 a short statement says that the workshop results were validated by two of the authors.  Is this rigorous enough to allow the research findings to be generalizable?  How was this validation undertaken? The advisory group are an expert group; their perspectives and their depth of knowledge are very important.  And yet, to achieve changes in behaviour, other perspectives need to be represented.  This challenge is acknowledged on p37 : “ Several rounds of review and refinement helped us to be confident in the validity of the maps from [the advisory group’s] perspective as stakeholders, although it is possible that the map would look different had different stakeholders been involved in each workshop .” I think this is worth further consideration and testing in future rounds of mapping.
4. I also wondered if we needed a simple definition of the behaviours that would lead to decarbonising homes – and how these behaviours might differ for different actors.  The team has clearly mapped out these behaviours in some detail in Appendix B but they are not described in the body of the paper.  The list in Appendix B1 covers a huge variety of behaviours, with different loci and impacts.  Some categorisation of these, perhaps in terms of how they relate to the prototype system map, might be helpful?  By describing the technical tasks of retrofit, but not the range of behaviours needed, the balance between C, O and M in policy measures risk becoming something of a pick’n’mix menu for policymakers, rather than a coherent and consistent package …  p28 mentions the “desired behaviour change of owner occupiers and private landlords” but this could cover a variety of behaviours including funding retrofit, commissioning retrofit, using a building renovation passport etc.  On p38 then it seems that other organisational behaviours are also within scope.
5. 10 headline recommendations are referred to on page 19, but 26 policy recommendations are identified in the results.  I understand that the 26 were generated by workshops which used the initial 10 headlines as stimulus.  Some critical discussion of the role of these headline recommendations would be valuable.  If a different mix of headline recommendations were used as the stimulus, how might this have changed the longer list generated through workshops?  I think that this might lead to some reflections on the completeness of the set of recommendations developed.
6. I’m also intrigued to see that in both the headline recommendations and the complete list, some recommendations do not connect to C, O or M.  These are lobbying recommendations.  One what basis are they justified if there is not a connection to the desired behaviours?  Or is there a secondary connection that lobbying would create WG opportunities, that would then translate into owner occupier or landlord opportunities?

Thank you for the opportunity to review this innovative work.