Lessons from working across fields to develop a framework for informed choices

Interdisciplinarity and informed choices

The Informed Health Choices (IHC) project team has developed key concepts for thinking critically about health-care claims, research and choices (Chalmers et al., 2018). In part because it is hard to change adults’ behaviours, they have focused largely on working with young people (Semakula et al., 2020). This reflects a general trend in this kind of work towards initiatives among schoolchildren or undergraduates (Oxman et al., 2021). The challenge, however, is not one that applies only in the health-care field, nor only to young people. Claims are made in the public arena every day about topics such as nutrition, ways of guaranteeing success in examinations, and how to make money quickly. Unfortunately, people often fail to question such claims, that is, to ask for or question supporting evidence. In recognition of the commonality of the problem of ensuring that decisions are informed by the best available evidence, and in the belief that solutions to our common problems are better sought together, the authors volunteered to join a cross-field group which convened in late 2018. Our aim was to develop a generic framework for assessing claims and supporting informed choices. The group included 24 researchers from a wide range of disciplines (see Table 1). What we had in common was that we all work to support the use of evidence in decision making in our respective fields.

This paper shares lessons of interest to researchers and others working in cross-disciplinary collaborations. Our aim in sharing our observations is to engage further with what we have learned and identify areas for improvement to enable greater trans-disciplinary and cross-field work in the future.

The extent and nature of agreement about the problems we were trying to solve

On the first day, we focused on understanding the work to date by those who had convened the meeting (the IHC project), and to develop shared understanding of the value of a cross-field framework. All participants acknowledged early on that our shared purpose centred on helping individuals and groups make well-informed decisions about interventions, primarily by improving the quality of communication of scientific information. We quickly recognised the value of working together, and the need to enable individuals to think critically about claims. At no point was the goal of developing a shared framework to support the use of evidence for informed choices challenged.

Despite the breadth of backgrounds and experiences within the group, several of us had taken part in cross-disciplinary work previously, and we self-selected to an extent in accepting the invitation to take part in this initiative. We were therefore each already convinced about the value of working across traditional boundaries. Discussions at the meeting were rich, varied and wide ranging. We agreed that across all fields, choices (when options are considered) and decisions (when one action is chosen) are wide ranging and differ from one another in degree, scale and nature. Discussions across fields about the need for a common framework helped us to think more broadly about the nature of the informed choices people make, and the kinds of claims with which they are faced. We learned from both our commonalities and our differences.

We also recognised the need to consider all levels of decision makers, from individuals making choices for themselves and their families, to health-care professionals and government policymakers making choices on behalf of individuals and whole populations. We recognised the extent to which public policy in its broadest sense (that is, policy that affects the public, whatever its source) is often shaped by a wide range of considerations. This included discussion of how policymakers understand public debate. We acknowledged that their decisions are shaped by perceived public support, official considerations, and a complex interaction between what the evidence suggests and how much policymakers are willing to risk political failure in following the evidence. We also perceived barriers for researchers/scientists in whether they inform policy, and how that occurs. We discussed the role that misinformation can play, and agreed on the importance of tackling it at all levels.

Discussions across fields helped us to see how the framework we were developing could be useful beyond health choices. For example, we discussed the strength of the available evidence and agreed that small-scale studies, which are common across disciplines, are part of the problem behind many misleading claims. We found examples from each of our fields, for example, a systematic review of management research that found a predominance of small studies with low internal validity (Barends et al., 2014).

The commonalities across fields not only related to experiences and ideas, but also to the degree of applicability of the framework across those fields. We all agreed that a mechanistic hypothesis, although important, is not on its own enough to justify implementation of an intervention until it has been tested. A key observation was that interventions, implemented in good faith based on a theory that something should work, can lead to wasted resources and even to harm. We did nevertheless observe that the development of theory, often using tools such as logic models, is a common precursor to any experiment, with differing levels of importance in different fields.

We observed that informed choices are not made in a power vacuum. The power and influence behind some ideas are forceful. For example, microloans to people living in poverty, with the intention of supporting them to generate income through small businesses, are usually provided at very high interest rates, and an overwhelming body of evidence suggests that this does more harm than good. Nevertheless, the desire to find solutions for poverty makes the intervention hugely attractive and very popular. Muhammad Yunus – considered by many as the founding father of microfinance – was awarded the Nobel Peace Prize in 2006. It is difficult to challenge accepted truths and change norms.

Other examples include the lack of global support for electric cars (Paine et al., 2006), the decades before the harms of smoking were accepted (Warner, 1991), and the limited action against the sugary drinks industry (Public Health England, 2019). Sometimes those with influence, whether big pharma, big donors or political parties, put their weight and their money behind an intervention, and it takes a lot to counter this. Most recently, we have observed persistence of the belief that hydroxychloroquine is an effective treatment for COVID-19, despite high-quality evidence to the contrary, following support for it from US President Donald Trump (Khuroo et al., 2020). A few public initiatives specifically challenge accepted truths, including Ask for Evidence (www.askforevidence.org). We agreed that all decision makers, including at national and international levels, have multiple factors to balance in decision making.

We found common ground when considering the important roles that the media (including social media) play in influencing the choices that people make, and explored how to counter the power of misinformation in all forms of media. Another shared challenge related to the power of commercial interests, and how well-resourced (but not necessarily evidence-based) organisations can use their financial weight to drive policies and interventions that are extremely difficult to challenge without considerable funding and effort.

The potential value of a generic framework of key concepts to inform choices was agreed, as reflected in a willingness to press on with understanding applications in different fields, and implications for the detailed sections of the framework (see more below).

How we identified and prioritised common problems for inclusion in the framework

When we considered adapting the original Informed Choices tool, which was developed for decisions in health care, into a framework for use across different fields, one of the main topics we discussed related to wording, the use of particular terminology and semantics, as opposed to differences in the concepts themselves. Colleagues in different disciplines use different jargon to characterise studies that focus on comparisons – randomised clinical trials, randomised social experiments, cluster-randomised trials, place-randomised trials. For example, in most disciplines, a randomised trial is referred to as an ‘experiment’, but in the domain of management, an experiment means ‘trying something out to see what happens’, and is a synonym for ‘pilot’. Nevertheless, it was understood that the phrases embody a common idea, that of fair comparisons in estimating outcomes. In another example, colleagues in each discipline alluded in varied ways to the tentativeness of evidence. We agreed that this concept of uncertainty is fundamental, and a reason for doing more than one study and for learning from mistakes.

Once we had aired, discussed and recognised the subtle differences in how we use words such as ‘trial’, ‘experiment’, ‘decision’, ‘policy’ and ‘harm’, we moved on to discussing the framework itself. This helped us to identify common factors that shape decisions, which in turn were included in the new cross-field framework.

We observed that across all fields, decision making involves trade-offs at several levels, balancing multiple benefits, harms and costs. We learned that in some fields it is routine to consider harms and costs (medicine, dentistry, veterinary medicine and international development, for example). In other fields, benefits, harms and costs might take longer to become apparent, and harms may not even be considered, for example, in education (Zhao, 2017). With some decisions it is necessary to balance benefits, harms and costs across multiple people, groups or populations of other species, for example, in environmental management. In some fields, such as planetary health, some people for whom this balance must be considered may not even have been born yet. Costs vary enormously across fields. They include both fiscal costs and opportunity costs, and they can be immediate or long term. In agriculture, for example, intensification through increased use of pesticides and/or fertilisers may meet increasing food production requirements, but can cause harm in the wider environment. In the longer term, there may also be cost implications, particularly to future generations, as damage to natural support services, such as soils or pollinators, can have economic consequences.

Opportunity costs vary according to context: what we have to give up in order to adopt a course of action can vary for each individual, and from group to group and place to place. In resource-poor settings, the opportunity cost of seemingly trivial actions can be enormous. Furthermore, we recognised that across fields, decisions can be made at many levels (personal, for family or friends, in a school or hospital, in a city, regionally, nationally or internationally), and the distribution of harms, benefits and costs varies with context. Who gains and who loses can sometimes be as important as maximising an outcome, if not more so. In evidence-based management, this is known as ‘stakeholder evidence’ (yet another term which means different things in different fields) – evidence that allows analysis of impact on different groups and introduces an ethical dimension. The need for fair comparisons to enable balancing of options cuts across fields and levels of decision being made.

Language commonly generates tension in all fields; for example, the contrast between warning language and neutral statements – ‘Beware false news’ versus ‘This is what an evidence-based claim looks like’. We agreed that the framework should contain statements that are as neutral as possible about the value of evidence, which could be adapted in different fields and for different uses. For example, the framework states: ‘Comparisons of interventions should be fair’ (Aronson et al., 2019: 304), rather than ‘Beware comparisons that are unfair’.

We developed the option for each field of enhancing the framework and sharing additional field-specific information. We agreed that claims are not made in a vacuum, and we discussed the importance of ethical issues. However, we decided not to include them in the framework, because they can be context specific. We agreed that irrespective of the field, there is a need for relevant examples of what happens when decision making is not evidence based, so that people will take notice. These have been included in field-specific sections of the project website (www.thatsaclaim.org).

The breadth of applicability of the framework

While we discussed how we might adapt the framework for different fields, there was never any suggestion that the framework would not be useful in any specific field. Rather, there was a lot of common ground across fields. Participants described cross-field work as a pleasure, reflecting a new way of working without disciplinary boundaries, engendering optimism. We felt reassured that we all speak the same broad language, and that the principles of evidence-informed decision making (and thus the framework that we developed) apply across all fields. To date, the project website includes field-specific enhancements for 8 of the 14 fields involved in the development of the framework (www.thatsaclaim.org).

Finally, there was a shared recognition that the framework is not an endpoint for any of our fields. Implementation of decisions is an essential aspect of informed choices, and is often lacking, whether we are discussing implementation of individual or policy choices. Policy choices in any field require implementation.

Research ethics statement

The authors conducted the research reported in this article in accordance with the ethical code of the University of Johannesburg where the first author is based.

Consent for publication statement

The author declares that research participants’ informed consent to publication of findings – including photos, videos and any personal or identifiable information – was secured prior to publication.

Further reading

Those interested in reading more about approaches for cross-field research might find the following of interest.

On the various perspectives that different groups bring to research, and why one perspective is not enough:

Crowe, S., Fenton, M., Hall, M., Cowan, K. and Chalmers, I. (2015) ‘Patients’, clinicians’ and the research communities’ priorities for treatment research: There is an important mismatch’. Research Involvement and Engagement, 1, 2. https://doi.org/10.1186/s40900-015-0003-x.

Morris, S. and Stevenson, O. (2021) ‘The dynamics of working at intersections: Reflections from exploring inequalities’. Research for All, 5 (2), 356–65. https://doi.org/10.14324/RFA.05.2.11.

On unpacking the involvement of different publics in research:

Oliver, S., Liabo, K., Stewart, R. and Rees, R. (2015) ‘Public involvement in research: Making sense of the diversity’. Journal of Health Services Research & Policy, 20 (1), 45–51. https://doi.org/10.1177/1355819614551848.

On the importance of titles and language when working with those outside your immediate professional circle:

Stewart, R., Dayal, H. and Langer, L. (2017) ‘Terminology and tensions within evidence-informed decision-making in South Africa over a 15-year period’. Research for All, 1 (2), 252–64. https://doi.org/10.18546/RFA.01.2.03.

On how unusual it has been for cross-field working to take place in some areas of the evidence approach:

Stewart, R. and Oliver, S. (2006) ‘Reviewing the potential for critical appraisal training to cater for professional practice’. Medical Teacher, 28 (2), e74–e79. https://doi.org/10.1080/01421590600617509.

On critical thinking in health care and education, and opportunities for collaboration:

Sharples, J.M., Oxman, A.D., Mahtani, K.R., Chalmers, I., Oliver, S., Collins, K., Austvoll-Dahlgren, A. and Hoffmann, T. (2017) ‘Critical thinking in healthcare and education’. BMJ, 357, j2234. https://doi.org/10.1136/bmj.j2234.

On the use of tools in interdisciplinary research in drug discovery:

Haldeman, M., Vieira, B., Winer, F. and Knutsen, L.J. (2005) ‘Exploration tools for drug discovery and beyond: Applying SciFinder^® to interdisciplinary research’. Current Drug Discovery Technologies, 2 (2), 69–74. https://doi.org/10.2174/1570163054064693.