Don Nutbeam: more paths to evidence-informed policymaking

By Don Nutbeam

Friday May 20, 2016

Nicholas Gruen’s recent two-part series on the problems with the use of evidence in policymaking, and with policy evaluation offered a thought-provoking interpretation of the challenges involved in effective policymaking today. However, rather than creating another arm of bureaucracy devoted solely to evaluation as he suggests, I would argue that improving the dialogue between policymakers and researchers is a more effective solution with longer term benefits.

It’s true, as he argues, that there are cultural differences between the worlds of research and policy that create barriers to policymakers making better use of research.

“Researchers care deeply about making a difference, but too often encounter barriers to communication with policymakers … ”

Too often, researchers claim their work is ignored without making an effort to understand that policymaking is complex, often messy, and driven by many considerations, of which available research findings are just one. And it’s true that there are some researchers so focussed on methodological purity that they run the risk of ending up with a perfect answer to the wrong question, often years later than the answer was actually needed.

But it’s too simplistic to suggest that researchers are a vain lot who care solely about displaying their academic prowess and who are indifferent to whether their work has impact. In general, they care deeply about their ability to make a difference, but too often they encounter barriers to communication with policymakers, and respond to career incentives that take them away from the practical application of their research.

Policy develops and changes on the basis of underlying beliefs about the cause of a problem and the potential effects of an intervention, as well as consideration of the social and political context in which action is taken.

At its simplest this means that the use of evidence is set alongside consideration of what is popular or at least acceptable to the population that is affected by a policy. Policy development and change is also significantly directed by the power and influence of the competing interests of “policy influencers”. Who wins, who loses, who will fight and who will compromise are significant determinants of this process. Consider for example current debates about reducing sugar consumption as a response to epidemics of overweight and obesity in many countries.

It follows that policymaking is rarely an “event”, or even an explicit set of decisions derived from an appraisal of evidence and following a pre-planned course. Policy tends to evolve through an iterative process, subject to continuous review and incremental change. Policymaking is an inherently “political” process, and the timing of decisions is usually dictated as much by political considerations as the state of the evidence.

In this context, policy change depends upon a point-in-time appraisal of what is scientifically plausible (evidence-based); what is politically acceptable or achievable; and what is practical for implementation. It follows that policy is more likely to be influenced by research evidence if it is available and accessible at the time it is needed; it is presented in a way that is sensitive to the political and social context; and the evidence points to actions for which powers and resources are (or could be) available, and the systems, structures and capacity for action exist.

Can researchers meet this brief? Gruen suggests not — but they can and they do, though not as much as they might. Those who do it well work effectively with policymakers who bring political experience and practical insights to the table and who, once engaged, are more likely to be invested the outcomes.

Many more benefits would flow if more researchers worked with policymakers to understand more clearly the type of questions that need answering, and to deploy the research methods that deliver the best possible answers to the questions of greatest public importance. This is more likely to happen if the incentives are right.

At present, our research grant funding system, especially in health and medicine, favours narrowly defined, methodologically pure applications. One unintended consequence is that many research questions of great public policy significance remain unanswered – we have become masters of learning more and more about less and less.  In such a system it is especially difficult to win funding for “real world”, complex and technically challenging intervention evaluation.

Impact assessment — for programs, funded research

In the UK, the introduction of “impact assessment” as a part of the regular national assessment of university research performance has positively influenced the dialogue between researchers and policymakers. With substantial resources and public reputation at stake, universities set about systematically understanding and describing the impact of their research on society and the economy, and in working with partners outside of the university to provide the evidence required to validate impact.

The outcomes were impressive. Universities across the UK provided well written case studies of social, health and economic impact. These are all now accessible and searchable online. With such a powerful advocacy tool it is unsurprising that the most recent UK government Public Spending Review saw continuing protection of national research spending at a time when almost all other government budgets were subject to significant reductions.

In Australia we also need to embrace the importance of research impact assessment. It will help optimise the usefulness of research to the policy community and support our advocacy for continued substantial taxpayer investment in publicly funded research. It may also help to reset the incentives for undertaking more complex and messy research that addresses complex and messy questions of great public importance.

The challenge of finding ways to ensure that evidence forms part of an inherently fluid political decision-making process is both a responsibility of those who generate evidence and those who use it. For those who generate evidence and those who wish to see it used, the key is to provide timely access to information, and to employ improved techniques for communicating and managing the inevitable uncertainties that arise through scientific research.

Knowledge brokers working for bridge building organisations such as the Sax Institute where I work play an important and nuanced role in identifying policymakers’ needs, helping to define research questions and design evaluations. They can also create productive linkages between policymakers and researchers. We should be aiming for more of these links and better dialogue between policy and research — not creating new barriers. Both policymaking and relevant, impactful research depend on it.

About the author
Newest Most Voted
Inline Feedbacks
View all comments
Peter Graves
Peter Graves
4 years ago

Thanks Don – always worthwhile being reminded about the benefits of knowledge-broking.

Can I comment on this thought of yours:
“However, rather than creating another arm of bureaucracy devoted solely to evaluation as he suggests, I would argue that improving the dialogue between policymakers and researchers is a more effective solution with longer term benefits”.

There is quite a difference between the two. Evaluation looks at what has worked, or what outcomes have been produced from (Government) programs. These two do not always coincide.

Knowledge broking surely is providing the research evidence needed for policy-making – a future prospect.

However, if you do mean evaluation research providing such evidence and having an influence on existing policy, then much better.

Don Nutbeam
Don Nutbeam
4 years ago
Reply to  Peter Graves

Thanks for your comment Peter. I agree and fully understand that knowledge transfer and evaluation are two quite distinct tasks. There is and will continue to be an important role for knowledge-broking as a way to increase policy makers’ access to and use of research to support policy development. We also work with colleagues in the formation of systematic approaches to policy evaluation and in undertaking evaluations, particularly of more innovative programs and pilot work. We see this as a logical extension of the knowledge-broking continuum, and as a further point of access for policy-makers to the scientific expertise that exists in our universities and institutes. As with more conventional knowledge transfer this evaluation support needs to be provided with a solid real-world understanding of the policy process, policy constraints and policy landscape if it going to be a useful tool to aid decision making.

Peter Graves
Peter Graves
4 years ago
Reply to  Don Nutbeam

Thanks Don, for that further background.

Your agency The SAX Institute does excellent work, but in the field of evidence for health policy. As does Professor John Lavis at McMaster University,

At one stage, the entire Australian Public Service was enjoined to undertake evidence-based policy, which suggests your comments (and those of Nicholas Gruen about an Evaluator-General) could merit a wider audience than health (as important as THAT is).

You ended up by suggesting “We should be aiming for more of these links and better dialogue between policy and research”, which I’m aware you and Professor Redmond undertake with the NHMRC.

Bigger question: how might your thoughts gain traction across the Australian Public Service ?

roger dennis scott
roger dennis scott
4 years ago

This is a valuable early contribution to a debate which is just starting inside universities. Professor Dunleavy from the LSE and visitor to the University of Canberra recently visited Queensland and placed this item on the local agenda, especially the need to respond to the Watt report. Think tanks sit at one end of a continuum in valuing academic input into public policy discussions; there are still plenty of senior and tenured staff members of universities who sit at the other end of the continuum and defend their right to ignore every other audience apart from their colleagues.

The essential resource for effective public sector leaders