If you thought that civil servants had a laser focus on data, policy research and evaluation before COVID-19, you’ll have clocked that the months since the start of the pandemic have taken things to a whole new level.
In the UK, the government has continually championed evidence-informed policy in its COVID-19 response. More and more, we’re told that decisions have been “guided by the scientific evidence” and, most recently, that “data, not dates” will decide the path out of lockdown.
It’s quite right that research should be front and centre for the civil service at present. If it wasn’t for biomedical researchers, we wouldn’t have a vaccine. If it wasn’t for statisticians, we wouldn’t understand how the virus has spread. Admittedly, research and evaluation are almost always front and centre for civil servants. Arguably, they come as a second nature in the profession. Proper research and evaluation techniques can help you to predict the impact of your policy proposals and secure buy-in from your department.
As a researcher at Newcastle University, I design, conduct and evaluate research on a daily basis. I recently worked with a British government department to investigate whether digital technologies, such as social media platforms, have the potential to increase the participation of young people in policy development.
While working in government, I came across a few manuals for conducting model evaluations, all seemingly named after different colours (the Magenta, Green and Aqua books are some examples). However, with some manuals running up to 150 pages, getting to grips with these was a daunting task. What was missing was a handy guide for designing, conducting and evaluating research to aid policy development.
The value of research for policy-making is clear. The more informed a proposal, the greater understanding of its impact, the greater chance that policy has of being adopted in the long term. All civil servants could consider running small-scale research projects to enhance their knowledge and skills, whether they have prior experience in research or not. Where then should they begin?
There are common themes to the processes academics use to design and carry out research. For this reason, I’d like to take you through the questions we ask ourselves — questions that all civil servants may wish to ask when considering research for policy development.
Decide what question(s) you want to ask
Think about what interests you in a particular policy area, and distil from this a question or set of questions. This is sometimes the hardest part. Try not to ask too big a question, as you might struggle to see what’s important within it. An example of this might be: ‘What are the challenges for civil servants engaging young people in policy-making?’ Don’t be afraid to ask a narrower version of this question, such as: ‘How could Instagram be best used to engage young people in care policy evaluation?’ There’s always the possibility of conducting follow-up research at a later stage.
What approach is best for carrying out your research?
There are two main types of research: qualitative and quantitative. Broadly speaking, qualitative research deals with individual case studies, words and meanings, while quantitative research deals with numbers and statistics. If you want to show how policies have affected the daily lives of a group of people, qualitative research may be a good way to go. If, on the other hand, you want to show how many people have engaged with a new initiative, you may be better off carrying out quantitative research. It’s important to say here that there’s no strictly right or wrong way, and many researchers combine these approaches.
How will you collect data?
If you’ve decided what you want to ask, and what approach you’re taking to get your answer, you now want to figure out your data collection methods. Different methods suit different approaches. Surveys, for instance, are unlikely to help you do research on historical archives, because surveys involve engaging with people rather than artefacts. Equally, ethnography may not be practical for a health study involving tens of thousands of participants.
The most common methods of research however tend to be surveys, interviews and focus groups. Workshops, observations or repetitive testing may also be useful, depending on your aims. Whatever you choose, make sure you’ve selected methods that allow you to get real answers to your questions, and that fit with your research approach.
Sometimes it doesn’t work out as hoped. In the past, I’ve planned research that depends on my being able to use digital technologies, only to find that the building doesn’t have Wi-Fi, plug points, or that my participants want to do something completely different. It’s important to be responsive to what’s happening in the field, and to have a back-up plan in case things don’t go as planned.
How will you ensure that you’re doing your research right?
How will you recruit people to your study and ensure your sample is representative? How will you ensure you are doing ethical research that fully protects those you’ve recruited? And how will you know whether your questions are the right ones for your research? These are all sensible worries. Running a pilot (sometimes called a ‘dry run’) may help you overcome any potential snags. Be sure to familiarise yourself with your department’s ethical guidelines and policies, and consider reaching out to colleagues or researchers in government to learn about best practice.
How will you check that you got what you needed from your research?
Once you have carried out your research, congratulations are in order. But steady on, because next you’ll need some way of analysing the data you have collected. If you have a lot of numbers to work with, you may want to use graphs and charts to visualise them. If you have a lot of descriptions and experiences, consider methods such as thematic analysis. This encourages you to break down bits of data into short descriptions to see more clearly the highlights in your data. Sometimes your research findings can surprise you. Try not to see this as a problem, but rather as an opportunity to learn more. Consider giving a presentation to your team or visualising your data in a creative way to communicate your findings and gain feedback on next steps.
Your research and evaluation needs might well be different from those suggested above. For example, you may be carrying out clinical research, or be responsible for just one part of a much larger project. However, understanding the process of designing and conducting research will help you to diagnose issues, communicate outputs and ensure your research gets you exactly what you need.
In the age of data-driven decision-making and mistrust of public institutions, getting the right information is more important than ever. I hope this article has given you the confidence to carry out your own research and evaluation project — however big or small — and get the information you need to deliver on your ideas.