Text size: A A A

Data labs: a low-cost way to understand service impact

As a society, we are good at sizing our social problems and reporting on them. For example, we know that there are around 250,000 admissions to hospitals for mental illness each year. But we are not nearly as adept at understanding and reporting on the impact of particular interventions.

At the same time, we collect vast amounts of data, especially in government.

A “data lab” offers a new, low-cost way to improve our understanding of service impact. It enables comparison of the impact of different types of services and providers by using data already collected.

Imagine you are part of an organisation that provides outpatient mental health services. You work hard to deliver your services but are unsure of what happens to your clients. For example, were they readmitted to hospital? And even when you know what happened to your client in key outcome areas such as readmission, you are unsure of the impact of your service on that outcome. Did your service contribute to them staying out of hospital? Or would they have stayed out anyway?

A data lab seeks to answer these questions by generating new insights from existing government data. Specifically, a data lab makes client outcomes visible by providing access to de-identified data on client outcomes. And it helps to compare the impact of a service against no service, by comparing outcomes of a service provider’s clients against a group of similar people.

At the highest level, the objectives of the data lab are to help improve services, and to fund more of what works.

“… data labs provide for the routine production of high-quality, low-cost evaluations of social policies and interventions.”

Currently, even where a service provider can collect the necessary data, services for evaluating causal impact typically cost around $100,000-200,000. By comparison, the UK data lab pilot is providing access to client outcomes and a report on comparative impact for about $5000 per program.

Tris Lumley, director of development at NPC, a key architect of the UK pilot, says data labs provide for the routine production of high-quality, low-cost evaluations of social policies and interventions.

In its first 18 months, the UK Justice Data Lab (or JDL) has provided data access and analysis to 121 service providers, and impacted service improvement priorities. In recognition of the JDL’s benefits, the Ministry of Justice has extended the pilot for a second year.

So, what are the potential applications in Australia?

There are many areas in which a data lab would have value here in Australia, especially where government has identified key outcomes that it would like to improve. For example, the New South Wales 2021 goal areas on crime and offending, preventative health and mental health could benefit from a data lab. At the federal level, a data lab could operate in employment or disability outcomes.

The primary constraint is whether outcomes data is currently collected at the individual level, and with sufficient other variables about individuals to construct comparison groups. These data conditions don’t exist in all areas but do exist with a few high-priority social issues.

The UK pilot has generated significant excitement in the social policy community. The data lab is a powerful tool in the impact measurement landscape and many of the conditions are ripe for the first one here.

More at The Mandarin: SAP’s head in the cloud with $150m government data centre

Author Bio

Olivia Hilton & Tarah Barzanji

Olivia Hilton is executive director of consulting at Social Ventures Australia, and managing dditor of the SVA Consulting Quarterly. Tarah Barzanji was manager, special projects at Social Ventures Australia and holds a Masters of Public Policy from Harvard University.