“Every politician loves evidence-based policy,” Senator Kim Carr told The Sydney Morning Herald in 2012. “I am told we have called for it, or claimed to have had it, on more than 1800 occasions in the Parliament this year.”
But as I wrote recently, good evidence is available less often than we would like. Into this breach steps Big Data. Always with capitals in case you weren’t sure it was Very Important.
Big Data is going to tell us what we don’t know about the things we need to know before we even know we don’t know it. When we ask silly questions like which clients exactly, within which programs, monitored using what data points held in which agency’s SAAP systems, the evangelists say things like, “2.5 quintillion bytes of data is being generated every day”. They say Information is coming from sensors, machine logs, mobile devices, GPS signals and transactional records and what we need to do is capture the potential of all this information.
The Australian Public Service Big Data Strategy says lots of things like this. It’s full of generic statements about the many large records that exist. Which is interesting in an abstract sort of way, but it reflects the persistent problem that big data is a solution looking for a problem.
That’s not to say there are no problems which may benefit from better or more data. It is simply to say that data only matters once there is a program methodology based on a plausible theory of change.
Data only matters when units of expenditure can be properly coded and tracked from allocation to output. Data only matters once you know what expected effect will indicate your program has delivered a result. When the result occurs — good or bad — it will generate a data point. But you can only know which data point to look for and where it will show up if you did the policy work up front.
Unfortunately, many consultants are selling an idea that the hard and urgent work of improving our program design and delivery methodologies can be superseded by mashing up thousands of data sets to reveal “insights” which usually amount to general correlations like geo-location and socio-economic status which any experienced public servant could have predicted.“We need the money allocated by Treasury to an agency to be coded and tracked as it flows to providers and then on to clients.”
What they usually don’t amount to is information about the performance of specific funding streams, programs and providers. What it doesn’t give an insight into are the measurable changes in the circumstances of clients clearly linked to good or bad public policy.
What we really need is small data. We need the money allocated by Treasury to an agency to be coded and tracked as it flows to providers and then on to clients. We need to be able to map those flows to a complementary set of numbers based on a program methodology. We need outcomes which are expected to be found absent or present in trackable data sets such as attendance at school, entry to hospital or entry into foster care.
Very often even the funky, innovative big data projects that get TED talks and governing features boil down to small-scale innovation. Harvard’s Data-Smart City Solutions is a good clearinghouse of best practice. While the current frontrunners in data are all fascinating, few profiled by Harvard are solving the problems which represent the biggest claims on public balance sheets.
Better data leads to more accessible public records: real-time public transport timetables; predictive modelling for environmental inspections and compliance; better geographical mapping of need to services. These are all useful but none suggest big data will solve our most pressing or most expensive problems.
Despite the massive changes in the sophistication of data, our challenge remains exactly as it always has been — connect the inputs to the outputs and the outputs to the outcomes. It’s hard work and it’s no wonder people are tempted to find a wonderful approach to numbers which can be interrogated for insights that get us out of the hard yakka of small data.
But small data is where big change is needed. And where the biggest gains remain.
More at The Mandarin: Cassandra Wilkinson: NSW leads on evidence-based approach