Cassandra Wilkinson: on data, bigger isn’t always better


Multi colored computer cables

Policymakers might have more data at their disposal, but do they really know how to use it? What government needs is small data — and program-specific innovation.

“Every politician loves evidence-based policy,” Senator Kim Carr told The Sydney Morning Herald in 2012. “I am told we have called for it, or claimed to have had it, on more than 1800 occasions in the Parliament this year.”

But as I wrote recently, good evidence is available less often than we would like. Into this breach steps Big Data. Always with capitals in case you weren’t sure it was Very Important.

Big Data is going to tell us what we don’t know about the things we need to know before we even know we don’t know it. When we ask silly questions like which clients exactly, within which programs, monitored using what data points held in which agency’s SAAP systems, the evangelists say things like, “2.5 quintillion bytes of data is being generated every day”. They say Information is coming from sensors, machine logs, mobile devices, GPS signals and transactional records and what we need to do is capture the potential of all this information.

The Australian Public Service Big Data Strategy says lots of things like this. It’s full of generic statements about the many large records that exist. Which is interesting in an abstract sort of way, but it reflects the persistent problem that big data is a solution looking for a problem.

FREE membership to The Mandarin

Receive unlimited access, get all the latest public sector news and features, plus The Juice, our daily news update sent direct to your inbox.

The Mandarin is where Australia's public sector leaders discuss their work and the issues faced within modern bureaucracy. Join today to discover the latest in public administration thinking and news from our dedicated reporters, current and former agency heads and senior executives.