Data integration is giving government a clearer understanding of everything from Indigenous life expectancy to commuting, says Australian Statistician David Kalisch — but cuts threaten data quality.
Governments are increasingly able to match up varied datasets to create a more detailed picture of a range of important indicators across the country.
This allows better decision making but also has the potential to reshape how government programs are evaluated, says David Kalisch, head of the Australian Bureau of Statistics.
“National statistical agencies across the world are combining data to deliver new information. Improved technology and statistical techniques are now enabling safe and efficient use of very large amalgamated data sets,” he told a Committee for the Economic Development of Australia event in Adelaide earlier this week.
“ABS has been developing its data integration expertise and capability now for over a decade. We worked across government to develop person- and business-centred integrated data.
“Data integration is already delivering new statistical insights such as improved estimates of Indigenous life expectancy, understanding outcomes for respective migrant cohorts and the importance of SMEs for job creation.
“Data integration has enabled the government to progress needs-based school funding using reliable information on parental income rather than the previous method, based on the geographic location of the school. 2016 census data combined with road network information is providing accurate information on journeys to work,” said the ABS head.
There is still “considerable untapped potential” for Australian governments to use data integration to assess the performance of their own policies and programs, enabling them to target funding to programs that are assessed as more likely to be effective. This already happens in New Zealand.
The government is working to improve Australia’s data infrastructure and integration capabilities, Kalisch noted, with the ABS contributing to the establishment of the Data Integration Partnership for Australia (DIPA).
“Australia does not have a strong tradition of rigorously evaluating outcomes of government programs and policies. DIPA provides an avenue to rectify this deficiency. Comparatively modest data integration capability can potentially improve the efficiency and effectiveness of government policies and programs costing taxpayers many billions of dollars.”
ABS ‘not a magic pudding’
The ABS is worried it won’t be able to continue delivering services to the same quality, as funding has been cut by nearly one-third over the past decade.
Independent benchmarking undertaken in 2016 found the ABS received about half the funding provided to Statistics Canada to deliver broadly equivalent statistical programs.
“Governments, the business sector and others across the community do value the national statistics that are produced each and every day. These statistics are keenly anticipated and used extensively to inform key decisions and public debates,” Kalisch said.
“Unfortunately, widespread reliance on our national statistics for many purposes hasn’t translated into secure funding for our national statistics over the past decade. And ABS funding is not a magic pudding that enables us to produce all of the essential statistics our key data users would like, from a reduced funding base.
“While ABS is now more efficient than it was a decade ago and technology now enables more opportunities to deliver some statistics, ultimately we do need to make choices about what we do, and what we can no longer do, constraining our national information infrastructure.”