ABS’s operational shift: from household and business surveys to integration of big, BIG data

By Harley Dennett

Tuesday August 28, 2018

The Australian Statistician has gone public with what his agency could offer national data users if it wasn’t being bled dry through cuts.

The phrase ‘smell of an oily rag’ is overused in an age of constant efficiency dividends and public service staffing cuts, but there are some organisations that are genuinely running on empty.

The Australian Bureau of Statistics, punished for the 2016 Census, has been at breaking point since the 2018-19 budget cuts, observers have noted — and not for lack of innovation.

David Kalisch has told a conference of market and social researchers the surveys they and governments rely on are at risk as the ABS balances increased costs of collection against the public value.

“If the ABS continues to be subject to efficiency dividends over the next decade, at the same trajectory as it has for the past decade, some of the core information currently taken for granted by governments, business and the community may no longer be available,” Kalisch told the Australian Market and Social Research Society Conference this month. “Our capacity to continue producing all of the detailed statistics around our labour market, industry activity and population would be increasingly at risk.”

Household surveys still have a return rate of about 90% but this is increasingly more difficult to sustain due to costs, availability and attitudes, he said, while they’re doing everything they can to reduce survey burden on businesses.

There is a longer-term shift away from those expensive surveys to using already collected very large data sets — including those for sale on the commercial market, scanner data, web-scraping, satellite data, GPS and telco data.

“While we inevitably do make trade-offs in terms of how we use our available (and generally declining) resources … my strong sense is that ABS has delivered more public value across all of these areas over recent years,” he said.

“We have continued to deliver high quality statistics, reprioritised our statistical program, enhanced data access for users alongside strengthened data secrecy protections, made greater use of available big data options rather than imposing directly on households and businesses, and improved the efficiency and capability of the ABS as an organisation.”

But still, the nation’s data users want more.

Far from offering a wish list reliant on imaginary financing, Kalisch said were a number of areas of high need where national statistics could be improved if resources were available. He highlighted two in particular: measurement of the services sector, and a survey on mental health that had not been done in more than a decade.

Data skilled staff will be hard to retain

Perhaps exceeding the demand for cyber skilled employees, data skills are even more valued in the commercial sector and the ABS could face a hard time keeping staff, Kalisch has warned.

“Over recent years, we have broadened our specialist expertise and drawn in staff with a broader range of expertise and organisational experiences, complementing those with deep ABS expertise. Our flexible work arrangements and positive workplace culture assist with staff attraction and retention as data skills become more highly valued across the economy.”

Engagement and privacy

The scatter-gun approach of the former Turnbull government resulted in data coordination, policy and governance being split across a multitude of entities, of which the ABS has only a small role, despite it’s 110 years of expertise (including its predecessor). Few, however, have been at the centre of a privacy debate as national and as defining as that of the Census.

As the ABS pivots to new sources of information, data integration, confidentiality and privacy are never far from Kalisch’s concerns. He thinks the ABS should be transparent and consultative as it inevitably dips into administrative and commercial data sources, noting it will take ongoing development of their statistical methods. Engaging with the community about how their data is being used for community benefit and protections that underpin its safe use need to improve, he adds.

Kalisch adds that data users across government need to more actively engage with the community about their data practices over coming years.

Top photo: a GovHack team plots how to use ABS Census data.

About the author
Inline Feedbacks
View all comments

The essential resource for effective
public sector professionals