Blogging a review to improve its quality
The Australian Bureau of Statistics has commissioned Lateral Economics to estimate the value of the Australian Census to the Australian community. As part of that exercise, we’ve got the go-ahead from ABS to do something that, it seems to me, all public inquiries, and most independent public agencies, should do as a matter of course: run a blog a little like the blog we ran when I chaired the Government 2.0 Taskforce.
Ten years on from the Taskforce, it’s amazing how slow the practice has been to catch on. It seems to me the kind of thing that the Productivity Commission should do, and the Reserve Bank. Anyway, as far as I know, there’s no such blog on offer there or anywhere else much.
One exception is the Bank of England which runs a lively staff blog in which staff explore all manner of questions relating to their research. It greatly enhances both its work and their staff’s morale and the willingness of some of the best to stay at the Bank rather than move off into academia or consultant land.
This exercise gives us an opportunity to trial it again. Accordingly, we will post from time to time outlining the progress of our investigations, particularly highlighting issues we are trying to understand to improve the quality of our review. We’ll be consulting ABS with draft blog posts before posting them to give the ABS an opportunity to provide input. We nevertheless remain responsible for the exercise, and for the content of each post, not the ABS.
If you do have anything to offer, please do so to show how productive this kind of exercise can be. I know myself how productive it is from my own experiences blogging myself not least as part of the Government 2.0 Taskforce.
However, as I’ve grown older it’s struck me how much adoption relies on, well, adoption. If no-one’s doing it, no one does it. Then a few people do it and the obviousness of doing it seems to become accessible to others, even as they keep their in-tray reasonably cleared. Meanwhile, as you ponder what the Romans ever did for us, what’s one thing these people forget to mention?
Valuing the Australian Census
Gene Tunny, Nicholas Gruen, and Matt Balmford, Lateral Economics
The five-yearly Australian Census provides a wealth of data on Australian households.
Those data are used extensively by all levels of governments, industry, and non-profits in decision making and planning.
The Census provides important information on household composition, journey-to-work flows, Indigenous status, and languages spoken at home, among other data.
It is also used extensively to ‘ground-truth’ and so assess and improve the representativeness of a wide variety of data sets.
The ABS understands that all this information is of substantial value, but no careful estimate of its total value has yet been made.
The ABS has engaged Lateral Economics to provide such an estimate.
In recent years, a combination of factors has led to renewed scrutiny of the value of the census, including burgeoning administrative data which could be tapped by national statistical agencies, as well as advances in computing power and data science.
Statistical agencies worldwide have come under pressure from ministers to provide greater justification for the hundreds of millions of dollars of costs of a national census.
For instance, in 2001, the UK House of Commons Treasury Select Committee called for the national census to be justified in cost-benefit terms.
Since then, the UK Office of National Statistics (ONS) has produced a Census Benefits Evaluation Report for the 2011 Census.
It was reported that the UK census yields annual benefits in the order of £500 million.
In 2013, Statistics New Zealand followed the example of the ONS and commissioned a valuation of the NZ census, which reported a return on investment (ROI) of around $5 of net benefits for every $1 of expenditure on the census.
In Australia, too, the issue of the cost of the census has come under scrutiny.
For example, in 1993, a Federal government interdepartmental committee investigated cost reduction opportunities for the census.
However, it recommended maintaining the current census format. Indeed, comprehensive five-yearly censuses are seen as necessary to ensure fair electoral representation given substantial population growth in newly-developed areas, such as Springfield and Yarrabilba in South East Queensland.
The requirement for a five-yearly Census was inserted into the Census and Statistics Act 1905 in 1977 (section 8(1)), following a 1976 High Court decision suggesting the need for more a more frequent Census to ensure fair electoral representation.
A critical question for Lateral Economics’ research project is the extent to which the Census is essential for developing high-quality estimates of populations at various levels, national, state, regional and for small areas, and for various groups in the population, including Indigenous and culturally and linguistically diverse people.
Such estimates are important for electoral representation, but they are also important for the allocation of:
- GST revenues across state and territories;
- resources in health, education, and other social services; and
- capital expenditures, both for social infrastructure and transport infrastructure, and in the private sector, too (e.g. shopping centres, retirement villages, etc.).
Such decisions may depend critically on information such as accurate estimates of Indigenous people in a region, or journey-to-work flows between regions, and the Census is a critical source of information to such estimates and for which there are not obvious substitutes.
Lateral Economics will attempt to quantify the benefits to the Australian community of more accurate decision making regarding the allocation of funds across regions for different purposes.
We will be guided, but not limited, by the methodologies employed in previous studies.
These methodologies rely largely on estimates of the inaccuracies of resource allocation and capital investment that would occur in the absence of Census data.
We will also draw on methodologies developed in these studies for estimating the dollar benefits of, among other things:
- improved public policy development in social policy areas where Census data are important, such as in policies for disadvantaged groups1 higher-quality academic and market research.
Even after the most stringent of efforts, considerable uncertainties will remain in parameterising these questions.
However, this underlines the importance of consulting as widely as possible – with the Australian Electoral Commission, Commonwealth Grants Commission, other federal and state agencies, local governments, and the private sector and NGOs – to develop defensible assumptions to underpin our estimates (guesstimates?).
It is likely we will provide reasonable lower and upper-bound estimates of this ROI.
In undertaking this project, Lateral Economics is not necessarily presuming that there will net benefits to the community from running the Australian Census in its current format and frequency.
We are doing our best to follow the evidence – something that the ABS has stressed its support for.
We note that, based on a cost-benefit analysis, South Africa decided not to run a census in 2016, so the question of whether a national census is valuable is an open one.
But wherever the evidence takes us, it’s critical we get the best evidence we can regarding users’ specific uses of Census data, the benefits they derive from that use, and any reflections they have on what the next best source of data would be in the absence of the Census, and the potential costs of any reduction in data quality.
In future articles, Lateral Economics will discuss our progress and seek feedback on our thinking and analysis to date regarding the different Census benefit values we will be estimating.
In the meantime, if you have any ideas or information that you think would help us, please get in touch via [email protected].
- This could be estimated by assuming a percentage improvement in the quality of related government services and applying that to total spending on those services. The assumption would be informed by consultations with stakeholders.
Nicholas Gruen on Twitter: @NGruen1