Chief statistician David Kalisch faces the media to explain #CensusFail. “Not an attack, nor a hack” says the Assistant Treasurer, so what was it?
Only a few short months ago, public trust wasn’t much of an issue for the Australian Bureau of Statistics.
Despite Tuesday being the third Census to allow completion online, the scale of the technological failure is at the point where IT security experts are openly suggesting chief statistician David Kalisch is not telling the whole truth about what went wrong.
There’s no evidence to cast doubt on the early-morning statement from the ABS that “the 2016 online Census form was subject to four denial of service attacks yesterday of varying nature and severity”.
On the other hand, informed commentators like RMIT University lecturer Mark Gregory, who was interviewed on the ABC’s light-hearted News Breakfast program this morning, see no reason to believe Kalisch until he provides evidence to back up the statement. He’s far from alone in suggesting the load testing the agency did on the site might have been inadequate, and the online form was not fit for purpose.
It’s unlikely in the extreme that Kalisch would come out and tell a bald-faced lie to the Australian public — but certainly a system being accessed by millions of Australians around the same time was always going to be especially vulnerable to a DDoS attack.
The Prime Minister’s cyber security adviser Alastair MacGibbon said this kind of mischief was fairly normal for a government website and the crash was the result of an unlucky “confluence of events” — and even suggested all the public controversy and “conjecture” around this year’s Census had made it more of a target. The increasingly sceptical reception to such explanations, counter-claims and reassurances offered by government figures this morning, and over the past few months, is surely an urgent concern.
Right now, the ABS might want to recall the advice of recently retired Department of Foreign Affairs and Trade secretary Peter Varghese, that the “distorted prism” of social media can give a false impression of “the centre of gravity of a policy issue” — because today, Twitter is aflame with outrage and ridicule.
Even though the overall mood of the nation is a lot more relaxed than Twitter is about the privacy and security concerns, and the online form’s meltdown, the way the decision to retain names and other personal information for four years was explained and defended has no doubt stoked considerable distrust in the ABS and the Census.
Last night’s technological failure obviously only adds to this unfortunate state of affairs, and has led to a major damage control effort starting today, which appears to include an increase in transparency about exactly what went wrong.
Inquiries and investigations to follow
Privacy commissioner Timothy Pilgrim rapidly announced he would investigate the attacks “to ensure that no personal information has been compromised as a result” and says the ABS confirmed to him it decided to shut down the online form to protect personal data. Prime Minister Malcolm Turnbull announced MacGibbon would also look at what went wrong, working with the ABS, Treasury and the Australian Signals Directorate, and upper house cross-bencher Nick Xenophon wants a Senate inquiry.
The first three attacks only “caused minor disruption” and over 2 million forms were successfully submitted and safely stored on the agency’s servers, says Kalisch. He said the system was then deliberately taken offline around 7.30pm as a “precaution” to “ensure the integrity of the data”.
Unhelpfully, the Census Twitter account kept telling people it was all going “smoothly” and they should keep trying to get online, it being August 9 and all, for several hours.
The public woke up to hear “steps have been taken during the night” and, as always in government, everything’s fine — Kalisch says he “can reassure Australians that their data are secure at the ABS”.
Another message is that there’s actually “plenty of time to complete the Census, to well into September, and … fines will not be imposed for completing the Census after Census night”.
There’s a lot of unanswered questions, and a promised 9am update from the ABS was not forthcoming, but Kalisch and the responsible minister, Assistant Treasurer Michael McCormack, faced the media just after 10.30am.
‘Not an attack, nor a hack’
Minister McCormack said this morning that “this was not an attack, nor was it a hack” but “rather, it was an attempt to frustrate the collection of ABS data” by unknown actors.
Security was not compromised and no data was lost, he reiterated, before giving a blow-by-blow account of the “rapid succession” of events that led to a “very cautious” response from the agency.
Kalisch said this was a “prudent precaution” and the ABS would continue to take this “cautious and conservative approach” into the future.
“I need to be sure about the robustness of our arrangements before I put it back online,” he said.
At one point, a router was overloaded by the “large-scale” DDoS attacks, McCormack said. The legitimate users filled in their forms at a peak submission rate of 150 per second, “well within” the capacity of 250 forms per second.
International geo-blocking failed to some degree, Kalisch said, fielding sharp questions from a largely incredulous media pack.
Before the crash: handling of privacy concerns criticised
Yesterday, the Statistical Society of Australia — not an organisation given to hyperbolic rhetoric — indicated its disappointment with the handling of the crucial project. President John Henstridge agrees with what the ABS is trying to do to create more valuable data sets, but said its communications strategy had endangered the whole thing:
“The Statistical Society of Australia is concerned that these changes, brought in with the 2011 Census and repeated in 2016, and that have many potential benefits, have not been handled well.
“In particular, the public whose cooperation is critical for a successful Census does not appear to have been adequately involved, and the reasons for the changes are even now not well publicised. This is an issue of transparency where the ABS needs to do better.”
University of Western Australia Centre for Software Practice director David Glance argues today the load testing may not have been adequate, and argues they have done the whole public sector a disservice:
“[The ABS] have not only damaged their own reputation and their ability to convince anyone to take seriously any of their technical claims, but they have brought into question the ability of any government agency to be able to run technology projects of this scale. This is specifically relevant given the recent discussions about running elections online.”
Yesterday, the chair of the UWA academic board, Associate Professor Cara McNish, rebutted the key arguments against the census boycott movement, in a letter to our sister website Crikey:
“It is clear that the more data you store, and the more tightly it is linked to individuals and can therefore be cross-referenced to other sources, the more effective research (and public benefit) can be achieved. For example, if we tracked all vehicles at all times, the data could be used to optimise traffic programming and road use. If we took this a step further and tracked all people at all times, we could improve public transport as well. And so on, ad absurdum.
“This argument fails, however, to balance potential benefits with issues of informed consent and privacy that are the cornerstones of modern research. One need look no further than the Australian Government’s own National Health and Medical Research Council’s (NHMRC) National Statement on Ethical Conduct in Human Research, which links the value of respect for human beings directly with the scope to make their own decisions.”
If the Census were a normal research project, she points out the only conclusion would be that it “clearly fails” to meet the research ethics guidelines’ requirement for “voluntary choice” or its requirement that participants understand what they’re getting into and its full implications. And she argues:
“Further, given the longitudinal tracking of information that will form the dynamical picture, along with data security questions, it is difficult, if not impossible, to predict the future implications of participation with any certainty.”