The Enhanced Commonwealth Reporting Framework is forcing public servants to rebuild a culture of evaluation and develop the skills to measure the real impact of the policies and programs they implement, rather than just basic outputs.
The bureaucracy is not very good at acknowledging when something isn’t working, Finance secretary Jane Halton told a recent seminar organised by her department and the Australasian Evaluation Society. Neither is the government of the day, one might observe, in which case a more meaningful reporting framework is all the more important in the long run.
For those worried that greater transparency and more genuine accountability sounds risky, Halton pointed out the Public Governance, Performance and Accountability Act also demands “a more mature approach to risk”.
It’s easy to explain where all the dollars go and what they’re spent on, harder to measure the actual impact a given program achieves in the real world. To do so, public servants must use a more diverse set of methodologies including evaluation, benchmarking and peer reviews, the Finance chief says.
“We need to move away from our focus on quantitative KPIs,” she said, explaining there are a staggering 3500 quantitative KPIs under previous performance reporting arrangements.
She reminded those present that parliamentarians are watching, and some want agencies held accountable for non-compliance with the PGPA Act. ACT Senator Katy Gallagher recently criticised the 11% of agencies which missed the August 31 deadline to have new corporate plans in place, in the Joint Committee of Public Accounts and Audit:
“I think in any other area of public life we look very dimly on people that do not observe their legal obligations, particularly those in senior leadership positions who should know better.”
Said Halton: “So [complete the plans] before that next Senate Estimates round is my strong advice.”
She suggested agencies were struggling to get the plans in place due to the difficulty of setting out those measures of real-world impact that they would later report against. A complete picture of performance requires information on the “how” and “why” — qualitative data — as well as the much simpler “what” that is described by quantitative data. Some Commonwealth entities already use “mature” methods of performance measurement that lead to meaningful monitoring and reporting, the Finance head acknowledged, but the standard of the APS as a whole is insufficient.
Despite the thousands of KPIs and reams of reporting being produced under the old system, parliamentarians are still not happy with the level of actual accountability.
“I don’t know how many of you have been grilled for hours and hours and hours in Senate Estimates, but you could really argue we’ve been spinning our wheels. And certainly there are many still-unhappy senators in relation to what we’re actually achieving with our reports.”
The lost art of evaluation
Earlier in the year, Infrastructure and Regional Development secretary Mike Mrdak said evaluation became something of a lost art in the APS around the year 2000, after Finance gave up central responsibility for it.
The PGPA has brought the need for evaluative thinking back in focus, and Halton advised senior bureaucrats to look to international expertise, share expertise with each other, and “plagiarise shamelessly” from successful work in other jurisdictions.
“Embedding best practice across the whole of the Commonwealth will take many years, but the commitment to do better is apparent,” she said.
“I feel a slight sense of déjà vu because I think I gave my first talk on evaluation in the public sector in about 1985, and we’ve come some distance but it has waxed and waned, so in my department we hope that it’s on the up. But to be frank, producing our first corporate plan under the new PGPA framework was challenging.”
Putting together the corporate plan required new “clarity of purpose” and a clear understanding of the department’s current operating environment, according to its secretary.
“This actually led to some very important, incredibly useful conversations in our leadership team, so of itself, it’s galvanised change,” she said.
Halton added that if asked for a self-assessment of her own department’s first effort at a corporate plan, she and her senior team would argue: “We’ve done OK.” She said they would give themselves a “pass” grade with a note: could do better next time.