03.02.2017

$40m for ‘robust, independent evaluation’ of Indigenous policy


The federal government has announced it is putting $40 million over four years towards better evaluation of federal Indigenous affairs policies, just as the auditor-general’s office reports the centrepiece of the portfolio has been fumbled by the Department of the Prime Minister and Cabinet.

The Australian National Audit Office reports “the department did not effectively implement” the Indigenous Advancement Strategy, which it developed after it took over the IA portfolio in September 2013.

PM&C acknowledged the findings and promised to implement all the recommendations, but also noted the very challenging timeframe imposed on it, and now is in an improved situation as a result: “Having moved through the transition period, with greater experience among applicants and stakeholders, the Strategy is moving into a more mature phase of implementation that draws on lessons learnt.”

The massive machinery-of-government change that followed the election of the Abbott government and the IAS have both been criticised heavily in the years since. The odd situation of having over 1800 staff from eight different agencies together bolted on to the government’s key central agency created a range of administrative difficulties, according to one research project.

The IAS was, according to PM&C, “a major reform in the administration and delivery of services and programs to Indigenous Australians in response to widespread criticism over many years.” It consolidated 150 funded activities under 27 separate programs into five streams, focused on specific priority outcomes. According to the audit conclusions:

“Planning and design for the Strategy was conducted in a seven week timeframe, which limited the department’s ability to fully implement key processes and frameworks, such as consultation, risk management and advice to Ministers, as intended.”

Implementation was also rushed, “and this affected the department’s ability to establish transitional arrangements and structures that focused on prioritising the needs of Indigenous communities” in the view of the auditors.

Administration of funding grants also “fell short of the standard required to effectively manage a billion dollars of Commonwealth resources” and the reasoning for recommending particular projects to the minister for funding was not clear — which means “limited assurance is available that the projects funded support the department’s desired outcomes”.

Funding applications were not assessed according to the guidelines issued by PM&C or the department’s public statements. It failed to meet some obligations under the rules and guidelines for all Commonwealth Grants, failed to keep records of some “key decisions” and did not establish performance targets for all projects that received funding.

Performance measures in general were not up to scratch and it seems likely the minister’s $40 million of new money for evaluation was encouraged by the report:

“The performance framework and measures established for the Strategy do not provide sufficient information to make assessments about program performance and progress towards achievement of the program outcomes. The monitoring systems inhibit the department’s ability to effectively verify, analyse or report on program performance.

“The department has commenced some evaluations of individual projects delivered under the Strategy but has not planned its evaluation approach after 2016-17.”

Minister for Indigenous Affairs Nigel Scullion said today the $10 million a year would pay for a “multi-year programme of evaluations … underpinned by a formal Evidence and Evaluation Framework to strengthen reporting, monitoring and evaluation at a contract, programme and outcome level” and put a positive spin on the IAS.

The minister also issued an unusual statement hitting back at the audit office, claiming the government was “a step ahead” of its latest recommendations and politicising the auditor-general’s independent role. Scullion said the report “should be viewed as a historical observation on a process carried out two-and-a-half years ago” and was a “premature” assessment of the success of the IAS. He said ANAO should have spent more time attacking his political opponents and suggested it was more concerned with contracted service providers than Indigenous communities:

“Any fair and reasonable assessment of the IAS needs to consider a timeframe well beyond the introductory period to give the strategy time to deliver the intended benefits.”By focusing its audit on the grants round, the ANAO has paid insufficient regard to the state Indigenous Affairs was in when the Coalition Government came to office in 2013 – and hence the need for the Government to implement its reforms.

“… It is unfortunate the ANAO, in undertaking its audit, consulted primarily with service providers that were competing for funding – not the communities that have benefited from the Government’s reforms.”

The Productivity Commission, among others, has previously identified a lack of genuine policy evaluation as a key problem in the IA policy space and after years of lacklustre achievements and Close the Gap reports, the new evaluation push is surely a welcome change for those in the sector. The manager of the Centre for Independent Studies’ Indigenous Research Program, Sara Hudson, said the announcement was “an acknowledgment that the current method of delivering programs is not working” and pointed to her previous finding that very few Indigenous affairs programs have been evaluated over the years.

“But the lack of evidence on the effectiveness of Indigenous programs is something the Australian government has been aware of for some time now. For years government has claimed to be focused on delivering evidence based policy, this has largely been empty rhetoric,” she added.

“If the government is serious about making a difference to Indigenous Australians’ lives, simply providing more money to evaluators will not be enough. The government must use the findings of the evaluations to improve the way programs and services are funded and delivered.”

Both of Scullion’s announcements and the PM&C response repeat the claim that the flagship policy “greatly improved the transparency and accountability of Indigenous Affairs funding” by making it clearer how much money is being spent and what outcomes it is expected to achieve. While it appears the IAS is clearly a troubled policy, the minister presents it as a good thing that will only get better with more evaluation:

“By establishing a multi-year funding allocation, we are ensuring there will be a long-term plan for evaluation and a formal strategy to monitor and review how individual contracts and programme streams are contributing to our efforts to deliver better outcomes for Indigenous Australians.

“Evaluation at the contract, programme and outcome level will ensure we not only know where the money is being spent, but we will know what works and why. This is important for the Government and taxpayers, but more important for communities in whose name the money is spent.

“It will also mean we will be better able to assess where our investment needs to be focused in the future – and ensure the IAS continues to deliver outcomes for Indigenous communities.”

Scullion’s funding announcement also put a positive spin on the unprecedented machinery-of-government change. In his view, Abbott’s bold move “created a regional network which is now working in partnership with Indigenous communities across urban, regional and remote Australia” and says the IAS set the stage for better outcomes:

“These reforms have enabled a far more strategic and flexible approach to the Government’s investment in Indigenous Affairs to achieve better outcomes on the ground while relieving administrative burden and red tape for organisations servicing Indigenous communities.”

The minister explains “experts will be engaged to undertake robust, independent evaluation of IAS activities to assess their impact and whether any improvements should be made” through the new $40m program.

However, the minister has shown a propensity to reject robust evaluation by independent experts that says anything negative about one of his programs. In December, Scullion was far from impressed by a research project about the Community Development Programme and attacked the two academics in question, claiming their report was not backed up by the “real” evidence.

Today, the minister’s office says the new evaluation framework will have a specific focus on “developing a better base of evidence that is accessible and useful for specific communities” and will be rolled out in “close consultation” with Aboriginal and Torres Strait Islander people:

“Under the IAS, First Australians are already involved in evaluation, including collecting data and undertaking surveys. Indigenous-run companies are currently delivering rigorous evaluation for the Government and this new framework will continue this partnership.

“This reflects the Coalition’s absolute commitment to working with Aboriginal and Torres Strait Islander people to improve outcomes for our First Australians.”

Meanwhile, the Prime Minister’s Indigenous Advisory Council has ceased functioning temporarily after the terms of its members expired on January 31. The ABC reports new members are likely to be appointed next week.