The Productivity Commission has called for the creation of an Office of Indigenous Policy Evaluation to help build evaluation and cultural capability across the Australian Public Service.
The recommendation is part of the PC’s proposed Indigenous Evaluation Strategy, which was recently delivered to the government.
Aimed at all APS agencies with a responsibility for designing or implementing policies and programs affecting Aboriginal and Torres Strait Islander people, the strategy sets out a new approach to evaluating those initiatives.
The paper notes that after decades of developing new policies and programs — and modifying existing ones — government agencies know “very little” about how those measures have actually impacted Aboriginal and Torres Strait Islander people, or how outcomes could be improved.
“There is currently no Australian government-wide approach to priority setting for evaluations of policies and programs affecting Aboriginal and Torres Strait Islander people,” the strategy says.
“And while policy makers agree that evidence is critical for good policies, many admit that in practice they do not rely heavily on evidence, or past experience, when formulating or modifying policies and programs.”
As a result, the policies and programs affecting Aboriginal and Torres Strait Islander people have fallen short, according to PC commissioner Romlie Mokak.
“Evaluation can play an important role filling this gap, but regrettably it is often an afterthought and of poor quality,” he says.
“Importantly, Aboriginal and Torres Strait Islander people are rarely asked about what, or how to evaluate, or what evaluation results mean.
“Working in partnership with Aboriginal and Torres Strait Islander people is fundamental to lifting the quality of evaluations, as is planning early so that the right questions are asked and the right data collected.”
The commission notes that reflecting the values, expertise and lived experiences of Aboriginal and Torres Strait Islander people in the evaluation process can improve outcomes. It has placed Aboriginal and Torres Strait Islander people at its centre and aligns with the National Agreement on Closing the Gap.
While the strategy doesn’t endorse particular evaluation approaches or methods — as different evaluation questions and contexts require different approaches — it does aim to complement and build on agencies’ evaluation processes and frameworks, providing a whole-of-government framework.
The overarching principle of the strategy is “centring Aboriginal and Torres Strait Islander people, perspectives, priorities and knowledges”, and acts as the lens through which the strategy’s other principles — credible, useful, ethical and transparent — should be interpreted. These principles frame how agencies should plan and conduct evaluation and how evaluations will be assessed, the paper says.
While the commission believes that agencies should continue to have primary responsibility for conducting evaluations, it has proposed an Office of Indigenous Policy Evaluation (OIPE) and an Indigenous Evaluation Council be established to provide centralised leadership, support, coordination and oversight of the strategy.
One of their joint roles, for example, would be to monitor the performance of agencies against the strategy and identify good and innovative practice in the evaluation of policies and programs.
The OIPE, working in partnership with the Indigenous Evaluation Council, would also establish and maintain a formal set of government-wide evaluation priorities.
The PC has also proposed 10 actions to support an evaluation culture, including:
- Agencies should systematically identify evaluation priorities,
- Departments should develop and publish three year evaluation forward work plans,
- Agencies should prepare an Aboriginal and Torres Strait Islander impact assessment and evaluation plan for new policy or program proposals,
- Agencies, supported by an OIPE, should ensure they have access to the skills required to undertake or commission evaluations consistent with the strategy,
- Agencies should ensure that they have access to, or are able to collect, the data they need to effectively undertake evaluations under the strategy,
- Agencies should strengthen and support Aboriginal and Torres Strait Islander people’s capability to engage, partner and lead in evaluation,
- All evaluation reports should be published,
- Agencies should publish accessible evaluation report summaries,
- Agencies should publish a management response to evaluation findings,
- The federal government should establish an Indigenous Evaluation Clearinghouse.
The commission has proposed an implementation timeline for the strategy, calling for some actions to be implemented within six months, and others within two years. It has also released guidance material for federal government agencies to use when they are selecting, planning, conducting and using evaluations of policies and programs affecting Aboriginal and Torres Strait Islander people.
If implemented, the strategy will improve the quality and usefulness of evaluations and strengthen the evidence base, according to PC chair Michael Brennan.
“Evaluation is about understanding what is working, what is not working, and what changes could be made to make policies and programs work better. Evaluation can also inform the direction of new policies and programs,” he says.
The commission began developing the strategy in 2019, and launched the draft version at the close of National Reconciliation Week earlier this year.