“The notion that we just bolt on evaluation at the back end, or close to the back end of a program, I’m hoping that those days might be passed.”
So said the Productivity Commission’s new Indigenous evaluation lead Romlie Mokak, discussing the project he now leads at a NAIDOC Week event in Canberra. The PC’s job aims to promote more high quality evaluation of any Commonwealth policies and programs in terms of their impact (or lack thereof) on Aboriginal and Torres Strait Islanders, but if this contributes to a more general evaluation renaissance in Canberra, then all the better.
Maybe it’s more realistic to hope those days have begun passing, suggested Ian Anderson, who has a similar responsibility as deputy chief executive of the National Indigenous Australians Agency.
“To analyse and monitor the effectiveness of programs and services for Aboriginal and Torres Strait Islander people, including programs and services delivered by bodies other than the Agency” is one of the NIAA’s new marching orders as an executive agency.
As a group within the Department of the Prime Minister and Cabinet, the same unit’s efforts to develop an evaluation framework fell years behind schedule and the success rate of Indigenous affairs policy is fairly low. But now there is a new minister who promises great things from the new administrative arrangements.
Anderson has only been in the fold since 2017 and was recruited for exactly this kind of purpose – as a university professor of Indigenous background with experience in highly relevant fields of research. At the NAIDOC forum, hosted by the Institute of Public Administration Australia, he noted a big gap between academic scholarship and the practice of evaluation in the public service.
“Research institutionalises knowledge through the universities and through the academic publishing system. That’s not quite the same in the public policy evaluation world.
“We tend to still have a very project-based approach to this [in government] and don’t think about what it is we need to do to create the sort of institutional capability that institutionalises that knowledge.”
As an example, Anderson recalled major efforts to improve “research and evaluation capability” in the Indigenous health sector during the 1990s.
“A recent piece of research work looking at where that evaluation in the Indigenous health space had gone found that the significant majority … had disappeared without trace. Just disappeared into the filing cabinets, non-published. What that said is that actually every time an evaluation was undertaken, it was like bunnies in the headlights.
“This is the first time we’re thinking about this problem in a critical, analytical way, rather than building on and drawing from a whole body of knowledge that is easily accessible and lends to that critical analysis at the front end of the design process.”
Mokak had linked this to another point that has been made regularly and was strongly reinforced by a panel of experts at the event – that Aboriginal and Torres Strait Islander views and knowledge must be involved in policy and program development as well as evaluation.
Evaluation needs to be an integral part of the policy process process from the start and all the time, not an afterthought. And Indigenous perspectives must be front and centre in all relevant targeted pieces of policy and program design, and evaluation.
There are “two broad agendas” that come into play as the PC goes about its new role, Anderson observed.
One is the various attempts at “repositioning the relationship between Indigenous Australians and the public policy process” – which is now generally seen as a vital step in improving outcomes in this specific area – and the other is the “professionalisation of public policy” in general.
He said there were significant opportunities to move from an old approach — “it feels good; it must be right” — to building and using a stronger base of evidence about what has and hasn’t worked in the past, but also challenges. One is the lack of Indigenous people employed in the wider “evaluation services sector” although Anderson sees that changing for the better.
“I think there’s a lot of work that we need to do within our public policy institutions,” the NIAA deputy CEO added.
“Rom went to it when he was talking about the culture of public service institutions around evaluation. There’s a deeper kind of need to really think about what is the capability that we need to do, to really start to build institutions that are evaluation-ready. I think particularly at the EL2 level in organisations, a critical understanding of what good evaluation design looks like and how that then shapes the public policy cycle [is needed].
“What we need to do is think about the analytical capability but also the methodological capability at that level of leadership in the organisation. We can’t create centralised units and do evaluation for big organisations. We need to embed that capability within the organisation.
“With that goes a cultural shift … about why we’re doing this – outward-looking, curious, an understanding of what is that broader world out there.”
For the public sector, he added, this meant not only building the “technical capability” but also the “social capability” to draw Indigenous thought and knowledge into the process.
Mokak emphasised that the PC’s project had only just begun but said he understood “the significance of the role, the weight of expectation but also the great opportunity that comes with [it]” and promised a highly consultative process. He spoke of the meaning of NAIDOC Week, his past experience in related areas of research and the public service, and his responsibility as a delegate to the national convention that produced the Uluru Statement from the Heart.
“In fact, my preference is to be a commissioner at the Productivity Commission, not the ‘Indigenous commissioner’,” he said, going slightly off the prepared script.
“Clearly my focus in my work will be about delivering benefit for our people but I sit around the commission table with my peers and we all work very closely together.”
The venue, Old Parliament House, triggered an anecdote that illustrated what being targeted as a special group by government policy has meant and continues to mean for many of the first people of Australia. Mokak recalled moving into the building’s office space in 2011, as chief executive of the Indigenous Doctors Association, and the significance of being “the first black organisation in this house” to all the staff.
“This is a place where many laws were made to the detriment of our people, so to have an Aboriginal, Torres Strait Islander organisation move into this place was very significant. What we did was cleanse the place – we couldn’t smoke it because of heritage values. We had two Ngangkari, traditional healers from the Anangu Pitjantjatjara Yankunytjatjara lands, come over in that wing and cleanse that place for us to do our good work.
“There were good spirits and not so good spirits. The good spirits the Ngangkari left, and they helped us along that journey. The bad spirits the Ngangkari got rid of, and forever grateful we were for that.”
Mokak said he was well aware of how challenging policy could be but said he had seen examples of success and social progress – like the introduction of low-aromatic fuel to tackle petrol sniffing, in which he was involved as a public servant, or the increasing number of Indigenous medical students, whose numbers are now proportionate to the population.
Bringing Indigenous knowledge to the fore was also a key part of his work as CEO of the Lowitja Institute, he said.
“My learnings over these years are fairly simple, really: that those who are most invested and most impacted must not be assigned to simply be policy render. They must be the designers, the architects, the builders and even the evaluators for impact and change.”
The full discussion, including video and transcript, is available from the IPAA ACT Division website.