NDIS case-study: 'build learning and adaptation into structure'

By David Donaldson

August 5, 2015

A greater openness to learning and risk through ‘adaptive management’ should be built into the National Disability Insurance Scheme to avoid some of the problems of recent policy implementation failures, argues Dr Gemma Carey, research fellow at the Regulatory Network at the Australian National University.

Gemma Carey
Gemma Carey

High-profile implementation failures such as the ‘pink batts’ scheme, the social inclusion agenda or the ever-evolving job network scheme highlight the risks to both government and citizens of complex, important programs built without sufficient adaptability to change course during implementation or when problems inevitably arise.

“Once you put a system in place it’s very hard to roll back and start again afresh,” Carey told The Mandarin. The key, then, is to make sure it’s built from the start to allow for continual evaluation and adjustment — something government still struggles with.

Given the lack of models on which to base the rollout of the NDIS, learning and adaptation will need to play an especially important role. No-one knows exactly how it will look at the end, as no-one has ever constructed a system quite like it.

Significant unknowns remain. In the South Australian pilot, even the fundamental question of how many people were expected to access services has presented challenges, with almost twice as many children being found to be eligible as anticipated.

Carey and Dr Mark Matthews have penned a working paper, Better methods for delivering adaptive regulation in public management: an application to the NDIS, in an effort “to bring more minds to the problem” of working out how the NDIS will work on the ground, says Carey. The paper states:

“Successful implementation of the NDIS confronts two of the largest, and currently unsolved, challenges of public administration — the use of market arrangements in service delivery and the ability to build flexibility and learning into public management structures while still maintaining a level of control.”

In addition to the need to adapt design to issues as they arise in this relatively new area of policy, Carey and Matthews hope an adaptive approach will help overcome some of the known problems of existing structures. Social programs have a tendency to become increasingly geared towards top-down compliance, ingraining rigidity and stifling the ability of programs to adapt. This in turn has undermined customer outcomes.

“What if we were to put in place a structure that can adapt and learn? No one should be expected to nail this out of the gate.”

Instead, argue Carey and Matthews, building adaptation and learning into the management structure — creating space for managed risk-taking in pursuit of better implementation and outcomes – could help overcome the endemic problems other programs have experienced.

“There can be a fear of being experimental,” thinks Carey. “We tend to select instruments and stick with them and don’t adapt when there are problems”.

“Let’s not go down that path with the NDIS. What if we were to put in place a structure that can adapt and learn? No one should be expected to nail this out of the gate. Instead, we could take risk earlier in the process, rather than then trying to manage risk in terms of fraud down the track.”

“The conducive and vibrant comparative testing environment … encourages this ‘bottom’ up approach by providing the concepts, methods and values needed.”

The paper — which Carey says is a working paper and far from the final word on the subject — applies Bayesian theory to propose an “adaptive management” approach to implementation. “Adaptive management is used a lot in ecological and resource management, but it hasn’t really been brought into the social policy space,” she says. “It’s about seeing how we can build in learning in a systematic way — putting in an architecture that puts learning at the forefront of the process.”

The authors propose the incorporation of specially created diagnostic unit which would feed analytical findings (based on Bayesian modelling, as is frequently used in medicine and healthcare provision) into the governance structures of the NDIS:

“To guide the implementation and delivery of the NDIS, an adaptive learning cycle could be established between the diagnostic unit and the governance architecture already in place (namely the Department of Social Services and the National Disability Insurance Agency).”

If this system functions how it is supposed to, service delivery will improve with time, despite the initial risks involved in implementation and delivery experimentation. The analysis facilitated by this process would also help demonstrate what impact the NDIS is having, and whether it is achieving its aims.

Carey and Matthews argue that incorporating their suggestions into program design would empower staff to find their own insights in the data. They believe it:

“…opens up a rich new avenue for people working on the ground at the coalface of service delivery to suggest imaginative new hypotheses based on their accumulating experience and rich tacit knowledge – both diagnostic and as experimental interventions. The conducive and vibrant comparative testing environment that we are alluding to encourages this ‘bottom’ up approach by providing the concepts, methods and values needed.”

Partly it’s also a question of changing the culture of government to embrace measured risk, says Carey. “What if you stopped being quite so scared about doing the wrong thing, because that’s probably going to make you do the wrong thing?”

She says she’s seeing a resurgence of interest in implementation after a period of neglect.

“It’s so often treated as though ‘implementation will just fall out once we legislate’. But what’s actually the worst case scenario? It’s not just you muck up this reform — you might also muck up the next reform, because this one sets the tone and the culture for future reforms. We might end up with worse services across the board as everyone works around this big, inflexible structure that the NDIS could become,” argues Carey.

“There’s a lot of good will that could be capitalised on around the NDIS, so let’s make sure we do this right. This may require government to slow down the rate of implementation and allow for more planning and development”

Read more at The Mandarin: The NDIS: it’s not (yet) about community impact

About the author
1 Comment
Newest Most Voted
Inline Feedbacks
View all comments
Julian Thomas
Julian Thomas
7 years ago

From an evaluator’s perspective, I think the adaptive approach to program delivery, informed by intelligent and integrated evaluative effort is critical to program sustainability and success. Happily, we are seeing more opportunities for evaluation professionals to engage with implementation in early stages. It might be fair to say that there is a non-negligible risk of ‘evaluation capture’ – but in my view this is far outweighed by the benefits that accrue from evaluators’ contributing as critical friends to the growth and success of new programs.


The Mandarin Premium

Insights & analysis that matter to you

Subscribe for only $5 a week


Get Premium Today